Transfer Learning for Wireless Fingerprinting Localization Based on Optimal Transport

The indoor localization systems based on wireless local area network received signal strength (RSS) have been widely applied due to the simplicity of system deployment as well as easy implementation on various mobile devices like the smartphones. However, they are often suffered by the major drawback of the extensive effort for location fingerprinting which is significantly labor-intensive and time-consuming. In response to this compelling problem, we design an improved manifold alignment approach to construct a cost-efficient radio map which consists of the sparsely collected location fingerprints and crowdsourcing RSS data with the purpose of reducing the overall fingerprints calibration effort. A new graph construction scheme which is proved to be the optimal choice to model the smoothness assumption in semi-supervised learning is proposed to explore the informativeness conveyed by location fingerprints during the process of radio map construction. In addition, the concept of execution characteristic function is considered to minimize the RSS sample capacity at each reference point to reduce fingerprints calibration effort further. Finally, the extensive experimental results demonstrate the performance improvement by the proposed system with the probability of localization errors within 3 m, 79.60%, which is at most 26.30 percentages higher than the one by the existing systems using location fingerprints solely.

Wi-Fi ingerprint-based localization is currently the most promising approach to building metropolitan-scale localization systems for both indoor and outdoor environments. A Wi-Fi fingerprint localization system makes use of a radio map that contains Wi-Fi signals received at various locations, and uses the map for location estimation. Thus, the accuracy of a Wi-Fi fingerprint localization system greatly depends on the quality of its radio map. This paper explores various properties of a metropolitan radio map that may affect the accuracy of Wi-Fi fingerprint localization systems. In this study, metropolitan-scale radio maps are obtained using war walking and war driving. These maps contain hundreds of thousands of access points and signal samples. A detailed comparison analysis of selected radio map properties reveals how different map properties affect the difference between the positional accuracies of the driving and walking radio maps in Wi-Fi fingerprint-based localization.

In Wi-Fi based location fingerprinting, the cost of constructing a radio map is high due to the calibration of measured RSS data at a large number of RPs and the update of database. Interpolating a coarse radio map into a dense one may reduce the cost. An interpolated radio map, however, has low accuracy, especially at space discontinuity such as a wall. We present a method of constructing a high-density radio map which preserves the discontinuity of RSS by localized smoothing in accordance with the layout of a building. Experimental results show that the radio map by discontinuity preserving smoothing has higher accuracy than conventional interpolating methods. With sampling density ≥ 35%, the performance is close to a genuine full density radio map. With sampling density > 60%, the performance is even better than the original full density map.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

Multi-source transfer learning via optimal transport feature ranking for EEG classification

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue.

The concept of transfer learning has received a great deal of concern and interest throughout the last decade. Selecting an ideal representational framework for instances of various domains to minimize the divergence among source and target domains is a fundamental research challenge in representative transfer learning. The domain adaptation approach is designed to learn more robust or higher-level features, required in transfer learning. This paper presents a novel transfer learning framework that employs a marginal probability-based domain adaptation methodology followed by a deep autoencoder. The proposed frame adapts the source and target domain by plummeting distribution deviation between the features of both domains. Further, we adopt the deep neural network process to transfer learning and suggest a supervised learning algorithm based on encoding and decoding layer architecture. Moreover, we have proposed two different variants of the transfer learning techniques for classification, which are termed as (i) Domain adapted transfer learning with deep autoencoder-1 (D-TLDA-1) using the linear regression and (ii) Domain adapted transfer learning with deep autoencoder-2 (D-TLDA-2) using softmax regression. Simulations have been conducted with two popular real-world datasets: ImageNet datasets for image classification problem and 20_Newsgroups datasets for text classification problem. Experimental findings have established and the resulting improvements in accuracy measure of classification shows the supremacy of the proposed D-TLDA framework over prominent state-of-the-art machine learning and transfer learning approaches.

Document classification is a challenging task to the data being high-dimensional and sparse. Many transfer learning methods have been investigated for improving the classification performance by effectively transferring knowledge from a source domain to a target domain, which is similar to but different from the source domain. However, most of the existing methods cannot handle the case that the training data of the target domain does not have labels. In this study, we propose a transductive transfer learning system, utilizing solutions evolved by genetic programming (GP) on a source domain to automatically pseudolabel the training data in the target domain in order to train classifiers. Different from many other transfer learning techniques, the proposed system pseudolabels target-domain training data to retrains classifiers using all target-domain features. The proposed method is examined on nine transfer learning tasks, and the results show that the proposed transductive GP system has better prediction accuracy on the test data in the target domain than existing transfer learning approaches including subspace alignment-domain adaptation methods, feature-level-domain adaptation methods, and one latest pseudolabeling strategy-based method.

The purpose of transfer learning is to utilize the knowledge gained from the existing (source) domain to enhance the performance on a distinct but related (target) domain. Existing works on transfer learning are not capable of optimizing different quality measures (components) such as minimizing the marginal distribution, minimizing the conditional distribution, maximizing the target domain variance, modeling the manifold by utilizing the common geometric properties in the source as well as the target domain at the same time. Moreover, existing transfer learning methods use conventional approaches to determine the appropriate values of their parameters, which is very hectic and time-consuming. Therefore, in order to overcome the drawbacks of existing approaches, we propose a Particle Swarm Optimization based Parameter Selection Approach for Unsupervised Discriminant Analysis (UDATL-PSO) in transfer learning framework. In UDATL-PSO, all the quality measures are considered at the same time, as well as the PSO approach has been used to select the best values of their parameters. Extensive experiments on various transfer learning tasks show that the proposed method has a significant influence on state-of-the-art methods.

Location fingerprinting provides localization for devices in indoor environments using existing Wireless Local Area Network (WLAN) infrastructure. However, the initial offline calibration which is required for these types of systems is non-trivial and requires significant labour cost. The current practice is to collect measurements at calibration points in a uniform grid for the area which is covered by the radio map. However, not all calibration points are resolvable in signal space and addition of unresolvable calibration points does not improve the localization accuracy. This paper presents Spatial Aware Signal Space Clustering (S3C) clustering algorithm which analyses walk test data for identifying these unresolvable calibration points prior to calibration phase. Simulation based studies shows that the algorithm is able to reduce the labour cost of calibration phase while preserving the localization accuracy.

Radio fingerprinting positioning is widely used for smartphone location-based services and Internet of Things applications given its high availability and low cost. Radio fingerprinting-based algorithms, however, are subject to the forced matching problem and often yield estimated positions even when a user is actually located outside of the fingerprint region. A positioning solution in multistory buildings should be able to locate positions accurately on the current floor; but these methods may generate unreasonable positioning trajectories, such as irrationally passing through a wall, when fingerprinting positioning is fused the with inertial measurement unit to further improve the accuracy. A radio map must be surveyed dynamically on-the-move, as a dynamic fingerprint, to reduce the time costs and on-site workload. Unlike static fingerprint-based methods, dynamic fingerprinting samples are sparse. Thus, we propose an enhanced indoor positioning solution using spatial context knowledge, extracted from the sparse dynamic fingerprints. In the offline stage of radio map calculation, we extract the dynamic fingerprint features and store them in a spatial features database to reduce the computational time and storage space complexity. The proposed floor detection, region recognition, and path correction algorithms identify the online spatial contexts from the stored spatial features to improve positioning performance. This solution was applied on smartphones combining WiFi and Bluetooth low-energy radio signals in two typical scenarios. The experimental results show that the floor detection accuracy reached 99% while region recognition accuracy reached 90.75%. The positioning path correction method enhances the accuracy of smartphone indoor positioning from 3.27 to 2.56 m.

Automated radio telemetry systems are one of the most widely applicable methods for tracking wildlife at fine spatiotemporal scales. A growing trend is for these systems to apply large networks of receivers to detect radio signal strength (RSS) from radio‐tagged individuals. Analytical methods to derive position estimates using RSS localisation, however, are relatively underdeveloped in the field of wildlife tracking. Here, we apply approaches from indoor positioning systems to develop a new method, radio fingerprinting, for localising radio‐tagged animals in structurally complex, outdoor environments. This method characterises the RSS patterns at known locations to generate a radio map of the study area, which is then used to predict the location of new RSS patterns. We conducted field tests to evaluate the localisation accuracy of radio fingerprinting relative to multilateration, a commonly used method for RSS localisation. To do so, we established an experimental receiver network covering multiple habitat types and compared the localisation accuracy of radio fingerprinting to multilateration. Additionally, we evaluated how a variety of features characteristic of typical tracking data sets affected the accuracy of both methods. While both methods had a similar median error under ideal conditions (~30 m), radio fingerprint localisation method offered several advantages over multilateration. Multilateration localisation estimates were highly affected by missed detections from the nearest receiver to the test point, which occurred in 30% of cases. In these cases, the median error was 103 m, over a 3.5‐fold increase. Distance to the nearest receiver also biased multilateration estimates with error increasing as the distance increased. Additionally, errors from multilateration estimates were higher in more densely vegetated areas. In contrast, fingerprinting position estimates were largely robust to each of these scenarios. Automated radio telemetry enables the fine‐scale, continuous tracking of range‐resident animals. We present radio fingerprinting as a localisation method in outdoor environments where standard localisation methods are error‐prone due to habitat heterogeneity and missed detections. This approach can be applied to any automated radio telemetry hardware and to any study system where a radio map can be generated.

Recently, transfer learning has gained popularity in the machine learning community. Transfer Learning (TL) has emerged as a promising paradigm that leverages knowledge learned from one or more related domains to improve prediction accuracy in a target domain with limited data. However, for time series forecasting (TSF) applications, transfer learning is relatively new. This paper addresses the need for empirical studies as identified in recent reviews advocating the need for practical guidelines for Transfer Learning approaches and method designs for time series forecasting. The main contribution of this paper is the suggestion of a comprehensive framework for Transfer Learning Sensitivity Analysis (SA) for time series forecasting. We achieve this by identifying various parameters seen from various angles of transfer learning applied to time series, aiming to uncover factors and insights that influence the performance of transfer learning in time series forecasting. Undoubtedly, symmetry appears to be a core aspect in the consideration of these factors and insights. A further contribution is the introduction of four TL performance metrics encompassed in our framework. These TL performance metrics provide insight into the extent of the transferability between the source and the target domains. Analyzing whether the benefits of transferred knowledge are equally or unequally accessible and applicable across different domains or tasks speaks to the requirement of symmetry or asymmetry in transfer learning. Moreover, these TL performance metrics inform on the possibility of the occurrence of negative transfers and also provide insight into the possible vulnerability of the network to catastrophic forgetting. Finally, we discuss a sensitivity analysis of an Ensemble TL technique use case (with Multilayer Perceptron models) as a proof of concept to validate the suggested framework. While the results from the experiments offer empirical insights into various parameters that impact the transfer learning gain, they also raise the question of network dimensioning requirements when designing, specifically, a neural network for transfer learning.

Transfer learning has been considered a key solution for the problem of learning when there is a lack of knowledge in some target domains. Its idea is to benefit from the learning on different (but related in some way) domains that have adequate knowledge and transfer what can improve the learning in the target domains. Although incompleteness is one of the main causes of knowledge shortage in many machine learning real-world tasks, it has received a little effort to be addressed by transfer learning. In particular, to the best of our knowledge, there is no single study to utilize transfer learning for the symbolic regression task when the underlying data are incomplete. The current work addresses this point by presenting a transfer learning method for symbolic regression on data with high ratios of missing values. A multi-tree genetic programming algorithm based feature-based transformation is proposed for transferring data from a complete source domain to a different, incomplete target domain. The experimental work has been conducted on real-world data sets considering different transfer learning scenarios each is determined based on three factors: missingness ratio, domain difference, and task similarity. In most cases, the proposed method achieved positive transductive transfer learning in both homogeneous and heterogeneous domains. Moreover, even with less significant success, the obtained results show the applicability of the proposed approach for inductive transfer learning.

Bearing is an important component in mechanical equipment. Its main function is to support the rotating mechanical body and reduce the friction coefficient and axial load. In the actual operating environment, the bearings are affected by complex working conditions and other factors. Therefore, it is very difficult to effectively obtain data that meets the conditions of independent and identical distribution of training data and test data, which result in unsatisfactory fault diagnosis results. As a transfer learning method, joint distribution adaptive (JDA) can effectively solve the learning problem of inconsistent distribution of training data and test data. In this paper, a new bearing fault diagnosis method based on JDA and deep belief network (DBN) with improved sparrow search algorithm (CWTSSA), namely JACADN is proposed. In the JACADN, the JDA is employed to carry out feature transfer between the source domain samples and target domain samples, that is, the source domain samples and target domain samples are mapped into the same feature space by the kernel function. Then the maximum mean difference is used as the metric to reduce the joint distribution difference between the samples in the two domains. Aiming at the parameter selection of the DBN, an improved sparrow search algorithm (CWTSSA) with global optimization ability is used to optimize the parameters of the DBN in order to construct an optimized DBN model. The obtained source domain samples and target domain samples are divided into training set and test set, which are input the optimized DBN to construct a bearing fault diagnosis model for improving the diagnosis accuracy. The effectiveness of the proposed method is verified by vibration data of QPZZ-II rotating machinery. The experimental results show that the proposed JACADN method can effectively improve the fault diagnosis accuracy of rolling bearings under variable operating conditions.