Unsupervised Domain Adaptive Fundus Image Segmentation with Category-Level Regularization

Sensors are devices that output signals for sensing physical phenomena and are widely used in all aspects of our social production activities. The continuous recording of physical parameters allows effective analysis of the operational status of the monitored system and prediction of unknown risks. Thanks to the development of deep learning, the ability to analyze temporal signals collected by sensors has been greatly improved. However, models trained in the source domain do not perform well in the target domain due to the presence of domain gaps. In recent years, many researchers have used deep unsupervised domain adaptation techniques to address the domain gap between signals collected by sensors in different scenarios, i.e., using labeled data in the source domain and unlabeled data in the target domain to improve the performance of models in the target domain. This survey first summarizes the background of recent research on unsupervised domain adaptation with time series sensor data, the types of sensors used, the domain gap between the source and target domains, and commonly used datasets. Then, the paper classifies and compares different unsupervised domain adaptation methods according to the way of adaptation and summarizes different adaptation settings based on the number of source and target domains. Finally, this survey discusses the challenges of the current research and provides an outlook on future work. This survey systematically reviews and summarizes recent research on unsupervised domain adaptation for time series sensor data to provide the reader with a systematic understanding of the field.

Bionic limb control through myoelectric pattern recognition, offering intuitive decoding of motor intent, can improve the quality of life for individuals with amputations. However, most work on pattern recognition only uses a small subset of the myoelectric data generated during daily life, to train an Artificial Neural Network (ANN) via Supervised Learning (SL). Scenarios substantially different from the recording session e.g. different limb positions, can lead to misclassifications by the ANN during everyday usage of the bionic limb. Recording labeled data from all scenarios encountered in daily life could alleviate the problem, but would be prohibitively time consuming. Unsupervised Domain Adaptation (UDA) offers a solution by leveraging unlabeled data from a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">target domain</i>, not represented in the labeled dataset i.e. the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">source domain</i>, to calibrate ANNs for improved performance. In this study we explore the potential of two UDA algorithms for domain shifts in myoelectric pattern recognition: Domain Adversarial Neural Networks (DANN) and Sliced Wasserstein Discrepancy (SWD). Offline evaluation identified SWD as the best-performing algorithm, which was subsequently validated in online experiments with 11 participants. Using UDA improved the performance on the target domain by 19% compared to an ANN trained through SL on data from the source domain only. Indeed, it nearly matched the performance of an ANN trained through SL on labeled data from both the source and target domain. This is the first time UDA was validated in online experiments as a viable approach to overcome domain shift problems caused by changes in limb position.

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.

Unsupervised domain adaptation aims to transfer knowledge from labeled source domain to unlabeled target domain. Recently, multisource domain adaptation (MDA) has begun to attract attention. Its performance should go beyond simply mixing all source domains together for knowledge transfer. In this article, we propose a novel prototype-based method for MDA. Specifically, for solving the problem that the target domain has no label, we use the prototype to transfer the semantic category information from source domains to target domain. First, a feature extraction network is applied to both source and target domains to obtain the extracted features from which the domain-invariant features and domain-specific features will be disentangled. Then, based on these two kinds of features, the named inherent class prototypes and domain prototypes are estimated, respectively. Then a prototype mapping to the extracted feature space is learned in the feature reconstruction process. Thus, the class prototypes for all source and target domains can be constructed in the extracted feature space based on the previous domain prototypes and inherent class prototypes. By forcing the extracted features are close to the corresponding class prototypes for all domains, the feature extraction network is progressively adjusted. In the end, the inherent class prototypes are used as a classifier in the target domain. Our contribution is that through the inherent class prototypes and domain prototypes, the semantic category information from source domains is transformed into the target domain by constructing the corresponding class prototypes. In our method, all source and target domains are aligned twice at the feature level for better domain-invariant features and more closer features to the class prototypes, respectively. Several experiments on public data sets also prove the effectiveness of our method.

Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.

Unsupervised domain adaptation is mainly focused on the tasks of transferring knowledge from a fully-labeled source domain to an unlabeled target domain. However, in some scenarios, the labeled data are expensive to collect, which cause an insufficient label issue in the source domain. To tackle this issue, some works have focused on few-shot unsupervised domain adaptation (FUDA), which transfers predictive models to an unlabeled target domain through a source domain that only contains a few labeled samples. Yet the relationship between labeled and unlabeled source domains are not well exploited in generating pseudo-labels. Additionally, the few-shot setting further prevents the transfer tasks as an excessive domain gap is introduced between the source and target domains. To address these issues, we newly proposed an adaptive dispersal and collaborative clustering (ADCC) method for FUDA. Specifically, for the shortage of the labeled source data, a collaborative clustering algorithm is constructed that expands the labeled source data to obtain more distribution information. Furthermore, to alleviate the negative impact of domain-irrelevant information, we construct an adaptive dispersal strategy that introduces an intermediate domain and pushes both the source and target domains to this intermediate domain. Extensive experiments on the Office31, Office-Home, miniDomainNet, and VisDA-2017 datasets showcase the superior performance of ADCC compared to the state-of-the-art FUDA methods.

Unsupervised partial domain adaptation (PDA) is a unsupervised domain adaptation problem which assumes that the source label space subsumes the target label space. A critical challenge of PDA is the negative transfer problem, which is triggered by learning to match the whole source and target domains. To mitigate negative transfer, we note a fact that, it is impossible for a source sample of outlier classes to find a target sample of the same category due to the absence of outlier classes in the target domain, while it is possible for a source sample of shared classes. Inspired by this fact, we exploit the cycle inconsistency, i.e., category discrepancy between the original features and features after cycle transformations, to distinguish outlier classes apart from shared classes in the source domain. Accordingly, we propose to filter out source samples of outlier classes by weight suppression and align the distributions of shared classes between the source and target domains by adversarial learning. To learn accurate weight assignment for filtering out outlier classes, we design cycle transformations based on domain prototypes and soft nearest neighbor, where center losses are introduced in individual domains to reduce the intra-class variation. Experiment results on three benchmark datasets demonstrate the effectiveness of our proposed method.KeywordsUnsupervised partial domain adaptationNegative transferCycle inconsistency

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.

The ability of supervised image analysis methods to provide state-of-the-art performance is limited by availability of high-quality labeled data in large quantities. Domain adaptation approaches propose a solution to this problem by leveraging quality labeled information from auxiliary data sources. In this work, we use adversarial learning for domain adaptation for remote sensing applications. First, we approached the problem of unavailable target domain labels with unsupervised domain adaptation and then extended our method for semisupervised domain adaptation to use a few available labels as well. We are using adversarial learning to extract discriminative target domain features that are aligned with source domain. We test our framework for two very different applications of remote sensing imagery, multiclass classification in hyperspectral images and semantic segmentation in large scale satellite images. For hyperspectral image analysis two datasets were used: the University of Houston shadow data was used for quantifying the efficacy of our approach to varying illumination, and the Botswana data was used to quantify the efficacy of our approach under multitemporal spectral shifts. Multisensor high-resolution images from National Agriculture Imagery Program and SpaceNet-Rio datasets were used as the source and target for the task of building extraction for large scale semantic segmentation based domain adaptation.

Electroencephalogram (EEG) signals convey abundant physiological and psychological information. Decoding EEG is fundamental for brain-computer interaction and medical rehabilitation. Nevertheless, the nonstationarity and inter-individual variability of EEG signals impedes current models from achieving robust crosssubject generalization without expensive subject-specific recalibration, thereby restricting practical deployment. Unsupervised domain adaptation (UDA) aims to improve generalization by minimizing distribution discrepancies between source and target domains. Recent studies regard different subjects as distinct domains and leverage UDA to facilitate cross-subject EEG classification by learning domain-invariant features through discrepancy minimization or adversarial training. However, these conventional methods are all developed in Euclidean space, which are insufficient to capture the non-linear structures inherent in EEG data. To address this limitation, we propose a Deep Adversarial Neural Network on the Symmetric Positive Definite (SPD) matrix manifold, referred to as SPD-DANN, which facilitates the extraction of subject-invariant features through adversarial learning. Additionally, we design an SPD domain feature alignment loss and an SPD class prototype pair loss to simultaneously promote feature alignment across subjects and enhance feature space discriminability. Extensive experiments on four brain-computer interface (BCI) datasets demonstrate that our method surpasses several state-of-the-art UDA techniques. Furthermore, the proposed loss functions are readily adaptable to broader unsupervised or semi-supervised domain adaptation frameworks.

Domain consistency regularization for unsupervised multi-source domain adaptive classification

Active Dynamic Weighting for multi-domain adaptation

With the rapid development of the remote sensing monitoring and computer vision technology, the deep learning method has made a great progress to achieve applications such as earth observation, climate change and even space exploration. However, the model trained on existing data cannot be directly used to handle the new remote sensing data, and labeling the new data is also time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) is one of the solutions to the aforementioned problems of labeled data defined as the source domain and unlabeled data as the target domain, i.e., its essential purpose is to obtain a well-trained model and tackle the problem of data distribution discrepancy defined as the domain shift between the source and target domain. There are a lot of reviews that have elaborated on UDA methods based on natural data, but few of these studies take into consideration thorough remote sensing applications and contributions. Thus, in this paper, in order to explore the further progress and development of UDA methods in remote sensing, based on the analysis of the causes of domain shift, a comprehensive review is provided with a fine-grained taxonomy of UDA methods applied for remote sensing data, which includes Generative training, Adversarial training, Self-training and Hybrid training methods, to better assist scholars in understanding remote sensing data and further advance the development of methods. Moreover, remote sensing applications are introduced by a thorough dataset analysis. Meanwhile, we sort out definitions and methodology introductions of partial, open-set and multi-domain UDA, which are more pertinent to real-world remote sensing applications. We can draw the conclusion that UDA methods in the field of remote sensing data are carried out later than those applied in natural images, and due to the domain gap caused by appearance differences, most of methods focus on how to use generative training (GT) methods to improve the model’s performance. Finally, we describe the potential deficiencies and further in-depth insights of UDA in the field of remote sensing.

Multi-source adversarial transfer learning for ultrasound image segmentation with limited similarity

Domain-transfer learning is a machine learning task to explore a source domain data set to help the learning problem in a target domain. Usually, the source domain has sufficient labeled data, while the target domain does not. In this paper, we propose a novel domain-transfer convolutional model by mapping a target domain data sample to a proxy in the source domain and applying a source domain model to the proxy for the purpose of prediction. In our framework, we firstly represent both source and target domains to feature vectors by two convolutional neural networks and then construct a proxy for each target domain sample in the source domain space. The proxy is supposed to be matched to the corresponding target domain sample convolutional representation vector well. To measure the matching quality, we proposed to maximize their squared-loss mutual information (SMI) between the proxy and target domain samples. We further develop a novel neural SMI estimator based on a parametric density ratio estimation function. Moreover, we also propose to minimize the classification error of both source domain samples and target domain proxies. The classification responses are also smoothened by manifolds of both the source domain and proxy space. By minimizing an objective function of SMI, classification error, and manifold regularization, we learn the convolutional networks of both source and target domains. In this way, the proxy of a target domain sample can be matched to the source domain data and thus benefits from the rich supervision information of the source domain. We design an iterative algorithm to update the parameters alternately and test it over benchmark data sets of abnormal behavior detection in video, Amazon product reviews sentiment analysis, etc.