An unsupervised domain adaptation brain CT segmentation method across image modalities and diseases

Summary Introduction : There has been a rapid development of deep learning (DL) models for medical imaging. However, DL requires a large labeled dataset for training the models. Getting large-scale labeled data remains a challenge, and multi-center datasets suffer from heterogeneity due to patient diversity and varying imaging protocols. Domain adaptation (DA) has been developed to transfer the knowledge from a labeled data domain to a related but unlabeled domain in either image space or feature space. DA is a type of transfer learning (TL) that can improve the performance of models when applied to multiple different datasets. Objective : In this survey, we review the state-of-the-art DL-based DA methods for medical imaging. We aim to summarize recent advances, highlighting the motivation, challenges, and opportunities, and to discuss promising directions for future work in DA for medical imaging. Methods : We surveyed peer-reviewed publications from leading biomedical journals and conferences between 2017-2020, that reported the use of DA in medical imaging applications, grouping them by methodology, image modality, and learning scenarios. Results : We mainly focused on pathology and radiology as application areas. Among various DA approaches, we discussed domain transformation (DT) and latent feature-space transformation (LFST). We highlighted the role of unsupervised DA in image segmentation and described opportunities for future development. Conclusion : DA has emerged as a promising solution to deal with the lack of annotated training data. Using adversarial techniques, unsupervised DA has achieved good performance, especially for segmentation tasks. Opportunities include domain transferability, multi-modal DA, and applications that benefit from synthetic data.

The adversarial domain adaptation has made many achievements in the field of Unsupervised Domain Adaptation (UDA). However, initial adversarial domain adaptation methods do not consider the category-level domain adaptation, causing confusion between different categories in the different domains. This paper propose a UDA approach called Minimizing Outputs' Differences of Classifiers with Different Responsibilities (MODCDR) to improve this situation. Its framework consists of one generator and two different task-specific classifiers, one working on the source domain, while the other working on the target domain. By training the feature generator to minimize the outputs' discrepancy of these two classifiers, this method not only generates discriminative features, but also achieves the category-level feature distribution alignment simultaneously. In addition, these two classifiers can also produce and filter the reliable pseudo-labels on the target domain, being used for assisting our training. This paper tested this method on three publicly available transfer learning object recognition datasets, with comparison to recent unsupervised adversarial domain adaptation approaches. The results show that the proposed method has superior performance.

The prime challenge in unsupervised domain adaptation (DA) is to mitigate the domain shift between the source and target domains. Prior DA works show that pretext tasks could be used to mitigate this domain shift by learning domain invariant representations. However, in practice, we find that most existing pretext tasks are ineffective against other established techniques. Thus, we theoretically analyze how and when a subsidiary pretext task could be leveraged to assist the goal task of a given DA problem and develop objective subsidiary task suitability criteria. Based on this criteria, we devise a novel process of sticker intervention and cast sticker classification as a supervised subsidiary DA problem concurrent to the goal task unsupervised DA. Our approach not only improves goal task adaptation performance, but also facilitates privacy-oriented source-free DA i.e. without concurrent source-target access. Experiments on the standard Office-31, Office-Home, DomainNet, and VisDA benchmarks demonstrate our superiority for both single-source and multi-source source-free DA. Our approach also complements existing non-source-free works, achieving leading performance.

Unsupervised deep domain adaptation algorithm for video based human activity recognition via recurrent neural networks

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

Unsupervised domain adaptive object detection aims to adapt a well-trained detector from its original source domain with rich labeled data to a new target domain with unlabeled data. Recently, mainstream approaches perform this task through adversarial learning, yet still suffer from two limitations. First, they mainly align marginal distribution by unsupervised cross-domain feature matching, and ignore each feature's categorical and positional information that can be exploited for conditional alignment; Second, they treat all classes as equally important for transferring cross-domain knowledge and ignore that different classes usually have different transferability. In this article, we propose a joint adaptive detection framework (JADF) to address the above challenges. First, an end-to-end joint adversarial adaptation framework for object detection is proposed, which aligns both marginal and conditional distributions between domains without introducing any extra hyper-parameter. Next, to consider the transferability of each object class, a metric for class-wise transferability assessment is proposed, which is incorporated into the JADF objective for domain adaptation. Further, an extended study from unsupervised domain adaptation (UDA) to unsupervised few-shot domain adaptation (UFDA) is conducted, where only a few unlabeled training images are available in unlabeled target domain. Extensive experiments validate that JADF is effective in both the UDA and UFDA settings, achieving significant performance gains over existing state-of-the-art cross-domain detection methods.

Domain adaptation assumes that samples from source and target domains are freely accessible during a training phase. However, such an assumption is rarely plausible in the real world and possibly causes data privacy issues, especially when the label of the source domain can be a sensitive attribute as an identifier. To avoid accessing source data that could contain sensitive information, we introduce source data free domain adaptation (SFDA). Our key idea is to leverage a pretrained model from the source domain and progressively update the target model in a self-learning manner. We observe that target samples with lower self-entropy measured by the pretrained source model are more likely to be classified correctly. From this, we select the reliable samples with the self-entropy criterion and define these as class prototypes. We then assign pseudolabels for every target sample based on the similarity score with class prototypes. We further propose point-to-set distance-based filtering, which does not require any tunable hyperparameters to reduce uncertainty from the pseudolabeling process. Finally, we train the target model with the filtered pseudolabels with regularization from the pretrained source model. Surprisingly, without direct usage of labeled source samples, our SFDA outperforms conventional domain adaptation methods on benchmark datasets. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b><i>Impact Statement</i></b>—This study addresses the data privacy issue, especially in unsupervised domain adaptation. Based on our privacy-preserving domain adaptation, various stakeholders, including enterprises and government organizations, can be free of concern about privacy issues with their labeled source dataset. Furthermore, the proposed data-free approach can contribute to creating a positive social impact, especially in large-scale datasets. Recently, since the size of data across various fields has been scaling up, it is almost incapable for individual researchers to directly utilize a large scale of data during training. For this reason, a new social trend of sharing pretrained models, e.g., EfficientNet and BERT, led by global enterprises with their huge amount of resources has been rising up. From this viewpoint, our approach thus enables more people to participate in the domain adaptation field specifically when the source data are large scale and contain sensitive attributes.

Remaining useful life prediction under variable operating conditions via multisource adversarial domain adaptation networks

Bionic limb control through myoelectric pattern recognition, offering intuitive decoding of motor intent, can improve the quality of life for individuals with amputations. However, most work on pattern recognition only uses a small subset of the myoelectric data generated during daily life, to train an Artificial Neural Network (ANN) via Supervised Learning (SL). Scenarios substantially different from the recording session e.g. different limb positions, can lead to misclassifications by the ANN during everyday usage of the bionic limb. Recording labeled data from all scenarios encountered in daily life could alleviate the problem, but would be prohibitively time consuming. Unsupervised Domain Adaptation (UDA) offers a solution by leveraging unlabeled data from a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">target domain</i>, not represented in the labeled dataset i.e. the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">source domain</i>, to calibrate ANNs for improved performance. In this study we explore the potential of two UDA algorithms for domain shifts in myoelectric pattern recognition: Domain Adversarial Neural Networks (DANN) and Sliced Wasserstein Discrepancy (SWD). Offline evaluation identified SWD as the best-performing algorithm, which was subsequently validated in online experiments with 11 participants. Using UDA improved the performance on the target domain by 19% compared to an ANN trained through SL on data from the source domain only. Indeed, it nearly matched the performance of an ANN trained through SL on labeled data from both the source and target domain. This is the first time UDA was validated in online experiments as a viable approach to overcome domain shift problems caused by changes in limb position.

An unsupervised domain adaptation approach for change detection and its application to deforestation mapping in tropical biomes

Electroencephalogram (EEG) signals convey abundant physiological and psychological information. Decoding EEG is fundamental for brain-computer interaction and medical rehabilitation. Nevertheless, the nonstationarity and inter-individual variability of EEG signals impedes current models from achieving robust crosssubject generalization without expensive subject-specific recalibration, thereby restricting practical deployment. Unsupervised domain adaptation (UDA) aims to improve generalization by minimizing distribution discrepancies between source and target domains. Recent studies regard different subjects as distinct domains and leverage UDA to facilitate cross-subject EEG classification by learning domain-invariant features through discrepancy minimization or adversarial training. However, these conventional methods are all developed in Euclidean space, which are insufficient to capture the non-linear structures inherent in EEG data. To address this limitation, we propose a Deep Adversarial Neural Network on the Symmetric Positive Definite (SPD) matrix manifold, referred to as SPD-DANN, which facilitates the extraction of subject-invariant features through adversarial learning. Additionally, we design an SPD domain feature alignment loss and an SPD class prototype pair loss to simultaneously promote feature alignment across subjects and enhance feature space discriminability. Extensive experiments on four brain-computer interface (BCI) datasets demonstrate that our method surpasses several state-of-the-art UDA techniques. Furthermore, the proposed loss functions are readily adaptable to broader unsupervised or semi-supervised domain adaptation frameworks.

Intelligent fault diagnosis (IFD) based on deep learning can achieve high accuracy from raw condition monitoring signals. However, models usually perform well on the training distribution only, and experience severe performance drops when applied to a different distribution. This is also observed in fault diagnosis, where assets are often operated in working conditions different from the ones in which the labeled data have been collected. The scenario where labeled data are available in a source domain and only unlabeled data are available in a target domain has been addressed recently by unsupervised domain adaptation (UDA) approaches for IFD. Recent methods have relied on self-training with confident pseudo-labels for the unlabeled target samples. However, the confidence-based selection of pseudo-labels is hindered by poorly calibrated uncertainty estimates in the target domain, primarily due to over-confident predictions, which limits the quality of pseudo-labels and leads to error accumulation. In this paper, we propose a novel method called Calibrated Adaptive Teacher (CAT), where we propose to calibrate the predictions of the teacher network on target samples throughout the self-training process, leveraging post hoc calibration techniques. We evaluate CAT on domain-adaptive IFD and perform extensive experiments on the Paderborn University (PU) benchmark for fault diagnosis of rolling bearings under varying operating conditions, using both time- and frequency-domain inputs. We compare four different calibration techniques within our framework, where temperature scaling is both the most effective and lightweight one. The resulting method—CAT+TempScaling—achieves state-of-the-art performance on most transfer tasks, with on average 7.5% higher accuracy and 4 times lower calibration error compared to domain-adversarial neural networks (DANNs) across the twelve PU transfer tasks.

With the rapid development of the remote sensing monitoring and computer vision technology, the deep learning method has made a great progress to achieve applications such as earth observation, climate change and even space exploration. However, the model trained on existing data cannot be directly used to handle the new remote sensing data, and labeling the new data is also time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) is one of the solutions to the aforementioned problems of labeled data defined as the source domain and unlabeled data as the target domain, i.e., its essential purpose is to obtain a well-trained model and tackle the problem of data distribution discrepancy defined as the domain shift between the source and target domain. There are a lot of reviews that have elaborated on UDA methods based on natural data, but few of these studies take into consideration thorough remote sensing applications and contributions. Thus, in this paper, in order to explore the further progress and development of UDA methods in remote sensing, based on the analysis of the causes of domain shift, a comprehensive review is provided with a fine-grained taxonomy of UDA methods applied for remote sensing data, which includes Generative training, Adversarial training, Self-training and Hybrid training methods, to better assist scholars in understanding remote sensing data and further advance the development of methods. Moreover, remote sensing applications are introduced by a thorough dataset analysis. Meanwhile, we sort out definitions and methodology introductions of partial, open-set and multi-domain UDA, which are more pertinent to real-world remote sensing applications. We can draw the conclusion that UDA methods in the field of remote sensing data are carried out later than those applied in natural images, and due to the domain gap caused by appearance differences, most of methods focus on how to use generative training (GT) methods to improve the model’s performance. Finally, we describe the potential deficiencies and further in-depth insights of UDA in the field of remote sensing.

A Two-Way alignment approach for unsupervised multi-Source domain adaptation

Domain consistency regularization for unsupervised multi-source domain adaptive classification