Joint Distribution Alignment via Adversarial Learning for Domain Adaptive Object Detection

Electroencephalogram (EEG) signals convey abundant physiological and psychological information. Decoding EEG is fundamental for brain-computer interaction and medical rehabilitation. Nevertheless, the nonstationarity and inter-individual variability of EEG signals impedes current models from achieving robust crosssubject generalization without expensive subject-specific recalibration, thereby restricting practical deployment. Unsupervised domain adaptation (UDA) aims to improve generalization by minimizing distribution discrepancies between source and target domains. Recent studies regard different subjects as distinct domains and leverage UDA to facilitate cross-subject EEG classification by learning domain-invariant features through discrepancy minimization or adversarial training. However, these conventional methods are all developed in Euclidean space, which are insufficient to capture the non-linear structures inherent in EEG data. To address this limitation, we propose a Deep Adversarial Neural Network on the Symmetric Positive Definite (SPD) matrix manifold, referred to as SPD-DANN, which facilitates the extraction of subject-invariant features through adversarial learning. Additionally, we design an SPD domain feature alignment loss and an SPD class prototype pair loss to simultaneously promote feature alignment across subjects and enhance feature space discriminability. Extensive experiments on four brain-computer interface (BCI) datasets demonstrate that our method surpasses several state-of-the-art UDA techniques. Furthermore, the proposed loss functions are readily adaptable to broader unsupervised or semi-supervised domain adaptation frameworks.

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

Unsupervised domain adaptation (UDA) is an important solution to reduce the bias between the labeled source domain and the unlabeled target domain. It has attracted more attention for optical remote sensing image scene classification and retrieval. Currently, most of the previous work is devoted to closed-set UDA. In fact, the target domain often contains unknown classes. Moreover, some open UDA methods mine structural information of the target domain directly from the type knowledge of the source domain, and less directly from the unlabeled data of the target domain. In this paper, we propose a new self-supervised-driven open-set UDA method combining contrastive self-supervised learning with consistency self-training for optical remote sensing scene classification and retrieval. Specifically, a contrastive self-supervised learning network is introduced to learn discriminative features from the unlabeled target domain data. Moreover, a novel open-set class learning module is developed based on two-level confidence rules and the consistency self-training strategy, which can obtain reliable unknown class samples for co-training. Finally, an open-set dataset including six cross-domain scenarios is constructed based on three public datasets and several experiments are conducted with eleven state-of-the-art domain adaptation methods. Experimental results demonstrate that our proposed method achieves superior performances on the six open-set cross-domain scenarios in both scene classification and retrieval. Especially, our method improves the overall classification accuracies by 9.72% to 24.06% and improve mean average retrieval precisions by 8.06% to 16.21% on the complex UCMD (source domain) → NWPU (target domain) scenario, compared with the other eleven state-of-the-art methods.

Unsupervised domain adaptation adapts classifiers to an un-labeled target domain by exploiting a labeled source domain. To reduce discrepancy between source and target domains, adversarial learning methods are typically selected to seek domain-invariant representations by confusing the domain discriminator. However, classifiers may not be well adapted to such a domain-invariant representation space, as the sample-level and class-level data structures could be distorted during adversarial learning. In this paper, we propose a novel Transferable Feature Learning approach on Graphs (TFLG) for unsupervised adversarial domain adaptation, which jointly incorporates sample-level and class-level structure information across two domains. TFLG first constructs graphs and identifies the class-wise correspondence across domains. A novel cross-domain graph convolutional operation is designed to jointly align the sample-level and class-level structures in two domains. Extensive experiments on benchmark datasets demonstrate the effectiveness of our approach, compared to the representative unsupervised domain adaptation methods.

The adversarial domain adaptation has made many achievements in the field of Unsupervised Domain Adaptation (UDA). However, initial adversarial domain adaptation methods do not consider the category-level domain adaptation, causing confusion between different categories in the different domains. This paper propose a UDA approach called Minimizing Outputs' Differences of Classifiers with Different Responsibilities (MODCDR) to improve this situation. Its framework consists of one generator and two different task-specific classifiers, one working on the source domain, while the other working on the target domain. By training the feature generator to minimize the outputs' discrepancy of these two classifiers, this method not only generates discriminative features, but also achieves the category-level feature distribution alignment simultaneously. In addition, these two classifiers can also produce and filter the reliable pseudo-labels on the target domain, being used for assisting our training. This paper tested this method on three publicly available transfer learning object recognition datasets, with comparison to recent unsupervised adversarial domain adaptation approaches. The results show that the proposed method has superior performance.

The prime challenge in unsupervised domain adaptation (DA) is to mitigate the domain shift between the source and target domains. Prior DA works show that pretext tasks could be used to mitigate this domain shift by learning domain invariant representations. However, in practice, we find that most existing pretext tasks are ineffective against other established techniques. Thus, we theoretically analyze how and when a subsidiary pretext task could be leveraged to assist the goal task of a given DA problem and develop objective subsidiary task suitability criteria. Based on this criteria, we devise a novel process of sticker intervention and cast sticker classification as a supervised subsidiary DA problem concurrent to the goal task unsupervised DA. Our approach not only improves goal task adaptation performance, but also facilitates privacy-oriented source-free DA i.e. without concurrent source-target access. Experiments on the standard Office-31, Office-Home, DomainNet, and VisDA benchmarks demonstrate our superiority for both single-source and multi-source source-free DA. Our approach also complements existing non-source-free works, achieving leading performance.

This work explores the usage of Fourier Transform for reducing the domain gap between the Source (e.g. Synthetic Image) and Target domain (e.g. Real Image) towards solving the Domain Adaptation problem. Most of the Unsupervised Domain Adaptation (UDA) algorithms reduce the global domain shift between labelled Source and unlabelled Target domain by matching the marginal distribution. UDA performance deteriorates for the cases where the domain gap between Source and Target is significant. To improve the overall performance of the existing UDA algorithms the proposed method attempts to bring the Source domain closer to the Target domain with the help of pseudo label based class consistent low-frequency swapping. This traditional image processing technique results in computational efficiency, especially compared to the state-of-the-art deep learning methods that use complex adversarial training. The proposed method Class Aware Frequency Transformation (CAFT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) can easily be plugged into any existing UDA algorithm to improve its performance. We evaluate CAFT on various domain adaptation datasets and algorithms and have achieved performance gains across all the popular benchmarks.

Unsupervised deep domain adaptation algorithm for video based human activity recognition via recurrent neural networks

Unsupervised domain adaptation (UDA) has been widely studied with the goal of transferring knowledge from a label-rich source domain to a related but unlabeled target domain. Most UDA techniques achieve this by reducing the feature discrepancies between the two domains to learn domain-invariant feature representations. While domain-invariant feature representations can reduce the differences between the source and target domains, excessively simplifying these differences may cause the model to overlook important domain-specific features, resulting in a decline in transfer learning effectiveness. To address this issue, this paper proposes a novel Gaussian Mixture Model for graph domain adaptation (GMM). This model effectively reduces the distributional bias between the source and target domains by modeling the distribution differences on a graph structure. GMM leverages the local structural information of the graph and the clustering capability of the Gaussian mixture model to automatically learn the latent mapping relationships between the source and target domains. To the best of our knowledge, this is the first work to introduce a Gaussian mixture model into UDA. Extensive experimental results on three standard benchmarks demonstrate that the proposed GMM algorithm outperforms state-of-the-art unsupervised domain adaptation methods in terms of performance.

Automated electrocardiogram classification can facilitate cheaper and more accessible arrhythmia and cardiovascular diagnoses for patients. However, clinical machine learning models fail in the real world when the data of interest is collected under different conditions to the training data e.g. due to a different hospital, demographic group, equipment. These differing data collection conditions are known as the target and source domains, respectively. While it is time-consuming and costly to obtain labeled data from the target domain, unlabeled target domain data is more readily available. Unsupervised domain adaptation from a source domain to an unlabeled target domain can unlock more widespread clinical use of machine learning models, including for arrhythmia classification. In this paper, we show that uncertainty objectives can improve arrhythmia classification on new datasets. We apply a Dirichlet Prior Network to test two novel approaches: the first minimizes target domain uncertainty to encourage domain alignment, while the second aligns the class prediction uncertainty of source and target heartbeats. We demonstrate that these methods improve heartbeat classification of electrocardiograms collected at different hospitals using the MIT-BIH and St. Petersburg INCART arrhythmia datasets. Our first approach improves the baseline target F1 score by 7% for ternary arrhythmia classification, while our second approach improves state-of-the-art domain adaptation performance by 3%.Clinical relevance- This proposes and validates methods applying uncertainty objectives to improve automated arrhythmia classification in new datasets.

Recognizing and locating objects by algorithms are essential and challenging issues for Intelligent Transportation Systems. However, the increasing demand for much labeled data hinders the further application of deep learning-based object detection. One of the optimal solutions is to train the target model with an existing dataset and then adapt it to new scenes, namely Unsupervised Domain Adaptation (UDA). However, most of existing methods at the pixel level mainly focus on adapting the model from source domain to target domain and ignore the essence of UDA to learn domain-invariant feature learning. Meanwhile, almost all methods at the feature level ignore to make conditional distributions matched for UDA while conducting feature alignment between source and target domain. Considering these problems, this paper proposes the ParaUDA, a novel framework of learning invariant representations for UDA in two aspects: pixel level and feature level. At the pixel level, we adopt CycleGAN to conduct domain transfer and convert the problem of original unsupervised domain adaptation to supervised domain adaptation. At the feature level, we adopt an adversarial adaption model to learn domain-invariant representation by aligning the distributions of domains between different image pairs with same mixture distributions. We evaluate our proposed framework in different scenes, from synthetic scenes to real scenes, from normal weather to challenging weather, and from scenes across cameras. The results of all the above experiments show that ParaUDA is effective and robust for adapting object detection models from source scenes to target scenes.

With the rapid development of the remote sensing monitoring and computer vision technology, the deep learning method has made a great progress to achieve applications such as earth observation, climate change and even space exploration. However, the model trained on existing data cannot be directly used to handle the new remote sensing data, and labeling the new data is also time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) is one of the solutions to the aforementioned problems of labeled data defined as the source domain and unlabeled data as the target domain, i.e., its essential purpose is to obtain a well-trained model and tackle the problem of data distribution discrepancy defined as the domain shift between the source and target domain. There are a lot of reviews that have elaborated on UDA methods based on natural data, but few of these studies take into consideration thorough remote sensing applications and contributions. Thus, in this paper, in order to explore the further progress and development of UDA methods in remote sensing, based on the analysis of the causes of domain shift, a comprehensive review is provided with a fine-grained taxonomy of UDA methods applied for remote sensing data, which includes Generative training, Adversarial training, Self-training and Hybrid training methods, to better assist scholars in understanding remote sensing data and further advance the development of methods. Moreover, remote sensing applications are introduced by a thorough dataset analysis. Meanwhile, we sort out definitions and methodology introductions of partial, open-set and multi-domain UDA, which are more pertinent to real-world remote sensing applications. We can draw the conclusion that UDA methods in the field of remote sensing data are carried out later than those applied in natural images, and due to the domain gap caused by appearance differences, most of methods focus on how to use generative training (GT) methods to improve the model’s performance. Finally, we describe the potential deficiencies and further in-depth insights of UDA in the field of remote sensing.

Adversarial domain adaptation is a powerful approach to transfer the knowledge of the label-rich source domain to the label-scarce target domain by mitigating domain shifts across distributions. Existing domain adaptation methods align either the marginal distribution with a single-domain discriminator or conditional distributions with multiple-domain discriminators. However, aligning both marginal (global) and conditional (local) distributions should be considered for domain adaptation. This paper proposes a novel adversarial distribution adaptation network (ADAN) to jointly reduce both the global and local distribution discrepancies between different domains for learning domain-invariant representations. ADAN utilizes a single-domain discriminator to adapt the global distribution between two domains, and source decision boundaries to align the local distributions between sub-domains. Furthermore, we extend our ADAN as improved ADAN (iADAN), in which we utilize a feature norm term to regularize the task-specific features to improve model generalization. Extensive experimental results show that our method outperforms other state-of-the-art domain adaptation methods on Office-Home and ImageCLEF-DA datasets.

Electroencephalograms (EEG) have shown to be a useful approach to measure the cognitive load in tasks where mental effort is involved. However, EEG signals present a high variability among subjects as well as a non-stationary behavior, so that distributions among samples of different subjects are mismatched. Methods based on Unsupervised Domain Adaptation (UDA) have been used as an effective solution to reduce such discrepancy, while the ones leveraged by deep learning (D-UDA) have improved the classification results over shallow approaches. However, most D-UDA methods assume that even though there are differences in marginal distributions between source and target domains, their conditional distributions remain fixed, which does not hold in many EEG databases. To address this problem, we propose a new D-UDA method, named Custom Domain Adaptation (CDA), which integrates Adaptive Batch Normalization (AdaBN) and Maximum Mean Discrepancy (MMD) into two independent deep neural networks in order to reduce the marginal and conditional distribution differences. CDA was compared with six popular D-UDA methods using a free-available dataset of cognitive loads and obtained an accuracy of $98.2\pm 2.67\%$ , which outperformed these state-of-the-art methods.

The ability of supervised image analysis methods to provide state-of-the-art performance is limited by availability of high-quality labeled data in large quantities. Domain adaptation approaches propose a solution to this problem by leveraging quality labeled information from auxiliary data sources. In this work, we use adversarial learning for domain adaptation for remote sensing applications. First, we approached the problem of unavailable target domain labels with unsupervised domain adaptation and then extended our method for semisupervised domain adaptation to use a few available labels as well. We are using adversarial learning to extract discriminative target domain features that are aligned with source domain. We test our framework for two very different applications of remote sensing imagery, multiclass classification in hyperspectral images and semantic segmentation in large scale satellite images. For hyperspectral image analysis two datasets were used: the University of Houston shadow data was used for quantifying the efficacy of our approach to varying illumination, and the Botswana data was used to quantify the efficacy of our approach under multitemporal spectral shifts. Multisensor high-resolution images from National Agriculture Imagery Program and SpaceNet-Rio datasets were used as the source and target for the task of building extraction for large scale semantic segmentation based domain adaptation.