- Research Article
45
- 10.1016/j.patcog.2021.108238
Universal multi-Source domain adaptation for image classification
- Aug 14, 2021
- Pattern Recognition
- Yueming Yin + 3 more +3
Universal multi-Source domain adaptation for image classification
Consistency-guided Multi-Source-Free Domain Adaptation
Universal multi-Source domain adaptation for image classification
Universal multi-Source domain adaptation for image classification
Source-Free Multi-Domain Adaptation with Generally Auxiliary Model Training
Unsupervised domain adaptation transfers gained knowledge from labeled source domain(s) to a similar unlabeled target domain by eliminating the domain shifts. Most existing domain adaptation methods require the access to source data to match the source and target distributions. However, data privacy concerns make it difficult or impossible to share source data, leading to failures in existing domain adaptation methods. Admittedly, a few previous studies deal with domain adaptation without source data, but they rarely pay heed to data free domain adaptation with multiple source domains containing richer knowledge. In this paper, we propose a new multi-source data-free domain adaptation method- generally auxiliary model training (GAM)- which fits the source models to the target domain under the supervision of pseudo target labels rather than matching data distributions. To collect high-quality initial pseudo target labels, our approach learns both specific and general source models to improve the generality of source models based on auxiliary learning. Going further, we introduce a class balanced coefficient of each category based on the number of samples to reduce the misclassification often caused by data imbalance. Experiments on real-world classification datasets show that the propsosed generally auxiliary training has a superiority over the baselines.
Read moreSource-Free Multidomain Adaptation With Fuzzy Rule-Based Deep Neural Networks
Unsupervised domain adaptation deals with a task from an unlabeled target domain by leveraging the knowledge gained from labeled source domain(s). The fuzzy system is adopted in domain adaptation to better tackle the uncertainty caused by information scarcity in the transfer. Most existing fuzzy and nonfuzzy domain adaptation methods depend on data-level distribution matching to eliminate the domain shift. However, data sharing can trigger privacy concerns. This situation results in the unavailability of source data, wherein most domain adaptation methods cannot be applied. Source-free domain adaptation is then proposed to handle this problem. But the existing source-free domain adaptation methods rarely deal with any soft information component due to data imprecision. Besides, fewer methods handle multiple source domains that provide richer transfer information. Thus, in this article, we propose source-free multidomain adaptation with fuzzy rule-based deep neural networks, which takes advantage of a fuzzy system to handle data uncertainty in domain adaptation without source data. To learn source private models with high generality, which is important to collect low-noise pseudotarget labels, auxiliary tasks are designed by jointly training source models from multiple domains, which share source parameters and fuzzy rules while protecting source data. To transfer fuzzy rules and fit source private parameters to the target domain, self-supervised learning and anchor-based alignment are built to force target data into source feature spaces. Experiments on real-world datasets under both homogeneous and heterogeneous label space scenarios are carried out to validate the proposed method. The results indicate the superiority of the proposed fuzzy rule-based source-free multidomain adaptation method.
Read moreMulti-Source Collaborative Contrastive Learning for Decentralized Domain Adaptation
Unsupervised multi-source domain adaptation aims to obtain a model working well on the unlabeled target domain by reducing the domain gap between the labeled source domains and the unlabeled target domain. Considering the data privacy and storage cost, data from multiple source domains and target domain are isolated and decentralized. This data decentralization scenario brings the difficulty of domain alignment for reducing the domain gap between the decentralized source domains and target domain, respectively. For conducting domain alignment under the data decentralization scenario, we propose Multi-source Collaborative Contrastive learning for decentralized Domain Adaptation (MCC-DA). The models from other domains are used as the bridge to reduce the domain gap. On the source domains and target domain, we penalize the inconsistency of data features extracted from the source domain models and target domain model by contrastive alignment. With the collaboration of source domain models and target domain model, the domain gap between decentralized source domains and target domain is reduced without accessing the data from other domains. The experiment results on multiple benchmarks indicate that our method can reduce the domain gap effectively and outperform the state-of-the-art methods significantly.
Read moreJoint Distribution Adaptation via Wasserstein Adversarial Training
This paper considers the unsupervised domain adaptation problem, in which we want to find a good prediction function on the unlabeled target domain, by utilizing the information provided in the labeled source domain. A common approach to the domain adaptation problem is to learn a representation space where the distributional discrepancy of the source and target domains is small. Existing methods generally tend to match the marginal distributions of the two domains, while the label information in the source domain is not fully exploited. In this paper, we propose a representation learning approach for domain adaptation, which is addressed as JODAWAT. We aim to adapt the joint distributions of the feature-label pairs in the shared representation space for both domains. In particular, we minimize the Wasserstein distance between the source and target domains, while the prediction performance on the source domain is also guaranteed. The proposed approach results in a minimax adversarial training procedure that incorporates a novel split gradient penalty term. A generalization bound on the target domain is provided to reveal the efficacy of representation learning for joint distribution adaptation. We conduct extensive evaluations on JODAWAT, and test its classification accuracy on multiple synthetic and real datasets. The experimental results justify that our proposed method is able to achieve superior performance compared with various domain adaptation methods.
Read moreLabel-efficient visual recognition with transfer learning
Visual recognition, encompassing tasks such as image classification, object detection, and semantic segmentation, has been a fundamental challenge in computer vision research. It also serves as the foundation for numerous applications, including autonomous driving, remote sensing, and robotics. The emergence of deep learning has significantly advanced this field, enabling remarkable achievements through the use of end-to-end trainable deep neural networks (DNNs). However, this remarkable progress comes at the cost of requiring vast amounts of annotated training data, the collection of which is often extremely labor-intensive and time-consuming. This constraint presents a significant barrier to the scalability and adaptability of various deep learning approaches. Transfer learning, which aims to transfer the off-the-shelf learned knowledge from one or multiple related source domains to target domains, is a promising way to mitigate this constraint. In this thesis, we explore label-efficient visual recognition with various transfer learning techniques, where the objective is to learn effective visual representations for target domains without any annotations by transferring the off-the-shelf knowledge learned in different but related source domains. Specifically, we investigate three representative types of transfer learning problems, including unsupervised domain adaptation, black-box unsupervised domain adaptation and test-time prompt tuning. As one of the most typical transfer learning problems, unsupervised domain adaptation aims to adapt a visual recognition model that is trained on labeled source-domain data to perform well on a related but unlabeled target-domain data. The primary challenge is to mitigate the distribution differences between the source and target domains, which are also referred to as domain gap or domain discrepancy, such that the model can generalize well on the target domain. In this thesis, we propose three innovative unsupervised domain adaptation techniques designed to efficiently transfer knowledge from labeled source domains to unlabeled target domains across a range of visual recognition tasks. Specifically, we explore diverse approaches to align domains from multiple perspectives (e.g., data or feature space alignment), thereby reducing the domain gap and enabling to perform well on unlabeled target domain. Black-box unsupervised domain adaptation aims to learn visual recognition model for target domain with the initial predictions of target data obtained by black-box source-domain model. In another word, during the adaptation process, only the model predictions of target data are available while either source data or source models are inaccessible. Compared to conventional unsupervised domain adaptation, black-box unsupervised domain adaptation offers notable advantages in terms of data privacy and flexibility, allowing adaptation to different target models regardless of the architecture or details of the source-trained black-box model. In this thesis, we analyze the challenges in black-box domain adaptation and design a novel approach for effectively exploiting the initial predictions of target data, leading to better adapted model on unlabeled target data. We further explore label-efficient visual recognition in the recent era of vision-language models. Despite the impressive zero-shot generalization capabilities of vision-language models, they always face unneglectable gaps while applied to target datasets, i.e., a target dataset typically features unique image styles and text formats tailored to its specific tasks. To circumvent this limitation, we explore an effective transfer learning approach for vision-language models, i.e., test-time prompt tuning, where the prompts are optimized as learnable vectors during the test stage without requiring any annotations for target data. Specifically, we examine the limitation of existing test-time prompt tuning methods, and propose a novel approach that could effectively exploit the learned knowledge in vision-language models and learn prompts for target data on-the-fly. Extensive experiments conducted over various visual recognition benchmarks indicate that our proposed methods achieve superior performance over target datasets, enabling to largely mitigate the dependence of labeled data across various visual recognition tasks.
Read moreAdaptive integrated weight unsupervised multi-source domain adaptation without source data
Unsupervised multi-source domain adaptation methods transfer knowledge learned from multiple labeled source domains to an unlabeled target domain. Existing methods assume that all source domain data can be accessed directly. However, such an assumption is unrealistic and causes data privacy concerns, especially when the source domain labels include personal information. In such a setting, it is prohibited to minimize domain gaps by pairwise calculation of the data from the source and target domains. Therefore, this work addresses the source-free unsupervised multi-source domain adaptation problem, where only the source models are available during the adaptation. We propose trust center sample-based source-free domain adaptation (TSDA) method to solve this problem. The key idea is to leverage the pre-trained models from the source domain and progressively train the target model in a self-learning manner. Because target samples with low entropy measured from the pre-trained source model achieve high accuracy, the trust center samples are selected first using the entropy function. Then pseudo labels are assigned for target samples based on a self-supervised pseudo-labeling strategy. For multiple source domains, corresponding target models are learned based on the assigned pseudo labels. Finally, multiple target models are integrated to predict the label for unlabeled target data. Extensive experiment results on some benchmark datasets and generated adversarial samples demonstrate that our approach outperforms existing UMDA methods, even though some methods can always access source data.
Read moreAttention Guided Multiple Source and Target Domain Adaptation.
Domain adaptation aims to alleviate the distribution discrepancy between source and target domains. Most conventional methods focus on one target domain setting adapted from one or multiple source domains while neglecting the multi-target domain setting. We argue that different target domains also have complementary information, which is very important for performance improvement. In this paper, we propose an Attention-guided Multiple source-and-target Domain Adaptation (AMDA) method to capture the context dependency information on transferable regions among multiple source and target domains. The innovation points of this paper are as follows: (1) We use numerous adversarial strategies to harvest sufficient information from multiple source and target domains, which extends the generalization and robustness of the feature pools. (2) We propose an intra-domain and inter-domain attention module to explore transferable context information. The proposed attention module can learn domain-invariant representations and reduce the negative transfer by focusing on transferable knowledge. Extensive experiments validate the effectiveness of our method with achieving state-of-the-art performance on several unsupervised domain adaptation datasets.
Read moreHeterogeneous Graph Attention Network for Unsupervised Multiple-Target Domain Adaptation.
Domain adaptation, which transfers the knowledge from label-rich source domain to unlabeled target domains, is a challenging task in machine learning. The prior domain adaptation methods focus on pairwise adaptation assumption with a single source and a single target domain, while little work concerns the scenario of one source domain and multiple target domains. Applying pairwise adaptation methods to this setting may be suboptimal, as they fail to consider the semantic association among multiple target domains. In this work we propose a deep semantic information propagation approach in the novel context of multiple unlabeled target domains and one labeled source domain. Our model aims to learn a unified subspace common for all domains with a heterogeneous graph attention network, where the transductive ability of the graph attention network can conduct semantic propagation of the related samples among multiple domains. In particular, the attention mechanism is applied to optimize the relationships of multiple domain samples for better semantic transfer. Then, the pseudo labels of the target domains predicted by the graph attention network are utilized to learn domain-invariant representations by aligning labeled source centroid and pseudo-labeled target centroid. We test our approach on four challenging public datasets, and it outperforms several popular domain adaptation methods.
Read moreMultitarget Domain Adaptation for Remote Sensing Classification Using Graph Neural Network
Remote sensing deals with huge variations in geography, acquisition season, and a plethora of sensors. Considering the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the above-mentioned aspects. Domain adaptation (DA) methods can adapt such model for applying on target domains with different distributions from the source domain. However, most remote sensing DA methods are designed for single-target, thus requiring a separate target classifier to be trained for each target domain. To mitigate this, we propose multitarget DA in which a single classifier is learned for multiple unlabeled target domains. To build a multitarget classifier, it may be beneficial to effectively aggregate features from the labeled source and different unlabeled target domains. Toward this, we exploit coteaching based on the graph neural network that is capable of leveraging unlabeled data. We use a sequential adaptation strategy that first adapts on the easier target domains assuming that the network finds it easier to adapt to the closest target domain. We validate the proposed method on two different datasets, representing geographical and seasonal variation. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://gitlab.lrz.de/ai4eo/da-multitarget-gnn/</uri> .
Read moreTask-Space Domain Adaptation for Cross-Scene Infant Cry Detection.
Infant cry detection is important for infant daily care as the cry signals provide useful information, such as hunger and discomfort, and for medical diagnosis, such as asphyxia. However, existing detection models suffer from performance degradation caused by biased data distribution across different scenes of deployment. This paper presents a Task-Space Domain Adaptation (TSDA) method that facilitates efficient knowledge transfer from the labeled source domain (e.g., home scene) to enhance the performance in the unlabeled target domain (e.g., hospital scene). TSDA first decomposes source domain features guided by cry detection labels to obtain task-relevant features. It then employs adversarial learning with nuclear norm-based representation to align target domain features with task-relevant source features, enabling unsupervised target domain adaptation. Extensive experiments conducted on home and clinical datasets show that TSDA outperforms the direct cross-dataset baseline test by approximately 30% in F1-score, and achieves a 1-12% improvement over existing domain adaptation methods, approaching the performance of supervised learning based on the labeled target domain. This study provides an effective solution for promoting the practical deployment of cry detection models across different scenes.
Read moreDomain adaptive remote sensing image semantic segmentation with prototype guidance
Domain adaptive remote sensing image semantic segmentation with prototype guidance
A New Progressive Multisource Domain Adaptation Network With Weighted Decision Fusion.
Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.
Read moreEnhancing Cross-Domain Remote Sensing Scene Classification by Multi-Source Subdomain Distribution Alignment Network
Multi-source domain adaptation (MSDA) in remote sensing (RS) scene classification has recently gained significant attention in the visual recognition community. It leverages multiple well-labeled source domains to train a model capable of achieving strong generalization on the target domain with little to no labeled data from the target domain. However, the distribution shifts among multiple source domains make it more challenging to align the distributions between the target domain and all source domains concurrently. Moreover, relying solely on global alignment risks losing fine-grained information for each class, especially in the task of RS scene classification. To alleviate these issues, we present a Multi-Source Subdomain Distribution Alignment Network (MSSDANet), which introduces novel network structures and loss functions for subdomain-oriented MSDA. By adopting a two-level feature extraction strategy, this model attains better global alignment between the target domain and multiple source domains, as well as alignment at the subdomain level. First, it includes a pre-trained convolutional neural network (CNN) as a common feature extractor to fully exploit the shared invariant features across one target and multiple source domains. Secondly, a dual-domain feature extractor is used after the common feature extractor, which maps the data from each pair of target and source domains to a specific dual-domain feature space and performs subdomain alignment. Finally, a dual-domain feature classifier is employed to make predictions by averaging the outputs from multiple classifiers. Accompanied by the above network, two novel loss functions are proposed to boost the classification performance. Discriminant Semantic Transfer (DST) loss is exploited to force the model to effectively extract semantic information among target and source domain samples, while Class Correlation (CC) loss is introduced to reduce the feature confusion from different classes within the target domain. It is noteworthy that our MSSDANet is developed in an unsupervised manner for domain adaptation, indicating that no label information from the target domain is required during training. Extensive experiments on four common RS image datasets demonstrate that the proposed method achieves state-of-the-art performance for cross-domain RS scene classification. Specifically, in the dual-source and three-source settings, MSSDANet outperforms the second-best algorithm in terms of overall accuracy (OA) by 2.2% and 1.6%, respectively.
Read moreHandling Domain Shift for Lesion Detection via Semi-supervised Domain Adaptation
As the community progresses towards automated Universal Lesion Detection (ULD), it is vital that the techniques developed are robust and easily adaptable across a variety of datasets coming from different scanners, hospitals, and acquisition protocols. In practice, this remains a challenge due to the complexities of the different types of domain shifts. In this paper, we address the domain-shift by proposing a novel domain adaptation framework for ULD. The proposed model allows for the transfer of lesion knowledge from a large labeled source domain to detect lesions on a new target domain with minimal labeled samples. The proposed method first aligns the feature distribution of the two domains by training a detector on the source domain using a supervised loss, and a discriminator on both source and unlabeled target domains using an adversarial loss. Subsequently, a few labeled samples from the target domain along with labeled source samples are used to adapt the detector using an over-fitting aware and periodic gradient update based joint few-shot fine-tuning technique. Further, we utilize a self-supervision scheme to obtain pseudo-labels having high-confidence on the unlabeled target domain which are used to further train the detector in a semi-supervised manner and improve the detection sensitivity. We evaluate our proposed approach on domain adaptation for lesion detection from CT-scans wherein a ULD network trained on the DeepLesion dataset is adapted to 3 target domain datasets such as LiTS, KiTS and 3Dircadb. By utilizing adversarial, few-shot and incremental semi-supervised training, our method achieves comparable detection sensitivity to the previous methods for few-shot and semi-supervised methods as well as to the Oracle model trained on the labeled target domain.
Read more