- Research Article
23
- 10.1016/j.knosys.2022.108903
Joint bi-adversarial learning for unsupervised domain adaptation
- Apr 30, 2022
- Knowledge-Based Systems
- Qing Tian + 2 more +2
Joint bi-adversarial learning for unsupervised domain adaptation
Unsupervised domain adaptation (UDA) aims to improve the classification performance on an unlabeled target domain by leveraging information from a fully labeled source domain. Recent approaches explore domain-invariant and class-discriminant representations to tackle this task. These methods, however, ignore the interaction between domain alignment learning and class discrimination learning. As a result, the missing or inadequate tradeoff between domain alignment and class discrimination are prone to the problem of negative transfer. In this paper, we propose Dynamic Weighted Learning (DWL) to avoid the discriminability vanishing problem caused by excessive alignment learning and domain misalignment problem caused by excessive discriminant learning. Technically, DWL dynamically weights the learning losses of alignment and discriminability by introducing the degree of alignment and discriminability. Besides, the problem of sample imbalance across domains is first considered in our work, and we solve the problem by weighing the samples to guarantee information balance across domains. Extensive experiments demonstrate that DWL has an excellent performance in several benchmark datasets. Code is available at https://github.com/NiXiao-cqu/TransferLearning-dwl-cvpr2021.
Joint bi-adversarial learning for unsupervised domain adaptation
Joint bi-adversarial learning for unsupervised domain adaptation
Deep Joint Discriminative Feature Learning and Class-Aware Domain Alignment for Unsupervised Domain Adaptation
Unsupervised domain adaptation aims to utilize the knowledge learned from source domain with labels to make predictions for target domain without labels. Most of existing methods are committed to aligning the global distribution information to narrow down the domain discrepancy between source and target domain, which show a promising achievement. However, these domain alignment methods can not completely eliminate domain shift because of the loose manifold structure, so that the adjacent samples located in a high-density data region from different classes are easily misclassified by the hyperplane learned from source domain. Another problem is that these methods only align the distributions in the domain level, while neglecting class-level distribution information, which may cause class misalignment among domains. In this paper, we propose to learn discriminative feature and align domain in class level which can learn better source discriminative feature representations and benefit domain alignment in the class level. The discriminative feature learning strategy is able to obtain a tight manifold structure and then promote a low-density separation between classes. Then, to perform fine-grained domain alignment and learn more transferable features, a class-aware domain alignment approach is proposed to align samples from the same class in source domain and target domain. Experiments on two datasets show that our proposal can improve the performance of deep unsupervised domain adaptation methods.
Read moreCross-Domain Contrastive Learning for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain. Most existing UDA methods learn domain-invariant feature representations by minimizing feature distances across domains. In this work, we build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets. Exploring the same set of categories shared by both domains, we introduce a simple yet effective framework CDCL, for domain alignment. In particular, given an anchor image from one domain, we minimize its distances to cross-domain samples from the same class relative to those from different categories. Since target labels are unavailable, we use a clustering-based approach with carefully initialized centers to produce pseudo labels. In addition, we demonstrate that CDCL is a general framework and can be adapted to the data-free setting, where the source data are unavailable during training, with minimal modification. We conduct experiments on two widely used domain adaptation benchmarks, i.e., Office-31 and VisDA-2017, for image classification tasks, and demonstrate that CDCL achieves state-of-the-art performance on both datasets.
Read moreStructured discriminative tensor dictionary learning for unsupervised domain adaptation
Structured discriminative tensor dictionary learning for unsupervised domain adaptation
Transferable regularization and normalization: Towards transferable feature learning for unsupervised domain adaptation
Transferable regularization and normalization: Towards transferable feature learning for unsupervised domain adaptation
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation
Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domain alignment approaches can only reduce, but not remove the domain shift, target domain samples distributed near the edge of the clusters, or far from their corresponding class centers are easily to be misclassified by the hyperplane learned from the source domain. To alleviate this issue, we propose to joint domain alignment and discriminative feature learning, which could benefit both domain alignment and final classification. Specifically, an instance-based discriminative feature learning method and a center-based discriminative feature learning method are proposed, both of which guarantee the domain invariant features with better intra-class compactness and inter-class separability. Extensive experiments show that learning the discriminative features in the shared feature space can significantly boost the performance of deep domain adaptation methods.
Read moreUnsupervised Multitarget Domain Adaptation With Dictionary-Bridged Knowledge Exploitation.
Unsupervised domain adaptation (UDA) is an emerging learning paradigm that models on unlabeled datasets by leveraging model knowledge built on other labeled datasets, in which the statistical distributions of these datasets are usually not identical. Formally, UDA is to leverage knowledge from a labeled source domain to promote an unlabeled target domain. Although there have been a variety of methods proposed to address the UDA problem, most of them are dedicated to single-source-to-single-target domain, while the works on single-source-to-multitarget domain are relatively rare. Compared to the single-source domain with single-target domain scenario, the UDA from single-source domain to multitarget domain is more challenging since it needs to consider not only the relationships between the source and the target domains but also those among the target domains. To this end, this article proposes a kind of dictionary learning-based unsupervised multitarget domain adaptation method (DL-UMTDA). In DL-UMTDA, a common dictionary is constructed to correlate the single-source and multitarget domains, while individual dictionaries are designed to exploit the private knowledge for the target domains. Through learning the corresponding dictionary representation coefficients in the UDA process, the correlations from the source to the target domains as well as these potential relationships between the target domains can be effectively exploited. In addition, we design an alternating algorithm to solve the DL-UMTDA model with theoretical convergence guarantee. Finally, extensive experiments on benchmark (Office + Caltech) and real datasets (AgeDB, Morph, and CACD) validate the superiority of the proposed method.
Read moreGraph Matching and Pseudo-Label Guided Deep Unsupervised Domain Adaptation
The goal of domain adaptation is to train a high-performance predictive model on the target domain data by using knowledge from the source domain data, which has different but related data distribution. In this paper, we consider unsupervised domain adaptation where we have labelled source domain data but unlabelled target domain data. Our solution to unsupervised domain adaptation is to learn a domain-invariant representation that is also category discriminative. Domain-invariant representations are realized by minimizing the domain discrepancy. To minimize the domain discrepancy, we propose a novel graph-matching metric between the source and target domain representations. Minimizing this metric allows the source and target representations to be in support of each other. We further exploit confident unlabelled target domain samples and their pseudo-labels to refine our proposed model. We expect the refining step to improve the performance further. This is validated by performing experiments on standard image classification adaptation datasets. Results showed our proposed approach out-perform previous domain-invariant representation learning approaches.
Read moreSelf-Distillation for Unsupervised 3D Domain Adaptation
Point cloud classification is a popular task in 3D vision. However, previous works, usually assume that point clouds at test time are obtained with the same procedure or sensor as those at training time. Unsupervised Domain Adaptation (UDA) instead, breaks this assumption and tries to solve the task on an unlabeled target domain, leveraging only on a supervised source domain. For point cloud classification, recent UDA methods try to align features across domains via auxiliary tasks such as point cloud reconstruction, which however do not optimize the discriminative power in the target domain in feature space. In contrast, in this work, we focus on obtaining a discriminative feature space for the target domain enforcing consistency between a point cloud and its augmented version. We then propose a novel iterative self-training methodology that exploits Graph Neural Networks in the UDA context to refine pseudo-labels. We perform extensive experiments and set the new state-of-the art in standard UDA benchmarks for point cloud classification. Finally, we show how our approach can be extended to more complex tasks such as part segmentation.
Read moreSelf-Supervised Cross-Video Temporal Learning for Unsupervised Video Domain Adaptation
We address the task of unsupervised domain adaptation (UDA) for videos with self-supervised learning. While UDA for images is a widely studied problem, UDA for videos is relatively unexplored. In this paper, we propose a novel self-supervised loss for the task of video UDA. The method is motivated by inverted reasoning. Many works on video classification have shown success with representations based on events in videos, e.g., ‘reaching’, ‘picking’, and ‘drinking’ events for ‘drinking coffee’. We argue that if we have event-based representations, we should be able to predict the relative distances between clips in videos. Inverting that, we propose a self-supervised task to predict the difference of the distance between two clips from the source video and the distance between two clips from the target video. We hope that such a task would encourage learning event-based representations of the videos, which is known to be beneficial for classification. Since we predict the difference of clip distances between clips from source videos and target videos, we ‘tie’ the two domains and expect to achieve well-adapted representations. We combine this purely self-supervised loss and the source classification loss to learn the model parameters. We give extensive empirical results on challenging video UDA benchmarks, i.e., UCF-HMDB and EPIC-Kitchens. The presented qualitative and quantitative results support our motivations and method.
Read moreLabel-efficient visual recognition with transfer learning
Visual recognition, encompassing tasks such as image classification, object detection, and semantic segmentation, has been a fundamental challenge in computer vision research. It also serves as the foundation for numerous applications, including autonomous driving, remote sensing, and robotics. The emergence of deep learning has significantly advanced this field, enabling remarkable achievements through the use of end-to-end trainable deep neural networks (DNNs). However, this remarkable progress comes at the cost of requiring vast amounts of annotated training data, the collection of which is often extremely labor-intensive and time-consuming. This constraint presents a significant barrier to the scalability and adaptability of various deep learning approaches. Transfer learning, which aims to transfer the off-the-shelf learned knowledge from one or multiple related source domains to target domains, is a promising way to mitigate this constraint. In this thesis, we explore label-efficient visual recognition with various transfer learning techniques, where the objective is to learn effective visual representations for target domains without any annotations by transferring the off-the-shelf knowledge learned in different but related source domains. Specifically, we investigate three representative types of transfer learning problems, including unsupervised domain adaptation, black-box unsupervised domain adaptation and test-time prompt tuning. As one of the most typical transfer learning problems, unsupervised domain adaptation aims to adapt a visual recognition model that is trained on labeled source-domain data to perform well on a related but unlabeled target-domain data. The primary challenge is to mitigate the distribution differences between the source and target domains, which are also referred to as domain gap or domain discrepancy, such that the model can generalize well on the target domain. In this thesis, we propose three innovative unsupervised domain adaptation techniques designed to efficiently transfer knowledge from labeled source domains to unlabeled target domains across a range of visual recognition tasks. Specifically, we explore diverse approaches to align domains from multiple perspectives (e.g., data or feature space alignment), thereby reducing the domain gap and enabling to perform well on unlabeled target domain. Black-box unsupervised domain adaptation aims to learn visual recognition model for target domain with the initial predictions of target data obtained by black-box source-domain model. In another word, during the adaptation process, only the model predictions of target data are available while either source data or source models are inaccessible. Compared to conventional unsupervised domain adaptation, black-box unsupervised domain adaptation offers notable advantages in terms of data privacy and flexibility, allowing adaptation to different target models regardless of the architecture or details of the source-trained black-box model. In this thesis, we analyze the challenges in black-box domain adaptation and design a novel approach for effectively exploiting the initial predictions of target data, leading to better adapted model on unlabeled target data. We further explore label-efficient visual recognition in the recent era of vision-language models. Despite the impressive zero-shot generalization capabilities of vision-language models, they always face unneglectable gaps while applied to target datasets, i.e., a target dataset typically features unique image styles and text formats tailored to its specific tasks. To circumvent this limitation, we explore an effective transfer learning approach for vision-language models, i.e., test-time prompt tuning, where the prompts are optimized as learnable vectors during the test stage without requiring any annotations for target data. Specifically, we examine the limitation of existing test-time prompt tuning methods, and propose a novel approach that could effectively exploit the learned knowledge in vision-language models and learn prompts for target data on-the-fly. Extensive experiments conducted over various visual recognition benchmarks indicate that our proposed methods achieve superior performance over target datasets, enabling to largely mitigate the dependence of labeled data across various visual recognition tasks.
Read moreTensorial multiview low-rank high-order graph learning for context-enhanced domain adaptation
Unsupervised Domain Adaptation (UDA) is a machine learning technique that facilitates knowledge transfer from a labeled source domain to an unlabeled target domain, addressing distributional discrepancies between these domains. Existing UDA methods often fail to effectively capture and utilize contextual relationships within the target domain. This research introduces a novel framework called Tensorial Multiview Low-Rank High-Order Graph Learning (MLRGL), which addresses these challenges by learning high-order graphs constrained by low-rank tensors to uncover contextual relations. The proposed framework ensures prediction consistency between randomly masked target images and their pseudo-labels by leveraging spatial context to generate multiview domain-invariant features through various augmented masking techniques. A high-order graph is constructed by combining Laplacian graphs to propagate these multiview features. Low-rank constraints are applied along both horizontal and vertical dimensions to better uncover inter-view and inter-class correlations among multiview features. This high-order graph is used to create an affinity matrix, mapping multiview features into a unified subspace. Prototype vectors and unsupervised clustering are then employed to calculate conditional probabilities for UDA tasks. We evaluated our approach using three different backbones across three benchmark datasets. The results demonstrate that the MLRGL framework outperforms current state-of-the-art methods in various UDA tasks. Additionally, our framework exhibits robustness to hyperparameter variations and demonstrates that multiview approaches outperform single-view solutions.
Read moreDiscriminative transfer feature learning for unsupervised domain adaptation.
Discriminative transfer feature learning for unsupervised domain adaptation.
Adversarial Domain Alignment Feature Similarity Enhancement Learning for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) attempts to transfer knowledge learned from labeled source domain to unlabeled target domain. Its main challenge is distribution gap between two domains. Most of works focus on reducing domain shift by domain alignment methods. Although these methods can reduce the domain shift, the samples far from the class center of target domain are still easily misclassified. To solve the problem, we propose a new approach named Adversarial Domain Alignment Feature Similarity Enhancement Learning (AASE). It learns domain invariant features by adversarial game and correlation alignment to reduce the domain gap, and makes these features having better discrimination via joint central discrimination and feature similarity enhancement. AASE makes the learned features have better intra-class compactness and inter-class separability. AASE is evaluated on two datasets, and the results show that AASE has critical improvement in the performance of UDA.
Read moreRobust Domain Adaptation: Representations, Weights and Inductive Bias (Extended Abstract)
Domain Invariant Representations (IR) has improved drastically the transferability of representations from a labelled source domain to a new and unlabelled target domain. Unsupervised Domain Adaptation (UDA) in presence of label shift remains an open problem. To this purpose, we present a bound of the target risk which incorporates both weights and invariant representations. Our theoretical analysis highlights the role of inductive bias in aligning distributions across domains. We illustrate it on standard benchmarks by proposing a new learning procedure for UDA. We observed empirically that weak inductive bias makes adaptation robust to label shift. The elaboration of stronger inductive bias is a promising direction for new UDA algorithms.
Read more