- Research Article
15
- 10.1016/j.knosys.2022.109320
Manifold discrimination partial adversarial domain adaptation
- Jun 25, 2022
- Knowledge-Based Systems
- Chunmei He + 4 more +4
Manifold discrimination partial adversarial domain adaptation
Domain adaptation aims to transfer knowledge across different domains and bridge the gap between them. While traditional knowledge transfer considers identical domain, a more realistic scenario is to transfer from a larger and more diverse source domain to a smaller target domain, which is referred to as partial domain adaptation (PDA). However, matching the whole source domain to the target domain for PDA might produce negative transfer. Samples in the shared classes should be carefully selected to mitigate negative transfer in PDA. We observe that the correlations between different target domain samples and source domain samples are diverse: classes are not equally correlated and moreover, different samples have different correlation strengthes even when they are in the same class. In this study, we propose ECDT, a novel PDA method that Exploits the Correlation Diversity for knowledge Transfer between different domains. We propose a novel method to estimate target domain label space that utilizes the label distribution and feature distribution of target samples, based on which outlier source classes can be filtered out and their negative effects on transfer can be mitigated. Moreover, ECDT combines class-level correlation and instance-level correlation to quantity sample-level transferability in domain adversarial network. Experimental results on three commonly used cross-domain object data sets show that ECDT is superior to previous partial domain adaptation methods.
Manifold discrimination partial adversarial domain adaptation
Manifold discrimination partial adversarial domain adaptation
A Stacked Auto-Encoder Based Partial Adversarial Domain Adaptation Model for Intelligent Fault Diagnosis of Rotating Machines
Fault diagnosis plays an indispensable role in prognostics and health management of rotating machines. In recent years, intelligent fault diagnosis methods based on domain adaptation technology have attracted the attention of researchers. However, a more extensive application scenario of fault diagnosis - partial domain adaptation (PDA) - has not been well-resolved. In this article, for the first time, a novel stacked auto-encoder based partial adversarial domain adaptation (SPADA) model is proposed to solve the fault diagnosis problem in PDA situations. Two deep stack auto-encoders are first designed to extract representative features from the training data (source domain) and test data (target domain), respectively. Then, a weighted classifier based on Softmax is used to weight the features from the source and target domains. Meanwhile, another domain discriminator and label predictor using the Softmax classifier are adopted to simultaneously implement domain adaptation and fault diagnosis. Comprehensive analysis is performed on real data to test the performance of the SPADA model and detailed comparisons are provided; the extensive experimental results show that the diagnosis performance of SPADA outperforms the existing deep learning and domain adaptation methods in dealing with the PDA problem.
Read morePartial Adversarial Domain Adaptation
Domain adversarial learning aligns the feature distributions across the source and target domains in a two-player minimax game. Existing domain adversarial networks generally assume identical label space across different domains. In the presence of big data, there is strong motivation of transferring deep models from existing big domains to unknown small domains. This paper introduces partial domain adaptation as a new domain adaptation scenario, which relaxes the fully shared label space assumption to that the source label space subsumes the target label space. Previous methods typically match the whole source domain to the target domain, which are vulnerable to negative transfer for the partial domain adaptation problem due to the large mismatch between label spaces. We present Partial Adversarial Domain Adaptation (PADA), which simultaneously alleviates negative transfer by down-weighing the data of outlier source classes for training both source classifier and domain adversary, and promotes positive transfer by matching the feature distributions in the shared label space. Experiments show that PADA exceeds state-of-the-art results for partial domain adaptation tasks on several datasets.
Read moreTight Source Domain Match for Partial Domain Adaptation Based on Maximum Density
Unsupervised domain adaptation (UDA) aims at transferring knowledge between a well-labelled ’source domain’ and an unlabelled ’target domain’ by decreasing distribution discrepancy. In real scenario, partial domain adaptation (PDA), where target domain only includes part of the classes of source domain, is adopted as fully-shared label space is often unavailable. Non-identical label spaces across domains lead to performance degradation due to source-unique classes being mis-matched to the target domain, i.e. negative transfer of the target domain. Although existing PDA approaches have produced promising results, they still confront with negative transfer problem without rigorous generalization bounds. In our work, a novel PDA model has been proposed based on margin disparity and maximum source intra-class density divergence (MDSD). It matches the feature distributions with shared labels and congregates source samples in the source with affirmative labels. Removing maximum target density with pseudo labels, it effectively avoids over-fitting and accelerates learning speed. In addition, we construct a new garbage classification dataset, which is comprised of source domain - Product (Pr) and target domain - Garbage (Ga) for PDA validation. Experiments show that our proposed unsupervised partial domain adaptation method has a good performance on Office-31, Office-Home and Pr-Ga datasets.
Read morePartial adversarial domain adaptation by dual-domain alignment for fault diagnosis of rotating machines
Partial adversarial domain adaptation by dual-domain alignment for fault diagnosis of rotating machines
Learning Target-Domain-Specific Classifier for Partial Domain Adaptation.
Unsupervised domain adaptation (UDA) aims at reducing the distribution discrepancy when transferring knowledge from a labeled source domain to an unlabeled target domain. Previous UDA methods assume that the source and target domains share an identical label space, which is unrealistic in practice since the label information of the target domain is agnostic. This article focuses on a more realistic UDA scenario, i.e., partial domain adaptation (PDA), where the target label space is subsumed to the source label space. In the PDA scenario, the source outliers that are absent in the target domain may be wrongly matched to the target domain (technically named negative transfer), leading to performance degradation of UDA methods. This article proposes a novel target-domain-specific classifier learning-based domain adaptation (TSCDA) method. TSCDA presents a soft-weighed maximum mean discrepancy criterion to partially align feature distributions and alleviate negative transfer. Also, it learns a target-specific classifier for the target domain with pseudolabels and multiple auxiliary classifiers to further address the classifier shift. A module named peers-assisted learning is used to minimize the prediction difference between multiple target-specific classifiers, which makes the classifiers more discriminant for the target domain. Extensive experiments conducted on three PDA benchmark data sets show that TSCDA outperforms other state-of-the-art methods with a large margin, e.g., 4% and 5.6% averagely on Office-31 and Office-Home, respectively.
Read moreImproving the Transferability of Deep Learning Models for Crop Yield Prediction: A Partial Domain Adaptation Approach
Over the past few years, there has been extensive exploration of machine learning (ML), especially deep learning (DL), for crop yield prediction, resulting in impressive levels of accuracy. However, such models are highly dependent on training samples with ground truth labels (i.e., crop yield records), which are not available in some regions. Additionally, due to the existence of domain shifts between different spatial regions, DL models trained within one region (i.e., source domain) tend to have poor performance when directly applied to other regions (i.e., target domain). Unsupervised domain adaptation (UDA) has become a promising strategy to improve the transferability of DL models by aligning the feature distributions in the source domain and the target domain. Despite the success, existing UDA models generally assume an identical label space across different domains. This assumption can be invalid in crop yield prediction scenarios, as crop yields can vary significantly in heterogeneous regions. Due to the mismatch between label spaces, negative transfer may occur if the entire source and target domains are forced to align. To address this issue, we proposed a novel partial domain adversarial neural network (PDANN), which relaxes the assumption of fully, equally shared label spaces across domains by downweighing the outlier source samples. Specifically, during model training, the PDANN weighs each labeled source sample based on the likelihood of its yield value given the expected target yield distribution. Instead of aligning the target domain to the entire source domain, the PDANN model downweighs the outlier source samples and performs partial weighted alignment of the target domain to the source domain. As a result, the negative transfer caused by source samples in the outlier label space would be alleviated. In this study, we assessed the model’s performance on predicting yields for two main commodities in the U.S., including corn and soybean, using the U.S. corn belt as the study region. The counties under study were divided into two distinct ecological zones and alternatively used as the source and target domains. Feature variables, including time-series vegetation indices (VIs) and sequential meteorological variables, were collected and aggregated at the county level. Next, the PDANN model was trained with the extracted features and corresponding crop yield records from the U.S. Department of Agriculture (USDA). Finally, the trained model was evaluated for three testing years from 2019 to 2021. The experimental results showed that the developed PDANN model had achieved a mean coefficient of determination (R2) of 0.70 and 0.67, respectively, in predicting corn and soybean yields, outperforming three other ML and UDA models by a large margin from 6% to 46%. As the first study performing partial domain adaptation for crop yield prediction, this research demonstrates a novel solution for addressing negative transfer and improving DL models’ transferability on crop yield prediction.
Read moreAddressing the Overfitting in Partial Domain Adaptation With Self-Training and Contrastive Learning
Partial domain adaptation (PDA) assumes that target domain class label set is a subset of that of source domain, while this problem setting is close to the actual scenario. At present, there are mainly two methods to solve the overfitting of source domain in PDA, namely the entropy minimization and the weighted self-training. However, the entropy minimization method may make the distribution prediction sharp but inaccurate for samples with relatively average prediction distribution, and cause the model to learn more error information. While the weighted self-training method will introduce erroneous noise information in the self-training process due to the existence of noise weights. Therefore, we address these issues in our work and propose self-training contrastive partial domain adaptation method (STCPDA). We present two modules to mine domain information in STCPDA. We first design self-training module based on simple samples in target domain to address the overfitting to source domain. We divide the target domain samples into simple samples with high reliability and difficult samples with low reliability, and the pseudo-labels of simple samples are selected for self-training learning. Then we construct the contrastive learning module for source and target domains. We embed contrastive learning into feature space of the two domains. By this contrastive learning module, we can fully explore the hidden information in all domain samples and make the class boundary more salient. Many experimental results on five datasets show the effectiveness and excellent classification performance of our method.
Read moreDiscriminative Partial Domain Adversarial Network
Domain adaptation (DA) has been a fundamental building block for Transfer Learning (TL) which assumes that source and target domain share the same label space. A more general and realistic setting is that the label space of target domain is a subset of the source domain, as termed by Partial domain adaptation (PDA). Previous methods typically match the whole source domain to target domain, which causes negative transfer due to the source-negative classes in source domain that does not exist in target domain. In this paper, a novel Discriminative Partial Domain Adversarial Network (DPDAN) is developed. We first propose to use hard binary weighting to differentiate the source-positive and source-negative samples in the source domain. The source-positive samples are those with labels shared by two domains, while the rest in the source domain are treated as source-negative samples. Based on the above binary relabeling strategy, our algorithm maximizes the distribution divergence between source-negative samples and all the others (source-positive and target samples), meanwhile minimizes domain shift between source-positive samples and target domain to obtain discriminative domain-invariant features. We empirically verify DPDAN can effectively reduce the negative transfer caused by source-negative classes, and also theoretically show it decreases negative transfer caused by domain shift. Experiments on four benchmark domain adaptation datasets show DPDAN consistently outperforms state-of-the-art methods.KeywordsPartial domain adaptationAdversarial learningDiscriminative learning
Read moreA Weighted Partial Domain Adaptation for Acoustic Scene Classification and Its Application in Fiber Optic Security System
Domain adaptation (DA) is a technology that transfers knowledge from the source domain to the target domain. General domain adaptation assume that the source and the target domain have the same label space. However, in practical application tasks, the label of the target domain is often only a subset of the source label. For this situation, partial domain adaptation is usually used as an effective solution to transfer knowledge from a large number of labeled data sets to unlabeled micro data sets. In this article, a weighted partial domain adaptation method is proposed to solve the Acoustic Scene Classification (ASC) problem. Our method establish a connection between source and target domains to do the partial domain adaptation. Experiments are carried out on TUT and ESC-50 datasets which show that our method achieves state-of-the-art results. What is more, we apply the algorithms to an optical fiber perimeter security system to complete early warning by identifying intrusion signals.
Read morePartial Domain Adaptation for Scene Classification From Remote Sensing Imagery
Although domain adaptation approaches have been proposed to tackle cross-regional, multitemporal, and multisensor remote sensing applications since they do not require any human interpretation in the target domain, most current works assume identical label space across the source and the target domains. However, in real-world applications, we often transfer knowledge from a large-scale dataset with rich annotations to a small-scale target dataset with scarcity of labels. In most cases, the label space of the source domain is usually large enough to subsume that of the target domain, which is termed partial domain adaptation. In this article, we propose a new partial domain adaptation algorithm for remote sensing scene classification and our proposed method contains three major parts. First, we employ a progressive auxiliary domain module to alleviate the negative transfer effect caused by outlier classes. Second, we adopt an improved domain adversarial neural network (DANN) with multiweights to better encourage domain confusion. Last but not least, we design an attentive complement entropy regularization to improve the prediction confidence for samples and avoid untransferable samples (such as the samples belonging to outlier classes in the source domain) being mistakenly classified. We collect three common remote sensing datasets to evaluate our proposed method. Our method achieves an average accuracy of 79.36%, which considerably outperforms other state-of-the-art partial domain adaptation methods with an average accuracy improvement of 1.90%–12.45% and attaining a 13.67% gain compared to the straightforward deep learning model (ResNet-50). The experiment results indicate that our approach shows promising prospects for solving more general and practical domain adaptation problems where the label space of the source domain subsumes that of the target domain.
Read moreMulti-source partial domain adaptation method based on pseudo-balanced target domain for fault diagnosis
Multi-source partial domain adaptation method based on pseudo-balanced target domain for fault diagnosis
Adversarial Partial Domain Adaptation by Cycle Inconsistency
Unsupervised partial domain adaptation (PDA) is a unsupervised domain adaptation problem which assumes that the source label space subsumes the target label space. A critical challenge of PDA is the negative transfer problem, which is triggered by learning to match the whole source and target domains. To mitigate negative transfer, we note a fact that, it is impossible for a source sample of outlier classes to find a target sample of the same category due to the absence of outlier classes in the target domain, while it is possible for a source sample of shared classes. Inspired by this fact, we exploit the cycle inconsistency, i.e., category discrepancy between the original features and features after cycle transformations, to distinguish outlier classes apart from shared classes in the source domain. Accordingly, we propose to filter out source samples of outlier classes by weight suppression and align the distributions of shared classes between the source and target domains by adversarial learning. To learn accurate weight assignment for filtering out outlier classes, we design cycle transformations based on domain prototypes and soft nearest neighbor, where center losses are introduced in individual domains to reduce the intra-class variation. Experiment results on three benchmark datasets demonstrate the effectiveness of our proposed method.KeywordsUnsupervised partial domain adaptationNegative transferCycle inconsistency
Read morePartial Domain Adaptation Method Based on Class-Weighted Alignment for Fault Diagnosis of Rotating Machinery
Domain adaptation (DA)-based methods for fault diagnosis (FD) of rotating machinery have achieved impressive results in recent years. Most methods hold the assumption that the source domain (SD) and target domain (TD) share the same label space, which is not always satisfied in actual situations. A more practical scenario called partial domain adaptation (PDA) needs to be given more attention, where the transferable knowledge learned from a larger SD is applied to a smaller but relevant TD. A PDA method called class-weighted alignment-based transfer network (CWATN) is proposed in this paper to adapt to this scenario. A novel weighting method is designed to adapt the data distributions of shared classes. Except for weighted class-level alignment, global-level feature adaptation is also considered to learn more general transferable knowledge. Moreover, a domain discrepancy learning block is plugged in the shared classifier as a residual block, which could enforce the network to learn and measure the discrepancy between SD and TD explicitly, thus improving the result of DA. Three case studies are implemented to verify the superiority of CWATN. Results demonstrate that the proposed method could obtain better diagnostic performance than the selected competitive methods in the PDA scenario.
Read moreSource Class Selection With Label Propagation For Partial Domain Adaptation
In traditional unsupervised domain adaptation problems, the target domain is assumed to share the same set of classes as the source domain. In practice, there exist situations where target-domain data are from only a subset of source-domain classes and it is not known which classes the target-domain data belong to since they are unlabeled. This problem has been formulated as Partial Domain Adaptation (PDA) in the literature and is a challenging task due to the negative transfer issue (i.e. source-domain data belonging to the irrelevant classes harm the domain adaptation). We address the PDA problem by detecting the outlier classes in the source domain progressively. As a result, the PDA is boiled down to an easier unsupervised domain adaptation problem which can be solved without the issue of negative transfer. Specifically, we employ the locality preserving projection to learn a latent common subspace in which a label propagation algorithm is used to label the target-domain data. The outlier classes can be detected if no target-domain data are labeled as these classes. We remove the detected outlier classes from the source domain and repeat the process for multiple iterations until convergence. Experimental results on commonly used datasets Office31 and Office-Home demonstrate our proposed method achieves state-of-the-art performance with an average accuracy of 98.1% and 75.4% respectively.
Read more