- Research Article
15
- 10.1016/j.knosys.2022.109320
Manifold discrimination partial adversarial domain adaptation
- Jun 25, 2022
- Knowledge-Based Systems
- Chunmei He + 4 more +4
Manifold discrimination partial adversarial domain adaptation
Domain adversarial learning aligns the feature distributions across the source and target domains in a two-player minimax game. Existing domain adversarial networks generally assume identical label space across different domains. In the presence of big data, there is strong motivation of transferring deep models from existing big domains to unknown small domains. This paper introduces partial domain adaptation as a new domain adaptation scenario, which relaxes the fully shared label space assumption to that the source label space subsumes the target label space. Previous methods typically match the whole source domain to the target domain, which are vulnerable to negative transfer for the partial domain adaptation problem due to the large mismatch between label spaces. We present Partial Adversarial Domain Adaptation (PADA), which simultaneously alleviates negative transfer by down-weighing the data of outlier source classes for training both source classifier and domain adversary, and promotes positive transfer by matching the feature distributions in the shared label space. Experiments show that PADA exceeds state-of-the-art results for partial domain adaptation tasks on several datasets.
Manifold discrimination partial adversarial domain adaptation
Manifold discrimination partial adversarial domain adaptation
Partial adversarial domain adaptation by dual-domain alignment for fault diagnosis of rotating machines
Partial adversarial domain adaptation by dual-domain alignment for fault diagnosis of rotating machines
A Stacked Auto-Encoder Based Partial Adversarial Domain Adaptation Model for Intelligent Fault Diagnosis of Rotating Machines
Fault diagnosis plays an indispensable role in prognostics and health management of rotating machines. In recent years, intelligent fault diagnosis methods based on domain adaptation technology have attracted the attention of researchers. However, a more extensive application scenario of fault diagnosis - partial domain adaptation (PDA) - has not been well-resolved. In this article, for the first time, a novel stacked auto-encoder based partial adversarial domain adaptation (SPADA) model is proposed to solve the fault diagnosis problem in PDA situations. Two deep stack auto-encoders are first designed to extract representative features from the training data (source domain) and test data (target domain), respectively. Then, a weighted classifier based on Softmax is used to weight the features from the source and target domains. Meanwhile, another domain discriminator and label predictor using the Softmax classifier are adopted to simultaneously implement domain adaptation and fault diagnosis. Comprehensive analysis is performed on real data to test the performance of the SPADA model and detailed comparisons are provided; the extensive experimental results show that the diagnosis performance of SPADA outperforms the existing deep learning and domain adaptation methods in dealing with the PDA problem.
Read morePartial Video Domain Adaptation with Partial Adversarial Temporal Attentive Network
Partial Domain Adaptation (PDA) is a practical and general domain adaptation scenario, which relaxes the fully shared label space assumption such that the source label space subsumes the target one. The key challenge of PDA is the issue of negative transfer caused by source-only classes. For videos, such negative transfer could be triggered by both spatial and temporal features, which leads to a more challenging Partial Video Domain Adaptation (PVDA) problem. In this paper, we propose a novel Partial Adversarial Temporal Attentive Network (PATAN) to address the PVDA problem by utilizing both spatial and temporal features for filtering source-only classes. Besides, PATAN constructs effective overall temporal features by attending to local temporal features that contribute more toward the class filtration process. We further introduce new benchmarks to facilitate research on PVDA problems, covering a wide range of PVDA scenarios. Empirical results demonstrate the state-of-the-art performance of our proposed PATAN across the multiple PVDA benchmarks. Code will be provided at: https://github.com/xuyu0010/PATAN.
Read moreRobust and efficient deep learning methods for vision-based action recognition
Vision-based action recognition, which performs action recognition based solely on RGB frames, has received strong research interest thanks to its wide applications in various fields, e.g. surveillance, smart homes, and autonomous driving. Significant progress has been made in vision-based action recognition thanks to the development of recognition technologies, particularly deep learning methods which have proven their effectiveness in visual recognition tasks, such as image classification. Compared to static images, videos contain additional information due to the additional temporal dimension, which includes both temporal and spatiotemporal correlation features. Therefore, the key to robust and efficient action recognition lies in the effective and efficient utilization of the temporal and spatiotemporal correlation features embedded within videos. \n \nIn this thesis, we first investigate extracting temporal features in a robust and efficient manner. While methods for extracting temporal features were proposed, these methods either require the computation or estimation of optical flow, which demand high computational power and large storage resource; or extracts only linear feature along the temporal dimension which results in inferior performances. To extract temporal features without the utilization of optical flow, we propose Attentive Correlated Temporal Feature (ACTF) which leverages inter-frame correlation feature and exploits both bilinear and linear correlations between successive frames on the regional level. By excluding optical flow estimation or calculation, ACTF can be combined with any spatial feature extraction network under the two-stream structure for end-to-end training. Meanwhile, capturing long-range spatiotemporal dependencies is an effective strategy for extracting spatiotemporal correlation features. Previous works have proposed methods utilizing either hand-crafted features or stacks of convolution or recurrent modules, both of which are computationally inefficient, and cause difficulty in network optimization. While the more recent non-local block inspired by the non-local means could extract long-range dependencies without affecting the networks' optimization, it significantly increases the parameter size and computational cost of the inserted networks. To extract robust long-range dependencies more efficiently, we explore on further improving the non-local neural network by proposing a novel long-range spatiotemporal dependencies extraction module, the Pyramid Non-Local (PNL) module. It extends the original non-local block by incorporating regional feature correlations at multiple scales. PNL upscales the effectiveness of the original non-local block by additionally addressing the spatiotemporal correlations between different regions, while improving the efficiency of the original non-local block with a significant decrease in computation cost. \n \nBesides the development of recognition technologies, the progress made in vision-based action recognition could also be attributed to the development of large-scale video datasets, which enable the effective training of deep learning models. However, it could be observed that the majority of current research focuses on videos in normal illumination, partly due to the fact that current benchmark datasets for vision-based action recognition are normally collected from web videos shot mostly under normal illumination. Yet, we argue that vision-based action recognition should not be constrained in normal illuminated videos. Vision-based action recognition in dark videos are also useful in various scenarios, e.g., night surveillance and self-driving at night. Such a task has rarely been researched, partly due to the lack of sufficient datasets for such a task. To this end, this thesis bridges the gap of the lack of data and pioneers vision-based action recognition in dark videos by collecting a novel dataset: the Action Recognition in the Dark (ARID) dataset. In this thesis, the ARID dataset is analyzed thoroughly with a comprehensive benchmark of current deep learning methods. \n \nMeanwhile, though the introduction of ARID pioneers vision-based action recognition in dark videos and bridges the gap between the absence of dark video datasets with the need for research, the scale of such a dataset is relatively small compared to current large-scale video datasets. One solution for training robust models for domains with less labeled data would be to transfer models learned in well-labeled domains. However, models trained in one domain would not generalize well in the other domain due to domain shift across domains, which is presented as distribution discrepancy between different domains. Domain adaptation (DA) approaches address domain shifts and enable networks to be applied to different scenarios. Although various image DA approaches have been proposed in recent years, there is limited research towards Video Domain Adaptation (VDA), owing to the complexity in adapting the different modalities of features in videos, which includes both temporal features and spatiotemporal correlation features. We argue that the correlation features are highly associated with action classes and proven their effectiveness in accurate video feature extraction through supervised vision-based action recognition. Yet correlation features of the same action would differ across domains due to domain shift. Adversarial Correlation Adaptation Network (ACAN) is developed in this thesis to align action videos by aligning pixel correlations, while a novel HMDB-ARID dataset with a larger domain shift is built in an effort to leverage current datasets for vision-based action recognition in dark videos. \n \nWe observe that while VDA methods enable the learning of transferable features across domains, these methods generally assume that the video source and target domains share an identical label space. Such an assumption may not hold in real-world applications. Instead, Partial Domain Adaptation (PDA) is a practical and general domain adaptation scenario, which relaxes the fully shared label space assumption such that the source label space subsumes the target one, and is more challenging than DA due to negative transfer caused by source-only classes. For videos, such negative transfer could be triggered by both spatial and temporal features, which leads to an even more challenging Partial Video Domain Adaptation (PVDA) problem. This thesis pioneers the PVDA problem by proposing a novel Partial Adversarial Temporal Attentive Network (PATAN) by utilizing both spatial and temporal features for filtering source-only classes. This thesis further introduces new benchmarks to facilitate research on PVDA problems, covering a wide range of PVDA scenarios. \n \nIn summary, this thesis contributes to robust and efficient vision-based action recognition by introducing two algorithms for extracting robust and efficient temporal or spatiotemporal correlation features and pioneering in the research of robust vision-based action recognition in dark videos, breaking through the constraint of current vision-based action recognition research conducted on only normal illuminated videos.
Read moreImproving the Transferability of Deep Learning Models for Crop Yield Prediction: A Partial Domain Adaptation Approach
Over the past few years, there has been extensive exploration of machine learning (ML), especially deep learning (DL), for crop yield prediction, resulting in impressive levels of accuracy. However, such models are highly dependent on training samples with ground truth labels (i.e., crop yield records), which are not available in some regions. Additionally, due to the existence of domain shifts between different spatial regions, DL models trained within one region (i.e., source domain) tend to have poor performance when directly applied to other regions (i.e., target domain). Unsupervised domain adaptation (UDA) has become a promising strategy to improve the transferability of DL models by aligning the feature distributions in the source domain and the target domain. Despite the success, existing UDA models generally assume an identical label space across different domains. This assumption can be invalid in crop yield prediction scenarios, as crop yields can vary significantly in heterogeneous regions. Due to the mismatch between label spaces, negative transfer may occur if the entire source and target domains are forced to align. To address this issue, we proposed a novel partial domain adversarial neural network (PDANN), which relaxes the assumption of fully, equally shared label spaces across domains by downweighing the outlier source samples. Specifically, during model training, the PDANN weighs each labeled source sample based on the likelihood of its yield value given the expected target yield distribution. Instead of aligning the target domain to the entire source domain, the PDANN model downweighs the outlier source samples and performs partial weighted alignment of the target domain to the source domain. As a result, the negative transfer caused by source samples in the outlier label space would be alleviated. In this study, we assessed the model’s performance on predicting yields for two main commodities in the U.S., including corn and soybean, using the U.S. corn belt as the study region. The counties under study were divided into two distinct ecological zones and alternatively used as the source and target domains. Feature variables, including time-series vegetation indices (VIs) and sequential meteorological variables, were collected and aggregated at the county level. Next, the PDANN model was trained with the extracted features and corresponding crop yield records from the U.S. Department of Agriculture (USDA). Finally, the trained model was evaluated for three testing years from 2019 to 2021. The experimental results showed that the developed PDANN model had achieved a mean coefficient of determination (R2) of 0.70 and 0.67, respectively, in predicting corn and soybean yields, outperforming three other ML and UDA models by a large margin from 6% to 46%. As the first study performing partial domain adaptation for crop yield prediction, this research demonstrates a novel solution for addressing negative transfer and improving DL models’ transferability on crop yield prediction.
Read moreTight Source Domain Match for Partial Domain Adaptation Based on Maximum Density
Unsupervised domain adaptation (UDA) aims at transferring knowledge between a well-labelled ’source domain’ and an unlabelled ’target domain’ by decreasing distribution discrepancy. In real scenario, partial domain adaptation (PDA), where target domain only includes part of the classes of source domain, is adopted as fully-shared label space is often unavailable. Non-identical label spaces across domains lead to performance degradation due to source-unique classes being mis-matched to the target domain, i.e. negative transfer of the target domain. Although existing PDA approaches have produced promising results, they still confront with negative transfer problem without rigorous generalization bounds. In our work, a novel PDA model has been proposed based on margin disparity and maximum source intra-class density divergence (MDSD). It matches the feature distributions with shared labels and congregates source samples in the source with affirmative labels. Removing maximum target density with pseudo labels, it effectively avoids over-fitting and accelerates learning speed. In addition, we construct a new garbage classification dataset, which is comprised of source domain - Product (Pr) and target domain - Garbage (Ga) for PDA validation. Experiments show that our proposed unsupervised partial domain adaptation method has a good performance on Office-31, Office-Home and Pr-Ga datasets.
Read morePartial domain adaptation based on shared class oriented adversarial network
Partial domain adaptation based on shared class oriented adversarial network
Learning Target-Domain-Specific Classifier for Partial Domain Adaptation.
Unsupervised domain adaptation (UDA) aims at reducing the distribution discrepancy when transferring knowledge from a labeled source domain to an unlabeled target domain. Previous UDA methods assume that the source and target domains share an identical label space, which is unrealistic in practice since the label information of the target domain is agnostic. This article focuses on a more realistic UDA scenario, i.e., partial domain adaptation (PDA), where the target label space is subsumed to the source label space. In the PDA scenario, the source outliers that are absent in the target domain may be wrongly matched to the target domain (technically named negative transfer), leading to performance degradation of UDA methods. This article proposes a novel target-domain-specific classifier learning-based domain adaptation (TSCDA) method. TSCDA presents a soft-weighed maximum mean discrepancy criterion to partially align feature distributions and alleviate negative transfer. Also, it learns a target-specific classifier for the target domain with pseudolabels and multiple auxiliary classifiers to further address the classifier shift. A module named peers-assisted learning is used to minimize the prediction difference between multiple target-specific classifiers, which makes the classifiers more discriminant for the target domain. Extensive experiments conducted on three PDA benchmark data sets show that TSCDA outperforms other state-of-the-art methods with a large margin, e.g., 4% and 5.6% averagely on Office-31 and Office-Home, respectively.
Read moreAdversarial Partial Domain Adaptation by Cycle Inconsistency
Unsupervised partial domain adaptation (PDA) is a unsupervised domain adaptation problem which assumes that the source label space subsumes the target label space. A critical challenge of PDA is the negative transfer problem, which is triggered by learning to match the whole source and target domains. To mitigate negative transfer, we note a fact that, it is impossible for a source sample of outlier classes to find a target sample of the same category due to the absence of outlier classes in the target domain, while it is possible for a source sample of shared classes. Inspired by this fact, we exploit the cycle inconsistency, i.e., category discrepancy between the original features and features after cycle transformations, to distinguish outlier classes apart from shared classes in the source domain. Accordingly, we propose to filter out source samples of outlier classes by weight suppression and align the distributions of shared classes between the source and target domains by adversarial learning. To learn accurate weight assignment for filtering out outlier classes, we design cycle transformations based on domain prototypes and soft nearest neighbor, where center losses are introduced in individual domains to reduce the intra-class variation. Experiment results on three benchmark datasets demonstrate the effectiveness of our proposed method.KeywordsUnsupervised partial domain adaptationNegative transferCycle inconsistency
Read moreDiscriminative Partial Domain Adversarial Network
Domain adaptation (DA) has been a fundamental building block for Transfer Learning (TL) which assumes that source and target domain share the same label space. A more general and realistic setting is that the label space of target domain is a subset of the source domain, as termed by Partial domain adaptation (PDA). Previous methods typically match the whole source domain to target domain, which causes negative transfer due to the source-negative classes in source domain that does not exist in target domain. In this paper, a novel Discriminative Partial Domain Adversarial Network (DPDAN) is developed. We first propose to use hard binary weighting to differentiate the source-positive and source-negative samples in the source domain. The source-positive samples are those with labels shared by two domains, while the rest in the source domain are treated as source-negative samples. Based on the above binary relabeling strategy, our algorithm maximizes the distribution divergence between source-negative samples and all the others (source-positive and target samples), meanwhile minimizes domain shift between source-positive samples and target domain to obtain discriminative domain-invariant features. We empirically verify DPDAN can effectively reduce the negative transfer caused by source-negative classes, and also theoretically show it decreases negative transfer caused by domain shift. Experiments on four benchmark domain adaptation datasets show DPDAN consistently outperforms state-of-the-art methods.KeywordsPartial domain adaptationAdversarial learningDiscriminative learning
Read morePartial Domain Adaptation for Scene Classification From Remote Sensing Imagery
Although domain adaptation approaches have been proposed to tackle cross-regional, multitemporal, and multisensor remote sensing applications since they do not require any human interpretation in the target domain, most current works assume identical label space across the source and the target domains. However, in real-world applications, we often transfer knowledge from a large-scale dataset with rich annotations to a small-scale target dataset with scarcity of labels. In most cases, the label space of the source domain is usually large enough to subsume that of the target domain, which is termed partial domain adaptation. In this article, we propose a new partial domain adaptation algorithm for remote sensing scene classification and our proposed method contains three major parts. First, we employ a progressive auxiliary domain module to alleviate the negative transfer effect caused by outlier classes. Second, we adopt an improved domain adversarial neural network (DANN) with multiweights to better encourage domain confusion. Last but not least, we design an attentive complement entropy regularization to improve the prediction confidence for samples and avoid untransferable samples (such as the samples belonging to outlier classes in the source domain) being mistakenly classified. We collect three common remote sensing datasets to evaluate our proposed method. Our method achieves an average accuracy of 79.36%, which considerably outperforms other state-of-the-art partial domain adaptation methods with an average accuracy improvement of 1.90%–12.45% and attaining a 13.67% gain compared to the straightforward deep learning model (ResNet-50). The experiment results indicate that our approach shows promising prospects for solving more general and practical domain adaptation problems where the label space of the source domain subsumes that of the target domain.
Read moreDeep Residual Correction Network for Partial Domain Adaptation.
Deep domain adaptation methods have achieved appealing performance by learning transferable representations from a well-labeled source domain to a different but related unlabeled target domain. Most existing works assume source and target data share the identical label space, which is often difficult to be satisfied in many real-world applications. With the emergence of big data, there is a more practical scenario called partial domain adaptation, where we are always accessible to a more large-scale source domain while working on a relative small-scale target domain. In this case, the conventional domain adaptation assumption should be relaxed, and the target label space tends to be a subset of the source label space. Intuitively, reinforcing the positive effects of the most relevant source subclasses and reducing the negative impacts of irrelevant source subclasses are of vital importance to address partial domain adaptation challenge. This paper proposes an efficiently-implemented Deep Residual Correction Network (DRCN) by plugging one residual block into the source network along with the task-specific feature layer, which effectively enhances the adaptation from source to target and explicitly weakens the influence from the irrelevant source classes. Specifically, the plugged residual block, which consists of several fully-connected layers, could deepen basic network and boost its feature representation capability correspondingly. Moreover, we design a weighted class-wise domain alignment loss to couple two domains by matching the feature distributions of shared classes between source and target. Comprehensive experiments on partial, traditional and fine-grained cross-domain visual recognition demonstrate that DRCN is superior to the competitive deep domain adaptation approaches.
Read moreA Weighted Partial Domain Adaptation for Acoustic Scene Classification and Its Application in Fiber Optic Security System
Domain adaptation (DA) is a technology that transfers knowledge from the source domain to the target domain. General domain adaptation assume that the source and the target domain have the same label space. However, in practical application tasks, the label of the target domain is often only a subset of the source label. For this situation, partial domain adaptation is usually used as an effective solution to transfer knowledge from a large number of labeled data sets to unlabeled micro data sets. In this article, a weighted partial domain adaptation method is proposed to solve the Acoustic Scene Classification (ASC) problem. Our method establish a connection between source and target domains to do the partial domain adaptation. Experiments are carried out on TUT and ESC-50 datasets which show that our method achieves state-of-the-art results. What is more, we apply the algorithms to an optical fiber perimeter security system to complete early warning by identifying intrusion signals.
Read morePartial Domain Adaptation Without Domain Alignment.
Unsupervised domain adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to a related and unlabeled target domain with identical label space. The main workhorse in UDA is domain alignment and has proven successful. However, it is practically difficult to find an appropriate source domain with identical label space. A more practical scenario is partial domain adaptation (PDA) where the source label space subsumes the target one. Unfortunately, due to the non-identity between label spaces, it is extremely hard to obtain an ideal alignment, conversely, easier resulting in mode collapse and negative transfer. These motivate us to find a relatively simpler alternative to solve PDA. To achieve this, we first explore a theoretical analysis, which says that the target risk is bounded by both model smoothness and between-domain discrepancy. Then, we instantiate the model smoothness as an intra-domain structure preserving (IDSP) while giving up possibly riskier domain alignment. To our best knowledge, this is the first naive attempt for PDA without alignment. Finally, our empirical results on benchmarks demonstrate that IDSP is not only superior to the PDA SOTAs (e.g., ∼+10% on Cl → Rw and ∼+8% on Ar → Rw), but also complementary to domain alignment in the standard UDA.
Read more