- Research Article
45
- 10.1016/j.patcog.2021.108238
Universal multi-Source domain adaptation for image classification
- Aug 14, 2021
- Pattern Recognition
- Yueming Yin + 3 more +3
Universal multi-Source domain adaptation for image classification
Reversed domain adaptation for nuclei segmentation-based pathological image classification
Universal multi-Source domain adaptation for image classification
Universal multi-Source domain adaptation for image classification
Deep Asymmetric Transfer Network for Unbalanced Domain Adaptation
Recently, domain adaptation based on deep models has been a promising way to deal with the domains with scarce labeled data, which is a critical problem for deep learning models. Domain adaptation propagates the knowledge from a source domain with rich information to the target domain. In reality, the source and target domains are mostly unbalanced in that the source domain is more resource-rich and thus has more reliable knowledge than the target domain. However, existing deep domain adaptation approaches often pre-assume the source and target domains balanced and equally, leading to a medium solution between the source and target domains, which is not optimal for the unbalanced domain adaptation. In this paper, we propose a novel Deep Asymmetric Transfer Network (DATN) to address the problem of unbalanced domain adaptation. Specifically, our model will learn a transfer function from the target domain to the source domain and meanwhile adapting the source domain classifier with more discriminative power to the target domain. By doing this, the deep model is able to adaptively put more emphasis on the resource-rich source domain. To alleviate the scarcity problem of supervised data, we further propose an unsupervised transfer method to propagate the knowledge from a lot of unsupervised data by minimizing the distribution discrepancy over the unlabeled data of two domains. The experiments on two real-world datasets demonstrate that DATN attains a substantial gain over state-of-the-art methods.
Read moreDomain-Adaptive TinyML Model for Efficient Pest and Disease Detection in Domestic Crops: A Practical Approach for Developing Countries
Artificial intelligence (AI) is a disruptive technology that has achieved remarkable breakthroughs across various domains and is reshaping our daily lives. However, AI’s resource-intensive, time-consuming, and domain-specific nature creates a significant divide between developed countries, with abundant resources, and developing countries, where computing power and datasets are limited. To bridge this gap, Tiny Machine Learning (TinyML), Transfer Learning (TL), and Domain Adaptation (DA) emerge as crucial approaches for implementing AI in resource-scarce environments. TinyML deploys compressed AI models on resource-constrained devices, including mobile devices and microcontrollers. TL leverages knowledge from one task to enhance performance on a related task, while DA, a sub-discipline of TL, applies models trained on one distribution to a related but different distribution. This paper presents a feasibility study on applying DA to develop a TinyML model for detecting and classifying diseases in domestic crops. We proposed a 2D-CNN model named "AgroNet," equipped with a domain classification head to predict the input domain, thus enhancing adaptability to different agricultural domains. We evaluated its performance with a TL-based 2D-CNN model and a traditional 2D-CNN baseline model in both source and target domains. The source domain dataset encompassed tea, apples, coffee, tomatoes, beans, and maize, while the target domain dataset included peas, cabbage, cassava, cowpea, and spinach—evaluation metrics comprised precision, f-measure, and accuracy. Results indicated superior performance in the source domain compared to the target domain. However, AgroNet exhibited satisfactory performance in the target domain, achieving an average score of over 0.7 across all metrics, while the baseline and TL-based models displayed significant performance drops, recording averages of less than 0.5 in most classes. The TL and baseline models’ reliance on static features limited their adaptability, particularly in the target domain characterized by diverse image characteristics. These findings underscore the necessity of extensive retraining for traditional approaches in new domains. Regarding memory and energy requirements, AgroNet consumed more resources than the other ML models, with 200MB and 500 joules expended over 61 seconds of training. We deployed the AgroNet model in an apple farm in Kericho County embedded in a mobile device. The energy and memory requirements were sufficient for execution on resource-scarce devices, as evidenced by the mobile devices, including embedded systems. The model recorded average results during real deployment with a significant drop in classification performance compared to simulation environments, likely due to noise introduced in the deployment environment, including background noise and differing lighting conditions. This study demonstrated the feasibility of DA for AI deployment in resource-constrained settings, as it can be trained on a source domain and applied in a comparable target domain after limited fine-tuning. This saves on computing costs and accelerates deployments. In the future, we will explore ensemble learning techniques, including boosting and bagging, and feature engineering methods, such as principal component analysis, feature selection, and data augmentation, to enhance machine learning classification performance.
Read moreEnhancing Cross-Domain Remote Sensing Scene Classification by Multi-Source Subdomain Distribution Alignment Network
Multi-source domain adaptation (MSDA) in remote sensing (RS) scene classification has recently gained significant attention in the visual recognition community. It leverages multiple well-labeled source domains to train a model capable of achieving strong generalization on the target domain with little to no labeled data from the target domain. However, the distribution shifts among multiple source domains make it more challenging to align the distributions between the target domain and all source domains concurrently. Moreover, relying solely on global alignment risks losing fine-grained information for each class, especially in the task of RS scene classification. To alleviate these issues, we present a Multi-Source Subdomain Distribution Alignment Network (MSSDANet), which introduces novel network structures and loss functions for subdomain-oriented MSDA. By adopting a two-level feature extraction strategy, this model attains better global alignment between the target domain and multiple source domains, as well as alignment at the subdomain level. First, it includes a pre-trained convolutional neural network (CNN) as a common feature extractor to fully exploit the shared invariant features across one target and multiple source domains. Secondly, a dual-domain feature extractor is used after the common feature extractor, which maps the data from each pair of target and source domains to a specific dual-domain feature space and performs subdomain alignment. Finally, a dual-domain feature classifier is employed to make predictions by averaging the outputs from multiple classifiers. Accompanied by the above network, two novel loss functions are proposed to boost the classification performance. Discriminant Semantic Transfer (DST) loss is exploited to force the model to effectively extract semantic information among target and source domain samples, while Class Correlation (CC) loss is introduced to reduce the feature confusion from different classes within the target domain. It is noteworthy that our MSSDANet is developed in an unsupervised manner for domain adaptation, indicating that no label information from the target domain is required during training. Extensive experiments on four common RS image datasets demonstrate that the proposed method achieves state-of-the-art performance for cross-domain RS scene classification. Specifically, in the dual-source and three-source settings, MSSDANet outperforms the second-best algorithm in terms of overall accuracy (OA) by 2.2% and 1.6%, respectively.
Read moreA Novel Domain Adaptation Bayesian Classifier for Updating Land-Cover Maps With Class Differences in Source and Target Domains
This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.
Read moreRobust Hyperspectral Image Domain Adaptation With Noisy Labels
In hyperspectral image (HSI) classification, domain adaptation (DA) methods have been proved effective to address unsatisfactory classification results caused by the distribution difference between training (i.e., source domain) and testing (i.e., target domain) pixels. However, these methods rely on accurate labels in source domain, and seldom consider the performance drop resulted by noisy label, which often happens since labeling pixel in HSI is a challenging task. To improve the robustness of DA method to label noise, we propose a new unsupervised HSI DA method, which is constructed from both feature-level and classifier-level. First, a linear transformation function is learned in feature-level to align the source (domain) subspace with the target (domain) subspace. Then, a robust low-rank representation based classifier is developed to well cope with the features obtained from the aligned subspace. Since both subspace alignment and the classifier are immune to noisy labels, the proposed method obtains good classification results when confronting with noisy labels in source domain. Experimental results on two DA benchmarks demonstrate the effectiveness of the proposed method.
Read moreDomain Adaptation for Face Recognition: Targetize Source Domain Bridged by Common Subspace
In many applications, a face recognition model learned on a source domain but applied to a novel target domain degenerates even significantly due to the mismatch between the two domains. Aiming at learning a better face recognition model for the target domain, this paper proposes a simple but effective domain adaptation approach that transfers the supervision knowledge from a labeled source domain to the unlabeled target domain. Our basic idea is to convert the source domain images to target domain (termed as targetize the source domain hereinafter), and at the same time keep its supervision information. For this purpose, each source domain image is simply represented as a linear combination of sparse target domain neighbors in the image space, with the combination coefficients however learnt in a common subspace. The principle behind this strategy is that, the common knowledge is only favorable for accurate cross-domain reconstruction, but for the classification in the target domain, the specific knowledge of the target domain is also essential and thus should be mostly preserved (through targetization in the image space in this work). To discover the common knowledge, specifically, a common subspace is learnt, in which the structures of both domains are preserved and meanwhile the disparity of source and target domains is reduced. The proposed method is extensively evaluated under three face recognition scenarios, i.e., domain adaptation across view angle, domain adaptation across ethnicity and domain adaptation across imaging condition. The experimental results illustrate the superiority of our method over those competitive ones.
Read moreUnsupervised Adversarial Domain Adaptation Network for Semantic Segmentation
With the rapid development of deep learning technology, semantic segmentation methods have been widely used in remote sensing data. A pretrained semantic segmentation model usually cannot perform well when the testing images (target domain) have an obvious difference from the training data set (source domain), while a large enough labeled data set is almost impossible to be acquired for each scenario. Unsupervised domain adaptation (DA) techniques aim to transfer knowledge learned from the source domain to a totally unlabeled target domain. By reducing the domain shift, DA methods have shown the ability to improve the classification accuracy for the target domain. Hence, in this letter, we propose an unsupervised adversarial DA network that converts deep features into 2-D feature curves and reduces the discrepancy between curves from the source domain and curves from the target domain based on a conditional generative adversarial networks (cGANs) model. Our proposed DA network is able to improve the semantic labeling accuracy when we apply a pretrained semantic segmentation model to the target domain. To test the effectiveness of the proposed method, experiments are conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) 2-D Semantic Labeling data set. Results show that our proposed network is able to stably improve overall accuracy not only when the source and target domains are from the same city but with different building styles but also when the source and target domains are from different cities and acquired by different sensors. By comparing with a few state-of-the-art DA methods, we demonstrate that our proposed method achieves the best cross-domain semantic segmentation performance.
Read moreHeterogeneous Domain Adaptation: An Unsupervised Approach.
Domain adaptation leverages the knowledge in one domain-the source domain-to improve learning efficiency in another domain-the target domain. Existing heterogeneous domain adaptation research is relatively well-progressed but only in situations where the target domain contains at least a few labeled instances. In contrast, heterogeneous domain adaptation with an unlabeled target domain has not been well-studied. To contribute to the research in this emerging field, this article presents: 1) an unsupervised knowledge transfer theorem that guarantees the correctness of transferring knowledge and 2) a principal angle-based metric to measure the distance between two pairs of domains: one pair comprises the original source and target domains and the other pair comprises two homogeneous representations of two domains. The theorem and the metric have been implemented in an innovative transfer model, called a Grassmann-linear monotonic maps-geodesic flow kernel (GLG), which is specifically designed for heterogeneous unsupervised domain adaptation (HeUDA). The linear monotonic maps (LMMs) meet the conditions of the theorem and are used to construct homogeneous representations of the heterogeneous domains. The metric shows the extent to which the homogeneous representations have preserved the information in the original source and target domains. By minimizing the proposed metric, the GLG model learns the homogeneous representations of heterogeneous domains and transfers knowledge through these learned representations via a geodesic flow kernel (GFK). To evaluate the model, five public data sets were reorganized into ten HeUDA tasks across three applications: cancer detection, the credit assessment, and text classification. The experiments demonstrate that the proposed model delivers superior performance over the existing baselines.
Read moreStructure-Preserved Multi-source Domain Adaptation
Domain adaptation has achieved promising results in many areas, such as image classification and object recognition. Although a lot of algorithms have been proposed to solve the task with different domain distributions, it remains a challenge for multi-source unsupervised domain adaptation. In addition, most of the existing algorithms learn a classifier on the source domain and predict the labels for the target data, which indicates that only the knowledge derived from the hyperplane is transferred to the target domain and the structure information is ignored. In light of this, we propose a novel algorithm for multi-source unsupervised domain adaptation. Generally speaking, we aim to preserve the whole structure from source domains and transfer it to serve the task on the target domain. The source and target data are put together for clustering, which simultaneously explores the structures of the source and target domains. The structure-preserved information from source domain further guides the clustering process on the target domain. Extensive experiments on two widely used databases on object recognition and face identification show the substantial improvement of our proposed approach over several state-of-the-art methods. Especially, our algorithm can take use of multi-source domains and achieve robust and better performance compared with the single source domain adaptation methods.
Read moreOvercoming learning bias via Prototypical Feature Compensation for source-free domain adaptation
The focus of Source-free Unsupervised Domain Adaptation (SFUDA) is to effectively transfer a well-trained model from the source domain to an unlabelled target domain. During the target domain adaptation, the source domain data is no longer accessible. Prevalent methodologies attempt to synchronize the data distributions between the source and target domains, utilizing pseudo-labels to impart categorical information, which has made some progress in improving the model’s performance. However, performance impairments persist due to the introduction of learning bias from the source model and the impact of noisy pseudo-labels generated for the target domain. In this research, we reveal that the central cause for feature misalignment during domain transition is the learning bias, which is generated by the discrepancy of information between source and target domain data. The source domain data may contain distinguishable features that do not appear on the target domain, which causes the pre-trained source model to fail to work during domain adaptation. To overcome the information discrepancy, we propose a Prototypical Feature Compensation (PFC) Network. The network extracts representative feature maps of the source domain. Then use them to minimize the discrepancy information in the target domain feature maps. This mechanism facilitates feature alignment across different domains, allowing the model to generate more accurate categorical data through pseudo-labelling. The experimental results and ablation studies demonstrate exceptional performance on three SFUDA datasets and provide evidence of the proposed PFC method’s ability to adjust the feature distribution of both source and target domain data, ensuring their overlap in the latent space.
Read moreCACP: Covariance-Aware Cross-Domain Prototypes for Domain Adaptive Semantic Segmentation
Domain adaptive semantic segmentation aims to reduce domain shifts / discrepancies between source and target domains, improving the source domain model's generalization ability to the target domain. Recently, prototypical methods, which primarily use single-source or single-target domain prototypes as category centers to aggregate features from both domains, have achieved competitive performance in this task. However, due to large domain shifts, single-source domain prototypes have finite generalization ability and not all source domain knowledge is conducive to model generalization. Single-target domain prototypes are noisy because they are prematurely initialized with all features filtered by pseudo labels, which causes error accumulation in the prototypes. To address these issues, we propose a covariance-aware cross-domain prototypes method (CACP) to achieve robust domain adaptation. We propose to use both domain prototypes to dynamically rectify pseudo labels in the target domain, effectively reducing the recognition difficulty of hard target domain samples and narrowing the gap between features of the same category in both domains. In addition, to further generalize the model to the target domain, we propose two modules based on covariance correlation, FSPC (Features Selection by Prototypes Covariances) and WSPC (Weighting Source by Prototypes Coefficients), to learn discriminative characteristics. FSPC selects highly correlated features to update target domain prototypes online, denoising and enhancing discriminativeness between categories. WSPC utilizes the correlation coefficients between target domain prototypes and source domain features to weight each point in the source domain, eliminating the information interference from the source domain. In particular, CACP achieves excellent performance on the GTA5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\to$</tex-math></inline-formula> Cityscapes and SYNTHIA <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\to$</tex-math></inline-formula> Cityscapes tasks with minimal computational resources and time.
Read moreLabel Space Embedding of Manifold Alignment for Domain Adaption
In recent years, domain adaptation methods have aroused much interest in the machine learning community which transfer labeled information from the source domain to the target domain. However, most of the domain adaptation methods require that the source domain and target domain share the same features which may limit the applications of these approaches. In the paper, we propose a novel domain adaptation approach for heterogeneous source and target domains. Our approach reconstructs each sample point using the feature line distances between the sample point and training classes. Then we build the connections between the points from the source and target domains using the reconstructed vectors. Finally, the points from the target domain are projected to label space, preserving the local geometries of the target manifold and the connections between the source and target data. Different with the general domain adaptation approaches, the labels of the unlabeled target points can be predicted automatically without using the classifiers. Results of numerical experiments illustrate the effectiveness of the proposed approach.
Read moreA New Progressive Multisource Domain Adaptation Network With Weighted Decision Fusion.
Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.
Read moreA multi-source domain adaption intelligent fault diagnosis method based on asymmetric adversarial training
To enhance the cross-domain diagnostic ability of the model, domain adaptation method is adopted. When using traditional domain adaption methods to extract domain invariant characteristics of axial flow fan faults, the characteristics of the source and target domains will be close to each other, thereby the distribution of trained source domain characteristics will be changed. When the fault characteristics of the source domain gather at the classification boundary, the trained model will incorrectly classify some target samples. In addition, single source domain adaptation can lead to poor model generalization ability. To resolve the above issues, a multi-source domain adaption intelligent fault diagnosis method based on asymmetric adversarial training is proposed. In this method, the asymmetric adversarial training method is used to realize unidirectional movement of fault characteristics from the target domain to the source domain; triplet-center loss is used to expand the inter-class distance and shorten the intra-class distance of fault characteristics in the source domain; domain invariant characteristics are extracted from different source and target domains, and they are inputted to their respective fault classifiers, then aligning the outputs of each classifier using cosine similarity. To improve the cross-domain diagnostic ability of the model, a strategy of aligning weights is adopted. The industrial actual data verification results indicate that this method is effective in solving relevant practical industrial problems.
Read more