Conditional Domain Adaptation with α-Rényi Entropy Regularization and Noise-Aware Label Weighting

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.

A unified framework for visual domain adaptation with covariance matching

Unsupervised domain adaptation (UDA) has been widely studied with the goal of transferring knowledge from a label-rich source domain to a related but unlabeled target domain. Most UDA techniques achieve this by reducing the feature discrepancies between the two domains to learn domain-invariant feature representations. While domain-invariant feature representations can reduce the differences between the source and target domains, excessively simplifying these differences may cause the model to overlook important domain-specific features, resulting in a decline in transfer learning effectiveness. To address this issue, this paper proposes a novel Gaussian Mixture Model for graph domain adaptation (GMM). This model effectively reduces the distributional bias between the source and target domains by modeling the distribution differences on a graph structure. GMM leverages the local structural information of the graph and the clustering capability of the Gaussian mixture model to automatically learn the latent mapping relationships between the source and target domains. To the best of our knowledge, this is the first work to introduce a Gaussian mixture model into UDA. Extensive experimental results on three standard benchmarks demonstrate that the proposed GMM algorithm outperforms state-of-the-art unsupervised domain adaptation methods in terms of performance.

Accurate gearbox fault diagnosis under varying operational speeds is critical for industrial predictive maintenance. A significant challenge is domain shift, where models trained under one condition fail to generalize to another, especially when only healthy data from the target domain is available for training. This study proposes a novel domain adaptation framework, CDANet, that directly leverages raw sensor data to perform multi-class fault classification without manual feature engineering. The model combines a lightweight CNNbased temporal feature extractor with a frozen DistilBERT encoder to capture transferable, domain-invariant representations, combined with a maximum mean discrepancy loss to align the feature distributions between the source and target domains using only healthy samples. Experimental results demonstrate that our proposed model significantly outperforms conventional deep learning approaches, achieving high classification accuracy across six domain adaptation tasks. This work validates the effectiveness of applying pre-trained models in domain adaptation for gearbox fault diagnosis under real-world domain shift constraints.

Recently, Semi-supervised Domain Adaptation (SSDA) has become more practical because a small number of labeled target samples can significantly boost the empirical target performance when using SSDA. Several current methods focus on prototype-based alignment to achieve cross-domain invariance in which the labeled samples from the source and target domains are concatenated to estimate the prototypes. The model is then trained to assign the unlabeled target data to the prototype within the same class. However, such methods fail to exploit the advantage of using few labeled target data because the labeled source data dominate the prototypes in the supervision process. Moreover, a recent method [1] showed that concatenating source and target samples for training can damage the semantic information of representations, which degrades the trained model’s ability to generate discriminative features. To solve these problems, in this paper, we divide labeled source and target samples into two subgroups for training. One group includes a large number of labeled source samples, and the other obtains a few labeled target samples. Then, we propose a novel SSDA framework that consists of two models. A model trained on the group that has the labeled source samples to provide an “inter-view” on the unlabeled target data is called the inter-view model. A model trained on a few labeled target samples that provides an “intra-view” of the unlabeled target data is called the intra-view model. Finally, both of these models collaborate to fully exploit information on the unlabeled target data. To the best of our knowledge, our proposed method achieves the state-of-the-art classification performance of SSDA in extensive experiments conducted on several visual benchmark domain adaptation datasets that utilize the advantages of multiple views and collaborative training.

PurposeThe study aims to cope with the problems confronted in the skin lesion datasets with less training data toward the classification of melanoma. The vital, challenging issue is the insufficiency of training data that occurred while classifying the lesions as melanoma and non-melanoma.Design/methodology/approachIn this work, a transfer learning (TL) framework Transfer Constituent Support Vector Machine (TrCSVM) is designed for melanoma classification based on feature-based domain adaptation (FBDA) leveraging the support vector machine (SVM) and Transfer AdaBoost (TrAdaBoost). The working of the framework is twofold: at first, SVM is utilized for domain adaptation for learning much transferrable representation between source and target domain. In the first phase, for homogeneous domain adaptation, it augments features by transforming the data from source and target (different but related) domains in a shared-subspace. In the second phase, for heterogeneous domain adaptation, it leverages knowledge by augmenting features from source to target (different and not related) domains to a shared-subspace. Second, TrAdaBoost is utilized to adjust the weights of wrongly classified data in the newly generated source and target datasets.FindingsThe experimental results empirically prove the superiority of TrCSVM than the state-of-the-art TL methods on less-sized datasets with an accuracy of 98.82%.Originality/valueExperiments are conducted on six skin lesion datasets and performance is compared based on accuracy, precision, sensitivity, and specificity. The effectiveness of TrCSVM is evaluated on ten other datasets towards testing its generalizing behavior. Its performance is also compared with two existing TL frameworks (TrResampling, TrAdaBoost) for the classification of melanoma.

Dual contrastive universal adaptation network for multi-source visual recognition

Learning deep neural networks that are generalizable across different domains remains a challenge due to the problem of domain shift. Unsupervised domain adaptation is a promising avenue which transfers knowledge from a source domain to a target domain without using any labels in the target domain. Contemporary techniques focus on extracting domain-invariant features using domain adversarial training. However, these techniques neglect to learn discriminative class boundaries in the latent representation space on a target domain and yield limited adaptation performance. To address this problem, we propose distance metric guided feature alignment (MetFA) to extract discriminative as well as domain-invariant features on both source and target domains. The proposed MetFA method explicitly and directly learns the latent representation without using domain adversarial training. Our model integrates class distribution alignment to transfer semantic knowledge from a source domain to a target domain. We evaluate the proposed method on fetal ultrasound datasets for cross-device image classification. Experimental results demonstrate that the proposed method outperforms the state-of-the-art and enables model generalization.

Sample-to-sample correspondence for unsupervised domain adaptation

Deep multi-Wasserstein unsupervised domain adaptation

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.

In the absence of the labeled samples in a domain referred to as target domain, Domain Adaptation (DA) techniques come in handy. Generally, DA techniques assume there are available source domains that share similar predictive function with the target domain. Two core challenges of DA typically arise, variance that exists between source and target domains, and the inherent source hypothesis bias. In this paper, we first propose a Stability Transfer criterion for selecting relevant source domains with low variance. With this criterion, we introduce a TARget learning Assisted by Source Classifier Adaptation (TARASCA) method to address the two core challenges that have impeded the performances of DA techniques. To verify the robustness of TARASCA, extensive experimental studies are carried out with comparison to several state-of-the-art DA methods on the real-world Sentiment and Newsgroups datasets, where various settings for the class ratios of the source and target domains are considered.

Population-based structural health monitoring (PBSHM) involves utilising knowledge from one set of structures in a population and applying it to a different set, such that predictions about the health states of each member in the population can be performed and improved. Central ideas behind PBSHM are those of knowledge transfer and mapping. In the context of PBSHM, knowledge transfer involves using information from a structure, defined as a source domain, where labels are known for a given feature, and mapping these onto the unlabelled feature space of a different, target domain structure. If the mapping is successful, a machine learning classifier trained on the transformed source domain data will generalise to the unlabelled target domain data; i.e. a classifier built on one structure will generalise to another, making Structural Heath Monitoring (SHM) cost-effective and applicable to a wide range of challenging industrial scenarios. This process of mapping features and labels across source and target domains is defined as domain adaptation, a subcategory of transfer learning. However, a key assumption in conventional domain adaptation methods is that there is consistency between the feature and label spaces. This means that the features measured from one structure must be the same dimension as the other (i.e. the same number of spectral lines of a transmissibility), and that labels associated with damage locations, classification and assessment, exist on both structures. These consistency constraints can be restrictive, limiting to which types of population domain adaptation can be applied. This paper, therefore, provides a mathematical underpinning for when domain adaptation is possible in a structural dynamics context, with reference to topology of a graphical representation of structures. By defining when conventional domain adaptation is applicable in a structural dynamics setting, approaches are discussed that could overcome these consistency restrictions. This approach provides a general means for performing transfer learning within a PBSHM context for structural dynamics-based features.KeywordsPopulation-based structural health monitoringTransfer learningDomain adaptation

All current deep learning-based prediction methods for remaining useful life (RUL) assume that training and testing data have similar distributions, but the existence of various operating conditions, failure modes, and noise lead to insufficient data with similar distributions during the training process, thereby reducing RUL prediction performance. Domain adaptation can effectively solve this problem by learning the cross-domain invariant features of the source domain and target domain to reduce the distribution difference. However, most domain adaptive methods extract the source domain and target domain features into a single space for feature alignment, which may leave out effective information and affect the accuracy of prediction. To address this problem, we propose a data-driven approach named long short-term memory network and multi-representation domain adaptation (LSTM-MRAN). We standardize and process the degraded sensor data with a sliding time window, use LSTM to extract features from the degraded data, and mine the time series information between the data. Then, we use multiple substructures in multi-representation domain adaptation to extract features of the source domain and target domain from different spaces and align features by minimizing conditional maximum mean difference (CMMD) loss functions. The effectiveness of the method is verified by the CMAPSS dataset. Compared with methods without domain adaptation and other transfer learning methods, the proposed method provides more reliable RUL prediction results under datasets with different operating conditions and failure modes.

Domain adaptation (DA) aims at facilitating the target model training by leveraging knowledge from related but distribution-inconsistent source domain. Most of the previous DA works concentrate on homogeneous scenarios, where the source and target domains are assumed to share the same feature space. Nevertheless, frequently, in reality, the domains are not consistent in not only data distribution but also the representation space and feature dimensions. That is, these domains are heterogeneous. Although many works have attempted to handle such heterogeneous DA (HDA) by transforming HDA to homogeneous counterparts or performing DA jointly with domain transformation, nearly all of them just concentrate on the feature and distribution alignment across domains, neglecting the structure and classification space preservation for domains themselves. In this work, we propose a novel HDA model, namely, heterogeneous classification space alignment (HCSA), which leverages knowledge from both the source samples and model parameters to the target. In HCSA, structure preservation, distribution, and classification space alignment are implemented, jointly with feature representation by transferring both the source-domain representation and model knowledge. Moreover, we design an alternating algorithm to optimize the HCSA model with guaranteed convergence and complexity analysis. In addition, the HCSA model is further extended with deep network architecture. Finally, we experimentally evaluate the effectiveness of the proposed method by showing its superiority to the compared approaches.