MDRN: Multi‐domain representation network for unsupervised domain generalization

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.

Medical image segmentation is vital for clinical diagnosis, yet deep learning models face domain shift challenges when test data distributions differ from training data. Single-source domain generalization (DG) has emerged as a solution to these limitations of deep learning models by training models on a single source domain to generalize to unseen target domains. Current single-source DG methods employ domain randomization (e.g., input/feature-space perturbations) to simulate unseen target domains, but face two key limitations: (1) Texture bias in CNN-based architectures due to their local processing nature, leading to overfitting and poor generalization; (2) Restricted augmented style diversity caused by solely source-dependent feature perturbations and input- or feature-only augmentation, resulting in insufficient coverage of target domain distributions and degraded model's generalizationability. Motivated by the Fourier transform's capability to model global dependency and decouple domain invariant features, we propose Random Domain Generalization (RandDG), a frequency-aware domain randomization method for single-source DG in medical image segmentation. This method enhances generalization ability through coordinated input and feature spaces perturbations while maintaining parameter efficiency via lightweight architectural design in the frequencydomain. The proposed RandDG incorporates a novel Global U-Shape Network (GUNet) segmentation architecture for efficient long-range dependency modeling and a Uniform Low Frequency spectrum Transform (ULoFT) filter for feature-space perturbation to enhance domain generalization ability, facilitating the learning of global domain invariant structural information through consistency constraints within the dual-space randomization framework. More specifically, first, the GUNet architecture combines a lightweight encoder with frequency-domain global filtering and forms a U-shape structure through a skip-connected decoder, utilizing Fourier transforms to capture long-range dependencies while minimizing parameters. Second, the ULoFT filter perturbs feature-space styles by Bernoulli mixing statistics of the source low-frequency amplitude spectrum with uniformly sampled values, thereby reducing sole reliance on source statistics and broadening the coverage of potential target domains. Third, a dual-space randomization framework simultaneously applies input-space style augmentation via global intensity nonlinear augmentation (GIN) filters and feature-space amplitude spectrum perturbation via ULoFT filter, regularized by a consistency loss to enforce domain invariant representationlearning. Experiments were conducted on an abdominal CT-MRI dataset for abdominal multi-organ segmentation from CT to T2 spectral presaturation with inversion recovery (T2-SPIR) MRI and a cross-center prostate dataset for prostate segmentation from one center to others. The proposed RandDG method demonstrated superior generalization ability, significantly improving robustness against domain shifts compared to competitive methods. When tested on unseen domains, the proposed method achieved an average DSC of 87.96% and an average HD of 4.82 mm on the abdominal CT-MRI dataset, as well as an average DSC of 75.95% and an average HD of 8.36 mm on the cross-center prostate dataset. Compared to the UNet baseline, the proposed method achieved an improvement of 10.65% in average DSC and a reduction of 5.23 mm in average HD on the abdominal CT-MRI dataset, as well as 19.36% higher average DSC and 3.39 mm lower average HD on the cross-center prostate dataset. Ablation studies confirmed the effectiveness of each component of the proposed RandDGmethod. The proposed RandDG method effectively addresses the critical challenges of texture bias and limited style diversity in single-source DG for medical image segmentation through frequency-aware dual-space randomization framework. This method shows promise for practical deployment in clinical settings, where target domain data may not always beavailable.

In an indoor positioning system (IPS), transfer learning (TL) methods are commonly used to predict the location of mobile devices under the assumption that all training instances of the target domain are given in advance. However, this assumption has been criticized for its shortcomings in dealing with the problem of signal distribution variations, especially in a dynamic indoor environment. The reasons are: collecting a sufficient number of training instances is costly, the training instances may arrive online, the feature spaces of the target and source domains may be different, and negative knowledge may be transferred in the case of a redundant source domain. In this work, we proposed an online heterogeneous transfer learning (OHetTLAL) algorithm for IPS-based RSS fingerprinting to improve the positioning performance in the target domain by fusing both source and target domain knowledge. The source domain was refined based on the target domain to avoid negative knowledge transfer. The co-occurrence measure of the feature spaces (Cmip) was used to derive the homogeneous new feature spaces, and the features with higher weight values were selected for training the classifier because they could positively affect the location prediction of the target. Thus, the objective function was minimized over the new feature spaces. Extensive experiments were conducted on two real-world scenarios of datasets, and the predictive power of the different modeling techniques were evaluated for predicting the location of a mobile device. The results have revealed that the proposed algorithm outperforms the state-of-the-art methods for fingerprint-based indoor positioning and is found robust to changing environments. Moreover, the proposed algorithm is not only resilient to fluctuating environments but also mitigates the model's overfitting problem.

Restoration towards decomposition: A simple approach for domain generalization

Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time, which is a realistic but challenging problem. In contrast to methods assuming an identical model for different domains, Mixture of Experts (MoE) exploits multiple domain-specific networks for leveraging complementary information between domains, obtaining impressive results. However, prior MoE-based DG ReID methods suffer from a large model size with the increase of the number of source domains, and most of them overlook the exploitation of domain-invariant characteristics. To handle the two issues above, this paper presents a new approach called Mimic Embedding via adapTive Aggregation (\(\textsf{META}\)) for DG person ReID. To avoid the large model size, experts in \(\textsf{META}\) do not adopt a branch network for each source domain but share all the parameters except for the batch normalization layers. Besides multiple experts, \(\textsf{META}\) leverages Instance Normalization (IN) and introduces it into a global branch to pursue invariant features across domains. Meanwhile, \(\textsf{META}\) considers the relevance of an unseen target sample and source domains via normalization statistics and develops an aggregation module to adaptively integrate multiple experts for mimicking unseen target domain. Benefiting from a proposed consistency loss and an episodic training algorithm, \(\textsf{META}\) is expected to mimic embedding for a truly unseen target domain. Extensive experiments verify that \(\textsf{META}\) surpasses state-of-the-art DG person ReID methods by a large margin. Our code is available at https://github.com/xbq1994/META. KeywordsDomain generalizationPerson re-identification

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.

Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.

Domain-transfer learning is a machine learning task to explore a source domain data set to help the learning problem in a target domain. Usually, the source domain has sufficient labeled data, while the target domain does not. In this paper, we propose a novel domain-transfer convolutional model by mapping a target domain data sample to a proxy in the source domain and applying a source domain model to the proxy for the purpose of prediction. In our framework, we firstly represent both source and target domains to feature vectors by two convolutional neural networks and then construct a proxy for each target domain sample in the source domain space. The proxy is supposed to be matched to the corresponding target domain sample convolutional representation vector well. To measure the matching quality, we proposed to maximize their squared-loss mutual information (SMI) between the proxy and target domain samples. We further develop a novel neural SMI estimator based on a parametric density ratio estimation function. Moreover, we also propose to minimize the classification error of both source domain samples and target domain proxies. The classification responses are also smoothened by manifolds of both the source domain and proxy space. By minimizing an objective function of SMI, classification error, and manifold regularization, we learn the convolutional networks of both source and target domains. In this way, the proxy of a target domain sample can be matched to the source domain data and thus benefits from the rich supervision information of the source domain. We design an iterative algorithm to update the parameters alternately and test it over benchmark data sets of abnormal behavior detection in video, Amazon product reviews sentiment analysis, etc.

Unsupervised domain adaptation aims to transfer knowledge from labeled source domain to unlabeled target domain. Recently, multisource domain adaptation (MDA) has begun to attract attention. Its performance should go beyond simply mixing all source domains together for knowledge transfer. In this article, we propose a novel prototype-based method for MDA. Specifically, for solving the problem that the target domain has no label, we use the prototype to transfer the semantic category information from source domains to target domain. First, a feature extraction network is applied to both source and target domains to obtain the extracted features from which the domain-invariant features and domain-specific features will be disentangled. Then, based on these two kinds of features, the named inherent class prototypes and domain prototypes are estimated, respectively. Then a prototype mapping to the extracted feature space is learned in the feature reconstruction process. Thus, the class prototypes for all source and target domains can be constructed in the extracted feature space based on the previous domain prototypes and inherent class prototypes. By forcing the extracted features are close to the corresponding class prototypes for all domains, the feature extraction network is progressively adjusted. In the end, the inherent class prototypes are used as a classifier in the target domain. Our contribution is that through the inherent class prototypes and domain prototypes, the semantic category information from source domains is transformed into the target domain by constructing the corresponding class prototypes. In our method, all source and target domains are aligned twice at the feature level for better domain-invariant features and more closer features to the class prototypes, respectively. Several experiments on public data sets also prove the effectiveness of our method.

Domain generalization (DG) aims at generalizing a classifier trained on multiple source domains to an unseen target domain with domain shift. A common pervasive theme in existing DG literature is domain-invariant representation learning with various invariance assumptions. However, prior works restrict themselves to an impractical assumption for real-world challenges: If a mapping induced by a deep neural network (DNN) could align the source domains well, then such a mapping aligns a target domain as well. In this paper, we simply take DNNs as feature extractors to relax the requirement of distribution alignment. Specifically, we put forward a novel angular invariance and the accompanied norm shift assumption. Based on the proposed term of invariance, we propose a novel deep DG method dubbed Angular Invariance Domain Generalization Network (AIDGN). The optimization objective of AIDGN is developed with a von-Mises Fisher (vMF) mixture model. Extensive experiments on multiple DG benchmark datasets validate the effectiveness of the proposed AIDGN method.

With various face presentation attacks emerging continually, face anti-spoofing (FAS) approaches based on domain generalization (DG) have drawn growing attention. Existing DG-based FAS approaches always capture the domain-invariant features for generalizing on the various unseen domains. However, they neglect individual source domains' discriminative characteristics and diverse domain-specific information of the unseen domains, and the trained model is not sufficient to be adapted to various unseen domains. To address this issue, we propose an Adaptive Mixture of Experts Learning (AMEL) framework, which exploits the domain-specific information to adaptively establish the link among the seen source domains and unseen target domains to further improve the generalization. Concretely, Domain-Specific Experts (DSE) are designed to investigate discriminative and unique domain-specific features as a complement to common domain-invariant features. Moreover, Dynamic Expert Aggregation (DEA) is proposed to adaptively aggregate the complementary information of each source expert based on the domain relevance to the unseen target domain. And combined with meta-learning, these modules work collaboratively to adaptively aggregate meaningful domain-specific information for the various unseen target domains. Extensive experiments and visualizations demonstrate the effectiveness of our method against the state-of-the-art competitors.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

Deep Joint Semantic Adaptation Network for Multi-source Unsupervised Domain Adaptation

The overfitting of domain signals results in poor domain generalization of face anti-spoofing. The current methods usually improve the diversity of source domains to alleviate this overfitting. However, this benefit is minimal, as even the most diverse domain signals will also be absent in the target domain. In this work, we propose a Domain-Guided Prompt Distribution Learning (DGPDL) built on Vision-Language Models like CLIP, which explores a unified representation of domain signals as a prompt across the source and target domain to alleviate the understanding bias caused by domain gaps. Specifically, we first define a learnable Domain-Specific Distribution (DSD) that covers as many domain elements as possible, such as image quality, color tone, camera settings, etc., which establish connections between different domains and linearly combinable prompt in any domain; Then, based on the style statistics of the given sample, we construct its optimal Domain-Specific Prompts (DSPs) from the defined DSD through the designed Prompt Assemble Attention (PAA) with the similarity matching; Finally, the assembled DSPs will act as carrier or agent to perform on both the vision and language branches, synergistically improving the model's recognition of domain signals. By using the prompt to represent domain signals uniformly, if the model can be robust to DSPs in the source domain, it should be applicable to target domain, as they share the same DSD. By representing domain signals as prompts rather than instantiation features, DGPDL effectively reduces the reliance on specific domain appearances. This design enables the model to dynamically adapt to unseen target domains without the need for retraining. Extensive experiments show that the DGPDL is effective and outperforms the state-of-the-art methods on several cross-domain benchmarks. Our code is available at https://github.com/liuajian/DGPDL-FAS.

A multi-agent reinforcement learning framework for cross-domain sequential recommendation.