Unsupervised Domain Adaptation with Implicit Pseudo Supervision for Semantic Segmentation

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.

Video domain adaptation for semantic segmentation using perceptual consistency matching

The adversarial domain adaptation has made many achievements in the field of Unsupervised Domain Adaptation (UDA). However, initial adversarial domain adaptation methods do not consider the category-level domain adaptation, causing confusion between different categories in the different domains. This paper propose a UDA approach called Minimizing Outputs' Differences of Classifiers with Different Responsibilities (MODCDR) to improve this situation. Its framework consists of one generator and two different task-specific classifiers, one working on the source domain, while the other working on the target domain. By training the feature generator to minimize the outputs' discrepancy of these two classifiers, this method not only generates discriminative features, but also achieves the category-level feature distribution alignment simultaneously. In addition, these two classifiers can also produce and filter the reliable pseudo-labels on the target domain, being used for assisting our training. This paper tested this method on three publicly available transfer learning object recognition datasets, with comparison to recent unsupervised adversarial domain adaptation approaches. The results show that the proposed method has superior performance.

Unsupervised Domain Adaptation (UDA) has attracted a surge of interest over the past decade but is difficult to be used in real-world applications. Considering the privacy-preservation issues and security concerns, in this work, we study a practical problem of Source-Free Domain Adaptation (SFDA), which eliminates the reliance on annotated source data. Current SFDA methods focus on extracting domain knowledge from the source-trained model but neglects the intrinsic structure of the target domain. Moreover, they typically utilize pseudo labels for self-training in the target domain, but suffer from the notorious error accumulation problem. To address these issues, we propose a new SFDA framework, called Region-to-Pixel Adaptation Network (RPANet), which learns the region-level and pixel-level discriminative representations through coarse-to-fine self-supervision. The proposed RPANet consists of two modules, Foreground-aware Contrastive Learning (FCL) and Confidence-Calibrated Pseudo-Labeling (CCPL), which explicitly address the key challenges of “how to distinguish” and “how to refine”. To be specific, FCL introduces a supervised contrastive learning paradigm in the region level to contrast different region centroids across different target images, which efficiently involves all pseudo labels while robust to noisy samples. CCPL designs a novel fusion strategy to reduce the overconfidence problem of pseudo labels by fusing two different target predictions without introducing any additional network modules. Extensive experiments on three cross-domain polyp segmentation tasks reveal that RPANet significantly outperforms state-of-the-art SFDA and UDA methods without access to source data, revealing the potential of SFDA in medical applications.

In this work, we address the task of unsupervised domain adaptation (UDA) for\nsemantic segmentation in presence of multiple target domains: The objective is\nto train a single model that can handle all these domains at test time. Such a\nmulti-target adaptation is crucial for a variety of scenarios that real-world\nautonomous systems must handle. It is a challenging setup since one faces not\nonly the domain gap between the labeled source set and the unlabeled target\nset, but also the distribution shifts existing within the latter among the\ndifferent target domains. To this end, we introduce two adversarial frameworks:\n(i) multi-discriminator, which explicitly aligns each target domain to its\ncounterparts, and (ii) multi-target knowledge transfer, which learns a\ntarget-agnostic model thanks to a multi-teacher/single-student distillation\nmechanism.The evaluation is done on four newly-proposed multi-target benchmarks\nfor UDA in semantic segmentation. In all tested scenarios, our approaches\nconsistently outperform baselines, setting competitive standards for the novel\ntask.\n

Domain consistency regularization for unsupervised multi-source domain adaptive classification

Superpixel-guided class-level denoising for unsupervised domain adaptive fundus image segmentation without source data

Domain adaptation (DA) is a representation learning methodology that transfers knowledge from a label-sufficient source domain to a label-scarce target domain. While most of early methods are focused on unsupervised DA (UDA), several studies on semi-supervised DA (SSDA) are recently suggested. In SSDA, a small number of labeled target images are given for training, and the effectiveness of those data is demonstrated by the previous studies. However, the previous SSDA approaches solely adopt those data for embedding ordinary supervised losses, overlooking the potential usefulness of the few yet informative clues. Based on this observation, in this paper, we propose a novel method that further exploits the labeled target images for SSDA. Specifically, we utilize labeled target images to selectively generate pseudo labels for unlabeled target images. In addition, based on the observation that pseudo labels are inevitably noisy, we apply a label noise-robust learning scheme, which progressively updates the network and the set of pseudo labels by turns. Extensive experimental results show that our proposed method outperforms other previous state-of-the-art SSDA methods.

Domain Adaptation (DA) is important for a segmentation model to deal with domain shift in a new target domain. Due to the privacy concern of medical data and the expensive annotation process, Source-Free Domain Adaptation (SFDA) is appealing without access to source data and labels of target domain images for the adaptation. However, existing SFDA methods have limited performance due to insufficient supervision and unreliable pseudo labels. In this paper, we propose an enhanced Iterative Pseudo Label Correction (IPLC+) SFDA framework guided by Segment Anything Model (SAM) for medical image segmentation. Specifically, with a pre-trained source model and SAM, we propose a Reliable SAM Pseudo-label Generator (RSPG) to obtain high-quality and reliable pseudo labels in the target domain based on multiple prompts randomly sampled from the model's prediction. To provide more efficient constraints during adaptation, we introduce self-training pseudo labels weighted by the uncertainty, and propose regularization using mean curvature minimization based on shape-prior knowledge for smoother segmentation. We also propose an Iterative Correction Learning (ICL) strategy to iteratively refine pseudo labels using SAM with updated prompts and combine supervisions to optimize the model sufficiently. Experiments on two public multi-site datasets for prostate and heart segmentation show that our method effectively outperformed ten state-of-the-art SFDA methods, improved the quality of pseudo labels, and even achieved better results than fully supervised learning in the target domain in some cases.

Deep models trained on large-scale RGB image datasets have shown tremendous success. It is important to apply such deep models to real-world problems. However, these models suffer from a performance bottleneck under illumination changes. Thermal IR cameras are more robust against such changes, and thus can be very useful for the real-world problems. In order to investigate efficacy of combining feature-rich visible spectrum and thermal image modalities, we propose an unsupervised domain adaptation method which does not require RGB-to-thermal image pairs. We employ large-scale RGB dataset MS-COCO as source domain and thermal dataset FLIR ADAS as target domain to demonstrate results of our method. Although adversarial domain adaptation methods aim to align the distributions of source and target domains, simply aligning the distributions cannot guarantee perfect generalization to the target domain. To this end, we propose a self-training guided adversarial domain adaptation method to promote generalization capabilities of adversarial domain adaptation methods. To perform self-training, pseudo labels are assigned to the samples on the target thermal domain to learn more generalized representations for the target domain. Extensive experimental analyses show that our proposed method achieves better results than the state-of-the-art adversarial domain adaptation methods. The code and models are publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>

This work explores the usage of Fourier Transform for reducing the domain gap between the Source (e.g. Synthetic Image) and Target domain (e.g. Real Image) towards solving the Domain Adaptation problem. Most of the Unsupervised Domain Adaptation (UDA) algorithms reduce the global domain shift between labelled Source and unlabelled Target domain by matching the marginal distribution. UDA performance deteriorates for the cases where the domain gap between Source and Target is significant. To improve the overall performance of the existing UDA algorithms the proposed method attempts to bring the Source domain closer to the Target domain with the help of pseudo label based class consistent low-frequency swapping. This traditional image processing technique results in computational efficiency, especially compared to the state-of-the-art deep learning methods that use complex adversarial training. The proposed method Class Aware Frequency Transformation (CAFT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) can easily be plugged into any existing UDA algorithm to improve its performance. We evaluate CAFT on various domain adaptation datasets and algorithms and have achieved performance gains across all the popular benchmarks.

Easy Domain Adaptation for cross-subject multi-view emotion recognition

Due to the unavailability of source domain data encountered in unsupervised domain adaptation, there has been an increasing number of studies on source-free domain adaptation (SFDA) in recent years. To better solve the SFDA problem and effectively leverage the multi-modal information in medical images, this paper presents a novel SFDA method for multi-modal stroke lesion segmentation in which evidential deep learning instead of convolutional neural network. Specifically, for multi-modal stroke images, we design a multi-modal opinion fusion module which uses Dempster-Shafer evidence theory for decision fusion of different modalities. Besides, for the SFDA problem, we use the pseudo label learning method, which obtains pseudo labels from the pre-trained source model to perform the adaptation process. To solve the unreliability of pseudo label caused by domain shift, we propose a pseudo label filtering scheme using shadowed sets theory and a pseudo label refining scheme using evidential uncertainty. These two schemes can automatically extract unreliable parts in pseudo labels and jointly improve the quality of pseudo labels with low computational costs. Experiments on two multi-modal stroke lesion datasets demonstrate the superiority of our method over other state-of-the-art SFDA methods.

Bionic limb control through myoelectric pattern recognition, offering intuitive decoding of motor intent, can improve the quality of life for individuals with amputations. However, most work on pattern recognition only uses a small subset of the myoelectric data generated during daily life, to train an Artificial Neural Network (ANN) via Supervised Learning (SL). Scenarios substantially different from the recording session e.g. different limb positions, can lead to misclassifications by the ANN during everyday usage of the bionic limb. Recording labeled data from all scenarios encountered in daily life could alleviate the problem, but would be prohibitively time consuming. Unsupervised Domain Adaptation (UDA) offers a solution by leveraging unlabeled data from a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">target domain</i>, not represented in the labeled dataset i.e. the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">source domain</i>, to calibrate ANNs for improved performance. In this study we explore the potential of two UDA algorithms for domain shifts in myoelectric pattern recognition: Domain Adversarial Neural Networks (DANN) and Sliced Wasserstein Discrepancy (SWD). Offline evaluation identified SWD as the best-performing algorithm, which was subsequently validated in online experiments with 11 participants. Using UDA improved the performance on the target domain by 19% compared to an ANN trained through SL on data from the source domain only. Indeed, it nearly matched the performance of an ANN trained through SL on labeled data from both the source and target domain. This is the first time UDA was validated in online experiments as a viable approach to overcome domain shift problems caused by changes in limb position.