• Home
  • Search
  • Incorporating Pre-Training Data Matters in Unsupervised Domain Adaptation.
  • Cite Icon2
  • https://doi.org/10.1109/tpami.2025.3572963Copy DOI Icon

Incorporating Pre-Training Data Matters in Unsupervised Domain Adaptation.

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In deep learning, initializing models with pre-trained weights has become the de facto practice for various downstream tasks. Many unsupervised domain adaptation (UDA) methods typically adopt a backbone pre-trained on ImageNet, and focus on reducing the source-target domain discrepancy. However, the impact of pre-training on adaptation received little attention. In this study, we delve into UDA from the novel perspective of pre-training. We first demonstrate the impact of pre-training by analyzing the dynamic distribution discrepancies between pre-training data domain and the source/ target domain during adaptation. Then, we reveal that the target error also stems from the pre-training in the following two factors: 1) empirically, target error arises from the gradually degenerative pre-trained knowledge during adaptation; 2) theoretically, the error bound depends on difference between the gradient of loss function, i.e., on the target domain and pre-training data domain. To address these two issues, we redefine UDA as a three-domain problem, i.e., source domain, target domain, and pre-training data domain; then we propose a novel framework, named TriDA. We maintain the pre-trained knowledge and improve the error bound by incorporating pre-training data into adaptation for both vanilla UDA and source-free UDA scenarios. For efficiency, we introduce a selection strategy for pre-training data, and offer a solution with synthesized images when pre-training data is unavailable during adaptation. Notably, TriDA is effective even with a small amount of pre-training or synthesized images, and seamlessly complements the two scenario UDA methods, demonstrating state-of-the-art performance across multiple benchmarks. We hope our work provides new insights for better understanding and application of domain adaptation.

Similar Papers
  • Research Article
  • Citations121

Divergence-Agnostic Unsupervised Domain Adaptation by Adversarial Attacks.

  • Nov 01, 2022
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Jingjing Li +5
  • Book Chapter
  • Citations8

Domain Adaptation for Eye Segmentation

  • Jan 01, 2020
  • Yiru Shen +2
  • Research Article

Systematic Analysis of Source-Free Domain Adaptation Methods

  • Mar 27, 2025
  • Frontiers in Computing and Intelligent Systems
  • Zhiyi Miao
  • Abstract

Unsupervised Domain Adaptation of Auto-Segmentation on Multi-Source MRIs

  • Sep 29, 2023
  • International Journal of Radiation Oncology*Biology*Physics
  • W Zhang +4
  • Research Article
  • Citations16

Overcoming learning bias via Prototypical Feature Compensation for source-free domain adaptation

  • Sep 17, 2024
  • Pattern Recognition
  • Zicheng Pan +3
  • Research Article
  • Citations6

Cross-Domain Urban Land Use Classification via Scenewise Unsupervised Multisource Domain Adaptation With Transformer

  • Jan 01, 2024
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Mengmeng Li +3
  • PDF
  • Supplementary Content
  • Citations68

Deep Unsupervised Domain Adaptation with Time Series Sensor Data: A Survey

  • Jul 23, 2022
  • Sensors (Basel, Switzerland)
  • Yongjie Shi +2
  • Research Article
  • Citations47

Adapt Everywhere: Unsupervised Adaptation of Point-Clouds and Entropy Minimization for Multi-Modal Cardiac Image Segmentation.

  • Jul 01, 2021
  • IEEE transactions on medical imaging
  • Sulaiman Vesal +4
  • PDF
  • Research Article
  • Citations58

The Eyes of the Gods: A Survey of Unsupervised Domain Adaptation Methods Based on Remote Sensing Data

  • Sep 03, 2022
  • Remote Sensing
  • Mengqiu Xu +4
  • Research Article
  • Citations148

Unsupervised Domain Adaptation With Label and Structural Consistency.

  • Sep 15, 2016
  • IEEE Transactions on Image Processing
  • Cheng-An Hou +3
  • Research Article
  • Citations31

PCLUDA: A Pseudo-Label Consistency Learning- Based Unsupervised Domain Adaptation Method for Cross-Domain Optical Remote Sensing Image Retrieval

  • Jan 01, 2023
  • IEEE Transactions on Geoscience and Remote Sensing
  • Dongyang Hou +3
  • PDF
  • Research Article
  • Citations2

Multi-Source Domain Adaptation Using Ambient Sensor Data

  • Nov 19, 2024
  • Applied Artificial Intelligence
  • Jawher Dridi +2
  • Research Article
  • Citations2

Source-Free Domain Adaptation for Question Answering with Masked Self-training

  • Jun 04, 2024
  • Transactions of the Association for Computational Linguistics
  • Maxwell J Yin +3
  • Conference Article
  • Citations3

Modeling Sequential Domain Shift through Estimation of Optimal Sub-spaces for Categorization

  • Jan 01, 2014
  • Suranjana Samanta +2
  • Research Article
  • Citations25

ACT: Semi-supervised Domain-adaptive Medical Image Segmentation with Asymmetric Co-Training.

  • Jan 01, 2022
  • Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
  • Xiaofeng Liu +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.