• Home
  • Search
  • Knowledge distillation for BERT unsupervised domain adaptation
  • Open Access IconOpen Access
  • Cite Icon33
  • https://doi.org/10.1007/s10115-022-01736-yCopy DOI Icon

Knowledge distillation for BERT unsupervised domain adaptation

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

A pre-trained language model, BERT, has brought significant performance improvements across a range of natural language processing tasks. Since the model is trained on a large corpus of diverse topics, it shows robust performance for domain shift problems in which data distributions at training (source data) and testing (target data) differ while sharing similarities. Despite its great improvements compared to previous models, it still suffers from performance degradation due to domain shifts. To mitigate such problems, we propose a simple but effective unsupervised domain adaptation method, adversarial adaptation with distillation (AAD), which combines the adversarial discriminative domain adaptation (ADDA) framework with knowledge distillation. We evaluate our approach in the task of cross-domain sentiment classification on 30 domain pairs, advancing the state-of-the-art performance for unsupervised domain adaptation in text sentiment classification.

Similar Papers
  • Research Article
  • Citations63

Improved Techniques for Adversarial Discriminative Domain Adaptation.

  • Nov 06, 2019
  • IEEE Transactions on Image Processing
  • Aaron Chadha +1
  • Research Article
  • Citations31

Robust adversarial discriminative domain adaptation for real-world cross-domain visual recognition

  • Jan 02, 2021
  • Neurocomputing
  • Jianfei Yang +3
  • Conference Article
  • Citations5693

Adversarial Discriminative Domain Adaptation

  • Jul 01, 2017
  • Eric Tzeng +3
  • Book Chapter
  • Citations43

M-ADDA: Unsupervised Domain Adaptation with Deep Metric Learning

  • Jan 01, 2020
  • Issam H Laradji +1
  • Research Article
  • Citations26

Transfer Learning for Bearing Fault Diagnosis based on Graph Neural Network with Dilated KNN and Adversarial Discriminative Domain Adaptation

  • Mar 12, 2024
  • Measurement Science and Technology
  • Tang Tang +4
  • PDF
  • Research Article
  • Citations11

Targeted adversarial discriminative domain adaptation

  • Jul 19, 2021
  • Journal of Applied Remote Sensing
  • Hua-Mei Chen +5
  • Conference Article
  • Citations80

Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation

  • Jan 01, 2021
  • L.T Nguyen-Meidine +5
  • Conference Article
  • Citations1

On the instability of unsupervised domain adaptation with ADDA

  • May 01, 2022
  • Kazuki Omi +1
  • Conference Article
  • Citations66

Cross-lingual Text-independent Speaker Verification Using Unsupervised Adversarial Discriminative Domain Adaptation

  • May 01, 2019
  • Wei Xia +2
  • Research Article
  • Citations15

Structural damage detection based on transmissibility functions with unsupervised domain adaptation

  • Oct 17, 2024
  • Engineering Structures
  • Xutong Zhang +3
  • Research Article
  • Citations10

Deep adversarial domain adaptation for hyperspectral calibration model transfer among plant phenotyping systems

  • Nov 03, 2022
  • Biosystems Engineering
  • Tanzeel U Rehman +1
  • Conference Article

Minimizing Outputs’ Differences of Classifiers with Different Responsibilities for Unsupervised Adversarial Domain Adaptation

  • Apr 09, 2021
  • Weiran Zhang +2
  • Conference Article
  • Citations5

Cross-Cultural Music Emotion Recognition by Adversarial Discriminative Domain Adaptation

  • Dec 01, 2018
  • Yi-Wei Chen +2
  • Preprint Article

Segmentation Cycle-GAN Loss for Structure-Preserving Domain Adaptation of Medical Images

  • Sep 06, 2024
  • Paolo Iacono
  • Preprint Article

Segmentation Cycle-GAN Loss for Structure-Preserving Domain Adaptation of Medical Images

  • Sep 06, 2024
  • Paolo Iacono
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.