• Home
  • Search
  • Multi-modal Domain Adaptation for Text Visual Question Answering Tasks
  • Cite Icon1
  • https://doi.org/10.1109/dicta60407.2023.00058Copy DOI Icon

Multi-modal Domain Adaptation for Text Visual Question Answering Tasks

  • Nov 28, 2023
  • Zhiyuan Li +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Domain adaptation aims to train a model on the labeled source data and unlabeled target data while improving the performance of the same model on the target domain. Recently, multi-modal domain adaptation is a relatively more challenging task compared with single-feature ones due to diverse forms of feature modalities. This paper aims to achieve transfer learning between two Visual Question Answering tasks: Text Visual Question Answering (Text-VQA) and Scene Text Visual Question Answering (ST-VQA). Compared to typical domain adaptation methods on single-modality analysis, the critical challenge for these tasks is that the input features contain complicated information from visual objects, question words, and optical character recognition (OCR) tokens in different forms. We consider a novel domain adaptation framework to facilitate knowledge transfer under the multiple modalities. Specially, the proposed Text-VQA domain adaptation framework (TDAF) consists of three important modules: (1) A mix-feature module fuses input features from multiple modalities and reduces the feature discrepancy for the cross-domain fused features. (2) A discrepancy discriminator focuses on keeping the source prediction correctly while detecting the ambiguous target samples by maximizing the discrepancy of probability outputs between two domains. (3) An self-supervised learning module is used for reducing information loss during representation learning by computing cosine similarity between collected inputs and decoded outputs. Through three joint modules, we successfully improve the performance of domain adaptation from Text-VQA to ST-VQA by raising 19.6% of accuracy and 18.2% of Average Normalized Levenshtein Similarity (ANLS). In addition, extra experiments were carried out to demonstrate the effectiveness of our proposed method compared with other SOTA approaches.

Similar Papers
  • Dissertation
  • Citations1

Domain Adaptation and Domain Generalization with Representation Learning

  • Jan 01, 2016
  • Muhammad Ghifary
  • Dissertation
  • Citations2

Domain Adaptation and Domain Generalization with Representation Learning

  • Jan 01, 2016
  • Muhammad Ghifary
  • Research Article

Trust-Aware Domain Adaptation Using Physics-Guided Reliability Learning for Cross-Condition Fault Diagnosis of Milling Machines

  • Jul 14, 2026
  • Sensors (Basel, Switzerland)
  • Saif Ullah +2
  • Research Article
  • Citations29

Beyond OCR + VQA: Towards end-to-end reading and reasoning for robust and accurate textvqa

  • Jan 21, 2023
  • Pattern Recognition
  • Gangyan Zeng +7
  • Conference Article
  • Citations22

Continual Unsupervised Domain Adaptation for Semantic Segmentation by Online Frequency Domain Style Transfer

  • Sep 19, 2021
  • Jan-Aike Termohlen +4
  • Research Article
  • Citations29

Supervised domain adaptation of decision forests: Transfer of models trained in vitro for in vivo intravascular ultrasound tissue characterization.

  • Mar 17, 2016
  • Medical Image Analysis
  • Sailesh Conjeti +7
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Conference Article
  • Citations7

Adversarial domain adaptation for multi-device retinal OCT segmentation

  • Mar 10, 2020
  • Yufan He +5
  • PDF
  • Conference Article
  • Citations23

Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation

  • Jan 01, 2020
  • Yiming Xu +4
  • Research Article
  • Citations12

Unsupervised Domain Adaptation via Discriminative Classes-Center Feature Learning in Adversarial Network

  • May 21, 2020
  • Neural Processing Letters
  • Wendong Chen +1
  • Research Article
  • Citations154

Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation.

  • Jan 01, 2019
  • Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
  • Junlin Yang +5
  • Research Article
  • Citations15

MobiVQA

  • Jul 04, 2022
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
  • Qingqing Cao +3
  • Conference Article
  • Citations10

Partially Zero-shot Domain Adaptation from Incomplete Target Data with Missing Classes

  • Mar 01, 2020
  • Masato Ishii +2
  • Research Article

Heterogeneous domain adaptation survival analysis with partially observed outcomes via dictionary learning and distribution alignment.

  • Aug 01, 2026
  • Journal of biomedical informatics
  • Dan Ni Lin +2
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.