• Home
  • Search
  • VQAMix: Conditional Triplet Mixup for Medical Visual Question Answering
  • Cite Icon72
  • https://doi.org/10.1109/tmi.2022.3185008Copy DOI Icon

VQAMix: Conditional Triplet Mixup for Medical Visual Question Answering

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Medical visual question answering (VQA) aims to correctly answer a clinical question related to a given medical image. Nevertheless, owing to the expensive manual annotations of medical data, the lack of labeled data limits the development of medical VQA. In this paper, we propose a simple yet effective data augmentation method, VQAMix, to mitigate the data limitation problem. Specifically, VQAMix generates more labeled training samples by linearly combining a pair of VQA samples, which can be easily embedded into any visual-language model to boost performance. However, mixing two VQA samples would construct new connections between images and questions from different samples, which will cause the answers for those new fabricated image-question pairs to be missing or meaningless. To solve the missing answer problem, we first develop the Learning with Missing Labels (LML) strategy, which roughly excludes the missing answers. To alleviate the meaningless answer issue, we design the Learning with Conditional-mixed Labels (LCL) strategy, which further utilizes language-type prior to forcing the mixed pairs to have reasonable answers that belong to the same category. Experimental results on the VQA-RAD and PathVQA benchmarks show that our proposed method significantly improves the performance of the baseline by about 7% and 5% on the averaging result of two backbones, respectively. More importantly, VQAMix could improve confidence calibration and model interpretability, which is significant for medical VQA models in practical applications. All code and models are available at https://github.com/haifangong/VQAMix.

Similar Papers
  • Research Article
  • Citations8

Parallel multi-head attention and term-weighted question embedding for medical visual question answering.

  • Mar 11, 2023
  • Multimedia Tools and Applications
  • Sruthy Manmadhan +1
  • Research Article
  • Citations11

Cross-Modal self-supervised vision language pre-training with multiple objectives for medical visual question answering

  • Nov 12, 2024
  • Journal of Biomedical Informatics
  • Gang Liu +4
  • Conference Article
  • Citations2

MVQAS

  • Oct 26, 2021
  • Haoyue Bai +3
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Research Article
  • Citations63

AMAM: An Attention-based Multimodal Alignment Model for Medical Visual Question Answering

  • Aug 27, 2022
  • Knowledge-Based Systems
  • Haiwei Pan +5
  • Research Article
  • Citations146

Medical visual question answering: A survey

  • Jun 08, 2023
  • Artificial Intelligence in Medicine
  • Zhihong Lin +7
  • Research Article
  • Citations4

Hierarchical Modeling for Medical Visual Question Answering with Cross-Attention Fusion

  • Apr 24, 2025
  • Applied Sciences
  • Junkai Zhang +2
  • Research Article

Medical Visual Question Answering: A Review

  • Apr 21, 2025
  • Recent Advances in Computer Science and Communications
  • Guan Yang +4
  • Research Article
  • Citations19

Anomaly Matters: An Anomaly-Oriented Model for Medical Visual Question Answering.

  • Nov 01, 2022
  • IEEE Transactions on Medical Imaging
  • Fuze Cong +3
  • Dissertation

Fine-tuning multimodal large language models for medical visual question answering

  • Jan 01, 2025
  • Ankitaben Mungalpara
  • Research Article

A Global Visual Information Intervention Model for Medical Visual Question Answering.

  • Jun 01, 2025
  • Computers in biology and medicine
  • Peixi Peng +4
  • Research Article

Medical question answering: A comprehensive multimodal and LLM-driven survey.

  • Oct 01, 2026
  • Computer methods and programs in biomedicine
  • Eya Mhedhbi +4
  • Research Article
  • Citations23

A Bi-level representation learning model for medical visual question answering

  • Aug 28, 2022
  • Journal of Biomedical Informatics
  • Yong Li +7
  • Book Chapter
  • Citations5

Neural Networks for Detecting Irrelevant Questions During Visual Question Answering

  • Jan 01, 2020
  • Mengdi Li +2
  • Research Article
  • Citations1

Caption-augmented reasoning model with Hierarchical rank LoRA finetuing for medical visual question Answering.

  • Dec 01, 2025
  • Journal of biomedical informatics
  • Yong Li +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.