• Home
  • Search
  • HCCL: Hierarchical Counterfactual Contrastive Learning for Robust Visual Question Answering
  • Cite Icon3
  • https://doi.org/10.1145/3673902Copy DOI Icon

HCCL: Hierarchical Counterfactual Contrastive Learning for Robust Visual Question Answering

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Despite most state-of-the-art models having achieved amazing performance in Visual Question Answering (VQA) , they usually utilize biases to answer the question. Recently, some studies synthesize counterfactual training samples to help the model to mitigate the biases. However, these synthetic samples need extra annotations and often contain noises. Moreover, these methods simply add synthetic samples to the training data to train the model with the cross-entropy loss, which cannot make the best use of synthetic samples to mitigate the biases. In this article, to mitigate the biases in VQA more effectively, we propose a Hierarchical Counterfactual Contrastive Learning (HCCL) method. Firstly, to avoid introducing noises and extra annotations, our method automatically masks the unimportant features in original pairs to obtain positive samples and create mismatched question-image pairs as negative samples. Then our method uses feature-level and answer-level contrastive learning to make the original sample close to positive samples in the feature space, while away from negative samples in both feature and answer spaces. In this way, the VQA model can learn the robust multimodal features and focus on both visual and language information to produce the answer. Our HCCL method can be adopted in different baselines, and the experimental results on VQA v2, VQA-CP, and GQA-OOD datasets show that our method is effective in mitigating the biases in VQA, which improves the robustness of the VQA model.

Similar Papers
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Research Article
  • Citations24

Enhancing robust VQA via contrastive and self-supervised learning

  • Nov 02, 2024
  • Pattern Recognition
  • Runlin Cao +4
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
  • Research Article
  • Citations16

Fair Attention Network for Robust Visual Question Answering

  • Sep 01, 2024
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Yandong Bi +4
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article
  • Citations26

See and Learn More: Dense Caption-Aware Representation for Visual Question Answering

  • Feb 01, 2024
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Yandong Bi +4
  • Research Article
  • Citations4

Self-supervised knowledge distillation in counterfactual learning for VQA

  • Nov 25, 2023
  • Pattern Recognition Letters
  • Yandong Bi +4
  • Research Article
  • Citations17

Test-Time Model Adaptation for Visual Question Answering With Debiased Self-Supervisions

  • Jan 01, 2024
  • IEEE Transactions on Multimedia
  • Zhiquan Wen +5
  • Research Article

Debiasing Visual Question Answering via Ensemble Gradient Detection and Iterative Attention Forgetting

  • Dec 18, 2025
  • Informatica
  • Qiuying Han +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.