• Home
  • Search
  • Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  • Open Access IconOpen Access
  • Cite Icon51
  • https://doi.org/10.1109/iccv48922.2021.00205Copy DOI Icon

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

  • Oct 1, 2021
  • Linjie Li +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Benefiting from large-scale pre-training, we have witnessed significant performance boost on the popular Visual Question Answering (VQA) task. Despite rapid progress, it remains unclear whether these state-of-the-art (SOTA) models are robust when encountering examples in the wild. To study this, we introduce Adversarial VQA, a new large-scale VQA benchmark, collected iteratively via an adversarial human-and-model-in-the-loop procedure. Through this new benchmark, we discover several interesting findings. (i) Surprisingly, we find that during dataset collection, non-expert annotators can easily attack SOTA VQA models successfully. (ii) Both large-scale pre-trained models and adversarial training methods achieve far worse performance on the new benchmark than over standard VQA v2 dataset, revealing the fragility of these models while demonstrating the effectiveness of our adversarial dataset. (iii) When used for data augmentation, our dataset can effectively boost model performance on other robust VQA benchmarks. We hope our Adversarial VQA dataset can shed new light on robustness study in the community and serve as a valuable benchmark for future work.

Similar Papers
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Book Chapter
  • Citations49

Visual Question Answering Using Deep Learning: A Survey and Performance Analysis

  • Jan 01, 2021
  • Yash Srivastava +3
  • Conference Article
  • Citations4

Incorporating 3D Information Into Visual Question Answering

  • Sep 01, 2019
  • Yue Qiu +3
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
  • Research Article
  • Citations127

NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tianwen Qian +4
  • Conference Article
  • Citations2

A Survey on Visual Question Answering

  • Oct 01, 2021
  • Mrinal Banchhor +1
  • Research Article

Incorporating Verb Semantic Information in Visual Question Answering Through Multitask Learning Paradigm

  • Jun 01, 2020
  • International Journal of Semantic Computing
  • Mehrdad Alizadeh +1
  • Research Article
  • Citations4

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Minjun Kim +4
  • Research Article
  • Citations82

Co-Attending Free-Form Regions and Detections With Multi-Modal Multiplicative Feature Embedding for Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Pan Lu +4
  • Research Article
  • Citations19

Achieving Human Parity on Visual Question Answering

  • Apr 04, 2023
  • ACM Transactions on Information Systems
  • Ming Yan +11
  • PDF
  • Conference Article
  • Citations4

Reasoning Over History: Context Aware Visual Dialog

  • Jan 01, 2020
  • Muhammad Shah +2
  • Research Article
  • Citations10

Robust visual question answering via semantic cross modal augmentation

  • Oct 16, 2023
  • Computer Vision and Image Understanding
  • Akib Mashrur +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.