• Home
  • Search
  • VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models
  • Cite Icon9
  • https://doi.org/10.1609/aaai.v38i7.28499Copy DOI Icon

VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) is a fundamental task in computer vision and natural language process fields. Although the “pre-training & finetuning” learning paradigm significantly improves the VQA performance, the adversarial robustness of such a learning paradigm has not been explored. In this paper, we delve into a new problem: using a pre-trained multimodal source model to create adversarial image-text pairs and then transferring them to attack the target VQA models. Correspondingly, we propose a novel VQATTACK model, which can iteratively generate both im- age and text perturbations with the designed modules: the large language model (LLM)-enhanced image attack and the cross-modal joint attack module. At each iteration, the LLM-enhanced image attack module first optimizes the latent representation-based loss to generate feature-level image perturbations. Then it incorporates an LLM to further enhance the image perturbations by optimizing the designed masked answer anti-recovery loss. The cross-modal joint attack module will be triggered at a specific iteration, which updates the image and text perturbations sequentially. Notably, the text perturbation updates are based on both the learned gradients in the word embedding space and word synonym-based substitution. Experimental results on two VQA datasets with five validated models demonstrate the effectiveness of the proposed VQATTACK in the transferable attack setting, compared with state-of-the-art baselines. This work reveals a significant blind spot in the “pre-training & fine-tuning” paradigm on VQA tasks. The source code can be found in the link https://github.com/ericyinyzy/VQAttack.

Similar Papers
  • Conference Article

Grounded Multi-modal Conversation for Zero-shot Visual Question Answering

  • Oct 05, 2025
  • Mohammad Reza Zarei +2
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Book Chapter

Multi-view Visual Question Answering Dataset for Real Environment Applications

  • Jan 01, 2020
  • Yue Qiu +3
  • Conference Article
  • Citations4

3DVQA: Visual Question Answering for 3D Environments

  • May 01, 2022
  • Yasaman Etesam +2
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • Research Article
  • Citations2

Unveiling the power of multimodal large language models for radio astronomical image understanding and question answering

  • Oct 10, 2025
  • Machine Learning: Science and Technology
  • Fuyong Zhao +7
  • Research Article

Analysis of Deep Learning Models on VQA(Toloka)Dataset

  • Oct 06, 2025
  • The Asian Bulletin of Big Data Management
  • Fariha Shoukat +3
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article
  • Citations297

Visual question answering: Datasets, algorithms, and future challenges

  • Jun 13, 2017
  • Computer Vision and Image Understanding
  • Kushal Kafle +1
  • Research Article
  • Citations53

Visual Question Answering: A Tutorial

  • Nov 01, 2017
  • IEEE Signal Processing Magazine
  • Damien Teney +2
  • Book Chapter
  • Citations3

Optimal Image Feature Ranking and Fusion for Visual Question Answering

  • Sep 09, 2020
  • Sruthy Manmadhan +1
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Research Article
  • Citations5

Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration

  • Jan 29, 2025
  • Remote Sensing
  • Fan Liu +5
  • Conference Article
  • Citations48

Coarse-to-Fine Reasoning for Visual Question Answering

  • Jun 01, 2022
  • Binh X Nguyen +5
  • Research Article
  • Citations2

VADS: Visuo-Adaptive DualStrike attack on visual question answer

  • Aug 31, 2024
  • Computer Vision and Image Understanding
  • Boyuan Zhang +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.