• Home
  • Search
  • Regularizing Attention Networks for Anomaly Detection in Visual Question Answering
  • Cite Icon15
  • https://doi.org/10.1609/aaai.v35i3.16279Copy DOI Icon

Regularizing Attention Networks for Anomaly Detection in Visual Question Answering

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

For stability and reliability of real-world applications, the robustness of DNNs in unimodal tasks has been evaluated. However, few studies consider abnormal situations that a visual question answering (VQA) model might encounter at test time after deployment in the real-world. In this study, we evaluate the robustness of state-of-the-art VQA models to five different anomalies, including worst-case scenarios, the most frequent scenarios, and the current limitation of VQA models. Different from the results in unimodal tasks, the maximum confidence of answers in VQA models cannot detect anomalous inputs, and post-training of the outputs, such as outlier exposure, is ineffective for VQA models. Thus, we propose an attention-based method, which uses confidence of reasoning between input images and questions and shows much more promising results than the previous methods in unimodal tasks. In addition, we show that a maximum entropy regularization of attention networks can significantly improve the attention-based anomaly detection of the VQA models. Thanks to the simplicity, attention-based anomaly detection and the regularization are model-agnostic methods, which can be used for various cross-modal attentions in the state-of-the-art VQA models. The results imply that cross-modal attention in VQA is important to improve not only VQA accuracy, but also the robustness to various anomalies.

Similar Papers
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Conference Article

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention

  • Jun 04, 2023
  • Ryo Shichida +4
  • PDF
  • Research Article
  • Citations19

Advancing surgical VQA with scene graph knowledge

  • May 23, 2024
  • International Journal of Computer Assisted Radiology and Surgery
  • Kun Yuan +5
  • Conference Article

Automatic multi-modal processing of language and vision to assist people with visual impairments

  • Jul 10, 2022
  • Hernán Maina +1
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • PDF
  • Research Article
  • Citations5

Visual Question Answering Models for Zero-Shot Pedestrian Attribute Recognition: A Comparative Study

  • Jun 28, 2024
  • SN Computer Science
  • Modesto Castrillón-Santana +5
  • Conference Article
  • Citations90

Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects

  • Jan 01, 2019
  • Gabriel Grand +1
  • Research Article
  • Citations33

Causal Inference with Knowledge Distilling and Curriculum Learning for Unbiased VQA

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yonghua Pan +3
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Research Article
  • Citations8

Zero-Shot Cross-Lingual Knowledge Transfer in VQA via Multimodal Distillation

  • Aug 01, 2025
  • IEEE Transactions on Computational Social Systems
  • Yu Weng +6
  • Video Transcripts

Question Agnostic Attention for Visual Question Answering

  • Dec 29, 2020
  • Underline Science Inc.
  • Moshiur Farazi
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.