• Home
  • Search
  • Exploring Human-Like Attention Supervision in Visual Question Answering
  • Cite Icon91
  • https://doi.org/10.1609/aaai.v32i1.12272Copy DOI Icon

Exploring Human-Like Attention Supervision in Visual Question Answering

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Attention mechanisms have been widely applied in the Visual Question Answering (VQA) task, as they help to focus on the area-of-interest of both visual and textual information. To answer the questions correctly, the model needs to selectively target different areas of an image, which suggests that an attention-based model may benefit from an explicit attention supervision. In this work, we aim to address the problem of adding attention supervision to VQA models. Since there is a lack of human attention data, we first propose a Human Attention Network (HAN) to generate human-like attention maps, training on a recently released dataset called Human ATtention Dataset (VQA-HAT). Then, we apply the pre-trained HAN on the VQA v2.0 dataset to automatically produce the human-like attention maps for all image-question pairs. The generated human-like attention map dataset for the VQA v2.0 dataset is named as Human-Like ATtention (HLAT) dataset. Finally, we apply human-like attention supervision to an attention-based VQA model. The experiments show that adding human-like supervision yields a more accurate attention together with a better performance, showing a promising future for human-like attention supervision in VQA.

Similar Papers
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Conference Article
  • Citations11

Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment

  • Oct 16, 2022
  • Argho Sarkar +1
  • Research Article
  • Citations24

Enhancing robust VQA via contrastive and self-supervised learning

  • Nov 02, 2024
  • Pattern Recognition
  • Runlin Cao +4
  • PDF
  • Conference Article
  • Citations132

Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions?

  • Jan 01, 2016
  • Abhishek Das +4
  • PDF
  • Research Article
  • Citations442

Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

  • Oct 01, 2017
  • Computer Vision and Image Understanding
  • Abhishek Das +4
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Research Article
  • Citations9

Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space

  • Mar 01, 2020
  • IEEE Journal of Selected Topics in Signal Processing
  • Difei Gao +3
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.