• Home
  • Search
  • Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment
  • Cite Icon11
  • https://doi.org/10.1109/icip46576.2022.9897381Copy DOI Icon

Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment

  • Oct 16, 2022
  • Argho Sarkar +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In this paper, we present a Grad-Cam aware supervised attention framework for visual question answering (VQA) tasks for post-disaster damage assessment purposes. Visual-attention in visual question-answering tasks aims to focus on relevant image regions according to questions to predict answers. However, the conventional attention mechanisms in VQA work in an unsupervised manner, learning to give importance to visual contents by minimizing only task-specific loss. This approach fails to provide appropriate visual attention where the visual contents are very complex. The content and nature of UAV images in FlooNet-VQA dataset are very complex as they depict the hazardous scenario after Hurricane Harvey from a high altitude. To tackle this, we propose a supervised attention mechanism that uses explainable features from Grad-Cam to supervise visual attention in the VQA pipeline. The mechanism we propose operates in two stages. In the first stage of learning, we derived the visual explanations through Grad-Cam by training a baseline attention-based VQA model. In the second stage, we supervise our visual content for each question by incorporating the Grad-Cam explanations from the previous phase of the training process. We have improved the model performance over the state-of-the-art VQA models by a considerable margin on FloodNet dataset.

Similar Papers
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
  • Research Article
  • Citations50

DRAU: Dual Recurrent Attention Units for Visual Question Answering

  • May 20, 2019
  • Computer Vision and Image Understanding
  • Ahmed Osman +1
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • Conference Article

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention

  • Jun 04, 2023
  • Ryo Shichida +4
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article
  • Citations63

AMAM: An Attention-based Multimodal Alignment Model for Medical Visual Question Answering

  • Aug 27, 2022
  • Knowledge-Based Systems
  • Haiwei Pan +5
  • Video Transcripts

Question Agnostic Attention for Visual Question Answering

  • Dec 29, 2020
  • Underline Science Inc.
  • Moshiur Farazi
  • Conference Article
  • Citations10

Question-Agnostic Attention for Visual Question Answering

  • Jan 10, 2021
  • Moshiur Farazi +2
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article
  • Citations9

Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space

  • Mar 01, 2020
  • IEEE Journal of Selected Topics in Signal Processing
  • Difei Gao +3
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Research Article
  • Citations55

Visual question answering model based on graph neural network and contextual attention

  • Mar 29, 2021
  • Image and Vision Computing
  • Himanshu Sharma +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.