• Home
  • Search
  • Transformer-based Sparse Encoder and Answer Decoder for Visual Question Answering
  • Cite Icon4
  • https://doi.org/10.1109/icsp56322.2022.9965298Copy DOI Icon

Transformer-based Sparse Encoder and Answer Decoder for Visual Question Answering

  • Oct 21, 2022
  • Longkun Peng +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) is all about understanding images and questions. Existing Transformer-based methods achieve excellent performance by associating questions with image region objects and directly using a special classification token for answer prediction. However, answering a question only needs to focus on some specific keywords and image regions, while excessively computing the attention of questions and image region objects will introduce unnecessary noise. Meanwhile, the information from these two modalities cannot be fully utilized when directly using the classification token to predict the answer. To this end, we propose a Transformer-based Sparse Encoder and Answer Decoder (SEAD) model for visual question answering, in which a two-stream sparse Transformer module based on co-attention is built to enhance the most relevant visual features and textual descriptions inter-modality. Furthermore, a single-step answer decoder is proposed to fully exploit the information of both modalities in the answer prediction stage, and a strategy is designed that fully utilizes the ground truth to correct the visual relevance scores in the decoder to focus on salient objects in the image. Our model performs magnificently, as shown by experiment results on the VQA v2.0 benchmark dataset.

Similar Papers
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Conference Article

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention

  • Jun 04, 2023
  • Ryo Shichida +4
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Research Article

Debiasing Visual Question Answering via Ensemble Gradient Detection and Iterative Attention Forgetting

  • Dec 18, 2025
  • Informatica
  • Qiuying Han +4
  • Conference Article
  • Citations13

Focal and Composed Vision-semantic Modeling for Visual Question Answering

  • Oct 17, 2021
  • Yudong Han +5
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • PDF
  • Research Article
  • Citations19

Advancing surgical VQA with scene graph knowledge

  • May 23, 2024
  • International Journal of Computer Assisted Radiology and Surgery
  • Kun Yuan +5
  • Conference Article
  • Citations11

Rankvqa: Answer Re-Ranking For Visual Question Answering

  • Jul 01, 2020
  • Yanyuan Qiao +2
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.