• Home
  • Search
  • Visual Question Answering as Reading Comprehension
  • Open Access IconOpen Access
  • Cite Icon59
  • https://doi.org/10.1109/cvpr.2019.00648Copy DOI Icon

Visual Question Answering as Reading Comprehension

  • Jun 1, 2019
  • Hui Li +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual question answering (VQA) demands simultaneous comprehension of both the image visual content and natural language questions. In some cases, the reasoning needs the help of common sense or general knowledge which usually appear in the form of text. Current methods jointly embed both the visual information and the textual feature into the same space. However, how to model the complex interactions between the two different modalities is not an easy task. In contrast to struggling on multimodal feature fusion, in this paper, we propose to unify all the input information by natural language so as to convert VQA into a machine reading comprehension problem. With this transformation, our method not only can tackle VQA datasets that focus on observation based questions, but can also be naturally extended to handle knowledge-based VQA which requires to explore large-scale external knowledge base. It is a step towards being able to exploit large volumes of text and natural language processing techniques to address VQA problem. Two types of models are proposed to deal with open-ended VQA and multiple-choice VQA respectively. We evaluate our models on three VQA benchmarks. The comparable performance with the state-of-the-art demonstrates the effectiveness of the proposed method.

Similar Papers
  • Research Article
  • Citations71

Multimodal feature fusion by relational reasoning and attention for visual question answering

  • Aug 19, 2019
  • Information Fusion
  • Weifeng Zhang +4
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Conference Article
  • Citations2

Estimating Viewed Images with Natural Language Question Answering from fMRI Data

  • Mar 01, 2020
  • Saya Takada +3
  • PDF
  • Conference Article
  • Citations150

ConceptBert: Concept-Aware Representation for Visual Question Answering

  • Jan 01, 2020
  • François Gardères +3
  • Research Article
  • Citations624

VQA: Visual Question Answering

  • Nov 08, 2016
  • International Journal of Computer Vision
  • Aishwarya Agrawal +6
  • Conference Article

A Crowdsourcing Tool for Data Augmentation in Visual Question Answering Tasks

  • Oct 16, 2018
  • Ramon Silva +4
  • Research Article
  • Citations1

An Enhanced Term Weighted Question Embedding for Visual Question Answering

  • May 05, 2022
  • Journal of Information & Knowledge Management
  • Sruthy Manmadhan +1
  • Book Chapter
  • Citations49

Visual Question Answering Using Deep Learning: A Survey and Performance Analysis

  • Jan 01, 2021
  • Yash Srivastava +3
  • Book Chapter
  • Citations16

Learning Convolutional Text Representations for Visual Question Answering

  • May 07, 2018
  • Zhengyang Wang +1
  • Research Article
  • Citations617

Beyond Bilinear: Generalized Multimodal Factorized High-Order Pooling for Visual Question Answering.

  • Apr 09, 2018
  • IEEE Transactions on Neural Networks and Learning Systems
  • Zhou Yu +4
  • Conference Article
  • Citations4921

VQA: Visual Question Answering

  • Dec 01, 2015
  • Stanislaw Antol +6
  • PDF
  • Conference Article
  • Citations27

Improving Visual Question Answering by Referring to Generated Paragraph Captions

  • Jan 01, 2019
  • Hyounghun Kim +1
  • Conference Article
  • Citations84

Boosting Visual Question Answering with Context-aware Knowledge Aggregation

  • Oct 12, 2020
  • Guohao Li +2
  • Conference Article
  • Citations2

A Survey on Visual Question Answering

  • Oct 01, 2021
  • Mrinal Banchhor +1
  • Research Article
  • Citations10

Visual Question Answering

  • Jul 13, 2022
  • International Journal of Advanced Research in Science, Communication and Technology
  • Dr Sai Madhavi D +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.