• Home
  • Search
  • Multimodal Encoder-Decoder Attention Networks for Visual Question Answering
  • Cite Icon59
  • https://doi.org/10.1109/access.2020.2975093Copy DOI Icon

Multimodal Encoder-Decoder Attention Networks for Visual Question Answering

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) is a multimodal task involving Computer Vision (CV) and Natural Language Processing (NLP), the goal is to establish a high-efficiency VQA model. Learning a fine-grained and simultaneous understanding of both the visual content of images and the textual content of questions is the heart of VQA. In this paper, a novel Multimodal Encoder-Decoder Attention Networks (MEDAN) is proposed. The MEDAN consists of Multimodal Encoder-Decoder Attention (MEDA) layers cascaded in depth, and can capture rich and reasonable question features and image features by associating keywords in question with important object regions in image. Each MEDA layer contains an Encoder module modeling the self-attention of questions, as well as a Decoder module modeling the question-guided-attention and self-attention of images. Experimental evaluation results on the benchmark VQA-v2 dataset demonstrate that MEDAN achieves state-of-the-art VQA performance. With the Adam solver, our best single model delivers 71.01% overall accuracy on the test-std set, and with the AdamW solver, we achieve an overall accuracy of 70.76% on the test-dev set. Additionally, extensive ablation studies are conducted to explore the reasons for MEDAN’s effectiveness.

Loading PDF

Similar Papers
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • Book Chapter
  • Citations3

Optimal Image Feature Ranking and Fusion for Visual Question Answering

  • Sep 09, 2020
  • Sruthy Manmadhan +1
  • Conference Article
  • Citations11

CS-VQA: Visual Question Answering with Compressively Sensed Images

  • Oct 01, 2018
  • Li-Chi Huang +5
  • PDF
  • Research Article
  • Citations24

Multi-Modal Explicit Sparse Attention Networks for Visual Question Answering

  • Nov 26, 2020
  • Sensors (Basel, Switzerland)
  • Zihan Guo +1
  • Research Article
  • Citations64

Computer Vision and Natural Language Processing

  • Dec 12, 2016
  • ACM Computing Surveys
  • Peratham Wiriyathammabhum +3
  • Research Article
  • Citations297

Visual question answering: Datasets, algorithms, and future challenges

  • Jun 13, 2017
  • Computer Vision and Image Understanding
  • Kushal Kafle +1
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Research Article
  • Citations5

Improved Blending Attention Mechanism in Visual Question Answering

  • Jan 01, 2023
  • Computer Systems Science and Engineering
  • Siyu Lu +6
  • Research Article
  • Citations55

Visual question answering model based on graph neural network and contextual attention

  • Mar 29, 2021
  • Image and Vision Computing
  • Himanshu Sharma +1
  • Research Article
  • Citations13

Local relation network with multilevel attention for visual question answering

  • Jan 20, 2020
  • Journal of Visual Communication and Image Representation
  • Bo Sun +3
  • Dissertation
  • Citations1

A Convolutional Neural Network Based Approach For Visual Question Answering

  • Apr 01, 2018
  • Lavanya Abhinaya Koduri
  • Conference Article
  • Citations1

Simple and effective visual question answering in a single modality

  • Sep 01, 2016
  • Yuetan Lin +3
  • Conference Article
  • Citations4

3DVQA: Visual Question Answering for 3D Environments

  • May 01, 2022
  • Yasaman Etesam +2
  • PDF
  • Research Article
  • Citations2

The Core of Smart Cities: Knowledge Representation and Descriptive Framework Construction in Knowledge-Based Visual Question Answering

  • Oct 14, 2022
  • Sustainability
  • Ruiping Wang +2
  • Book Chapter
  • Citations2

A VQA System for Medical Image Classification Using Transfer Learning

  • Jan 01, 2021
  • C Dhanush +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.