• Home
  • Search
  • A Convolutional Neural Network Based Approach For Visual Question Answering
  • Cite Icon1
  • https://doi.org/10.31979/etd.g2w5-abudCopy DOI Icon

A Convolutional Neural Network Based Approach For Visual Question Answering

  • Apr 1, 2018
  • Lavanya Abhinaya Koduri
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Computer Vision is a scientific discipline which involves the development of an algorithmic basis for the construction of intelligent systems that aim at analysis, understanding and extraction of useful information from visual data. This visual data can be plain images, video sequences, views from multiple cameras, etc. Natural Language Processing (NLP), is the ability of machines to read and understand human languages. Visual Question Answering (VQA), is a multi-discipline Artificial Intelligence (AI) research problem, which is a combination of Natural Language Processing (NLP), Computer Vision (CV), and Knowledge Reasoning (KR). Given an image and a question related to the image in natural language, the algorithm has to output an accurate natural language answer. Since the questions are open-ended, the system requires a very detailed understanding of the image, its context and a broad set of AI capabilities – object detection, activity recognition and knowledge-based reasoning. Since the release of the VQA dataset in 2014, numerous datasets and algorithms for VQA have been put forward. In this work, we propose a new baseline for the problem of visual question answering. Our model uses a deep residual network (ResNet) to compute the image features and ByteNet to compute question embeddings. A soft attention mechanism is used to focus on most relevant image features and a classifier is used to generate probabilities over an answer set. We implemented the solution in TensorFlow, which is an open source deep-learning platform, developed by Google. iv Prior to using deep residual network (ResNet) and ByteNet, we tried using VGG16 for extracting image features and long short-term memory units (LSTM) for extracting question features. We observed that using ResNet and ByteNet resulted in an improved accuracy when compared to using VGG16 and LSTM. We evaluate our model on three major image question answering datasets: DAQUAR-ALL, COCO-QA and The VQA Dataset. Our model, despite having a relatively simple architecture, achieves 64.6% accuracy on VQA 1.0 dataset and 59.7% accuracy on VQA 2.0 dataset.

Similar Papers
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • Research Article

Overcoming the Limitations of Learning-Based VQA for Counting Questions with Zero-Shot Learning

  • Aug 20, 2024
  • International Journal on Artificial Intelligence Tools
  • A Lubna +1
  • Research Article
  • Citations297

Visual question answering: Datasets, algorithms, and future challenges

  • Jun 13, 2017
  • Computer Vision and Image Understanding
  • Kushal Kafle +1
  • Conference Article
  • Citations2

A Survey on Visual Question Answering

  • Oct 01, 2021
  • Mrinal Banchhor +1
  • PDF
  • Research Article
  • Citations24

Multi-Modal Explicit Sparse Attention Networks for Visual Question Answering

  • Nov 26, 2020
  • Sensors (Basel, Switzerland)
  • Zihan Guo +1
  • Conference Article
  • Citations4

3DVQA: Visual Question Answering for 3D Environments

  • May 01, 2022
  • Yasaman Etesam +2
  • Conference Article
  • Citations3

NLP Meets Vision for Visual Interpretation - A Retrospective Insight and Future directions

  • May 20, 2021
  • Ahmed Jamshed +1
  • Conference Article
  • Citations2

Beyond Words: Exploring Co-Attention with BERT in Visual Question Answering

  • May 24, 2024
  • Vibhashree B S +4
  • Book Chapter

Multi-view Visual Question Answering Dataset for Real Environment Applications

  • Jan 01, 2020
  • Yue Qiu +3
  • Research Article
  • Citations17

Inner Knowledge-based Img2Doc Scheme for Visual Question Answering

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Qun Li +4
  • Research Article

Analysis of Deep Learning Models on VQA(Toloka)Dataset

  • Oct 06, 2025
  • The Asian Bulletin of Big Data Management
  • Fariha Shoukat +3
  • Conference Article
  • Citations3

Object-based classification for Visual Question Answering

  • Feb 01, 2020
  • Akshra Gupta +4
  • PDF
  • Conference Article
  • Citations150

ConceptBert: Concept-Aware Representation for Visual Question Answering

  • Jan 01, 2020
  • François Gardères +3
  • Book Chapter
  • Citations3

Optimal Image Feature Ranking and Fusion for Visual Question Answering

  • Sep 09, 2020
  • Sruthy Manmadhan +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.