• Home
  • Search
  • Rich Visual Knowledge-Based Augmentation Network for Visual Question Answering
  • Cite Icon78
  • https://doi.org/10.1109/tnnls.2020.3017530Copy DOI Icon

Rich Visual Knowledge-Based Augmentation Network for Visual Question Answering

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual question answering (VQA) that involves understanding an image and paired questions develops very quickly with the boost of deep learning in relevant research fields, such as natural language processing and computer vision. Existing works highly rely on the knowledge of the data set. However, some questions require more professional cues other than the data set knowledge to answer questions correctly. To address such an issue, we propose a novel framework named a knowledge-based augmentation network (KAN) for VQA. We introduce object-related open-domain knowledge to assist the question answering. Concretely, we extract more visual information from images and introduce a knowledge graph to provide the necessary common sense or experience for the reasoning process. For these two augmented inputs, we design an attention module that can adjust itself according to the specific questions, such that the importance of external knowledge against detected objects can be balanced adaptively. Extensive experiments show that our KAN achieves state-of-the-art performance on three challenging VQA data sets, i.e., VQA v2, VQA-CP v2, and FVQA. In addition, our open-domain knowledge is also beneficial to VQA baselines. Code is available at https://github.com/yyyanglz/KAN.

Similar Papers
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article

An Adaptive Multimodal Fusion Network Based on Multilinear Gradients for Visual Question Answering

  • Dec 24, 2024
  • Electronics
  • Chengfang Zhao +3
  • PDF
  • Research Article
  • Citations24

Multi-Modal Explicit Sparse Attention Networks for Visual Question Answering

  • Nov 26, 2020
  • Sensors (Basel, Switzerland)
  • Zihan Guo +1
  • Dissertation
  • Citations1

A Convolutional Neural Network Based Approach For Visual Question Answering

  • Apr 01, 2018
  • Lavanya Abhinaya Koduri
  • Research Article
  • Citations13

GFSNet: Gaussian Fourier with sparse attention network for visual question answering

  • Mar 15, 2025
  • Artificial Intelligence Review
  • Xiang Shen +4
  • Research Article

Overcoming the Limitations of Learning-Based VQA for Counting Questions with Zero-Shot Learning

  • Aug 20, 2024
  • International Journal on Artificial Intelligence Tools
  • A Lubna +1
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
  • Research Article
  • Citations3

HCCL: Hierarchical Counterfactual Contrastive Learning for Robust Visual Question Answering

  • Oct 29, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Dongze Hao +3
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Research Article
  • Citations43

Multitask Learning for Visual Question Answering.

  • Mar 01, 2023
  • IEEE Transactions on Neural Networks and Learning Systems
  • Jie Ma +5
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Book Chapter
  • Citations1

Towards Open Ended and Free Form Visual Question Answering: Modeling VQA as a Factoid Question Answering Problem

  • Jan 01, 2021
  • Abhishek Narayanan +3
  • Conference Article
  • Citations3

NLP Meets Vision for Visual Interpretation - A Retrospective Insight and Future directions

  • May 20, 2021
  • Ahmed Jamshed +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.