• Home
  • Search
  • Optimal Image Feature Ranking and Fusion for Visual Question Answering
  • Cite Icon3
  • https://doi.org/10.1007/978-981-15-5788-0_10Copy DOI Icon

Optimal Image Feature Ranking and Fusion for Visual Question Answering

  • Sep 9, 2020
  • Sruthy Manmadhan +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) is a moderately new and challenging multi-modal task, which endeavors to discover an answer for a given pair of an image and a relating question. This AI-complete task gains attraction from numerous researchers from the areas computer vision (CV) and natural language processing (NLP) due to its various potential applications. The general flow of VQA algorithms consists of image feature extraction, question feature extraction and joint comprehension of these two to generate an appropriate answer. Existing VQA systems did not pay attention to input feature extraction, but only celebrated different ways of multi-modal embedding. This paper proposes to improve the task of VQA by feature-level fusion of visual information. The goal of feature fusion is to consolidate relevant information from two or more feature vectors into a solitary one with additional discriminative power. Unlike simple concatenation, this paper uses discriminative correlation analysis (DCA) for fusion, which is the only method that incorporates the class structure into the feature-level fusion. Since the VQA systems are generally modeled as classification systems by treating the correct answers as classes, class-specific DCA suits well here. The newly created fused feature vectors are close to the right answers and thus raise the role of image understanding in VQA. The experimental results show the effectiveness of the new approach on DAQUAR dataset with mutual information (MI) as an evaluation metric.

Similar Papers
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article
  • Citations1

Counting in Visual Question Answering: Methods, Datasets, and Future Work

  • Oct 20, 2023
  • International Journal of Image and Graphics
  • Tesfayee Meshu Welde +1
  • Research Article
  • Citations12

Visual Question Answering using Transformer Architectures: Applying Transformer Models to Improve Performance in VQA Tasks

  • Mar 31, 2022
  • Journal of Artificial Intelligence & Cloud Computing
  • Vedant Singh
  • Research Article
  • Citations45

A survey of methods, datasets and evaluation metrics for visual question answering

  • Oct 15, 2021
  • Image and Vision Computing
  • Himanshu Sharma +1
  • Abstract

Understanding the diagnosis and management of external eye disease: Gurwood AS: Br J Optom Disp 1995;3:155–163

  • May 01, 1996
  • International Contact Lens Clinic
  • PDF
  • Research Article
  • Citations412

Multiscale Feature Extraction and Fusion of Image and Text in VQA

  • Apr 11, 2023
  • International Journal of Computational Intelligence Systems
  • Siyu Lu +5
  • Book Chapter
  • Citations2

A VQA System for Medical Image Classification Using Transfer Learning

  • Jan 01, 2021
  • C Dhanush +2
  • Book Chapter
  • Citations276

Revisiting Visual Question Answering Baselines

  • Jan 01, 2016
  • Allan Jabri +2
  • Book Chapter
  • Citations5

Neural Networks for Detecting Irrelevant Questions During Visual Question Answering

  • Jan 01, 2020
  • Mengdi Li +2
  • PDF
  • Research Article
  • Citations86

MedFuseNet:\xa0An\xa0attention-based multimodal deep learning model for visual question answering in the medical domain

  • Oct 06, 2021
  • Scientific Reports
  • Dhruv Sharma +2
  • Conference Article

Bridging the Gap: Leveraging Textual and Visual Contexts for PreciseMedical Visual Question Answering

  • Aug 08, 2024
  • Pratiksh Kumar +3
  • Research Article
  • Citations472

The multi-modal fusion in visual question answering: a review of attention mechanisms.

  • May 30, 2023
  • PeerJ Computer Science
  • Siyu Lu +5
  • Research Article

An Adaptive Multimodal Fusion Network Based on Multilinear Gradients for Visual Question Answering

  • Dec 24, 2024
  • Electronics
  • Chengfang Zhao +3
  • PDF
  • Research Article
  • Citations2

Visual Question Answering for Intelligent Interaction

  • Jul 06, 2022
  • Mobile Information Systems
  • Panpan Gao +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.