• Home
  • Search
  • Question Answering from Brain Activity Data via Decoder Based on Neural Networks
  • Cite Icon1
  • https://doi.org/10.1109/lifetech52111.2021.9391890Copy DOI Icon

Question Answering from Brain Activity Data via Decoder Based on Neural Networks

  • Mar 9, 2021
  • Saya Takada +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We build a model that can estimate what subjects recognize from functional magnetic resonance imaging (fMRI) data via a visual question answering (VQA) model. The VQA model can generate an answer to a question about an image. We convert fMRI signals into image features via an fMRI decoder based on the relationship between the fMRI signals and the image features extracted from the gazed image. Then this allows the VQA model to answer a visual question from the fMRI signals measured while the subject is gazing at the image. Though brain decoding, which interprets what humans recognize, has become overwhelmingly popular in neuroscience, they often suffer from the small datasets of brain activity data. To overcome the small size of datasets of fMRI signals, we introduce an fMRI decoder based on neural networks that have a high expressive ability. Even when we do not have enough fMRI signals, the proposed method derives the answer to what a person is looking at from fMRI signals. Experimental results on several datasets show that our method allows us to answer a question about gazed images from fMRI signals.

Similar Papers
  • Conference Article

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention

  • Jun 04, 2023
  • Ryo Shichida +4
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • PDF
  • Conference Article
  • Citations2

Can Pre-training help VQA with Lexical Variations?

  • Jan 01, 2020
  • Shailza Jolly +1
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Conference Article
  • Citations7

Learning to Ask Informative Sub-Questions for Visual Question Answering

  • Jun 01, 2022
  • Kohei Uehara +2
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Conference Article
  • Citations47

Quantifying and Alleviating the Language Prior Problem in Visual Question Answering

  • Jul 18, 2019
  • Yangyang Guo +5
  • PDF
  • Conference Article
  • Citations37

The Promise of Premise: Harnessing Question Premises in Visual Question Answering

  • Jan 01, 2017
  • Aroma Mahendru +4
  • PDF
  • Research Article
  • Citations19

Advancing surgical VQA with scene graph knowledge

  • May 23, 2024
  • International Journal of Computer Assisted Radiology and Surgery
  • Kun Yuan +5
  • Research Article
  • Citations8

Parallel multi-head attention and term-weighted question embedding for medical visual question answering.

  • Mar 11, 2023
  • Multimedia Tools and Applications
  • Sruthy Manmadhan +1
  • Video Transcripts

Question Agnostic Attention for Visual Question Answering

  • Dec 29, 2020
  • Underline Science Inc.
  • Moshiur Farazi
  • Conference Article
  • Citations10

Question-Agnostic Attention for Visual Question Answering

  • Jan 10, 2021
  • Moshiur Farazi +2
  • Research Article
  • Citations33

Causal Inference with Knowledge Distilling and Curriculum Learning for Unbiased VQA

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yonghua Pan +3
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.