• Home
  • Search
  • Visual Question Answering Based on Position Alignment
  • Cite Icon1
  • https://doi.org/10.1109/cisp-bmei53629.2021.9624447Copy DOI Icon

Visual Question Answering Based on Position Alignment

  • Oct 23, 2021
  • Qihao Xia +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The alignment of information from images and questions is of great significance in the visual question answering task. Whether an object in image is related to the question or not is the basic judgement relied on the feature alignment. Many previous works have proposed different alignment methods to build better cross modality interaction. The attention mechanism is the most used method in making alignment. The classical bottom up and top down model builds a top down attention distribution by concatenating question features to each image features and calculates the attention weights between question and image. However, the bottom up and top down model didn't consider the positional information in image and question. In this paper, we revisit the attention distribution from a position perspective which aligns question to object's positional information. We first embed the positional information of each object in image and calculate a position attention distribution to indicate the relevance between objects' positions in context of the current question. Through the attention distribution model can select the related position in image to answer the given question. The position attention distribution is concatenated to the feature attention distribution to get the final distribution. We evaluate our method on visual question answering (VQA2.0) dataset, and show that our method is effective in multimodal alignment.

Similar Papers
  • Research Article
  • Citations38

Positional Attention Guided Transformer-Like Architecture for Visual Question Answering

  • Jan 01, 2023
  • IEEE Transactions on Multimedia
  • Aihua Mao +4
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article
  • Citations29

OpenViVQA: Task, dataset, and multimodal fusion models for visual question answering in Vietnamese

  • Jul 04, 2023
  • Information Fusion
  • Nghia Hieu Nguyen +3
  • Conference Article
  • Citations48

Coarse-to-Fine Reasoning for Visual Question Answering

  • Jun 01, 2022
  • Binh X Nguyen +5
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations2

Hierarchical Attention Networks for Multimodal Machine Learning

  • Mar 01, 2022
  • Journal of Physics Conference Series
  • Haotian Liang +1
  • Book Chapter

Multi-view Visual Question Answering Dataset for Real Environment Applications

  • Jan 01, 2020
  • Yue Qiu +3
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • Book Chapter
  • Citations5

Neural Networks for Detecting Irrelevant Questions During Visual Question Answering

  • Jan 01, 2020
  • Mengdi Li +2
  • Research Article
  • Citations3

Enhancing visual question answering with a two‐way co‐attention mechanism and integrated multimodal features

  • Dec 21, 2023
  • Computational Intelligence
  • Mayank Agrawal +2
  • Research Article
  • Citations53

Visual Question Answering: A Tutorial

  • Nov 01, 2017
  • IEEE Signal Processing Magazine
  • Damien Teney +2
  • Conference Article
  • Citations1

A Study of Visual Question Answering Techniques Based on Collaborative Multi-Head Attention

  • Feb 01, 2023
  • Yingli Yang +2
  • Research Article

Visual Question Answering Using Semantic Information from Image Descriptions

  • Apr 18, 2021
  • The International FLAIRS Conference Proceedings
  • Tasmia Tasmia +2
  • Research Article
  • Citations5

Improved Blending Attention Mechanism in Visual Question Answering

  • Jan 01, 2023
  • Computer Systems Science and Engineering
  • Siyu Lu +6
  • Research Article
  • Citations472

The multi-modal fusion in visual question answering: a review of attention mechanisms.

  • May 30, 2023
  • PeerJ Computer Science
  • Siyu Lu +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.