• Home
  • Search
  • Hierarchical Attention Networks for Multimodal Machine Learning
  • Cite Icon2
  • https://doi.org/10.1088/1742-6596/2218/1/012020Copy DOI Icon

Hierarchical Attention Networks for Multimodal Machine Learning

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The Visual Question Answering (VQA) task is to infer the correct answer to a free-form question based on the given image. This task is challenging because it requires model handling both visual and textual information. Most successful attempts on VQA task have been achieved by using attention mechanism which can capture inter-modal and intra-modal dependencies. In this paper, we raise a new attention-based model to solve VQA. We use question information to guide model concentrate on special regions and attribute and hierarchically reason the answer. We also propose multi-modal fusion strategy based on co-attention method to fuse both visual and textual information. Under the same experimental conditions, extensive experiments on VQA-v2.0 dataset illustrate our method performance exceeds the performance of some state-of-the-art methods of the same experimental conditions.

Similar Papers
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Conference Article
  • Citations48

Coarse-to-Fine Reasoning for Visual Question Answering

  • Jun 01, 2022
  • Binh X Nguyen +5
  • PDF
  • Conference Article
  • Citations23

Recommending Themes for Ad Creative Design via Visual-Linguistic Representations

  • Apr 20, 2020
  • Yichao Zhou +4
  • Research Article

Visual Question Answering Using Semantic Information from Image Descriptions

  • Apr 18, 2021
  • The International FLAIRS Conference Proceedings
  • Tasmia Tasmia +2
  • Research Article
  • Citations6

Explanation vs. attention: A two-player game to obtain attention for VQA and visual dialog

  • Jul 23, 2022
  • Pattern Recognition
  • Badri N Patro +2
  • Research Article
  • Citations29

Explanation vs Attention: A Two-Player Game to Obtain Attention for VQA

  • Apr 03, 2020
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Badri Patro +2
  • Research Article
  • Citations27

Toward Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baseline

  • Jun 01, 2023
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Lichen Zhao +8
  • Conference Article
  • Citations1

A Study of Visual Question Answering Techniques Based on Collaborative Multi-Head Attention

  • Feb 01, 2023
  • Yingli Yang +2
  • Conference Article
  • Citations1

An Visual Question Answering System using Cross-Attention Network

  • Aug 17, 2023
  • R Dhana Lakshmi +1
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • Book Chapter
  • Citations5

Neural Networks for Detecting Irrelevant Questions During Visual Question Answering

  • Jan 01, 2020
  • Mengdi Li +2
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Research Article
  • Citations19

Achieving Human Parity on Visual Question Answering

  • Apr 04, 2023
  • ACM Transactions on Information Systems
  • Ming Yan +11
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Book Chapter
  • Citations4

Multi-modal Feature Fusion Based on Variational Autoencoder for Visual Question Answering

  • Jan 01, 2019
  • Liqing Chen +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.