• Home
  • Search
  • Co-Attending Free-Form Regions and Detections With Multi-Modal Multiplicative Feature Embedding for Visual Question Answering
  • Cite Icon82
  • https://doi.org/10.1609/aaai.v32i1.12240Copy DOI Icon

Co-Attending Free-Form Regions and Detections With Multi-Modal Multiplicative Feature Embedding for Visual Question Answering

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Recently, the Visual Question Answering (VQA) task has gained increasing attention in artificial intelligence. Existing VQA methods mainly adopt the visual attention mechanism to associate the input question with corresponding image regions for effective question answering. The free-form region based and the detection-based visual attention mechanisms are mostly investigated, with the former ones attending free-form image regions and the latter ones attending pre-specified detection-box regions. We argue that the two attention mechanisms are able to provide complementary information and should be effectively integrated to better solve the VQA problem. In this paper, we propose a novel deep neural network for VQA that integrates both attention mechanisms. Our proposed framework effectively fuses features from free-form image regions, detection boxes, and question representations via a multi-modal multiplicative feature embedding scheme to jointly attend question-related free-form image regions and detection boxes for more accurate question answering. The proposed method is extensively evaluated on two publicly available datasets, COCO-QA and VQA, and outperforms state-of-the-art approaches. Source code is available at https://github.com/lupantech/dual-mfa-vqa.

Similar Papers
  • Research Article
  • Citations33

ALSA: Adversarial Learning of Supervised Attentions for Visual Question Answering.

  • Nov 11, 2020
  • IEEE transactions on cybernetics
  • Yun Liu +5
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations13

Local relation network with multilevel attention for visual question answering

  • Jan 20, 2020
  • Journal of Visual Communication and Image Representation
  • Bo Sun +3
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations3

Enhancing visual question answering with a two‐way co‐attention mechanism and integrated multimodal features

  • Dec 21, 2023
  • Computational Intelligence
  • Mayank Agrawal +2
  • PDF
  • Research Article
  • Citations24

Multi-Modal Explicit Sparse Attention Networks for Visual Question Answering

  • Nov 26, 2020
  • Sensors (Basel, Switzerland)
  • Zihan Guo +1
  • Conference Article
  • Citations48

Coarse-to-Fine Reasoning for Visual Question Answering

  • Jun 01, 2022
  • Binh X Nguyen +5
  • Research Article
  • Citations50

DRAU: Dual Recurrent Attention Units for Visual Question Answering

  • May 20, 2019
  • Computer Vision and Image Understanding
  • Ahmed Osman +1
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Conference Article
  • Citations10

Multimodal Continuous Visual Attention Mechanisms

  • Oct 01, 2021
  • Antonio Farinhas +2
  • Research Article
  • Citations617

Beyond Bilinear: Generalized Multimodal Factorized High-Order Pooling for Visual Question Answering.

  • Apr 09, 2018
  • IEEE Transactions on Neural Networks and Learning Systems
  • Zhou Yu +4
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Conference Article
  • Citations11

Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment

  • Oct 16, 2022
  • Argho Sarkar +1
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.