• Home
  • Search
  • SegEQA: Video Segmentation Based Visual Attention for Embodied Question Answering
  • Cite Icon38
  • https://doi.org/10.1109/iccv.2019.00976Copy DOI Icon

SegEQA: Video Segmentation Based Visual Attention for Embodied Question Answering

  • Oct 1, 2019
  • Haonan Luo +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Embodied Question Answering (EQA) is a newly defined research area where an agent is required to answer the user's questions by exploring the real world environment. It has attracted increasing research interests due to its broad applications in automatic driving system, in-home robots, and personal assistants. Most of the existing methods perform poorly in terms of answering and navigation accuracy due to the absence of local details and vulnerability to the ambiguity caused by complicated vision conditions. To tackle these problems, we propose a segmentation based visual attention mechanism for Embodied Question Answering. Firstly, We extract the local semantic features by introducing a novel high-speed video segmentation framework. Then by the guide of extracted semantic features, a bottom-up visual attention mechanism is proposed for the Visual Question Answering (VQA) sub-task. Further, a feature fusion strategy is proposed to guide the training of the navigator without much additional computational cost. The ablation experiments show that our method boosts the performance of VQA module by 4.2% (68.99% vs 64.73%) and leads to 3.6% (48.59% vs 44.98%) overall improvement in EQA accuracy.

Similar Papers
  • Research Article
  • Citations23

Depth and Video Segmentation Based Visual Attention for Embodied Question Answering.

  • Jun 01, 2023
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Haonan Luo +5
  • Research Article
  • Citations82

Co-Attending Free-Form Regions and Detections With Multi-Modal Multiplicative Feature Embedding for Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Pan Lu +4
  • Research Article
  • Citations17

Accuracy vs. complexity: A trade-off in visual question answering models

  • Jun 12, 2021
  • Pattern Recognition
  • Moshiur Farazi +2
  • Research Article
  • Citations16

Image Segmentation Based on Visual Attention Mechanism

  • Dec 01, 2009
  • Journal of Multimedia
  • Qiaorong Zhang +2
  • Conference Article
  • Citations5696

Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering

  • Jun 01, 2018
  • Peter Anderson +6
  • Research Article
  • Citations13

Local relation network with multilevel attention for visual question answering

  • Jan 20, 2020
  • Journal of Visual Communication and Image Representation
  • Bo Sun +3
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Research Article
  • Citations16

Robust-EQA: Robust Learning for Embodied Question Answering With Noisy Labels.

  • Sep 01, 2024
  • IEEE transactions on neural networks and learning systems
  • Haonan Luo +5
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations45

Exploring region relationships implicitly: Image captioning with visual relationship attention

  • Mar 05, 2021
  • Image and Vision Computing
  • Zongjian Zhang +3
  • Conference Article
  • Citations10

Multimodal Continuous Visual Attention Mechanisms

  • Oct 01, 2021
  • Antonio Farinhas +2
  • Research Article
  • Citations17

SAVAnet: Surgical Action-Driven Visual Attention Network for Autonomous Endoscope Control

  • Oct 01, 2023
  • IEEE Transactions on Automation Science and Engineering
  • Huxin Gao +8
  • Conference Article
  • Citations852

Dual Attention Networks for Multimodal Reasoning and Matching

  • Jul 01, 2017
  • Hyeonseob Nam +2
  • Conference Article
  • Citations5

Sparse And Structured Visual Attention

  • Sep 19, 2021
  • Pedro Henrique Martins +3
  • Research Article
  • Citations18

A hierarchical recurrent approach to predict scene graphs from a visual‐attention‐oriented perspective

  • Mar 22, 2019
  • Computational Intelligence
  • Wenjing Gao +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.