• Home
  • Search
  • SceneGATE: Scene-Graph Based Co-Attention Networks for Text Visual Question Answering
  • Cite Icon8
  • https://doi.org/10.3390/robotics12040114Copy DOI Icon

SceneGATE: Scene-Graph Based Co-Attention Networks for Text Visual Question Answering

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) models fail catastrophically on questions related to the reading of text-carrying images. However, TextVQA aims to answer questions by understanding the scene texts in an image–question context, such as the brand name of a product or the time on a clock from an image. Most TextVQA approaches focus on objects and scene text detection, which are then integrated with the words in a question by a simple transformer encoder. The focus of these approaches is to use shared weights during the training of a multi-modal dataset, but it fails to capture the semantic relations between an image and a question. In this paper, we proposed a Scene Graph-Based Co-Attention Network (SceneGATE) for TextVQA, which reveals the semantic relations among the objects, the Optical Character Recognition (OCR) tokens and the question words. It is achieved by a TextVQA-based scene graph that discovers the underlying semantics of an image. We create a guided-attention module to capture the intra-modal interplay between the language and the vision as a guidance for inter-modal interactions. To permit explicit teaching of the relations between the two modalities, we propose and integrate two attention modules, namely a scene graph-based semantic relation-aware attention and a positional relation-aware attention. We conduct extensive experiments on two widely used benchmark datasets, Text-VQA and ST-VQA. It is shown that our SceneGATE method outperforms existing ones because of the scene graph and its attention modules.

Loading PDF

Similar Papers
  • Conference Article
  • Citations12

Occluded Text Detection and Recognition in the Wild

  • May 01, 2022
  • Zobeir Raisi +1
  • Research Article
  • Citations117

Scene text detection and recognition with advances in deep learning: a survey

  • Mar 27, 2019
  • International Journal on Document Analysis and Recognition (IJDAR)
  • Xiyan Liu +2
  • Conference Article
  • Citations5

Scene text detection and recognition system for visually impaired people in real world

  • Oct 09, 2018
  • Lei Fei +5
  • Research Article
  • Citations1

Novel Study on Localization in Scene Text Detection

  • Feb 15, 2023
  • International Journal of Next-Generation Computing
  • Pravinkumar Sonsare +4
  • Research Article
  • Citations29

Beyond OCR + VQA: Towards end-to-end reading and reasoning for robust and accurate textvqa

  • Jan 21, 2023
  • Pattern Recognition
  • Gangyan Zeng +7
  • Book Chapter
  • Citations3

Scene Text Detection with Text Statistical Characteristics and Deep Neural Network

  • Jan 01, 2017
  • Yanyun Qu +2
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations3

Performance Evaluation of Efficient and Accurate Text Detection and Recognition in Natural Scenes Images Using EAST and OCR Fusion

  • Jan 01, 2025
  • International Journal of Advanced Computer Science and Applications
  • Vishnu Kant Soni +5
  • Conference Article
  • Citations6

Vietnamese Scene Text Detection and Recognition using Deep Learning: An Empirical Study

  • Jul 29, 2022
  • Nhat Truong Pham +5
  • Research Article
  • Citations5

A Deep Learning-Based Text Detection and Recognition Approach for Natural Scenes

  • Oct 27, 2022
  • Journal of Circuits, Systems and Computers
  • Xuexiang Li
  • Research Article
  • Citations2

A crowdsource based framework for Bengali scene text data collection and detection

  • Nov 15, 2023
  • Computers and Electrical Engineering
  • Md Yearat Hossain +1
  • Research Article
  • Citations2

Scene Text Detection and Recognition Using Maximally Stable Extremal Region

  • Dec 15, 2024
  • Journal of Applied Engineering and Technological Science (JAETS)
  • Golda Jeyasheeli P +3
  • Conference Article
  • Citations138

TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

  • Jun 01, 2021
  • Amanpreet Singh +5
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Conference Article
  • Citations17

A Unified Video Text Detection Method with Network Flow

  • Nov 01, 2017
  • Xue-Hang Yang +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.