• Home
  • Search
  • DSGEM: Dual scene graph enhancement module‐based visual question answering
  • https://doi.org/10.1049/cvi2.12186Copy DOI Icon

DSGEM: Dual scene graph enhancement module‐based visual question answering

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Abstract Visual Question Answering (VQA) aims to appropriately answer a text question by understanding the image content. Attention‐based VQA models mine the implicit relationships between objects according to the feature similarity, which neglects the explicit relationships between objects, for example, the relative position. Most Visual Scene Graph‐based VQA models exploit the relative positions or visual relationships between objects to construct the visual scene graph, while they suffer from the semantic insufficiency of visual edge relations. Besides, the scene graph of text modality is often ignored in these works. In this article, a novel Dual Scene Graph Enhancement Module (DSGEM) is proposed that exploits the relevant external knowledge to simultaneously construct two interpretable scene graph structures of image and text modalities, which makes the reasoning process more logical and precise. Specifically, the authors respectively build the visual and textual scene graphs with the help of commonsense knowledge and syntactic structure, which explicitly endows the specific semantics to each edge relation. Then, two scene graph enhancement modules are proposed to propagate the involved external and structural knowledge to explicitly guide the feature interaction between objects (nodes). Finally, the authors embed such two scene graph enhancement modules to existing VQA models to introduce the explicit relation reasoning ability. Experimental results on both VQA V2 and OK‐VQA datasets show that the proposed DSGEM is effective and compatible to various VQA architectures.

Similar Papers
  • Conference Article
  • Citations22

Lightweight Visual Question Answering using Scene Graphs

  • Oct 26, 2021
  • Sai Vidyaranya Nuthalapati +6
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article

A Semi-automatic Annotation of Scene Graph

  • May 29, 2025
  • INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Ashwini J K
  • Conference Article
  • Citations28

Visual Question Answering over Scene Graph

  • Sep 01, 2019
  • Soohyeong Lee +3
  • Research Article
  • Citations28

Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task

  • Jun 26, 2023
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Stan Weixian Lei +6
  • Research Article
  • Citations337

A Comprehensive Survey of Scene Graphs: Generation and Application.

  • Jan 01, 2023
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Xiaojun Chang +5
  • Research Article

Visual dialog with semantic consistency: An external knowledge-driven approach.

  • May 01, 2026
  • Neural networks : the official journal of the International Neural Network Society
  • Shanshan Du +1
  • Research Article
  • Citations6

SEMScene: Semantic-Consistency Enhanced Multi-Level Scene Graph Matching for Image-Text Retrieval

  • Aug 16, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yuankun Liu +7
  • Conference Article

Jointly Representing Images and Text

  • Oct 16, 2016
  • Frank Keller
  • Conference Article

Cross-modal Image Retrieval Based on Scene Graphs

  • Jul 26, 2024
  • Bochuan He +2
  • Conference Article
  • Citations2

EdgeNet for efficient scene graph classification

  • Jul 18, 2022
  • Vivek B.S +4
  • PDF
  • Research Article
  • Citations2

Enhancing visual question answering with common sense knowledge: a data-driven neurosymbolic graph routing approach

  • Jun 14, 2025
  • International Journal of Data Science and Analytics
  • Muhammad Junaid Khan +3
  • Research Article
  • Citations62

Scene Graph Refinement Network for Visual Question Answering

  • Jan 01, 2023
  • IEEE Transactions on Multimedia
  • Tianwen Qian +4
  • Conference Article

Graph Feature Generation based on Scene Graph Benchmark Application on Video

  • Oct 19, 2022
  • Jiyoun Lim +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.