• Home
  • Search
  • Enhancing visual question answering with common sense knowledge: a data-driven neurosymbolic graph routing approach
  • Cite Icon2
  • https://doi.org/10.1007/s41060-025-00827-7Copy DOI Icon

Enhancing visual question answering with common sense knowledge: a data-driven neurosymbolic graph routing approach

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Abstract Visual question answering (VQA) requires models to comprehend both visual and textual inputs, often necessitating multi-hop reasoning and external knowledge beyond image content. Despite recent advances, current VQA models struggle with complex questions that require reasoning over both structured knowledge and unstructured visual information. To address these limitations, we propose the neurosymbolic graph routing network (NeSyGRN), which integrates a graph routing network for stepwise reasoning with an enrichment mechanism leveraging the common sense knowledge graph (CSKG). This approach enriches scene graphs and question representations with common sense knowledge, enhancing the reasoning capabilities of VQA through structured data augmentation. NeSyGRN is evaluated on benchmark datasets, demonstrating substantial improvements over state-of-the-art baselines. On the KR-VQA dataset, NeSyGRN achieves an overall accuracy of 74.52%, surpassing prior best results by 8%. Ablation studies validate the critical role of knowledge enrichment, with scene graph enrichment contributing most significantly to the improvements. On the GQA dataset, NeSyGRN achieves an accuracy of 83.43%, setting new state-of-the-art results in consistency, plausibility, and contextual validity. These promising results highlight the effectiveness of integrating structured common sense knowledge for VQA, reinforcing its potential for applications in data-rich environments such as health care, education, and decision support systems.

Loading PDF

Similar Papers
  • PDF
  • Research Article
  • Citations9

A survey of neurosymbolic visual reasoning with scene graphs and common sense knowledge

  • Mar 21, 2025
  • Neurosymbolic Artificial Intelligence
  • M Jaleed Khan +3
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • PDF
  • Research Article
  • Citations8

MKBQA: Question Answering over Knowledge Graph Based on Semantic Analysis and Priority Marking Method

  • May 16, 2023
  • Applied Sciences
  • Xiang Wang +3
  • Conference Article
  • Citations59

Visual Question Answering as Reading Comprehension

  • Jun 01, 2019
  • Hui Li +3
  • Book Chapter
  • Citations14

Expressive Scene Graph Generation Using Commonsense Knowledge Infusion for Visual Understanding and Reasoning

  • Jan 01, 2022
  • Muhammad Jaleed Khan +2
  • Conference Article
  • Citations3

NLP Meets Vision for Visual Interpretation - A Retrospective Insight and Future directions

  • May 20, 2021
  • Ahmed Jamshed +1
  • Conference Article
  • Citations1

Improving Generation of Sentiment Commonsense by Bias Mitigation

  • Feb 01, 2023
  • Jinkyu Lee +1
  • Conference Article

Bridging the Gap: Leveraging Textual and Visual Contexts for PreciseMedical Visual Question Answering

  • Aug 08, 2024
  • Pratiksh Kumar +3
  • PDF
  • Research Article
  • Citations11

An Interpretable Fake News Detection Method Based on Commonsense Knowledge Graph

  • May 30, 2023
  • Applied Sciences
  • Xiang Gao +11
  • Research Article
  • Citations80

Long Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance

  • Jan 01, 2024
  • IEEE Transactions on Multimedia
  • Weizhi Nie +3
  • Research Article

Philosophy and Common Sense Knowledge

  • Jan 01, 2026
  • American Philosophical Quarterly
  • Noah Lemos
  • Conference Article
  • Citations22

Lightweight Visual Question Answering using Scene Graphs

  • Oct 26, 2021
  • Sai Vidyaranya Nuthalapati +6
  • Research Article
  • Citations7

NeuSyRE: Neuro-symbolic visual understanding and reasoning framework based on scene graph enrichment

  • Oct 04, 2024
  • Semantic Web
  • M Jaleed Khan +2
  • Research Article
  • Citations1

Investigation of Available Datasets and Techniques for Visual Question Answering

  • Aug 03, 2023
  • INTERNATIONAL JOURNAL OF NEXT-GENERATION COMPUTING
  • Lata Bhavnani +1
  • Research Article

DSGEM: Dual scene graph enhancement module‐based visual question answering

  • Mar 07, 2023
  • IET Computer Vision
  • Boyue Wang +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.