• Home
  • Search
  • Neurosymbolic Visual Reasoning with Scene Graphs and Multimodal LLMs
  • https://doi.org/10.3233/faia250227Copy DOI Icon

Neurosymbolic Visual Reasoning with Scene Graphs and Multimodal LLMs

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

This chapter explores the advancements and challenges in achieving comprehensive scene understanding and visual reasoning through neurosymbolic integration and Multimodal Large Language Models (MLLMs). It begins by highlighting the limitations of basic vision tasks in extracting contextual and relational information from scenes, introducing scene graphs as a structured representation to bridge this gap. The chapter delves into Scene Graph Generation (SGG) methods, emphasising the importance of incorporating common sense knowledge from knowledge graphs to enhance the accuracy and expressiveness of scene graphs. The NeuSyRE framework is presented as a neurosymbolic approach for enriched scene graph generation and reasoning, demonstrating its effectiveness in downstream tasks such as image captioning and visual question answering. The chapter also examines the role of MLLMs in visual reasoning, discussing their architectures, performance on zero-shot tasks and challenges in handling fine-grained visual details. MARVEL, a novel benchmark for abstract visual reasoning, is introduced to evaluate the perceptual and reasoning capabilities of MLLMs. Insights from MARVEL highlight the limitations of current MLLMs in solving complex reasoning tasks and underscore the potential of neurosymbolic systems to address these challenges. The chapter concludes by emphasising the synergy between neurosymbolic approaches and MLLMs in advancing visual intelligence and achieving robust and explainable AI systems.

Similar Papers
  • PDF
  • Research Article
  • Citations9

A survey of neurosymbolic visual reasoning with scene graphs and common sense knowledge

  • Mar 21, 2025
  • Neurosymbolic Artificial Intelligence
  • M Jaleed Khan +3
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Book Chapter
  • Citations14

Expressive Scene Graph Generation Using Commonsense Knowledge Infusion for Visual Understanding and Reasoning

  • Jan 01, 2022
  • Muhammad Jaleed Khan +2
  • Research Article
  • Citations7

NeuSyRE: Neuro-symbolic visual understanding and reasoning framework based on scene graph enrichment

  • Oct 04, 2024
  • Semantic Web
  • M Jaleed Khan +2
  • Research Article
  • Citations5

KnowZRel: Common Sense Knowledge-Based Zero-Shot Relationship Retrieval for Generalized Scene Graph Generation

  • Dec 01, 2025
  • IEEE Transactions on Artificial Intelligence
  • M Jaleed Khan +2
  • Conference Article
  • Citations22

Lightweight Visual Question Answering using Scene Graphs

  • Oct 26, 2021
  • Sai Vidyaranya Nuthalapati +6
  • Research Article
  • Citations2

Adaptive Image-to-Video Scene Graph Generation via Knowledge Reasoning and Adversarial Learning

  • Jun 28, 2022
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Jin Chen +2
  • Research Article
  • Citations28

Boosting Scene Graph Generation with Visual Relation Saliency

  • Jan 05, 2023
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yong Zhang +5
  • Conference Article
  • Citations2

EdgeNet for efficient scene graph classification

  • Jul 18, 2022
  • Vivek B.S +4
  • Research Article
  • Citations20

Generating Triples With Adversarial Networks for Scene Graph Construction

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Matthew Klawonn +1
  • Book Chapter
  • Citations4

Scene Graph Generation with Geometric Context

  • Jan 01, 2022
  • Vishal Kumar +2
  • Conference Article
  • Citations757

Unbiased Scene Graph Generation From Biased Training

  • Jun 01, 2020
  • Kaihua Tang +4
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article

Human-like scene graph generation and evaluation

  • Jan 01, 2026
  • Multimedia Tools and Applications
  • Victor Milewski +2
  • Research Article

A Semi-automatic Annotation of Scene Graph

  • May 29, 2025
  • INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Ashwini J K
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.