• Home
  • Search
  • Generating Triples With Adversarial Networks for Scene Graph Construction
  • Cite Icon20
  • https://doi.org/10.1609/aaai.v32i1.12321Copy DOI Icon

Generating Triples With Adversarial Networks for Scene Graph Construction

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Driven by successes in deep learning, computer vision research has begun to move beyond object detection and image classification to more sophisticated tasks like image captioning or visual question answering. Motivating such endeavors is the desire for models to capture not only objects present in an image, but more fine-grained aspects of a scene such as relationships between objects and their attributes. Scene graphs provide a formal construct for capturing these aspects of an image. Despite this, there have been only a few recent efforts to generate scene graphs from imagery. Previous works limit themselves to settings where bounding box information is available at train time and do not attempt to generate scene graphs with attributes. In this paper we propose a method, based on recent advancements in Generative Adversarial Networks, to overcome these deficiencies. We take the approach of first generating small subgraphs, each describing a single statement about a scene from a specific region of the input image chosen using an attention mechanism. By doing so, our method is able to produce portions of the scene graphs with attribute information without the need for bounding box labels. Then, the complete scene graph is constructed from these subgraphs. We show that our model improves upon prior work in scene graph generation on state-of-the-art data sets and accepted metrics. Further, we demonstrate that our model is capable of handling a larger vocabulary size than prior work has attempted.

Similar Papers
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Conference Article
  • Citations22

Lightweight Visual Question Answering using Scene Graphs

  • Oct 26, 2021
  • Sai Vidyaranya Nuthalapati +6
  • Book Chapter
  • Citations14

Expressive Scene Graph Generation Using Commonsense Knowledge Infusion for Visual Understanding and Reasoning

  • Jan 01, 2022
  • Muhammad Jaleed Khan +2
  • Conference Article
  • Citations2

EdgeNet for efficient scene graph classification

  • Jul 18, 2022
  • Vivek B.S +4
  • Research Article
  • Citations2

Adaptive Image-to-Video Scene Graph Generation via Knowledge Reasoning and Adversarial Learning

  • Jun 28, 2022
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Jin Chen +2
  • Research Article
  • Citations4

Adversarial Attacks on Scene Graph Generation

  • Jan 01, 2024
  • IEEE Transactions on Information Forensics and Security
  • Mengnan Zhao +4
  • Book Chapter
  • Citations4

Scene Graph Generation with Geometric Context

  • Jan 01, 2022
  • Vishal Kumar +2
  • Research Article
  • Citations7

NeuSyRE: Neuro-symbolic visual understanding and reasoning framework based on scene graph enrichment

  • Oct 04, 2024
  • Semantic Web
  • M Jaleed Khan +2
  • Book Chapter

Neurosymbolic Visual Reasoning with Scene Graphs and Multimodal LLMs

  • Mar 17, 2025
  • Frontiers in artificial intelligence and applications
  • M Jaleed Khan +2
  • Research Article

Balanced scene graph generation assisted by an additional biased predictor

  • Nov 01, 2022
  • SCIENTIA SINICA Informationis
  • 文彬 王 +2
  • PDF
  • Research Article
  • Citations9

A survey of neurosymbolic visual reasoning with scene graphs and common sense knowledge

  • Mar 21, 2025
  • Neurosymbolic Artificial Intelligence
  • M Jaleed Khan +3
  • Research Article
  • Citations16

DRAKE: Deep Pair-Wise Relation Alignment for Knowledge-Enhanced Multimodal Scene Graph Generation in Social Media Posts

  • Jul 01, 2023
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Ze Fu +6
  • Dissertation

Leveraging deep learning for visual understanding of videos

  • Jan 01, 2024
  • Clement Xian Ren Tan
  • Conference Article
  • Citations12

Scene Understanding for Safety Analysis in Human-Robot Collaborative Operations

  • Apr 01, 2020
  • Hassam Riaz +4
  • Conference Article
  • Citations363

Scene Graph Generation With External Knowledge and Image Reconstruction

  • Jun 01, 2019
  • Jiuxiang Gu +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.