• Home
  • Search
  • GoG: Relation-aware Graph-over-Graph Network for Visual Dialog
  • Cite Icon32
  • https://doi.org/10.18653/v1/2021.findings-acl.20Copy DOI Icon

GoG: Relation-aware Graph-over-Graph Network for Visual Dialog

  • Jan 1, 2021
  • Feilong Chen +4 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual dialog, which aims to hold a meaningful conversation with humans about a given image, is a challenging task that requires models to reason the complex dependencies among visual content, dialog history, and current questions. Graph neural networks are recently applied to model the implicit relations between objects in an image or dialog. However, they neglect the importance of 1) coreference relations among dialog history and dependency relations between words for the question representation; and 2) the representation of the image based on the fully represented question. Therefore, we propose a novel relation-aware graph-over-graph network (GoG) for visual dialog. Specifically, GoG consists of three sequential graphs: 1) H-Graph, which aims to capture coreference relations among dialog history; 2) History-aware Q-Graph, which aims to fully understand the question through capturing dependency relations between words based on coreference resolution on the dialog history; and 3) Questionaware I-Graph, which aims to capture the relations between objects in an image based on fully question representation. As an additional feature representation module, we add GoG to the existing visual dialogue model. Experimental results show that our model outperforms the strong baseline in both generative and discriminative settings by a significant margin.

Loading PDF

Similar Papers
  • Research Article
  • Citations4

You should know more: Learning external knowledge for visual dialog

  • Feb 12, 2022
  • Neurocomputing
  • Lei Zhao +4
  • Book Chapter
  • Citations224

Visual Coreference Resolution in Visual Dialog Using Neural Module Networks

  • Jan 01, 2018
  • Satwik Kottur +4
  • Research Article

Visual dialog with semantic consistency: An external knowledge-driven approach.

  • May 01, 2026
  • Neural networks : the official journal of the International Neural Network Society
  • Shanshan Du +1
  • Conference Article
  • Citations17

Improving Cross-Modal Understanding in Visual Dialog Via Contrastive Learning

  • May 23, 2022
  • Feilong Chen +3
  • Research Article
  • Citations35

Visual Dialog.

  • May 01, 2019
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Abhishek Das +8
  • Conference Article
  • Citations435

Visual Dialog

  • Jul 01, 2017
  • Abhishek Das +7
  • Supplementary Content

Visual Dialog: Towards Communicative Visual Agents

  • May 30, 2019
  • Figshare
  • Satwik Kottur
  • Conference Article
  • Citations149

Reasoning Visual Dialogs With Structural and Partial Observations

  • Jun 01, 2019
  • Zilong Zheng +3
  • Conference Article

A GuessWhat?! Game for Goal-Oriented Visual Dialog: A Survey

  • Dec 05, 2021
  • Ghada M Elshamy +2
  • Research Article

A fine-grained deconfounding study for knowledge-based visual dialog

  • Dec 01, 2024
  • Visual Informatics
  • An-An Liu +5
  • Conference Article
  • Citations163

Recursive Visual Attention in Visual Dialog

  • Jun 01, 2019
  • Yulei Niu +5
  • Video Transcripts

An interpretable representation that visually grounds dialog history

  • Oct 21, 2021
  • Underline Science Inc.
  • Thomas Vadora +5
  • PDF
  • Conference Article
  • Citations68

History for Visual Dialog: Do we really need it?

  • Jan 01, 2020
  • Shubham Agarwal +4
  • Conference Article
  • Citations5

SKANet: Structured Knowledge-Aware Network for Visual Dialog

  • Jul 05, 2021
  • Lei Zhao +4
  • Research Article
  • Citations27

Multi-View Attention Network for Visual Dialog

  • Mar 27, 2021
  • Applied Sciences
  • Sungjin Park +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.