• Home
  • Search
  • Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference.
  • Cite Icon4
  • https://doi.org/10.1109/tpami.2024.3425222Copy DOI Icon

Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference.

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The visual question generation (VQG) task aims to generate human-like questions from an image and potentially other side information (e.g., answer type). Previous works on VQG fall in two aspects: i) They suffer from one image to many questions mapping problem, which leads to the failure of generating referential and meaningful questions from an image. ii) They fail to model complex implicit relations among the visual objects in an image and also overlook potential interactions between the side information and image. To address these limitations, we first propose a novel learning paradigm to generate visual questions with answer-awareness and region-reference. Concretely, we aim to ask the right visual questions with Double Hints - textual answers and visual regions of interests, which could effectively mitigate the existing one-to-many mapping issue. Particularly, we develop a simple methodology to self-learn the visual hints without introducing any additional human annotations. Furthermore, to capture these sophisticated relationships, we propose a new double-hints guided Graph-to-Sequence learning framework, which first models them as a dynamic graph and learns the implicit topology end-to-end, and then utilizes a graph-to-sequence model to generate the questions with double hints. Experimental results demonstrate the priority of our proposed method.

Similar Papers
  • Conference Article
  • Citations4

UAV-VQG: Visual Question Generation Framework on UAV Images

  • Dec 15, 2021
  • Argho Sarkar +1
  • Conference Article
  • Citations8

Deconfounded Visual Question Generation with Causal Inference

  • Oct 26, 2023
  • Jiali Chen +4
  • Research Article
  • Citations22

Visual Question Generation

  • May 28, 2020
  • ACM Computing Surveys
  • Charulata Patil +1
  • Conference Article
  • Citations7

Generating Natural Questions from Images for Multimodal Assistants

  • Jun 06, 2021
  • Alkesh Patel +4
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Research Article

An approach to medical visual question generation based on question categories

  • Oct 16, 2025
  • Journal of the Chinese Institute of Engineers
  • N Sheerin Sitara +3
  • PDF
  • Conference Article
  • Citations16

Visual Question Generation from Radiology Images

  • Jan 01, 2020
  • Mourad Sarrouti +2
  • Research Article
  • Citations9

Cross-modality Multiple Relations Learning for Knowledge-based Visual Question Answering

  • Oct 23, 2023
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yan Wang +6
  • PDF
  • Research Article
  • Citations8

Goal-Driven Visual Question Generation from Radiology Images

  • Aug 20, 2021
  • Information
  • Mourad Sarrouti +2
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
  • Book Chapter
  • Citations8

Look, Read and Ask: Learning to Ask Questions by Reading Text in Images

  • Jan 01, 2021
  • Soumya Jahagirdar +2
  • Conference Article
  • Citations4

Attention-based Visual Question Generation

  • Mar 05, 2021
  • Charulata Patil +1
  • Conference Article
  • Citations9

K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition

  • Jan 01, 2023
  • Kohei Uehara +1
  • Conference Article
  • Citations353

Generating Natural Questions About an Image

  • Jan 01, 2016
  • Nasrin Mostafazadeh +5
  • Research Article
  • Citations74

Radial Graph Convolutional Network for Visual Question Generation.

  • Apr 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xing Xu +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.