• Home
  • Search
  • ConVQG: Contrastive Visual Question Generation with Multimodal Guidance
  • Cite Icon1
  • https://doi.org/10.1609/aaai.v38i5.28216Copy DOI Icon

ConVQG: Contrastive Visual Question Generation with Multimodal Guidance

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Asking questions about visual environments is a crucial way for intelligent agents to understand rich multi-faceted scenes, raising the importance of Visual Question Generation (VQG) systems. Apart from being grounded to the image, existing VQG systems can use textual constraints, such as expected answers or knowledge triplets, to generate focused questions. These constraints allow VQG systems to specify the question content or leverage external commonsense knowledge that can not be obtained from the image content only. However, generating focused questions using textual constraints while enforcing a high relevance to the image content remains a challenge, as VQG systems often ignore one or both forms of grounding. In this work, we propose Contrastive Visual Question Generation (ConVQG), a method using a dual contrastive objective to discriminate questions generated using both modalities from those based on a single one. Experiments on both knowledge-aware and standard VQG benchmarks demonstrate that ConVQG outperforms the state-of-the-art methods and generates image-grounded, text-guided, and knowledge-rich questions. Our human evaluation results also show preference for ConVQG questions compared to non-contrastive baselines.

Similar Papers
  • Conference Article
  • Citations8

Deconfounded Visual Question Generation with Causal Inference

  • Oct 26, 2023
  • Jiali Chen +4
  • PDF
  • Research Article
  • Citations8

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation

  • Jan 17, 2023
  • Sensors (Basel, Switzerland)
  • He Zhu +3
  • PDF
  • Conference Article
  • Citations16

Visual Question Generation from Radiology Images

  • Jan 01, 2020
  • Mourad Sarrouti +2
  • Supplementary Content
  • Citations114

Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications

  • Jan 01, 2023
  • Progress in Artificial Intelligence
  • Nikahat Mulla +1
  • Conference Article
  • Citations9

K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition

  • Jan 01, 2023
  • Kohei Uehara +1
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
  • Research Article

An approach to medical visual question generation based on question categories

  • Oct 16, 2025
  • Journal of the Chinese Institute of Engineers
  • N Sheerin Sitara +3
  • Conference Article
  • Citations7

Generating Natural Questions from Images for Multimodal Assistants

  • Jun 06, 2021
  • Alkesh Patel +4
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Conference Article

Learning to Ask: Multi-Decoder Fine-Tuning for Multi-Hop Visual Question Generation with External Knowledge

  • Jan 01, 2026
  • Arpan Phukan +2
  • Research Article
  • Citations22

Visual Question Generation

  • May 28, 2020
  • ACM Computing Surveys
  • Charulata Patil +1
  • Research Article
  • Citations4

Diverse Visual Question Generation Based on Multiple Objects Selection

  • Mar 08, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Wenhao Fang +4
  • Research Article
  • Citations6

Explicitly diverse visual question generation.

  • Apr 01, 2025
  • Neural networks : the official journal of the International Neural Network Society
  • Jiayuan Xie +4
  • PDF
  • Research Article
  • Citations8

Goal-Driven Visual Question Generation from Radiology Images

  • Aug 20, 2021
  • Information
  • Mourad Sarrouti +2
  • Book Chapter
  • Citations8

Difficulty-Controllable Visual Question Generation

  • Jan 01, 2021
  • Feng Chen +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.