• Home
  • Search
  • Generating Natural Questions from Images for Multimodal Assistants
  • Cite Icon7
  • https://doi.org/10.1109/icassp39728.2021.9413599Copy DOI Icon

Generating Natural Questions from Images for Multimodal Assistants

  • Jun 6, 2021
  • Alkesh Patel +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Generating natural, diverse, and meaningful questions from images is an essential task for multimodal assistants as it confirms whether they have understood the object and scene in the images properly. The research in visual question answering (VQA) and visual question generation (VQG) is a great step. However, this research does not capture questions that a visually-abled person would ask multimodal assistants. Recently published datasets such as KB-VQA, FVQA, and OK-VQA try to collect questions that look for external knowledge which makes them appropriate for multimodal assistants. However, they still contain many obvious and common-sense questions that humans would not usually ask a digital assistant. In this paper, we provide a new benchmark dataset that contains questions generated by human annotators keeping in mind what they would ask multimodal digital assistants. Large scale annotations for thousands of images are expensive and time-consuming, so we also present an effective way of automatically generating questions from unseen images. In this paper, we present an approach for generating diverse and meaningful questions that consider image content and metadata of image (e.g., location, associated keyword). We evaluate our approach using standard evaluation metrics such as BLEU, METEOR, ROUGE, and CIDEr to show the relevance of generated questions with human-provided questions. We also measure the diversity of generated questions using generative strength and inventiveness metrics. We report new state-of-the-art results on the public and our datasets.

Similar Papers
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
  • Conference Article
  • Citations4

UAV-VQG: Visual Question Generation Framework on UAV Images

  • Dec 15, 2021
  • Argho Sarkar +1
  • Research Article
  • Citations74

Radial Graph Convolutional Network for Visual Question Generation.

  • Apr 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xing Xu +4
  • Research Article
  • Citations21

Exploring Duality in Visual Question-Driven Top-Down Saliency.

  • Jan 01, 2019
  • IEEE Transactions on Neural Networks and Learning Systems
  • Shengfeng He +3
  • PDF
  • Conference Article
  • Citations16

Visual Question Generation from Radiology Images

  • Jan 01, 2020
  • Mourad Sarrouti +2
  • Research Article

An approach to medical visual question generation based on question categories

  • Oct 16, 2025
  • Journal of the Chinese Institute of Engineers
  • N Sheerin Sitara +3
  • Research Article
  • Citations22

Visual Question Generation

  • May 28, 2020
  • ACM Computing Surveys
  • Charulata Patil +1
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations4

Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference.

  • Dec 01, 2024
  • IEEE transactions on pattern analysis and machine intelligence
  • Kai Shen +6
  • Research Article
  • Citations15

MobiVQA

  • Jul 04, 2022
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
  • Qingqing Cao +3
  • Book Chapter
  • Citations8

Look, Read and Ask: Learning to Ask Questions by Reading Text in Images

  • Jan 01, 2021
  • Soumya Jahagirdar +2
  • Research Article
  • Citations20

Visual Question Generation From Remote Sensing Images

  • Jan 01, 2023
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Laila Bashmal +5
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations37

Answer Again: Improving VQA with Cascaded-Answering Model

  • Jan 01, 2020
  • IEEE Transactions on Knowledge and Data Engineering
  • Liang Peng +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.