• Home
  • Search
  • Knowledge-Based Visual Question Generation
  • Cite Icon27
  • https://doi.org/10.1109/tcsvt.2022.3189242Copy DOI Icon

Knowledge-Based Visual Question Generation

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual question generation task aims to generate meaningful questions about an image targeting an answer. Existing methods focus on the visual concepts in the image for question generation. However, humans inevitably use their knowledge related to visual objects in images to construct questions. In this paper, we propose a knowledge-based visual question generation model that can integrate visual concepts and non-visual knowledge to generate questions. To obtain visual concepts, we utilize a pre-trained object detection model to obtain object-level features of each object in the image. To obtain useful non-visual knowledge, we first retrieve the knowledge from the knowledge-base related to the visual objects in the image. Considering that not all retrieved knowledge is helpful for this task, we introduce an answer-aware module to capture the candidate knowledge related to the answer from the retrieved knowledge, which ensures that the generated content can be targeted at the answer. Finally, object-level representations containing visual concepts and non-visual knowledge are sent to a decoder module to generate questions. Extensive experiments on the FVQA and KBVQA datasets show that the proposed model outperforms the state-of-the-art models.

Similar Papers
  • Conference Article
  • Citations8

Deconfounded Visual Question Generation with Causal Inference

  • Oct 26, 2023
  • Jiali Chen +4
  • Research Article
  • Citations25

Re-Attention for Visual Question Answering

  • Apr 03, 2020
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Wenya Guo +5
  • Research Article
  • Citations1

Interactive Visual Task Learning for Robots

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Weiwei Gu +2
  • Research Article
  • Citations4

Diverse Visual Question Generation Based on Multiple Objects Selection

  • Mar 08, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Wenhao Fang +4
  • Research Article
  • Citations70

An adaptive nonlinear diffusion algorithm for filtering medical images.

  • Jan 01, 2000
  • IEEE Transactions on Information Technology in Biomedicine
  • J.S Jin +2
  • Conference Article
  • Citations4

<title>Adaptive nonlinear diffusion algorithm for image filtering</title>

  • Apr 03, 1997
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Yung Wang +2
  • Conference Article
  • Citations2

AutoSFX: Automatic Sound Effect Generation for Videos

  • Oct 28, 2024
  • Yujia Wang +2
  • Conference Article

An algorithm of image retrieval based on content similarity

  • Jun 01, 2011
  • Zhiyong Xiong +1
  • Conference Article
  • Citations3

Object detection based on LHOG feature matching

  • Jul 19, 2013
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Xiaoyuan Huang +1
  • Conference Article
  • Citations13

English

  • Jan 01, 2008
  • Norihiko Kawai +2
  • Conference Article
  • Citations28

Trilingual Semantic Embeddings of Visually Grounded Speech with Self-Attention Mechanisms

  • May 01, 2020
  • Yasunori Ohishi +5
  • Conference Article

Polyline-driven stop sign detection

  • Dec 01, 2016
  • Qi Li +1
  • Research Article
  • Citations6

Explicitly diverse visual question generation.

  • Apr 01, 2025
  • Neural networks : the official journal of the International Neural Network Society
  • Jiayuan Xie +4
  • Conference Article
  • Citations9

K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition

  • Jan 01, 2023
  • Kohei Uehara +1
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.