• Home
  • Search
  • Look, Read and Ask: Learning to Ask Questions by Reading Text in Images
  • Cite Icon8
  • https://doi.org/10.1007/978-3-030-86549-8_22Copy DOI Icon

Look, Read and Ask: Learning to Ask Questions by Reading Text in Images

  • Jan 1, 2021
  • Soumya Jahagirdar +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We present a novel problem of text-based visual question generation or TextVQG in short. Given the recent growing interest of the document image analysis community in combining text understanding with conversational artificial intelligence, e.g., text-based visual question answering, TextVQG becomes an important task. TextVQG aims to generate a natural language question for a given input image and an automatically extracted text also known as OCR token from it such that the OCR token is an answer to the generated question. TextVQG is an essential ability for a conversational agent. However, it is challenging as it requires an in-depth understanding of the scene and the ability to semantically bridge the visual content with the text present in the image. To address TextVQG, we present an OCR-consistent visual question generation model that Looks into the visual content, Reads the scene text, and Asks a relevant and meaningful natural language question. We refer to our proposed model as OLRA. We perform an extensive evaluation of OLRA on two public benchmarks and compare them against baselines. Our model – OLRA automatically generates questions similar to the public text-based visual question answering datasets that were curated manually. Moreover, we ‘significantly’ outperform baseline approaches on the performance measures popularly used in text generation literature.

Similar Papers
  • Research Article
  • Citations20

Visual Question Generation From Remote Sensing Images

  • Jan 01, 2023
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Laila Bashmal +5
  • Research Article

An approach to medical visual question generation based on question categories

  • Oct 16, 2025
  • Journal of the Chinese Institute of Engineers
  • N Sheerin Sitara +3
  • PDF
  • Research Article
  • Citations8

Goal-Driven Visual Question Generation from Radiology Images

  • Aug 20, 2021
  • Information
  • Mourad Sarrouti +2
  • Conference Article
  • Citations85

Automatic Generation of Grounded Visual Questions

  • Aug 01, 2017
  • Shijie Zhang +4
  • Conference Article
  • Citations7

Generating Natural Questions from Images for Multimodal Assistants

  • Jun 06, 2021
  • Alkesh Patel +4
  • Research Article
  • Citations22

Visual Question Generation

  • May 28, 2020
  • ACM Computing Surveys
  • Charulata Patil +1
  • Research Article
  • Citations18

Robust visual question answering via polarity enhancement and contrast

  • Nov 01, 2024
  • Neural Networks
  • Dahe Peng +1
  • PDF
  • Conference Article
  • Citations16

Visual Question Generation from Radiology Images

  • Jan 01, 2020
  • Mourad Sarrouti +2
  • Research Article
  • Citations18

VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage

  • Jul 12, 2023
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Federico Becattini +6
  • Conference Article
  • Citations4

UAV-VQG: Visual Question Generation Framework on UAV Images

  • Dec 15, 2021
  • Argho Sarkar +1
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Research Article
  • Citations4

Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference.

  • Dec 01, 2024
  • IEEE transactions on pattern analysis and machine intelligence
  • Kai Shen +6
  • Conference Article
  • Citations2

Estimating Viewed Images with Natural Language Question Answering from fMRI Data

  • Mar 01, 2020
  • Saya Takada +3
  • Conference Article
  • Citations59

Visual Question Answering as Reading Comprehension

  • Jun 01, 2019
  • Hui Li +3
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.