• Home
  • Search
  • Cascade Reasoning Network for Text-based Visual Question Answering
  • Cite Icon59
  • https://doi.org/10.1145/3394171.3413924Copy DOI Icon

Cascade Reasoning Network for Text-based Visual Question Answering

  • Oct 12, 2020
  • Fen Liu +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We study the problem of text-based visual question answering (T-VQA) in this paper. Unlike general visual question answering (VQA) which only builds connections between questions and visual contents, T-VQA requires reading and reasoning over both texts and visual concepts that appear in images. Challenges in T-VQA mainly lie in three aspects: 1) It is difficult to understand the complex logic in questions and extract specific useful information from rich image contents to answer them; 2) The text-related questions are also related to visual concepts, but it is difficult to capture cross-modal relationships between the texts and the visual concepts; 3) If the OCR (optical character recognition) system fails to detect the target text, the training will be very difficult. To address these issues, we propose a novel Cascade Reasoning Network (CRN) that consists of a progressive attention module (PAM) and a multimodal reasoning graph (MRG) module. Specifically, the PAM regards the multimodal information fusion operation as a stepwise encoding process and uses the previous attention results to guide the next fusion process. The MRG aims to explicitly model the connections and interactions between texts and visual concepts. To alleviate the dependence on the OCR system, we introduce an auxiliary task to train the model with accurate supervision signals, thereby enhancing the reasoning ability of the model in question answering. Extensive experiments on three popular T-VQA datasets demonstrate the effectiveness of our method compared with SOTA methods. The source code is available at https://github.com/guanghuixu/CRN_tvqa.

Similar Papers
  • Book Chapter
  • Citations8

Soft Computing Techniques for Optical Character Recognition Systems

  • Dec 24, 2016
  • Arindam Chaudhuri +3
  • Book Chapter
  • Citations5

JPEG for Arabic Handwritten Character Recognition: Add a Dimension of Application

  • Oct 01, 2008
  • Abdurazzag Ali +1
  • Conference Article

Determination of optimal features database for OCR of printed Telugu text

  • Dec 01, 2015
  • C Vasantha Lakshmi +2
  • Conference Article
  • Citations45

OmniPage vs. Sakhr: paired model evaluation of two Arabic OCR products

  • Jan 07, 1999
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Tapas Kanungo +2
  • Conference Article
  • Citations1

Generation of training database using a noise model for OCR systems

  • Feb 20, 2012
  • Do Yen +6
  • Conference Article
  • Citations1

<title>Preliminary evaluation of histogram-based binarization algorithms</title>

  • Mar 30, 1995
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Junichi Kanai +1
  • Research Article
  • Citations33

Optical Character Recognition System for Nastalique Urdu-Like Script Languages Using Supervised Learning

  • Sep 01, 2019
  • International Journal of Pattern Recognition and Artificial Intelligence
  • S S R Rizvi +3
  • Conference Article
  • Citations8

Developing Automated Optical Character Recognition System Using Machine Learning Algorithm to Solve Payment Verification Issues

  • Oct 25, 2021
  • Michael Siek +1
  • Conference Article
  • Citations3

End-to-End Trainable Thai OCR System Using Hidden Markov Models

  • Sep 01, 2008
  • Kriste Krstovski +3
  • Conference Article
  • Citations19

Adaptive image restoration of text images that contain touching or broken characters

  • Nov 19, 2002
  • P Stubberud +2
  • Research Article
  • Citations24

OCR in Indian Scripts: A Survey

  • May 01, 2005
  • IETE Technical Review
  • Peeta Basa Pati +1
  • Conference Article
  • Citations8

Retrieving and combining repeated passages to improve OCR

  • Jun 19, 2017
  • Shaobin Xu +1
  • Conference Article
  • Citations11

Retrieving and Combining Repeated Passages to Improve OCR

  • Jun 01, 2017
  • Shaobin Xu +1
  • Conference Article
  • Citations11

An Open Vocabulary OCR System with Hybrid Word-Subword Language Models

  • Nov 01, 2017
  • Meng Cai +6
  • Conference Article
  • Citations1

<title>Identification and correction of rejection and substitution errors in optical character recognition systems</title>

  • Apr 14, 1993
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Glenn S Himes +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.