• Home
  • Search
  • Can Pre-training help VQA with Lexical Variations?
  • Cite Icon2
  • https://doi.org/10.18653/v1/2020.findings-emnlp.257Copy DOI Icon

Can Pre-training help VQA with Lexical Variations?

  • Jan 1, 2020
  • Shailza Jolly +1 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Rephrasings or paraphrases are sentences with similar meanings expressed in different ways. Visual Question Answering (VQA) models are closing the gap with the oracle performance for datasets like VQA2.0. However, these models fail to perform well on rephrasings of a question, which raises some important questions like Are these models robust towards linguistic variations? Is it the architecture or the dataset that we need to optimize? In this paper, we analyzed VQA models in the space of paraphrasing. We explored the role of language & cross-modal pre-training to investigate the robustness of VQA models towards lexical variations. Our experiments find that pre-trained language encoders generate efficient representations of question rephrasings, which help VQA models correctly infer these samples. We empirically determine why pre-training language encoders improve lexical robustness. Finally, we observe that although pre-training all VQA components obtain state-of-the-art results on the VQA-Rephrasings dataset, it still fails to completely close the performance gap between original and rephrasing validation splits.

Loading PDF

Similar Papers
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Research Article
  • Citations1

Adversarial Sample Synthesis for Visual Question Answering

  • Nov 21, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Chuanhao Li +4
  • Conference Article
  • Citations668

Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering

  • Jun 01, 2018
  • Aishwarya Agrawal +3
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • PDF
  • Research Article
  • Citations19

Advancing surgical VQA with scene graph knowledge

  • May 23, 2024
  • International Journal of Computer Assisted Radiology and Surgery
  • Kun Yuan +5
  • Conference Article

Estimation of Visual Contents from Human Brain Signals via VQA Based on Brain-Specific Attention

  • Jun 04, 2023
  • Ryo Shichida +4
  • Conference Article
  • Citations7

Learning to Ask Informative Sub-Questions for Visual Question Answering

  • Jun 01, 2022
  • Kohei Uehara +2
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • PDF
  • Research Article
  • Citations5

Visual Question Answering Models for Zero-Shot Pedestrian Attribute Recognition: A Comparative Study

  • Jun 28, 2024
  • SN Computer Science
  • Modesto Castrillón-Santana +5
  • Conference Article
  • Citations90

Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects

  • Jan 01, 2019
  • Gabriel Grand +1
  • Research Article
  • Citations33

Causal Inference with Knowledge Distilling and Curriculum Learning for Unbiased VQA

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yonghua Pan +3
  • Conference Article
  • Citations15

Feature Enhancement in Attention for Visual Question Answering

  • Jul 01, 2018
  • Yuetan Lin +3
  • Research Article
  • Citations8

Zero-Shot Cross-Lingual Knowledge Transfer in VQA via Multimodal Distillation

  • Aug 01, 2025
  • IEEE Transactions on Computational Social Systems
  • Yu Weng +6
  • Conference Article
  • Citations47

Quantifying and Alleviating the Language Prior Problem in Visual Question Answering

  • Jul 18, 2019
  • Yangyang Guo +5
  • Video Transcripts

Question Agnostic Attention for Visual Question Answering

  • Dec 29, 2020
  • Underline Science Inc.
  • Moshiur Farazi
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.