• Home
  • Search
  • Asking Questions to Alleviate Object Hallucination in Large Vision-Language Models
  • Cite Icon1
  • https://doi.org/10.1109/tcsvt.2025.3618949Copy DOI Icon

Asking Questions to Alleviate Object Hallucination in Large Vision-Language Models

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Large vision language models (LVLMs) have achieved rapid development. However, just like large language models (LLMs), LVLMs face the critical challenge of hallucination, which refers to the phenomenon that the generation text containing references or descriptions of the input image is incorrect or inconsistent. The causes of hallucinations are complex and therefore difficult to avoid directly during the generation process. To alleviate hallucinations, existing studies mainly employ an instruction-tuning approach that requires model retraining with specific data. Other methods use decoding constraints to penalize specific tokens during the decoding process. These will incur expensive annotation costs and computation burden. In this paper, we propose a framework named AQAH to alleviate hallucinations without relying on manual data and large-scale parameter tuning. AQAH compares multiple generated samples to locate the hallucination factors, and then asks questions about the uncertain information. Finally, the answers to the questions are used to add auxiliary information to the prompt to correct the hallucination of LVLMs during regeneration. To facilitate this process, we constructed an automatic process that involves the training of a small model for question generation, and the agent collaboration framework including the small question generation model and large question answering foundation model. Since AQAH does not directly constrain the decoding, it will not cause a significant degradation in inference efficiency, nor force LVLMs to suffer the notorious problem of shortened text generation length. We experimentally demonstrate the effectiveness of AQAH in hallucination alleviation through the proposed “active questioning & answer verification” paradigm in various multimodal tasks such as captioning and visual question answering. Beyond the promising performance and fewer training/inference time costs against other hallucination reduction methods, our method is highly interpretable and flexible, showing great potential in improving LVLMs by exploiting small-scale models. The code is available at https://github.com/bcxbg/AQAH.

Similar Papers
  • Research Article
  • Citations5

Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration

  • Jan 29, 2025
  • Remote Sensing
  • Fan Liu +5
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Conference Article

CrossLM: A Data-Free Collaborative Fine-Tuning Framework for Large and Small Language Models

  • Jun 23, 2025
  • Yongheng Deng +5
  • Research Article
  • Citations6

Tax Intelligent Decision-Making Language Model

  • Jan 01, 2024
  • IEEE Access
  • Yan Zhong +2
  • Conference Article

Grounded Multi-modal Conversation for Zero-shot Visual Question Answering

  • Oct 05, 2025
  • Mohammad Reza Zarei +2
  • Research Article
  • Citations11

How Can IJDS Authors, Reviewers, and Editors Use (and Misuse) Generative AI?

  • Apr 01, 2023
  • INFORMS Journal on Data Science
  • Galit Shmueli +7
  • Research Article
  • Citations2

LMT++: Adaptively Collaborating LLMs With Multi-Specialized Teachers for Continual VQA in Robotic Surgical Videos.

  • Jan 01, 2025
  • IEEE transactions on medical imaging
  • Yuyang Du +8
  • Research Article
  • Citations8

Prophet: Prompting Large Language Models With Complementary Answer Heuristics for Knowledge-Based Visual Question Answering.

  • Aug 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Zhou Yu +4
  • Research Article

Analysis of the Integration Strategies of LLM and VLM Models with the Transformer Architecture

  • Mar 27, 2025
  • Journal of Computer Science and Artificial Intelligence
  • Yao Zhang
  • Research Article
  • Citations41

Prompting large language model with context and pre-answer for knowledge-based VQA

  • Mar 06, 2024
  • Pattern Recognition
  • Zhongjian Hu +3
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Book Chapter
  • Citations78

Towards Human-Like Educational Question Generation with Large Language Models

  • Jan 01, 2022
  • Zichao Wang +3
  • Research Article
  • Citations12

DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark

  • Feb 19, 2025
  • Remote Sensing
  • Haodong Li +2
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article
  • Citations26

ZVQAF: Zero-shot visual question answering with feedback from large language models

  • Mar 06, 2024
  • Neurocomputing
  • Cheng Liu +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.