• Home
  • Search
  • Detecting and Preventing Hallucinations in Large Vision Language Models
  • Cite Icon124
  • https://doi.org/10.1609/aaai.v38i16.29771Copy DOI Icon

Detecting and Preventing Hallucinations in Large Vision Language Models

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Instruction tuned Large Vision Language Models (LVLMs) have significantly advanced in generalizing across a diverse set of multi-modal tasks, especially for Visual Question Answering (VQA). However, generating detailed responses that are visually grounded is still a challenging task for these models. We find that even the current state-of-the-art LVLMs (InstructBLIP) still contain a staggering 30 percent of the hallucinatory text in the form of non-existent objects, unfaithful descriptions, and inaccurate relationships. To address this, we introduce M-HalDetect, a Multimodal Hallucination Detection Dataset that can be used to train and benchmark models for hallucination detection and prevention. M-HalDetect consists of 16k fine-grained annotations on VQA examples, making it the first comprehensive multi-modal hallucination detection dataset for detailed image descriptions. Unlike previous work that only consider object hallucination, we additionally annotate both entity descriptions and relationships that are unfaithful. To demonstrate the potential of this dataset for hallucination prevention, we optimize InstructBLIP through our novel Fine-grained Direct Preference Optimization (FDPO). We also train fine-grained multi-modal reward models from InstructBLIP and evaluate their effectiveness with best-of-n rejection sampling (RS). We perform human evaluation on both FDPO and rejection sampling, and find that they reduce hallucination rates in InstructBLIP by 41% and 55% respectively. We also find that our reward model generalizes to other multi-modal models, reducing hallucinations in LLaVA and mPLUG-OWL by 15% and 57% respectively, and has strong correlation with human evaluated accuracy scores. The dataset is available at https://github.com/hendryx-scale/mhal-detect.

Similar Papers
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Research Article
  • Citations2

LMT++: Adaptively Collaborating LLMs With Multi-Specialized Teachers for Continual VQA in Robotic Surgical Videos.

  • Jan 01, 2025
  • IEEE transactions on medical imaging
  • Yuyang Du +8
  • Research Article
  • Citations8

Prophet: Prompting Large Language Models With Complementary Answer Heuristics for Knowledge-Based Visual Question Answering.

  • Aug 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Zhou Yu +4
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article
  • Citations9

VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Ziyi Yin +7
  • Supplementary Content
  • Citations50

Multimodal Large Language Models in Medical Imaging: Current State and Future Directions

  • Aug 08, 2025
  • Korean Journal of Radiology
  • Yoojin Nam +14
  • Research Article

Comparative Performance of Seven Mainstream Large Language Models on the 2022 American College of Radiology Diagnostic Imaging In-Training Examination.

  • Apr 01, 2026
  • Cureus
  • Kian A Huang +6
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Research Article

Enhanced Reasoning via Multimodal LLMs and Collaborative Inference

  • Jan 01, 2025
  • IEEE Transactions on Multimedia
  • Zhiquan Wen +4
  • Research Article
  • Citations41

Prompting large language model with context and pre-answer for knowledge-based VQA

  • Mar 06, 2024
  • Pattern Recognition
  • Zhongjian Hu +3
  • Research Article

Chain-of-Cognition Through Multi-Agent Collaboration Against Cross-Modal Cognitive Interference

  • Jan 01, 2026
  • IEEE Transactions on Audio, Speech and Language Processing
  • Liqi He +4
  • Research Article
  • Citations2

Large Language Model Adaptation Strategies in Speech-Based Cognitive Screening: Systematic Evaluation.

  • Mar 26, 2026
  • JMIR AI
  • Fatemeh Taherinezhad +8
  • PDF
  • Conference Article
  • Citations23

Context-aware chatbot using MLLMs for Cultural Heritage

  • Apr 15, 2024
  • Pavan Kartheek Rachabatuni +3
  • Research Article
  • Citations41

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

  • Mar 05, 2025
  • ACM Computing Surveys
  • Jiayi Kuang +9
  • Research Article

Research on Multimodal Large Language Models for Visual Question Answering: Advances and Challenges

  • Jan 26, 2026
  • Applied and Computational Engineering
  • Liangyu Mei
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.