• Home
  • Search
  • Context-aware chatbot using MLLMs for Cultural Heritage
  • Cite Icon23
  • https://doi.org/10.1145/3625468.3652193Copy DOI Icon

Context-aware chatbot using MLLMs for Cultural Heritage

  • Apr 15, 2024
  • Pavan Kartheek Rachabatuni +3 more
Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Multi-modal Large Language Models (MLLMs) are currently an extremely active research topic for the multimedia and computer vision communities, and show a significant impact in visual analysis and text generation tasks. MLLM's are well-versed in integrated understanding, analysis of complex data from cross modalities (i.e. text-image) and text generation with chat abilities. Almost all MLLM's, focus on alignment of image features to textual features for downstream text generation tasks includes detailed image description, visual question answering, stories and poems generation, phrase grounding, etc.. However, when focusing on visual question answering, questions that are highly relevant to the context of an image may not be answered correctly with the existing MLLM's, contrary to questions that are related to visual aspects. Moreover, generating meta data (context) for an image using present day MLLM's is hard task due to hallucinating characteristic of underlying Large Language Models (LLM's), and adequate contextual information cannot be directly derived from an image based perspective.

Loading PDF

Similar Papers
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Research Article
  • Citations2

LMT++: Adaptively Collaborating LLMs With Multi-Specialized Teachers for Continual VQA in Robotic Surgical Videos.

  • Jan 01, 2025
  • IEEE transactions on medical imaging
  • Yuyang Du +8
  • Research Article
  • Citations8

Prophet: Prompting Large Language Models With Complementary Answer Heuristics for Knowledge-Based Visual Question Answering.

  • Aug 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Zhou Yu +4
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Research Article

Enhanced Reasoning via Multimodal LLMs and Collaborative Inference

  • Jan 01, 2025
  • IEEE Transactions on Multimedia
  • Zhiquan Wen +4
  • Research Article
  • Citations41

Prompting large language model with context and pre-answer for knowledge-based VQA

  • Mar 06, 2024
  • Pattern Recognition
  • Zhongjian Hu +3
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Supplementary Content
  • Citations50

Multimodal Large Language Models in Medical Imaging: Current State and Future Directions

  • Aug 08, 2025
  • Korean Journal of Radiology
  • Yoojin Nam +14
  • Research Article
  • Citations124

Detecting and Preventing Hallucinations in Large Vision Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Anisha Gunjal +2
  • Research Article
  • Citations9

VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Ziyi Yin +7
  • Research Article
  • Citations4

Tackling Vision Language Tasks through Learning Inner Monologues

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Diji Yang +6
  • Research Article
  • Citations41

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

  • Mar 05, 2025
  • ACM Computing Surveys
  • Jiayi Kuang +9
  • Conference Article

Grounded Multi-modal Conversation for Zero-shot Visual Question Answering

  • Oct 05, 2025
  • Mohammad Reza Zarei +2
  • Research Article
  • Citations9

M4CXR: Exploring Multitask Potentials of Multimodal Large Language Models for Chest X-Ray Interpretation.

  • Oct 01, 2025
  • IEEE transactions on neural networks and learning systems
  • Jonggwon Park +4
  • Research Article
  • Citations5

SigPhi-Med: A lightweight vision-language assistant for biomedicine.

  • Jul 01, 2025
  • Journal of biomedical informatics
  • Feizhong Zhou +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.