• Home
  • Search
  • PLMAS: Adaptive Sample Selection for Prompting LLMs in Knowledge-Based Visual Question Answering
  • https://doi.org/10.1145/3777476Copy DOI Icon

PLMAS: Adaptive Sample Selection for Prompting LLMs in Knowledge-Based Visual Question Answering

  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

With the rapid advancement of large-scale model technology, Visual Question Answering (VQA) — a core subfield of multimodal research — increasingly relies on these models to address complex challenges. This trend is especially evident in Knowledge-based VQA (KB-VQA), which requires integrating external knowledge. While most studies approach KB-VQA using explicit or implicit knowledge bases, recent studies employ in-context learning to guide large language models (LLMs) with implicit knowledge (e.g., PICa and Prophet). However, existing sample selection strategies for in-context learning are oversimplified and fail to adequately leverage the tacit knowledge encoded within LLMs. To address this limitation, we propose an adaptive sample selection strategy that integrates triple similarity calculations (question-image, question-caption, and question-pre-answer) and dynamically assembles the most relevant samples using weighted combinations, thereby effectively activating the large model’s implicit knowledge. To evaluate the performance of our proposed approach, we conducted experiments on benchmark datasets. Results demonstrate that our method (PLMAS) achieves state-of-the-art performance on both the OK-VQA and A-OKVQA datasets.

Similar Papers
  • Research Article
  • Citations8

Prophet: Prompting Large Language Models With Complementary Answer Heuristics for Knowledge-Based Visual Question Answering.

  • Aug 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Zhou Yu +4
  • Research Article
  • Citations41

Prompting large language model with context and pre-answer for knowledge-based VQA

  • Mar 06, 2024
  • Pattern Recognition
  • Zhongjian Hu +3
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Conference Article
  • Citations53

Adaptive Sample Selection for Robust Learning under Label Noise

  • Jan 01, 2023
  • Deep Patel +1
  • Research Article
  • Citations2

LMT++: Adaptively Collaborating LLMs With Multi-Specialized Teachers for Continual VQA in Robotic Surgical Videos.

  • Jan 01, 2025
  • IEEE transactions on medical imaging
  • Yuyang Du +8
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Conference Article
  • Citations7

How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering

  • Jan 01, 2024
  • Jinxin Liu +7
  • Research Article
  • Citations5

Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration

  • Jan 29, 2025
  • Remote Sensing
  • Fan Liu +5
  • Conference Article
  • Citations40

A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA

  • Oct 10, 2022
  • Yangyang Guo +5
  • Research Article
  • Citations2

Promoting Knowledge Base Question Answering by Directing LLMs to Generate Task-relevant Logical Forms

  • Apr 11, 2025
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Jianqi Gao +5
  • Research Article
  • Citations26

ZVQAF: Zero-shot visual question answering with feedback from large language models

  • Mar 06, 2024
  • Neurocomputing
  • Cheng Liu +3
  • Conference Article

Grounded Multi-modal Conversation for Zero-shot Visual Question Answering

  • Oct 05, 2025
  • Mohammad Reza Zarei +2
  • Research Article
  • Citations9

M4CXR: Exploring Multitask Potentials of Multimodal Large Language Models for Chest X-Ray Interpretation.

  • Oct 01, 2025
  • IEEE transactions on neural networks and learning systems
  • Jonggwon Park +4
  • Conference Article
  • Citations2

Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment

  • Jan 01, 2024
  • Proceedings of the Annual Meeting of the Association for Computational Linguistics
  • Yunxin Li +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.