• Home
  • Search
  • Multimodal Large Language Models in Medical Imaging: Current State and Future Directions
  • Cite Icon50
  • https://doi.org/10.3348/kjr.2025.0599Copy DOI Icon

Multimodal Large Language Models in Medical Imaging: Current State and Future Directions

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Multimodal large language models (MLLMs) are emerging as powerful tools in medicine, particularly in radiology, with the potential to serve as trusted artificial intelligence (AI) partners for clinicians. In radiology, these models integrate large language models (LLMs) with diverse multimodal data sources by combining clinical information and text with radiologic images of various modalities, ranging from 2D chest X-rays to 3D CT/MRI. Methods for achieving this multimodal integration are rapidly evolving, and the high performance of freely available LLMs may further accelerate MLLM development. Current applications of MLLMs now span automatic generation of preliminary radiology report, visual question answering, and interactive diagnostic support. Despite these promising capabilities, several significant challenges hinder widespread clinical adoption. MLLMs require access to large-scale, high-quality multimodal datasets, which are scarce in the medical domain. Risks of hallucinated findings, lack of transparency in decision-making processes, and high computational demands further complicate implementation. This review summarizes the current capabilities and limitations of MLLMs in medicine—particularly in radiology—and outlines key directions for future research. Critical areas include incorporating region-grounded reasoning to link model outputs to specific image regions, developing robust foundation models pre-trained on large-scale medical datasets, and establishing strategies for the safe and effective integration of MLLMs into clinical practice.

Similar Papers
  • Research Article
  • Citations35

Interpretable medical image Visual Question Answering via multi-modal relationship graph learning

  • Jul 20, 2024
  • Medical Image Analysis
  • Xinyue Hu +7
  • Research Article
  • Citations2

LMT++: Adaptively Collaborating LLMs With Multi-Specialized Teachers for Continual VQA in Robotic Surgical Videos.

  • Jan 01, 2025
  • IEEE transactions on medical imaging
  • Yuyang Du +8
  • Research Article
  • Citations11

How Can IJDS Authors, Reviewers, and Editors Use (and Misuse) Generative AI?

  • Apr 01, 2023
  • INFORMS Journal on Data Science
  • Galit Shmueli +7
  • Research Article
  • Citations2

Evaluating the Accuracy and Diagnostic Reasoning of Multimodal Large Language Models in Interpreting Neuroradiology Cases From RadioGraphics.

  • Jan 01, 2026
  • Korean journal of radiology
  • Pae Sun Suh +6
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Research Article
  • Citations8

Prophet: Prompting Large Language Models With Complementary Answer Heuristics for Knowledge-Based Visual Question Answering.

  • Aug 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Zhou Yu +4
  • Research Article

Research on Multimodal Large Language Models for Visual Question Answering: Advances and Challenges

  • Jan 26, 2026
  • Applied and Computational Engineering
  • Liangyu Mei
  • Research Article

A review of multimodal large language models for smart grids management and control

  • Jun 01, 2026
  • Sustainable Energy, Grids and Networks
  • G Cirrincione +6
  • Research Article
  • Citations25

Multi-modal large language models in radiology: principles, applications, and potential.

  • Dec 02, 2024
  • Abdominal radiology (New York)
  • Yiqiu Shen +6
  • Research Article

Enhanced Reasoning via Multimodal LLMs and Collaborative Inference

  • Jan 01, 2025
  • IEEE Transactions on Multimedia
  • Zhiquan Wen +4
  • Research Article
  • Citations17

A Review of the Opportunities and Challenges with Large Language Models in Radiology: The Road Ahead.

  • Nov 21, 2024
  • AJNR. American journal of neuroradiology
  • Neetu Soni +4
  • Research Article
  • Citations41

Prompting large language model with context and pre-answer for knowledge-based VQA

  • Mar 06, 2024
  • Pattern Recognition
  • Zhongjian Hu +3
  • Research Article
  • Citations124

Detecting and Preventing Hallucinations in Large Vision Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Anisha Gunjal +2
  • Research Article
  • Citations6

Deep Learning and Methods Based on Large Language Models Applied to Stellar Light Curve Classification

  • Jan 01, 2025
  • Intelligent Computing
  • Yu-Yang Li +6
  • Research Article
  • Citations1

Evaluating Multiple Input Strategies of Large Language Models for Gallbladder Polyps on Ultrasound: Comparative Study

  • Dec 23, 2025
  • JMIR Medical Informatics
  • Lin Jiang +8
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.