Enhancing Radiology Report Generation via Multi-Phased Supervision.

Development and evaluation of a retrieval-augmented large language model framework for enhancing endodontic education.

Inspired by the tremendous success of Large Language Models (LLMs), existing Radiology report generation methods attempt to leverage large models to achieve better performance. They usually adopt a Transformer to extract the visual features of a given X-ray image, and then, feed them into the LLM for text generation. How to extract more effective information for the LLMs to help them improve final results is an urgent problem that needs to be solved. Additionally, the use of visual Transformer models also brings high computational complexity. To address these issues, this paper proposes a novel context-guided efficient radiology report generation framework. Specifically, we introduce the Mamba as the vision backbone with linear complexity, and the performance obtained is comparable to that of the strong Transformer model. More importantly, we perform context retrieval from the training set for samples within each mini-batch during the training phase, utilizing both positively and negatively related samples to enhance feature representation and discriminative learning. Subsequently, we feed the vision tokens, context information, and prompt statements to invoke the LLM for generating high-quality medical reports. Extensive experiments on three X-ray report generation datasets (i.e., IU X-Ray, MIMIC-CXR, CheXpert Plus) fully validated the effectiveness of our proposed model. The source code is available at https://github.com/Event-AHU/Medical_ Image_Analysis.

Radiology report generation (RRG) produces detailed textual descriptions for radiographs, serving as a crucial task for medical analysis and diagnosis. Most existing RRG approaches naturally follow the multimodal text generation paradigm, where autoregressive models are utilized to perform token-by-token report generation and thus are potentially risky in generating invalid content while being limited in low information processing speed. Although advanced architectures, such as pre-trained models and large language models (LLMs), are applied for RRG and achieve good performance, they still face the aforementioned risk and speed limitation, especially that LLMs may introduce hallucinations. Consider that radiology reports are highly patternized, sentences in them convey specific meanings independently and are frequently reused, we propose a new extractive radiograph reporting (ERR) workflow and design a dedicated framework that efficiently and accurately extracts appropriate sentences from existing radiological cases for report generation. Our approach employs a memory module to store important medical information and enhance the encoding for input radiograph with better cross-modal representations, which are used to match sentences for the extraction process. We conducted experiments on two widely used benchmark datasets, with the results demonstrating that our approach outperforms strong baselines and achieves comparable results with existing state-of-the-art generative models. Analyses further confirm that our ERR approach not only produces reports with reliable content but also ensures high training and inference efficiency.

The rapid development of Multimodal Large Language Models (MLLMs) has advanced Radiology Report Generation (RRG). While much of this progress is driven by increasingly powerful Large Language Models (LLMs), the roles of both the vision encoder and the LLM remain underexplored, especially in domain-specific contexts. In this work, we systematically study how different vision encoders and LLMs affect RRG performance, analyzing the task from both vision- and languagecentric perspectives. Through extensive evaluation, we show that domain-adapted vision encoders and LLMs significantly enhance the quality and clinical relevance of generated reports. These findings offer practical guidance for building effective MLLMs in medical imaging.

Large language models (LLMs) have made a significant impact on the fields of general artificial intelligence. General purpose LLMs exhibit strong logic and reasoning skills and general world knowledge but can sometimes generate misleading results when prompted on specific subject areas. LLMs trained with domain-specific knowledge can reduce the generation of misleading information (i.e. hallucinations) and enhance the precision of LLMs in specialized contexts. Training new LLMs on specific corpora however can be resource intensive. Here we explored the use of a retrieval-augmented generation (RAG) model which we tested on literature specific to a biomedical research area. OpenAI's GPT-3.5, GPT-4, Microsoft's Prometheus, and a custom RAG model were used to answer 19 questions pertaining to diffuse large B-cell lymphoma (DLBCL) disease biology and treatment. Eight independent reviewers assessed LLM responses based on accuracy, relevance, and readability, rating responses on a 3-point scale for each category. These scores were then used to compare LLM performance. The performance of the LLMs varied across scoring categories. On accuracy and relevance, the RAG model outperformed other models with higher scores on average and the most top scores across questions. GPT-4 was more comparable to the RAG model on relevance versus accuracy. By the same measures, GPT-4 and GPT-3.5 had the highest scores for readability of answers when compared to the other LLMs. GPT-4 and 3.5 also had more answers with hallucinations than the other LLMs, due to non-existent references and inaccurate responses to clinical questions. Our findings suggest that an oncology research-focused RAG model may outperform general-purpose LLMs in accuracy and relevance when answering subject-related questions. This framework can be tailored to Q&A in other subject areas. Further research will help understand the impact of LLM architectures, RAG methodologies, and prompting techniques in answering questions across different subject areas.

In response to the worldwide COVID-19 pandemic, advanced automated technologies have emerged as valuable tools to aid healthcare professionals in managing an increased workload by improving radiology report generation and prognostic analysis. This study proposes a Multi-modality Regional Alignment Network (MRANet), an explainable model for radiology report generation and survival prediction that focuses on high-risk regions. By learning spatial correlation in the detector, MRANet visually grounds region-specific descriptions, providing robust anatomical regions with a completion strategy. The visual features of each region are embedded using a novel survival attention mechanism, offering spatially and risk-aware features for sentence encoding while maintaining global coherence across tasks. A cross-domain LLMs-Alignment is employed to enhance the image-to-text transfer process, resulting in sentences rich with clinical detail and improved explainability for radiologists. Multi-center experiments validate the overall performance and each module's composition within the model, encouraging further advancements in radiology report generation research emphasizing clinical interpretation and trustworthiness in AI models applied to medical studies.

It is becoming increasingly important to evaluate the effectiveness of large language models (LLMs) and query-assisted platforms like Google and ChatGPT in providing clinically relevant and accurate information to patient-initiated inquiries. Limited studies have characterized the performance of these platforms on semi-elective orthopedic trauma surgeries. This study evaluates the function of both interactive online models on frequently queried patient searches on ankle and clavicle fracture operations. An inquiry set consisting of ten prevalent patient questions for each fracture type was extracted using Google and ChatGPT. Responses, alongside their source attributions, were curated for clinical relevance and accuracy. This assessment was conducted in a double-blind fashion. Grading was completed by two researchers with academic and clinical experience in orthopedic trauma (D.E.P, H.F.B), with oversight and validation provided by a board-certified orthopedic trauma surgeon (A.N.M). Descriptive and comparative statistics were used to understand the dataset. ChatGPT performed better than Google in terms of clinical relevance (p = 0.001) and accuracy (p = 0.004) in ankle fracture patient queries. Clavicle fracture queries showed a significant difference favoring ChatGPT for accuracy (p = 0.04), though relevance did not reach statistical significance (p = 0.06). When answer ground-truth was analyzed, GPT outperformed Google by providing more academic sources (p < 0.05). Large Language Models outperformed traditionally used online platforms in delivering clinically precise and contextually relevant information on semi-elective, common orthopedic trauma surgeries. The ability of LLMs to synthesize responses from credible medical sources significantly diminishes the variability and potential inaccuracies inherent in conventional web searches. These insights strongly suggest that LLMs could play a pivotal role in enhancing patient engagement and comprehension in trauma care, meriting further exploration of their integration within clinical frameworks. Therapeutic Level III.

ObjectivesThis study evaluated the efficacy of integrating a retrieval-augmented generation (RAG) model and a large language model (LLM) to improve the accuracy of drug name mapping across international vocabularies.MethodsDrug ingredient names were translated into English using the Japanese Accepted Names for Pharmaceuticals. Drug concepts were extracted from the standard vocabulary of OHDSI, and the accuracy of mappings between translated terms and RxNorm was assessed by vector similarity, using the BioBERT-generated embedded vectors as the baseline. Subsequently, we developed LLMs with RAG that distinguished the final candidates from the baseline. We assessed the efficacy of the LLM with RAG in candidate selection by comparing it with conventional methods based on vector similarity.ResultsThe evaluation metrics demonstrated the superior performance of the combined LLM + RAG over traditional vector similarity methods. Notably, the hit rates of the Mixtral 8x7b and GPT-3.5 models exceeded 90%, significantly outperforming the baseline rate of 64% across stratified groups of PO drugs, injections, and all interventions. Furthermore, the r-precision metric, which measures the alignment between model judgment and human evaluation, revealed a notable improvement in LLM performance, ranging from 41% to 50% compared to the baseline of 23%.ConclusionsIntegrating an RAG and an LLM outperformed conventional string comparison and embedding vector similarity techniques, offering a more refined approach to global drug information mapping.

Intelligent medicine is eager to automatically generate radiology reports to ease the tedious work of radiologists. Previous researches mainly focused on the text generation with encoder-decoder structure, while CNN networks for visual features ignored the long-range dependencies correlated with textual information. Besides, few studies exploit cross-modal mappings to promote radiology report generation. To alleviate the above problems, we propose a novel end-to-end radiology report generation model dubbed Self-Supervised dual-Stream Network (S3-Net). Specifically, a Dual-Stream Visual Feature Extractor (DSVFE) composed of ResNet and SwinTransformer is proposed to capture more abundant and effective visual features, where the former focuses on local response and the latter explores long-range dependencies. Then, we introduced the Fusion Alignment Module (FAM) to fuse the dual-stream visual features and facilitate alignment between visual features and text features. Furthermore, the Self-Supervised Learning with Mask(SSLM) is introduced to further enhance the visual feature representation ability. Experimental results on two mainstream radiology reporting datasets (IU X-ray and MIMIC-CXR) show that our proposed approach outperforms previous models in terms of language generation metrics.

The object of this study is the process that generates machine-readable Country-by-Country reports in XML format using large language models. This paper addresses the task related to the current dependence of the process that generates these reports on specialized software, which leads to additional financial costs. The research and analysis of the effectiveness of publicly available large language models for generating Country-by-Country reports with new data showed high results, provided that an example model of such generation was prompted. Three large language models out of nine studied yielded results close to ideal (obtained by manual preparation or specialized systems), namely 96 points out of 100 according to the devised evaluation methodology. Four other studied models demonstrated slightly lower efficiency, but their level is also sufficient for practical use. At the same time, the resulting average cost of generating one report (US cents 4.2) is significantly lower than in the case of using specialized systems. Regarding the effectiveness of general-purpose large language models for generating Country-by-Country reports in the absence of a generation example, it is currently insufficient for practical use. In this case, all of the models studied showed results close to 0 points, i.e., completely incorrect reports were obtained. Such results are attributed to the insufficient amount of sample data during training of publicly available models. Thus, publicly available large language models could in practice replace specialized software systems designed to generate Country-by-Country reports in XML format, at least in the case of generating new reports

This study aimed to evaluate the performance of four large language models (LLMs)-ChatGPT, Gemini, Copilot, and Claude-in responding to upper eyelid blepharoplasty-related questions, focusing on medical accuracy, clinical relevance, response length, and readability. A set of queries regarding upper eyelid blepharoplasty, covering six categories (anatomy, surgical procedure, additional intraoperative procedures, postoperative monitoring, follow-up, and postoperative complications) were posed to each LLM. An identical prompt establishing clinical context was provided before each question. Responses were evaluated by three ophthalmologists using a 5-point Likert scale for medical accuracy and a 3-point Likert scale for clinical relevance. The length of the responses was assessed. Readability was also evaluated using the Flesch Reading Ease Score, Flesch-Kincaid Grade Level, Coleman-Liau Index, Gunning Fog Index, and Simple Measure of Gobbledygook grade. A total of 30 standardized questions were presented to each LLM. None of the responses from any LLM received a score of 1 regarding medical accuracy for any question. ChatGPT achieved an 80% 'highly accurate' response rate, followed by Claude (60%), Gemini (40%), and Copilot (20%). None of the responses from ChatGPT and Claude received a score of1 regarding clinical relevance, whereas 10% of Gemini's responses and 26.7% of Copilot's responses received a score of 1. ChatGPT also provided the most clinically 'relevant' responses (86.7%), outperforming the other LLMs. Copilot generated the shortest responses, while ChatGPT generated the longest. Readability analyses revealed that all responses required advanced reading skills at a 'college graduate' level or higher, with Copilot's responses being the most complex. ChatGPT demonstrated superior performance in both medical accuracy and clinical relevance among evaluated LLMs regarding upper eyelid blepharoplasty, particularly excelling in postoperative monitoring and follow-up categories. While all models generated complex texts requiring advanced literacy, ChatGPT's detailed responses offer valuable guidance for ophthalmologists managing upper eyelid blepharoplasty cases. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Aim: Large language models (LLMs) are increasingly integrated into medical education and decision support systems. However, their capabilities in acute care settings, such as emergency rheumatology, require systematic evaluation. This study aimed to compare the educational performance of four LLMs ChatGPT-4o, DeepSeek v3.2, Gemini 2.5 Pro, and Perplexity Academic across five domains: clinical accuracy, safety, diagnostic reasoning, realism, and use of evidence. Materials and Methods: Each model generated responses to 20 standardized emergency rheumatology scenarios. Two board-certified rheumatologists independently evaluated the outputs using a 10-point scoring rubric. Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC). Differences among models were assessed using Friedman tests with post-hoc Wilcoxon signed-rank tests corrected for multiple comparisons. A Likert scale was also used to assess scenario complexity and educational utility. Results: ChatGPT-4o and DeepSeek v3.2 achieved the highest average total scores (mean: 7.75 each), significantly outperforming Perplexity Academic (mean: 6.70; p &lt; 0.001). Although both models showed higher mean scores than Gemini 2.5 Pro (p ≈ 0.03), these differences were not statistically significant after correction. DeepSeek v3.2 showed slightly greater performance consistency. ICC analyses confirmed high inter-rater agreement across all domains (ICC&gt;0.80). Conclusion: ChatGPT-4o and DeepSeek v3.2 demonstrated superior clinical reasoning, safety, and educational utility in emergency rheumatology scenarios. These findings support their potential role as adjunctive tools in medical training, provided expert oversight and validation mechanisms are in place.

Objective: To compare the diagnostic accuracy of two advanced large language models (LLMs), ChatGPT-o1 and DeepSeek-V3, in expert-validated simulated otorhinolaryngology cases, and to assess subspecialty-specific performance and inter-rater agreement relative to human specialists.Methods: A cross-sectional diagnostic accuracy study was conducted using 70 expert-validated clinical vignettes across five ENT subspecialties. Two academic otolaryngologists and two LLMs independently evaluated each case. All LLMs operated in deterministic mode (temperature = 0) with standardized single-pass prompting in isolated sessions. Diagnostic accuracy, inter-rater agreement (Cohen’s κ), and subspecialty-specific performance were analyzed. A post hoc power analysis (Cohen’s h = 0.22; α = 0.05) assessed the ability to detect moderate effect sizes.Results: Both LLMs achieved a diagnostic accuracy of 90.0% (63/70), with no significant difference between them (p = 1.00) and substantial inter-model agreement (κ = 0.68). Human evaluators achieved accuracies of 97.1% and 92.9%, with fair inter-rater agreement (κ = 0.26). Subspecialty performance was highest in otology and pediatric ENT (100%) and rhinology (92.3%), with greater variability observed in laryngology and head and neck surgery. Shared error patterns included overestimation of malignancy in high-risk patients. Post hoc power analysis demonstrated 78% power to detect moderate differences.Conclusion: In controlled, vignette-based evaluations, ChatGPT-o1 and DeepSeek-V3 demonstrated diagnostic accuracy approaching expert-level performance across simulated ENT scenarios, with strong inter-model agreement and subspecialty-dependent variability. These findings highlight the potential of LLMs as diagnostic decision-support tools while underscoring the need for multimodal and real-world validation before clinical implementation.

No systematic review has previously examined the application of large language models (LLMs) for generating impressions from radiology report findings. This study systematically reviews the performance of LLMs on this task and their associated evaluation methodologies. A search of seven electronic databases on 7 August 2025 identified 15 eligible papers (average quality score: 71.4%). These articles evaluated 35 LLMs, including 21 base models. The reported performance ranges were as follows: Recall-Oriented Understudy for Gisting Evaluation (ROUGE)-1, 35.9% (Generative Pre-Trained Transformer (GPT)-4) to 69.7% (Baichuan2-13B); ROUGE-2, 13.4% (Large Language Model Meta AI (Llama)) to 52.4% (Baichuan2-13B); and ROUGE-L, 16.5% (Chat General Language Model–Medical (ChatGLM-Med)) to 63.8% (finetuned Text-to-Text Transfer Transformer (T5)). The finetuned T5 consistently demonstrated high performance, based on Bidirectional Encoder Representations from Transformers Score (BERTScore): 89.2%; BiLingual Evaluation Understudy (BLEU)-1: 65.2%; BLEU-2: 57.9%; BLEU-3: 52.5%; BLEU-4: 48.3%; Metric for Evaluation of Translation with Explicit ORdering (METEOR): 38.1%; ROUGE-1: 59.9%; ROUGE-2: 50.9%; ROUGE-L: 63.8%; and subjective metrics (clinical usability: 4.5/5.0; completeness: 4.3/5.0; conciseness: 4.3/5.0; fluency: 4.4/5.0). These results, based on 132,043 computed tomography, echocardiography, magnetic resonance imaging, and X-ray reports, indicate its strong clinical potential for assisting radiologists in impression generation through supervised finetuning rather than prompting techniques used in closed-source LLMs.

MP24-02 EVALUATING THE PERFORMANCE OF DIFFERENT LARGE LANGUAGE MODELS ON HEALTH CONSULTATION AND PATIENT EDUCATION IN UROLITHIASIS