Quantifying Innovation in Stroke: Large Language Model Bibliometric Analysis

The paper studies the problem of software code migration during the transition from the old version of the SAP ERP 6.0 system to modern ERP platforms, in particular SAP S/4HANA. The necessity of automation of the migration process using artificial intelligence methods and tools is substantiated. A comprehensive analysis of existing academic and industrial case studies confirmed the feasibility of using large language models for code migration. A number of advantages of the approach based on large language models are revealed, in particular, the ability to handle various migration scenarios. The key risks and limitations common in the enterprise environment are highlighted, including data security, privacy, high cost of API calls and limited knowledge of models regarding the subject area. A method is developed for selecting a large open source language model that can be safely used in the enterprise landscape, thereby mitigating the risks associated with cloud solutions. The method is based on the developed system of criteria, including: model size, context window size, license type, support for fine-tuning training, focus on code quality (measured using a reference dataset), support for model quantization and community support. The hybrid AHP-TOPSIS method is proposed for systematic ranking of candidate models. The AHP method was used to check the weights of the criteria, while TOPSIS was used to rank the models based on their proximity to the ideal solution. Based on the developed method, three large language models were selected: Qwen 2.5 Coder 14B, DeepSeek-Coder-V2 16B and Llama 3.1 8B. A comprehensive testing plan for the selected models was developed for further experimental studies. The scientific novelty of the work lies in the development of a method for selecting large language models for custom code migration tasks, which differs from the existing system of model selection criteria and the use of a hybrid AHP-TOPSIS approach for ranking candidate models. The practical value of the work is that the developed method allows you to select a model taking into account the constraints of the corporate environment and information security requirements, which will increase the level of automation in the migration of AVAP when switching to new versions of ERP systems from SAP SE. The developed method can be applied to the selection of large language models when upgrading any large computer systems developed in both open source and proprietary programming languages.

Large language models illuminate a progressive pathway to artificial intelligent healthcare assistant

Documenting Disclosure: Limited Reporting of Generative AI Usage in Radiology Research Manuscripts.

Background Large language models (LLMs) hold substantial promise for medical imaging interpretation. However, there is a lack of studies on their feasibility in handling reasoning questions associated with medical diagnosis. Purpose To investigate the viability of leveraging three publicly available LLMs to enhance consistency and diagnostic accuracy in medical imaging based on standardized reporting, with pathology as the reference standard. Materials and Methods US images of thyroid nodules with pathologic results were retrospectively collected from a tertiary referral hospital between July 2022 and December 2022 and used to evaluate malignancy diagnoses generated by three LLMs-OpenAI's ChatGPT 3.5, ChatGPT 4.0, and Google's Bard. Inter- and intra-LLM agreement of diagnosis were evaluated. Then, diagnostic performance, including accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC), was evaluated and compared for the LLMs and three interactive approaches: human reader combined with LLMs, image-to-text model combined with LLMs, and an end-to-end convolutional neural network model. Results A total of 1161 US images of thyroid nodules (498 benign, 663 malignant) from 725 patients (mean age, 42.2 years ± 14.1 [SD]; 516 women) were evaluated. ChatGPT 4.0 and Bard displayed substantial to almost perfect intra-LLM agreement (κ range, 0.65-0.86 [95% CI: 0.64, 0.86]), while ChatGPT 3.5 showed fair to substantial agreement (κ range, 0.36-0.68 [95% CI: 0.36, 0.68]). ChatGPT 4.0 had an accuracy of 78%-86% (95% CI: 76%, 88%) and sensitivity of 86%-95% (95% CI: 83%, 96%), compared with 74%-86% (95% CI: 71%, 88%) and 74%-91% (95% CI: 71%, 93%), respectively, for Bard. Moreover, with ChatGPT 4.0, the image-to-text-LLM strategy exhibited an AUC (0.83 [95% CI: 0.80, 0.85]) and accuracy (84% [95% CI: 82%, 86%]) comparable to those of the human-LLM interaction strategy with two senior readers and one junior reader and exceeding those of the human-LLM interaction strategy with one junior reader. Conclusion LLMs, particularly integrated with image-to-text approaches, show potential in enhancing diagnostic medical imaging. ChatGPT 4.0 was optimal for consistency and diagnostic accuracy when compared with Bard and ChatGPT 3.5. © RSNA, 2024 Supplemental material is available for this article.

Computer-aided diagnosis (CAD) has advanced medical image analysis, while large language models (LLMs) have shown potential in clinical applications. However, LLMs struggle to interpret medical images, which are critical for decision-making. Here we show a strategy integrating LLMs with CAD networks. The framework uses LLMs’ medical knowledge and reasoning to enhance CAD network outputs, such as diagnosis, lesion segmentation, and report generation, by summarizing information in natural language. The generated reports are of higher quality and can improve the performance of vision-based CAD models. In chest X-rays, an LLM using ChatGPT improved diagnosis performance by 16.42 percentage points compared to state-of-the-art models, while GPT-3 provided a 15.00 percentage point F1-score improvement. Our strategy allows accurate report generation and creates a patient-friendly interactive system, unlike conventional CAD systems only understood by professionals. This approach has the potential to revolutionize clinical decision-making and patient communication.

IntroductionThe increasing volume of radiological images and the associated workload of report generation necessitate efficient solutions, making artificial intelligence (AI) a crucial tool to streamline this process for radiologists. Recent years have seen a surge in research exploring AI-driven radiological report generation directly from images, particularly with the emergence of large vision language models. However, a comprehensive understanding of the current landscape, including specific limitations and the extent to which efforts move beyond abnormality detection to full textual report generation, remains unclear. This scoping review aims to systematically map the existing literature to provide an overview of the current state of AI in generating radiological reports from medical images, including the scope and limitations of existing research. To our knowledge, no prior scoping review has comprehensively mapped this landscape, especially considering recent advancements in foundation models in medicine and related AI architectures. Considering the explosive growth of related studies in recent years, a comprehensive scoping review will be significant in mapping the current research status and understanding relevant limitations.Methods and analysisThis scoping review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews guidelines to map the literature on AI generating radiological reports from medical images. We will search PubMed, Scopus and Web of Science for peer-reviewed articles (January 2016 to March 2025) using keywords related to AI, radiological reports and medical images. Original research in English focusing on AI-driven report generation from images will be included and studies without report generation or not using medical images as input will be excluded. Two independent reviewers will perform a two-stage screening. Data extraction, guided by the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist and focusing on study characteristics, AI methods, image modalities, report features, limitations and key findings, will be analysed using narrative and descriptive synthesis, with results presented in tables, figures and a narrative summary.Ethics and disseminationThis protocol describes a scoping literature review methodology that does not involve research on humans, animals or their data; therefore, no ethical approval is required. Following the review, the results will be considered for publication in a relevant peer-reviewed journal and may be shared with stakeholders through reports or summaries.

<p>Artificial intelligence (AI) methods, especially machine learning and natural language processing, are increasingly affecting health professions education (HPE), including the medical school application and selection processes, assessment, and scholarship production. The rise of large language models over the past 18 months, such as ChatGPT, has raised questions about how best to incorporate these methods into HPE. The lack of training in AI among most HPE faculty and scholars poses an important challenge in facilitating such discussions. In this commentary, the authors provide a primer on the AI methods most often used in the practice and scholarship of HPE, discuss the most pressing challenges and opportunities these tools afford, and underscore that these methods should be understood as part of the larger set of statistical tools available.Despite their ability to process huge amounts of data and their high performance completing some tasks, AI methods are only as good as the data on which they are trained. Of particular importance is that these models can perpetuate the biases that are present in those training datasets, and they can be applied in a biased manner by human users. A minimum set of expectations for the application of AI methods in HPE practice and scholarship is discussed in this commentary, including the interpretability of the models developed and the transparency needed into the use and characteristics of such methods.The rise of AI methods is affecting multiple aspects of HPE including raising questions about how best to incorporate these models into HPE practice and scholarship. In this commentary, we provide a primer on the AI methods most often used in HPE and discuss the most pressing challenges and opportunities these tools afford.</p> <p><br></p> <p><br></p>

Although traditional statistical techniques and machine learning methods have contributed significantly to genetics and, in particular, inherited disease diagnosis, they often struggle with complex, high-dimensional data, a challenge now addressed by state-of-the-art deep learning models. Large language models (LLMs), based on transformer architectures, have excelled in tasks requiring contextual comprehension of unstructured medical data. This systematic review examines the role of generative Artificial Intelligence (AI) methods in human medical genomics, focusing on the genetic research and diagnostics of both rare and common diseases. Automated keyword-based search in PubMed, bioRxiv, medRxiv, and arXiv was conducted, targeting studies on LLM applications in diagnostics and education within genetics and removing irrelevant or outdated models. A total of 195 studies were analyzed, highlighting the prospects of their applications in knowledge navigation, analysis of clinical and genetic data, and interaction with patients and medical professionals. Key findings indicate that while transformer-based models perform well across a diverse range of tasks (such as identification of tentative molecular diagnosis from clinical data or genetic variant interpretation), major challenges persist in integrating multimodal data (genomic sequences, imaging, and clinical records) into unified and clinically robust pipelines, facing limitations in generalizability and practical implementation in clinical settings. This review provides a comprehensive classification and assessment of the current capabilities and limitations of LLMs in transforming hereditary disease diagnostics and supporting genetic education, serving as a guide to navigate this rapidly evolving field, while outlining application use cases, implementation guidance, and forward-looking research directions.

We fine‐tuned and compared several encoder‐based Transformer large language models (LLM) to predict differential item functioning (DIF) from the item text. We then applied explainable artificial intelligence (XAI) methods to identify specific words associated with the DIF prediction. The data included 42,180 items designed for English language arts and mathematics summative state assessments among students in grades 3 to 11. Prediction ranged from .04 to .32 among eight focal and reference group pairs. Our findings suggest that many words associated with DIF reflect minor subdomains included in the test blueprint by design, rather than construct‐irrelevant content that may need to be removed from assessments. This may explain why qualitative reviews of DIF items often yield inconclusive results. Our approach can be used to (1) screen words associated with DIF during the item‐writing process for immediate revision to reduce preventable adverse DIF, (2) assist traditional DIF item reviews by highlighting key words, or (3) use DIF prediction as an alternative when obtaining sufficient sample size for traditional DIF analyses is impossible. Extensions of this research can enhance the assessment fairness, especially programs that lack resources to build high‐quality items, and among smaller subpopulations with insufficient sample sizes for traditional DIF analyses. See source code here .

Motivation: Classifying MRI sequences automatically is critical for developing labeled datasets for deep learning applications in medical imaging. Goal(s): The study evaluates large language models(LLMs) performance in classifying MRI sequences, comparing them to current methods like CNNs and string-matching, focusing on accuracy and interpretability. Approach: We applied a GPT-4-based LLM to classify 1490 brain MRI sequences from UCSF and compared to CNN and string-matching classifiers using sensitivity, specificity, and accuracy. Results: The LLM classifier outperformed both CNN and string-matching methods, achieving 0.83 accuracy, with high sensitivity and specificity across sequence types. Its interpretability offered additional insights, improving classification transparency and minimizing false positives. Impact: LLMs provide a more accurate and interpretable approach for MRI sequence classification, offering clinicians and researchers a more reliable tool. This could enhance research workflows, reduce manual labeling time, and allow for more robust deep learning models in medical imaging.

Medical report generation demands accurate abnormality detection and precise description generation from CT images. While large language models have shown promising results in natural language processing tasks, their application in medical imaging analysis faces challenges due to the complexity of fine-grained feature detection and the requirement for domain-specific knowledge. This paper presents a novel framework integrating large language models with specialized medical image processing techniques for fine-grained abnormality detection and natural language description generation. Our approach incorporates a multi-modal knowledge enhancement module and a hierarchical attention mechanism to bridge the gap between visual understanding and textual description. The framework employs an adapter-based architecture for efficient domain adaptation and introduces a medical knowledge-enhanced loss function to improve description accuracy. Experimental results on three public datasets demonstrate the effectiveness of our approach, achieving 94.6% detection accuracy and a BLEU-4 score of 0.421 for description generation, surpassing current state-of-the-art methods. The system shows particular strength in handling subtle abnormalities, with a 91.2% average precision in fine-grained detection tasks. Comprehensive ablation studies validate the contribution of each component, while qualitative analysis demonstrates the clinical relevance of generated descriptions. The proposed framework represents a significant advancement in automated medical image analysis, offering potential benefits for clinical workflow optimization and diagnostic support.

Background Artificial intelligence (AI) offers significant opportunities to improve the field of implementation science by supporting key activities such as evidence synthesis, contextual analysis, and decision-making to promote the adoption and sustainability of evidence-based practices. This living scoping review aims to: (1) map applications of AI in implementation research and practice; (2) identify evaluation approaches, reported outcomes, and potential risks; and (3) synthesize reported research gaps and opportunities for advancing the use of AI in implementation science. Methods This scoping review will follow the Joanna Briggs Institute (JBI) methodology and the Cochrane guidance for living systematic reviews. A living scoping review is warranted to keep up with the rapid changes in AI and its growing use in implementation science. We will include empirical studies, systematic reviews, grey literature, and policy documents that describe or evaluate applications of AI to support implementation science across the steps of the Knowledge-to-Action (KTA) Model. AI methods and models of interest include machine learning, deep learning, natural language processing, large language models, and related technologies and approaches. A search strategy will be applied to bibliographic databases (MEDLINE, Embase, CINAHL, PsycINFO, IEEE Xplore, Web of Science), relevant journals, conference proceedings, and preprint servers. Two reviewers will independently screen studies and extract data on AI characteristics, specific implementation task according to the KTA Model, evaluation methods, outcome domains, risks, and research gaps. Extracted data will be analyzed descriptively and synthesized narratively using a mapping approach aligned with the KTA Model. Discussion This living review will consolidate the evidence base on how AI is applied across the spectrum of implementation science. It will inform researchers, policymakers, and practitioners seeking to harness AI to improve the adoption, scale-up, and sustainability of evidence-based interventions, while identifying areas for methodological advancement and risk mitigation. Review registration Open Science Framework, May 2025: https://doi.org/10.17605/OSF.IO/2Q5DV

High-throughput extraction of quantitative image features, known as radiomics, has the potential to improve clinical radiology by revealing hidden features in medical images. In this review, we discuss the convergence of radiomics with artificial intelligence (AI), specifically the development of large language models (LLMs) and agentic AI models, alongside improvements in radiomics methods, standardisation, and clinical uptake pathways. Our aim is to summarise these developments, highlight current challenges, and suggest potential future directions for the widespread adoption of radiomics in clinical practice. The study conducted an extensive literature review, focusing on radiomics research, particularly investigations that examined validation frameworks, standardisation efforts, deep learning, LLMs, multi-centre studies, and the emerging library of agentic pipelines. The review compares key publications to outline common findings affecting diagnostic accuracy, prognostic performance, reproducibility, and integration into clinical workflows. Advances in AI, particularly LLMs that work with both text and images to enhance understanding, and agentic AI systems that tailor workflows to specific conditions, offer promising opportunities to improve radiomic analysis and its clinical applicability. Issues of reproducibility are being addressed through standardisation and robust validation, paving the way for broader clinical implementation. Radiomics has the potential to become an essential component of precision imaging in the coming years, though this will require resolving challenges related to reproducibility, interpretability, and seamless integration into clinical practice through cross-disciplinary collaboration. CLINICAL RELEVANCE: Radiologists and imaging scientists should anticipate the future use of radiomic AI solutions, potentially incorporating LLMs, to support diagnostic, prognostic, and therapeutic decision-making, ultimately enhancing precision medicine.

Introduction Large language models (LLMs) have the potential to realize accurate risk stratification and disease prediction by integrating multimodal data, such as electronic health records, medical images, and genomic profiles. However, in complex tasks like atherosclerosis risk estimation, the design of the prompt is critical to bring out the performance of LLMs. Several researches have demonstrated that in-context learning, where the prompt contains some cases, improves the performance of LLMs when the contained cases are carefully selected. Because manual selection usually requires enormous costs, automatic case selection has been desired. Methods This study aims to improve the effectiveness of in-context learning in LLM for atherosclerosis risk prediction by introducing a strategy for case selection. Our method involves two key components: feature selection and case selection. In the feature selection phase, we compute the mutual information between the clinical features and atherosclerosis risk using the database to extract the essential clinical features. This feature selection removes the unimportant features that disturb the performance of the case selection mechanism. Then, in the case selection phase, we select several cases from the database that are the most similar to the patient being diagnosed. This study compares three metrics to compute the similarity score: - Mahalanobis distance (trained via large margin nearest neighbor classification; LMNN [1]) - Cosine similarity (between the vector representations in KNN-augmented in-context example selection; KATE [2]) - Euclidean distance (between the prompt features) This combined strategy supplies the LLM with informative cases and thereby enhances the diagnostic performance. Results We used a fine-tuned Llama3-8B model for our target language and analyzed a dataset of 117,709 cases. Each case contained 96 features that were collected from annual health check-up data. From this dataset, 1,000 cases each were randomly selected for validation and test datasets. The remaining data were used as the database in our system. We considered a binary classification task and assigned the positive label when the Cardio-Ankle Vascular Index (CAVI) was greater than 8.0. The proportion of selected features was adjusted in 10% increments to maximize the F1 score on the validation dataset. The result shows that our proposed method, the strategic feature and case selections, significantly improved the F1 score for the test dataset by around 2% compared with zero-shot prompt and random selection (see the attached table). Among three similarity metrics, Euclidean distance yielded the highest F1 scores. We also observed that the feature selection phase with the adjusted number of selected features improved the F1 score in almost all settings. Conclusion Our strategy for feature and case selections improved the performance of in-context learning for LLM-based atherosclerosis risk prediction.

The rapid development of Multimodal Large Language Models (MLLMs) has advanced Radiology Report Generation (RRG). While much of this progress is driven by increasingly powerful Large Language Models (LLMs), the roles of both the vision encoder and the LLM remain underexplored, especially in domain-specific contexts. In this work, we systematically study how different vision encoders and LLMs affect RRG performance, analyzing the task from both vision- and languagecentric perspectives. Through extensive evaluation, we show that domain-adapted vision encoders and LLMs significantly enhance the quality and clinical relevance of generated reports. These findings offer practical guidance for building effective MLLMs in medical imaging.