Explainable AI for Transparent MRI Segmentation: Deep Learning and Visual Attribution in Clinical Decision Support

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both 'traditional' machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support. This article is protected by copyright. All rights reserved.

Background: The implementation of the Indonesian Nursing Diagnosis Standards (IDHS) in clinical practice continues to encounter obstacles, including limited time for assessment, documentation burden, and inconsistency in diagnostic interpretation among nurses. Advancements in artificial intelligence (AI) offer opportunities to design clinical decision support systems that may enhance the accuracy, uniformity, and efficiency of nursing diagnoses. Objective: This study aimed to explore nurses’ experiences, challenges, and expectations regarding the use of IDHS and to assess the initial feasibility of developing an AI-based system to support standardized nursing diagnoses. Methods: A descriptive qualitative design was employed using in-depth interviews with five nurses from diverse clinical settings, including emergency, critical care, inpatient, outpatient, and community services. Thematic analysis was conducted to identify perceptions and needs related to IDHS and AI. Content validity of the preliminary system design was evaluated by three expert validators using the Item-Level Content Validity Index (I-CVI) and the Scale-Level Content Validity Index/Average (S-CVI/Ave). Results: Five major themes emerged: (1) varied experiences in formulating nursing diagnoses, (2) inconsistent use of IDHS in practice, (3) challenges related to documentation and diagnostic interpretation, (4) positive attitudes toward AI integration in clinical workflows, and (5) the need for features such as automated diagnostic suggestions, clarification of clinical criteria, and integration with electronic medical records. Content validity testing demonstrated strong agreement among experts, with I-CVI values ranging from 0.80 to 1.00 and an S-CVI/Ave of 0.92. Conclusion: The study indicates that integrating AI into the IDHS-based diagnostic process holds substantial potential to improve diagnostic quality, standardization, and clinical efficiency. The content validity results support further development and prototype testing in subsequent research phases

Chapter 15 - Clinical decision support systems to improve the diagnosis and management of respiratory diseases

SP0042 Artificial Intelligence in Medicine

By incorporating the colored MRI identification synthesis into the MRI segmentation model with transfer learning AI Y-Net, this study clearly shows the high potential of a multidisciplinary system-level study for diagnoses. This way, such a system can provide integrity of the goal without compromising the quality of each one and saving time consumption. Another alternative to such integration is to be used for enhancement and segmentation that is accurate and robust to the variabilities in scanner and acquisition protocols. System Level Simulator is the deep learning based on Kearse AI deep learning network specified to Y-VGG16 net results of outstanding performance in medical image segmentation. Based on the literature, there are different AI models for the diagnosis system, which are different of what is proposed in this paper. A partial-frozen network is applied to the U-net to compare results between different fine-tuning FT strategies. The network operation is also evaluated depending on the dataset size, showing the importance of the combination of dataset, TL and data augmentation (DA). Transfer learning (TL) helps us for MRI medical image segmentation deep learning with more accurate performances of the TL technique. The system hybrid the Y-Net architecture with Transfer learning to reduce the domain-shift effect in brain MRI segmentation results of the automated deep learning segmentation approach.

Clinical Decision Support Tools

Machine Learning and Artificial Intelligence (AI) more broadly have great immediate and future potential for transforming almost all aspects of medicine. However, in many applications, even outside medicine, a lack of transparency in AI applications has become increasingly problematic. This is particularly pronounced where users need to interpret the output of AI systems. Explainable AI (XAI) provides a rationale that allows users to understand why a system has produced a given output. The output can then be interpreted within a given context. One area that is in great need of XAI is that of Clinical Decision Support Systems (CDSSs). These systems support medical practitioners in their clinic decision-making and in the absence of explainability may lead to issues of under or over-reliance. Providing explanations for how recommendations are arrived at will allow practitioners to make more nuanced, and in some cases, life-saving decisions. The need for XAI in CDSS, and the medical field in general, is amplified by the need for ethical and fair decision-making and the fact that AI trained with historical data can be a reinforcement agent of historical actions and biases that should be uncovered. We performed a systematic literature review of work to-date in the application of XAI in CDSS. Tabular data processing XAI-enabled systems are the most common, while XAI-enabled CDSS for text analysis are the least common in literature. There is more interest in developers for the provision of local explanations, while there was almost a balance between post-hoc and ante-hoc explanations, as well as between model-specific and model-agnostic techniques. Studies reported benefits of the use of XAI such as the fact that it could enhance decision confidence for clinicians, or generate the hypothesis about causality, which ultimately leads to increased trustworthiness and acceptability of the system and potential for its incorporation in the clinical workflow. However, we found an overall distinct lack of application of XAI in the context of CDSS and, in particular, a lack of user studies exploring the needs of clinicians. We propose some guidelines for the implementation of XAI in CDSS and explore some opportunities, challenges, and future research needs.

Lung cancer remains the leading cause of cancer-related mortality worldwide, highlighting the urgent need for earlier detection within real-world screening and patient management pathways. Recent advances in multi-omics technologies have created new opportunities for identifying biomarkers associated with early-stage lung cancer, particularly in high-risk populations under clinical surveillance. This review systematically evaluates early diagnostic biomarkers across multiple omics layers, including genomics, epigenomics, transcriptomics, proteomics, metabolomics and microbiomics. It also summarises the application of artificial intelligence (AI), particularly machine learning and deep learning approaches, for integrating and analysing complex multi-omics datasets to support biomarker discovery and clinical decision-making. Multi-omics strategies are accelerating the identification of molecular signatures relevant to early lung cancer detection. AI-driven methods enable the extraction of latent patterns from high-dimensional data, facilitating risk stratification, diagnostic refinement, histological subtyping and treatment planning. The review highlights the clinical utility of these biomarkers and their potential incorporation into screening algorithms, as well as the development of AI-based clinical decision support systems (CDSS) aligned with real-world clinical workflows. However, major barriers to clinical translation remain, including multi-centre data heterogeneity, limited model interpretability affecting clinical trust, regulatory and cost-effectiveness challenges and insufficient validation in prospective cohorts. Emerging technologies, such as single-cell and spatial multi-omics, along with federated learning frameworks, offer promising solutions to bridge the gap between computational discovery and clinical implementation. The integration of AI and multi-omics approaches has the potential to advance risk-adapted and personalised early detection strategies for lung cancer.

Artificial intelligence (AI) technologies, including machine learning (ML), deep learning, and large language models, are increasingly applied in medical diagnostics. In rhinology, these tools are being evaluated for tasks such as image interpretation, cytology classification, and clinical decision support. To systematically evaluate the application and diagnostic accuracy of AI technologies in rhinology, with a focus on clinical utility and implementation barriers. This review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines. Seventeen full-text studies were screened based on predefined eligibility criteria, focusing on AI applications with diagnostic metrics in rhinology. Data on AI type, diagnostic task, performance outcomes, and study quality were extracted and synthesized narratively. Twelve studies met the inclusion criteria. Image-based diagnostic tools using convolutional neural networks demonstrated high accuracy (81%-99%) in nasal polyp detection, cytology classification, and computed tomography (CT) scan interpretation. ML models using patient-reported data achieved accuracies of 74.5%-85.5% for chronic rhinosinusitis prediction. Large language models like ChatGPT and Gemini were evaluated for clinical question answering, with performance exceeding 80% in some domains. Risk of bias was moderate in most primary studies, and none reported clinical integration beyond prototype stages. AI exhibits promising diagnostic accuracy across several applications in rhinology. However, significant challenges persist, including limited validation, methodological heterogeneity, and lack of clinical implementation. Future research should focus on prospective trials, explainability, and regulatory frameworks to ensure safe integration into clinical workflows.

Chest pain is the leading cause of presentations to the emergency department (ED), and the number of patients presenting with this symptom may cause overcrowding. To accelerate triage in the ED, clinical decision pathways using fast diagnostic protocols and high sensitivity cTn (hs-cTn) assays are being recommended by guidelines. However, their implementation is challenging. To develop an electronic health records (EHR)-embedded clinical decision support system (CDSS) that helps to follow protocol recommendations, improve safety and effectiveness of triage and improve guideline-adherence regarding appropriate use of fast troponin-based protocols in patients presenting with suspected non-ST-elevation acute coronary syndrome (NSTE-ACS). The CDSS was designed as a knowledge-based decision support tool based on the latest version of the 2023 ESC guideline recommended clinical decision pathways, i.e. ESC 0/1h and ESC 0/2h algorithm, and can be programmed to consider sex-specific upper limit of normal and different commercially available hs-cTn assays. Data can be entered manually or automatically. Input variables include time since symptom onset, baseline and follow-up hs-cTn levels and time intervals between blood samples. The system's output ultimately provides an unequivocal classification into a triage category, adjusted for the exact time elapsed between measurements. The tool further provides information on the need and timing of a second, or even a third hs-cTn, if two measurements are inconclusive. A prospective, single-centre validation study using a sequential roll-out design with alternating 2-week blocks of standard-of-care and CDSS-supported care is planned over a 24-week enrolment period. The primary endpoint is the proportion of protocol adherence. Key secondary endpoints include rates of correct triage, length of ED stay, proportion of direct discharge after rule-out in patients at low risk and the proportion of inappropriate blood draws (excessive or deferred) as an estimate for cost-effectiveness. A novel CDSS that represents a Class IIa medical software device and has already received CE mark for use in clinical routine. It is expected to improve guideline adherence, effectiveness and safety of patients admitted to an ED with suspected NSTE-ACS. A prospective pilot study for validation is planned.

BackgroundDespite the unprecedented performance of deep learning algorithms in clinical domains, full reviews of algorithmic predictions by human experts remain mandatory. Under these circumstances, artificial intelligence (AI) models are primarily designed as clinical decision support systems (CDSSs). However, from the perspective of clinical practitioners, the lack of clinical interpretability and user-centered interfaces hinders the adoption of these AI systems in practice.ObjectiveThis study aims to develop an AI-based CDSS for assisting polysomnographic technicians in reviewing AI-predicted sleep staging results. This study proposed and evaluated a CDSS that provides clinically sound explanations for AI predictions in a user-centered manner.MethodsOur study is based on a user-centered design framework for developing explanations in a CDSS that identifies why explanations are needed, what information should be contained in explanations, and how explanations can be provided in the CDSS. We conducted user interviews, user observation sessions, and an iterative design process to identify three key aspects for designing explanations in the CDSS. After constructing the CDSS, the tool was evaluated to investigate how the CDSS explanations helped technicians. We measured the accuracy of sleep staging and interrater reliability with macro-F1 and Cohen κ scores to assess quantitative improvements after our tool was adopted. We assessed qualitative improvements through participant interviews that established how participants perceived and used the tool.ResultsThe user study revealed that technicians desire explanations that are relevant to key electroencephalogram (EEG) patterns for sleep staging when assessing the correctness of AI predictions. Here, technicians wanted explanations that could be used to evaluate whether the AI models properly locate and use these patterns during prediction. On the basis of this, information that is closely related to sleep EEG patterns was formulated for the AI models. In the iterative design phase, we developed a different visualization strategy for each pattern based on how technicians interpreted the EEG recordings with these patterns during their workflows. Our evaluation study on 9 polysomnographic technicians quantitatively and qualitatively investigated the helpfulness of the tool. For technicians with <5 years of work experience, their quantitative sleep staging performance improved significantly from 56.75 to 60.59 with a P value of .05. Qualitatively, participants reported that the information provided effectively supported them, and they could develop notable adoption strategies for the tool.ConclusionsOur findings indicate that formulating clinical explanations for automated predictions using the information in the AI with a user-centered design process is an effective strategy for developing a CDSS for sleep staging.

Artificial intelligence (AI) has increasingly influenced healthcare, yet its adoption in nursing research remains underexplored. Although interest in AI applications, such as robotics, decision support, and machine learning, is growing, a comprehensive bibliometric mapping of global scholarship in nursing remains lacking. To examine the global trends, intellectual structures, and thematic evolution of AI-related research in nursing from 1984 to 2025 (through May 2025) and to identify key authors, institutions, and research foci. A systematic search was conducted in the Web of Science Core Collection using predefined title-based keywords ("nurse" OR "nursing") AND ("artificial intelligence" OR "machine learning" OR "deep learning" OR "robotic∗" OR "chatbot∗" OR "neural network∗"). Studies were included if they explicitly addressed nursing practice, management, education, or research applications of AI. Two independent reviewers screened all records by title and abstract to confirm nursing relevance using predefined inclusion and exclusion criteria. This yielded 799 records; after removing duplicates and uncited papers, 530 publications were retained as nursing-related. Bibliometric analysis employed performance and science-mapping techniques via VOSviewer and Biblioshiny. Indicators included publication trends, citation distributions, keyword co-occurrence, authorship patterns, and collaborations. The number of publications increased substantially after 2018, with the United States, China, and the United Kingdom being the most productive countries. Dominant research themes included robotics in elder care, clinical decision support systems, and nursing education enhanced by AI tools. Coauthorship analysis revealed limited international collaboration, and keyword mapping identified "robotics," "machine learning," and "nursing education" as leading focal points. Despite decades of development, AI remains an emerging and underutilized area within nursing research. The rapid growth of publications in recent years signals expanding interest, yet the field lacks consolidated efforts and cohesive global collaboration. Greater interdisciplinary and international engagement is needed to accelerate innovation.

Health systems challenges that emerged during the COVID-19 pandemic, such as a lack of resources and medical staff, are forcing solutions which optimize healthcare performance. One of the solutions is the development of clinical decision support systems (CDSS) based on artificial intelligence (AI). We classified AI-based clinical decision-supporting systems used during the pandemic and evaluated the mathematical algorithms present in these systems. Materials and methods: we searched for articles relevant to the aim of the study in the Scopus publication database. Results: depending on the purpose of the development a clinical decision support system based on artificial intelligence during pandemic, we identified three groups of tasks: organizational, scientific and diagnostic. Tasks such as predicting of pandemic parameters, searching of analogies in pandemic progression, prioritization of patients, use of telemedicine are solved for the purposes of healthcare organization. Artificial intelligence in drugs and vaccine development, alongside personalized treatment programs, apply to new scientific knowledge acquisition. Diagnostic tasks include the development of mathematical models for assessing COVID-19 outcomes, prediction of disease severity, analysis of factors influencing COVID-19 complications. Conclusion: artificial intelligence methods can be effectively implemented for decision support systems in solving tasks that face healthcare during pandemic.

Accurate diagnosis of kidney disease is essential, as it remains a major health concern affecting individuals across all age groups. Early detection is critical to ensuring timely and effective treatment. With the advancement of deep learning, powerful models now offer end-to-end learning capabilities, extracting relevant patterns directly from complex medical data. These models significantly improve diagnostic precision and support clinical decision-making. However, as deep learning models like ANN, CNN, and LSTM become more complex, the need for interpretability has become increasingly important. The trade-off between model performance and transparency remains a persistent gap in the healthcare domain. To address this concern, the present article uses three black-box models to predict the existence of kidney stones. The models were trained using a dataset of unbalanced kidney stones that was first preprocessed using established methods to increase accuracy and balance. Then ANN, CNN, and LSTM models were used, yielding outstanding accuracies of 97 %, 96 %, and 98 %, respectively. To enhance the interpretability of these black-box models, three XAI technique were applied individually to each model which are Saliency Maps, Ablation, and Permutation Feature Importance. These methods successfully identified the most influential features contributing to the predictions which are Serum Creatinine (sc) and Hemoglobin (hemo). The model's interpretability and high accuracy lay the groundwork for energy-efficient, trustworthy AI applications in 6G-based medical ecosystems. The results demonstrate that combining deep learning with XAI enables both high predictive performance and meaningful model transparency. This integration addresses the critical need for interpretable AI in healthcare and contributes to the development of reliable clinical decision support systems.

Neural network based clinical decision support system for the calculation the initial continuous subcutaneous insulin infusion settings