Unlocking Machine Learning Model Decisions: A Comparative Analysis of LIME and SHAP for Enhanced Interpretability

BACKGROUND Acute kidney injury (AKI) is a common adverse outcome following nephrectomy. The progression from AKI to acute kidney disease (AKD) and subsequently to chronic kidney disease (CKD) remains a concern; yet, the predictive mechanisms for these transitions are not fully understood. Interpretable machine learning (ML) models offer insights into how clinical features influence long-term renal function outcomes after nephrectomy, providing a more precise framework for identifying patients at risk and supporting improved clinical decision-making processes. OBJECTIVE This study aimed to (1) evaluate postnephrectomy rates of AKI, AKD, and CKD, analyzing long-term renal outcomes along different trajectories; (2) interpret AKD and CKD models using Shapley Additive Explanations values and Local Interpretable Model-Agnostic Explanations algorithm; and (3) develop a web-based tool for estimating AKD or CKD risk after nephrectomy. METHODS We conducted a retrospective cohort study involving patients who underwent nephrectomy between July 2012 and June 2019. Patient data were randomly split into training, validation, and test sets, maintaining a ratio of 76.5:8.5:15. Eight ML algorithms were used to construct predictive models for postoperative AKD and CKD. The performance of the best-performing models was assessed using various metrics. We used various Shapley Additive Explanations plots and Local Interpretable Model-Agnostic Explanations bar plots to interpret the model and generated directed acyclic graphs to explore the potential causal relationships between features. Additionally, we developed a web-based prediction tool using the top 10 features for AKD prediction and the top 5 features for CKD prediction. RESULTS The study cohort comprised 1559 patients. Incidence rates for AKI, AKD, and CKD were 21.7% (n=330), 15.3% (n=238), and 10.6% (n=165), respectively. Among the evaluated ML models, the Light Gradient-Boosting Machine (LightGBM) model demonstrated superior performance, with an area under the receiver operating characteristic curve of 0.97 for AKD prediction and 0.96 for CKD prediction. Performance metrics and plots highlighted the model’s competence in discrimination, calibration, and clinical applicability. Operative duration, hemoglobin, blood loss, urine protein, and hematocrit were identified as the top 5 features associated with predicted AKD. Baseline estimated glomerular filtration rate, pathology, trajectories of renal function, age, and total bilirubin were the top 5 features associated with predicted CKD. Additionally, we developed a web application using the LightGBM model to estimate AKD and CKD risks. CONCLUSIONS An interpretable ML model effectively elucidated its decision-making process in identifying patients at risk of AKD and CKD following nephrectomy by enumerating critical features. The web-based calculator, found on the LightGBM model, can assist in formulating more personalized and evidence-based clinical strategies.

BackgroundTuberculosis spondylitis (TS), commonly known as Pott’s disease, is a severe type of skeletal tuberculosis that typically requires surgical treatment. However, this treatment option has led to an increase in healthcare costs due to prolonged hospital stays (PLOS). Therefore, identifying risk factors associated with extended PLOS is necessary. In this research, we intended to develop an interpretable machine learning model that could predict extended PLOS, which can provide valuable insights for treatments and a web-based application was implemented.MethodsWe obtained patient data from the spine surgery department at our hospital. Extended postoperative length of stay (PLOS) refers to a hospitalization duration equal to or exceeding the 75th percentile following spine surgery. To identify relevant variables, we employed several approaches, such as the least absolute shrinkage and selection operator (LASSO), recursive feature elimination (RFE) based on support vector machine classification (SVC), correlation analysis, and permutation importance value. Several models using implemented and some of them are ensembled using soft voting techniques. Models were constructed using grid search with nested cross-validation. The performance of each algorithm was assessed through various metrics, including the AUC value (area under the curve of receiver operating characteristics) and the Brier Score. Model interpretation involved utilizing methods such as Shapley additive explanations (SHAP), the Gini Impurity Index, permutation importance, and local interpretable model-agnostic explanations (LIME). Furthermore, to facilitate the practical application of the model, a web-based interface was developed and deployed.ResultsThe study included a cohort of 580 patients and 11 features include (CRP, transfusions, infusion volume, blood loss, X-ray bone bridge, X-ray osteophyte, CT-vertebral destruction, CT-paravertebral abscess, MRI-paravertebral abscess, MRI-epidural abscess, postoperative drainage) were selected. Most of the classifiers showed better performance, where the XGBoost model has a higher AUC value (0.86) and lower Brier Score (0.126). The XGBoost model was chosen as the optimal model. The results obtained from the calibration and decision curve analysis (DCA) plots demonstrate that XGBoost has achieved promising performance. After conducting tenfold cross-validation, the XGBoost model demonstrated a mean AUC of 0.85 ± 0.09. SHAP and LIME were used to display the variables’ contributions to the predicted value. The stacked bar plots indicated that infusion volume was the primary contributor, as determined by Gini, permutation importance (PFI), and the LIME algorithm.ConclusionsOur methods not only effectively predicted extended PLOS but also identified risk factors that can be utilized for future treatments. The XGBoost model developed in this study is easily accessible through the deployed web application and can aid in clinical research.

Explainable artificial intelligence for predicting dengue outbreaks in Bangladesh using eco-climatic triggers.

Background Predicting the prognosis of patients with acute myocardial infarction (AMI) combined with diabetes mellitus (DM) is crucial due to high in-hospital mortality rates. This study aims to develop and validate a mortality risk prediction model for these patients by interpretable machine learning (ML) methods. Methods Data were sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV, version 2.2). Predictors were selected by Least absolute shrinkage and selection operator (LASSO) regression and checked for multicollinearity with Spearman’s correlation. Patients were randomly assigned to training and validation sets in an 8:2 ratio. Seven ML algorithms were used to construct models in the training set. Model performance was evaluated in the validation set using metrics such as area under the curve (AUC) with 95% confidence interval (CI), calibration curves, precision, recall, F1 score, accuracy, negative predictive value (NPV), and positive predictive value (PPV). The significance of differences in predictive performance among models was assessed utilising the permutation test, and 10-fold cross-validation further validated the model’s performance. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) were applied to interpret the models. Results The study included 2,828 patients with AMI combined with DM. Nineteen predictors were identified through LASSO regression and Spearman’s correlation. The Random Forest (RF) model was demonstrated the best performance, with an AUC of 0.823 (95% CI: 0.774–0.872), high precision (0.867), accuracy (0.873), and PPV (0.867). The RF model showed significant differences (p < 0.05) compared to the K-Nearest Neighbours and Decision Tree models. Calibration curves indicated that the RF model’s predicted risk aligned well with actual outcomes. 10-fold cross-validation confirmed the superior performance of RF model, with an average AUC of 0.828 (95% CI: 0.800–0.842). Significant Variables in RF model indicated that the top eight significant predictors were urine output, maximum anion gap, maximum urea nitrogen, age, minimum pH, maximum international normalised ratio (INR), mean respiratory rate, and mean systolic blood pressure. Conclusion This study demonstrates the potential of ML methods, particularly the RF model, in predicting in-hospital mortality risk for AMI patients with DM. The SHAP and LIME methods enhance the interpretability of ML models.

Age is a significant risk factor for mild cognitive impairment (MCI) and Alzheimer’s disease (AD) and identifying brain age patterns is critical for comprehending the normal aging and MCI/AD processes. Prior studies have widely established the univariate relationships between brain regions and age, while multivariate associations remain largely unexplored. Herein, various artificial intelligence (AI) models were used to perform brain age prediction using an MRI dataset (n = 825). The optimal AI model was then integrated with the feature importance methods, namely Shapley additive explanations (SHAP), local interpretable model-agnostic explanations, and layer-wise relevance propagation, to identify the significant multivariate brain regions hierarchically involved in this prediction. Our results showed that the deep learning model (referred to as AgeNet) outperformed conventional machine learning models for brain age prediction, and that AgeNet integrated with SHAP (referred to as AgeNet-SHAP) identified all ground-truth perturbed regions as key predictors of brain age in semi-simulation, demonstrating the validity of our methodology. In the experimental dataset, when compared to cognitively normal (CN) participants, MCI exhibited moderate differences in brain regions, whereas AD showed highly robust and widely distributed regional differences. Individualized AgeNet-SHAP regional features further showed associations with clinical severity scores in the AD continuum. These results collectively facilitate data-driven explainable AI approaches for disease progression, diagnostics, prognostics, and personalized medicine efforts.

The increasing sophistication and frequency of cyberattacks necessitate advanced and transparent decision-making frameworks for threat intelligence and risk assessment. While artificial intelligence (AI) has demonstrated significant potential in automating threat detection, predicting attack patterns, and prioritizing incident response, the opaque nature of many AI models particularly deep learning poses challenges in trust, interpretability, and regulatory compliance. This paper presents an Explainable AI (XAI)-driven framework for enhancing Cyber Threat Intelligence (CTI) and risk assessment, enabling security analysts to understand, validate, and act upon AI-generated insights with confidence. The proposed approach integrates state-of-the-art machine learning algorithms with model-agnostic and model-specific interpretability techniques, such as SHapley Additive exPlanations (SHAP), Local Interpretable Model-agnostic Explanations (LIME), Layer-wise Relevance Propagation (LRP), and attention-based visualization. The framework processes diverse CTI data sources including network traffic logs, vulnerability feeds, malware signatures, and open-source intelligence (OSINT) to identify malicious patterns, quantify risks, and generate actionable intelligence. By coupling predictive accuracy with transparency, the system not only improves detection and classification of advanced persistent threats (APTs), phishing campaigns, and zero-day exploits, but also facilitates compliance with emerging AI governance frameworks, such as the EU AI Act and NIST AI Risk Management Framework. Experimental evaluations on benchmark cybersecurity datasets demonstrate that the XAI-enabled models maintain high precision and recall while offering interpretable outputs that significantly reduce analyst decision latency and improve collaborative threat response. The paper further discusses the role of XAI in mitigating bias, enhancing accountability, and fostering human–machine teaming in security operations centers (SOCs). Practical implementation considerations including scalability, computational efficiency, and integration with existing Security Information and Event Management (SIEM) platforms are addressed, along with recommendations for future research in multi-modal XAI for cybersecurity. This research underscores the critical value of explainability in AI-driven CTI, ensuring that automated systems are not only effective but also transparent, trustworthy, and aligned with organizational and regulatory requirements for responsible AI deployment in cyber defense.

In the field of network security, botnet attacks pose a significant challenge, exploiting networks of infected devices to launch sophisticated threats. As these attacks evolve, the need for effective detection methods becomes increasingly critical. This study proposes an explainable Machine Learning (ML) model that aims to identify and categorize botnet attacks, and it investigates the efficiency of different Explainable Artificial Intelligence (XAI) techniques for Intrusion Detection Systems (IDS). To train our models, we employed a subset of the NCC-2 dataset, which includes a mix of normal traffic and seven different types of botnet attacks across three sensors. The ML techniques selected for this research are Random Forest (RF), Extra Trees (ET), Decision Tree (DT), and K-Nearest Neighbors (KNN), with a GridSearch cross-validation approach for optimal hyperparameter tuning. We also explored the effects of class balance through Synthetic Minority Oversampling (SMOTE) and Random Undersampling. The models’ performance was rigorously tested using the Matthews Correlation Coefficient (MCC) and Macro F1-score, with the ET model demonstrating superior results of 99% MCC and 97% F1-score, respectively. To enhance the interpretability of the ET model’s decision-making process, we integrated three XAI techniques: SHapley Additive exPlanations (SHAP), Dalex, and Local Interpretable Model-agnostic Explanations (LIME), and evaluated their efficiency. The LIME and Dalex techniques showed efficient construction times for IDS.

Introduction: The connection between green innovation and firm performance on the industrial sector of Pakistan is explored by this study, which implements the versatile explainable artificial intelligence (XAI) approaches to reveal the knowledge path with transparent explanations of complex decisions. Objective: It adopts a comparative approach to SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) model in explaining green innovation endeavors into an elucidation of practical business gains guaranteed. Method: On a large scale, study access a comprehensive dataset of 284 manufacturing companies of diverse industries within Pakistan and develop a machine learning model to forecast the firm indicators of performance whilst being interpretable using XAI methods. Results: Research has found that green innovation has a positive impact that is statistically significant on firm performance and the drivers which have been identified are described as environmental compliance, resource efficiency and sustainable product development. The above comparative analysis of SHAP and LIME shows that both methods, in their turn, yield good insights, but SHAP shows more consistent global explanations whereas LIME is better at explaining individual instances. Conclusions: By having the contribution in the growing literature on sustainable business practices in emerging markets, this research is practical in giving the right directions to policymakers and business leaders in Pakistan. Suggestions are given to the policy makers and the firms in Pakistan indicating that strategic investments need to be made in process innovation and regulatory frameworks. This study highlights the promise of XAI to help in de-mystifying the complex nature of relationships in sustainability-related performance.

This study presents a comprehensive methodology for developing a Fault Detection and Diagnosis (FDD) system for an industrial distillation column using advanced machine learning algorithms. Steady state and dynamic simulations in Aspen Plus® generate extensive datasets under normal and faulty conditions. Feature engineering, using the Minimum Redundancy Maximum Relevance (MRMR) algorithm, selects the most relevant features for fault detection. Various machine learning models, including Decision Trees, Support Vector Machines, k-nearest Neighbours, and Neural Networks, were trained and evaluated based on performance metrics such as accuracy, recall, precision, and F1 score. The top models were integrated into a stacked classifier system with a voting mechanism to enhance fault detection reliability. Explainable Artificial Intelligence (XAI) techniques, such as Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP), were incorporated to improve model interpretability, allowing engineers to understand and validate the FDD system's decision-making process. Simulation results confirm that the proposed methodology accurately identifies and classifies faults. By integrating dynamic simulations, advanced machine learning, and XAI techniques, a robust and scalable solution is achieved for fault detection in distillation columns, improving operational reliability, safety, and reducing downtime. Future work could extend this approach to other industrial processes and explore additional machine learning algorithms to further enhance performance.

Electrocardiogram signal analysis can be time-consuming, tedious, and error-prone. Therefore, automated analysis is need of time that will assist clinicians in detecting cardiac abnormalities accurately and efficiently. Recently, deep learning models have shown unprecedented progress and strong arrhythmia classification capabilities, but their deployment in the medical sector is constrained due to their “black-box” nature. This paper proposes a robust explainability method to assist in explaining the underlying decision-making process in deep neural networks (DNN) and help provide feedback on biases that would benefit in improving DNN models. To achieve these objectives, initially, a deep learning model is trained on the MIT-BIH Arrhythmia Database and their classification performance is evaluated. The classification findings are then interpreted using the post-hoc explanation methods such as the SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) and Gradient-weighted Class Activation Mapping (Grad-CAM) that interpret the decision rationale. As these methods are initially proposed for image applications therefore a new masking approach is proposed to cater these post-hoc explainability methods for ECG time-series data. After evaluating these methods for ECG arrhythmia classification, several drawbacks are drawn such as they fail to locate a feature’s importance if there are multiple occurrences of the same feature in a signal and also, SHAP and LIME perform random perturbations that sometimes produce unreliable explanations. Therefore, to overcome the drawbacks associated with these post-hoc explainability methods on time-series data, a novel K-GradCam method is proposed. The proposed K-GradCam method ensemble the benefits of these gradient based and perturbation based approaches and has demonstrated advantages over SHAP, Grad-CAM and LIME in terms of interpreting the models’ decisions. To compare the the proposed technique with post-hoc explainability methods quantitatively, the confidence index of the proposed method is evaluated using dice loss. The proposed method shares 71% similarity with SHAP and 81% similarity with Grad-CAM methods; however, it shares the benefits of both methods and is computationally faster than SHAP and LIME.

The rise of sophisticated cyber threats, such as malware and network intrusions, necessitates the use of Artificial Intelligence (AI) for efficient and accurate detection. However, traditional AI models often operate as black boxes, leaving security analysts without insights into the reasoning behind critical decisions. Explainable AI (XAI) addresses this challenge by providing interpretability and transparency in AI-driven cybersecurity solutions. This paper explores the role of XAI in malware analysis and network intrusion detection, highlighting how interpretable models enhance trust, improve decision-making, and facilitate regulatory compliance. It examines state-of-the-art XAI techniques, including Shapley Additive Explanations (SHAP), Local Interpretable Model-agnostic Explanations (LIME), and rule-based systems, for their application in identifying malicious software and detecting network anomalies. Furthermore, the paper discusses challenges, such as computational overhead and scalability, while presenting future directions for integrating XAI in real-time and hybrid security frameworks. By advancing the adoption of interpretable AI, cybersecurity systems can achieve greater effectiveness and reliability, addressing both technical and organizational needs in combating evolving cyber threats.

ObjectivePostoperative complications remain a major concern in laryngeal cancer surgery, often requiring invasive interventions or intensive care. This study aimed to develop and validate an interpretable machine learning (ML) model to preoperatively predict Clavien-Dindo Grade ≥ III complications and support risk-informed perioperative decision-making.MethodsWe conducted a retrospective study using a temporally split cohort of laryngeal cancer patients. Postoperative complications were graded using the Clavien-Dindo (CD) classification. Eight ML algorithms were trained and evaluated using receiver operating characteristic (ROC) curves, calibration plots, and decision curve analysis (DCA). Model interpretability was assessed using SHapley Additive exPlanations (SHAP). A web-based calculator was deployed for clinical use.ResultThe random forest (RF) model achieved the best performance, with an area under the curve (AUC) of 0.935 in the training set and 0.842 in the test set. The model demonstrated robust sensitivity and specificity for both surgical and medical complications. Calibration curves indicated strong agreement between predicted and actual outcomes. SHAP analysis identified eight key predictors—such as vocal cord mobility, tumor subsite, and nutritional status—that contributed most to risk estimation. A user-friendly web calculator was developed and is accessible at: https://qilushiny.shinyapps.io/qilupredicate/.ConclusionWe developed a clinically interpretable ML model that accurately predicts major postoperative complications in patients undergoing laryngeal cancer surgery. This tool provides individualized risk assessments that can guide surgical planning, optimize perioperative strategies, and enhance shared decision-making. Prospective multicenter validation is needed to confirm its utility in routine practice.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12893-025-03213-z.

Introduction: The integration of Artificial Intelligence (AI) into personalized banking has enhanced service delivery in areas such as loan processing, credit assessment, and fraud detection. Despite these advancements, ethical concerns, especially algorithmic bias and lack of fairness, pose significant challenges. This study addresses the need for equitable AI systems that promote transparency, fairness, and regulatory compliance in the banking sector. Objective: This study aims to develop and implement a comprehensive framework for integrating ethical principles into AI-driven banking systems, with a focus on mitigating algorithmic bias, enhancing fairness, and improving transparency in personalized banking services. Methods: A comprehensive methodology is proposed that integrates bias-aware data collection, fairness-constrained machine-learning models, and explainable AI (XAI) techniques. Tools such as Shapley Additive Explanations (SHAPs) and Local Interpretable Model-Agnostic Explanations (LIMEs) are applied to interpret model outputs. Adversarial debiasing and fairness-aware learning algorithms were employed to identify and mitigate systemic biases in financial data. Alternative data sources, including utility and rental payment histories, were incorporated to enhance inclusivity. Results: The implementation of the proposed framework demonstrates improved fairness in decision-making without significantly compromising model accuracy. Bias metrics show measurable reductions in disparate impacts across the demographic groups. Explainability tools enhance transparency, enabling a more transparent communication of AI decisions to both users and regulators. Conclusions: Embedding ethical principles into AI-driven banking systems is critical to ensuring fairness, regulatory alignment, and public trust. The structured framework presented in this study supports the development of responsible AI systems to mitigate bias, enhance explainability, and foster financial inclusion. This approach serves as the foundation for building equitable and accountable AI applications in modern banking.

The accurate detection and assessment of stress play a pivotal role in enhancing individual well-being and healthcare outcomes. Traditional methods of stress detection often grapple with limitations in accuracy and scalability. With the advent of machine learning (ML), the potential to revolutionize stress detection has emerged. This paper presents a comprehensive study on the application of ML algorithms for stress detection, with a focus on physiological and behavioral data analysis. Central to our approach is the integration of SHapley Additive exPlanations (SHAP), an Explainable Artificial Intelligence (XAI) technique, to interpret ML models. SHAP provides a novel lens to understand the impact of individual features in the complex decision-making processes of ML models, thereby enhancing the transparency and reliability of stress predictions. We demonstrate how SHAP not only aids in elucidating model decisions but also contributes to refining the models for greater accuracy. Our results highlight the effectiveness of ML in detecting stress and the pivotal role of XAI in making these models more interpretable and trustworthy. This study underscores the synergy between advanced ML techniques and XAI, paving the way for more nuanced and reliable stress detection methodologies that are essential in diverse settings, from healthcare to workplace environments.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak