Enhancing Transparency in Healthcare Machine Learning Models Using Shap and Deeplift a Methodological Approach

12074 Background: Skeletal-related events (SREs) are detrimental clinical events in bone metastasis from advanced cancer, defined by pathologic fracture, spinal cord compression, and inevitable surgical or radiational intervention to the bone. Given their negative impact on quality of life and prognosis and interpatient heterogeneity in the SRE risk, accurate identification of patients with high SRE risk is critical. Methods: The patient-level data from three randomized clinical trials that administered zoledronic acid (ZA) to patients with bone-metastatic breast cancer, castration-resistant prostate cancer (CRPC), and other types of cancer were analyzed. (N = 460, 452, and 315, respectively) Machine learning (ML) models to predict SREs within 18 months (breast cancer), 12 months (CRPC), and 9 months (other cancers) were developed based on more than 40 baseline clinical and laboratory data. Seven ML algorithms and five feature selection methods were utilized to develop multiple models. The ML model with the best performance was identified based on the F1 score and the area under the receiver operating characteristic curve (AUC-ROC) for each cancer type and interrogated for important features with Shapley additive explanations. Lastly, the ML models’ ability to stratify patients by the cumulative SRE risk was evaluated by calculating hazard ratio (HR) with Cox-proportional hazards models. Results: Among the multiple ML models developed with different algorithms and feature selection methods, the model developed utilizing the random forests algorithm and the Boruta method for selecting features demonstrated the best performance in all types of cancer (F1 0.70, 0.67, and 0.67, and AUC-ROC 0.72, 0.68, and 0.73 for breast cancer, CRPC, and other cancers, respectively). In the ML model for breast cancer, performance status (PS), history of SRE, serum alkaline phosphatase (ALP), history of anti-neoplastic surgery, radiation therapy, and pathologic fracture were included as important features. Serum ALP, albumin, sodium, Gleason scores, and geographic regions were shown to be relevant in the CRPC model. For other cancers, serum ALP, albumin, total protein, phosphorus, red blood cell count, white blood cell count, visceral metastases, and history of arthritis were incorporated in the ML model. The ML model prediction successfully stratified the patients for cumulative SRE risk in all three cohorts. (HR with 95% confidence interval: 2.43 [1.86 – 3.18], 1.92 [1.51 – 2.45], and 3.06 [2.29 – 4.09] for breast cancer, CRPC, and other types of cancer, respectively). Conclusions: ML models incorporating baseline clinical and laboratory data can identify patients with bone metastasis on ZA harboring a high SRE risk.

This study focuses on using various machine learning (ML) models to evaluate the shear behaviors of ultra-high-performance concrete (UHPC) beams reinforced with glass fiber-reinforced polymer (GFRP) bars. The main objective of the study is to predict the shear strength of UHPC beams reinforced with GFRP bars using ML models. We use four different ML models: support vector machine (SVM), artificial neural network (ANN), random forest (R.F.), and extreme gradient boosting (XGBoost). The experimental database used in the study is acquired from various literature sources and comprises 54 test observations with 11 input features. These input features are likely parameters related to the composition, geometry, and properties of the UHPC beams and GFRP bars. To ensure the ML models' generalizability and scalability, random search methods are utilized to tune the hyperparameters of the algorithms. This tuning process helps improve the performance of the models when predicting the shear strength. The study uses the ACI318M-14 and Eurocode 2 standard building codes to predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams. The ML models' predictions are compared to the results obtained from these building code standards. According to the findings, the XGBoost model demonstrates the highest predictive test performance among the investigated ML models. The study employs the SHAP (SHapley Additive exPlanations) analysis to assess the significance of each input parameter in the ML models' predictive capabilities. A Taylor diagram is used to statistically compare the accuracy of the ML models. This study concludes that ML models, particularly XGBoost, can effectively predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams.

Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosis.

Chronic pain around the temporomandibular joint (TMJ) and masticatory muscles is a primary symptom of temporomandibular disorders (TMD). However, the clinical significance of magnetic resonance imaging (MRI) features in predicting TMD-related pain remains unclear. This study aimed to develop and interpret machine learning (ML) models based on MRI characteristics for predicting pain severity in patients with TMD. The present retrospective study included 584 patients with TMD between January 2022 and December 2024, yielding a total of 755 TMJ MRI data sets. Pain severity was classified using the visual analogue scale (VAS). Demographic variables (age, sex) and MRI features-including lesion side, disc position, disc morphology, disc signal, disc perforation, bilaminar zone tear, joint space, joint effusion, condylar movement, bony changes and morphology/signal of the lateral pterygoid muscle-were collected. Eleven ML models based on demographic and MRI features were developed: logistic regression (LR), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), adaptive boosting (AdaBoost), gradient boosting classifier (GBC), bagging classifier (BC), extremely randomised trees (ETC), decision tree classifier (DTC) and multilayer perceptron (MLP). Model performance was evaluated using multiple metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity and F1 score. Precision-recall (PR) curves and calibration curves were plotted to assess discrimination and model calibration. Decision curve analysis (DCA) was conducted to evaluate the clinical net benefit across a range of threshold probabilities. Model interpretability was enhanced using Shapley Additive Explanations (SHAP), which quantified the contribution of each feature to individual predictions. Feature selection was conducted based on mean SHAP values, and separate LightGBM models were constructed using the Top 3, 5, and 9 most important features, as well as the full-feature set, for performance comparison. The data set was randomly divided into a training set (n = 604) and a test set (n = 151). Among the 11 ML models, the LightGBM model demonstrated the best predictive performance, with an AUC of 0.899, and was therefore identified as the optimal model. SHAP analysis identified age, disc position and condylar movement as the top three contributing features. Feature selection analysis indicated that selecting the top nine SHAP-ranked variables led to the highest diagnostic performance, with an AUC of 0.829. This study developed an interpretable, high-performing MRI-based ML model incorporating SHAP analysis to integrate imaging and clinical features for objective pain assessment, which may help identify high-risk TMD patients and guide personalised treatment strategies.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak

Perinatal depression (PND) is one of the most common medical complications during pregnancy and postpartum period, affecting 10–20% of pregnant individuals. Black and Latina women have higher rates of PND, yet they are less likely to be diagnosed and receive treatment. Machine learning (ML) models based on Electronic Medical Records (EMRs) have been effective in predicting postpartum depression in middle-class White women but have rarely included sufficient proportions of racial and ethnic minorities, which contributed to biases in ML models for minority women. Our goal is to determine whether ML models could serve to predict depression in early pregnancy in racial/ethnic minority women by leveraging EMR data. We extracted EMRs from a hospital in a large urban city that mostly served low-income Black and Hispanic women (N=5,875) in the U.S. Depressive symptom severity was assessed from a self-reported questionnaire, PHQ-9. We investigated multiple ML classifiers, used Shapley Additive Explanations (SHAP) for model interpretation, and determined model prediction bias with two metrics, Disparate Impact, and Equal Opportunity Difference. While ML model (Elastic Net) performance was low (ROCAUC=0.67), we identified well-known factors associated with PND, such as unplanned pregnancy and being single, as well as underexplored factors, such as self-report pain levels, lower levels of prenatal vitamin supplement intake, asthma, carrying a male fetus, and lower platelet levels blood. Our findings showed that despite being based on a sample mostly composed of 75% low-income minority women (54% Black and 27% Latina), the model performance was lower for these communities. In conclusion, ML models based on EMRs could moderately predict depression in early pregnancy, but their performance is biased against low-income minority women.

BACKGROUND Acute kidney injury (AKI) is a common adverse outcome following nephrectomy. The progression from AKI to acute kidney disease (AKD) and subsequently to chronic kidney disease (CKD) remains a concern; yet, the predictive mechanisms for these transitions are not fully understood. Interpretable machine learning (ML) models offer insights into how clinical features influence long-term renal function outcomes after nephrectomy, providing a more precise framework for identifying patients at risk and supporting improved clinical decision-making processes. OBJECTIVE This study aimed to (1) evaluate postnephrectomy rates of AKI, AKD, and CKD, analyzing long-term renal outcomes along different trajectories; (2) interpret AKD and CKD models using Shapley Additive Explanations values and Local Interpretable Model-Agnostic Explanations algorithm; and (3) develop a web-based tool for estimating AKD or CKD risk after nephrectomy. METHODS We conducted a retrospective cohort study involving patients who underwent nephrectomy between July 2012 and June 2019. Patient data were randomly split into training, validation, and test sets, maintaining a ratio of 76.5:8.5:15. Eight ML algorithms were used to construct predictive models for postoperative AKD and CKD. The performance of the best-performing models was assessed using various metrics. We used various Shapley Additive Explanations plots and Local Interpretable Model-Agnostic Explanations bar plots to interpret the model and generated directed acyclic graphs to explore the potential causal relationships between features. Additionally, we developed a web-based prediction tool using the top 10 features for AKD prediction and the top 5 features for CKD prediction. RESULTS The study cohort comprised 1559 patients. Incidence rates for AKI, AKD, and CKD were 21.7% (n=330), 15.3% (n=238), and 10.6% (n=165), respectively. Among the evaluated ML models, the Light Gradient-Boosting Machine (LightGBM) model demonstrated superior performance, with an area under the receiver operating characteristic curve of 0.97 for AKD prediction and 0.96 for CKD prediction. Performance metrics and plots highlighted the model’s competence in discrimination, calibration, and clinical applicability. Operative duration, hemoglobin, blood loss, urine protein, and hematocrit were identified as the top 5 features associated with predicted AKD. Baseline estimated glomerular filtration rate, pathology, trajectories of renal function, age, and total bilirubin were the top 5 features associated with predicted CKD. Additionally, we developed a web application using the LightGBM model to estimate AKD and CKD risks. CONCLUSIONS An interpretable ML model effectively elucidated its decision-making process in identifying patients at risk of AKD and CKD following nephrectomy by enumerating critical features. The web-based calculator, found on the LightGBM model, can assist in formulating more personalized and evidence-based clinical strategies.

The number of deaths from cardiovascular disease is projected to reach 23.3 million by 2030. As a contribution to preventing this phenomenon, this paper proposed a machine learning (ML) model to predict patients with arteriosclerotic heart disease (AHD). We also interpreted the prediction model results based on the ML approach and deployed modelagnostic ML methods to identify informative features and their interpretations. We used a hematology Electronic Health Record (EHR) with information on erythrocytes, hematocrit, hemoglobin, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, leukocytes, thrombocytes, age, and sex. To detect and predict AHD, we explored random forest (RF), XGBoost, and AdaBoost models. We examined the prediction model results based on the confusion matrix and accuracy measures. We used the Shapley Additive exPlanations (SHAP) framework to interpret the ML model and quantify the contribution of features to predictions. Our study included data from 6,837 patients, with 4,702 records from patients diagnosed with AHD and 2,135 records from patients without an AHD diagnosis. AdaBoost outperformed RF and XGBoost, achieving an accuracy of 0.78, precision of 0.82, F1-score of 0.85, and recall of 0.88. According to the SHAP summary bar plot method, hemoglobin was the most important attribute for detecting and predicting AHD patients. The SHAP local interpretability bar plot revealed that hemoglobin and mean corpuscular hemoglobin concentration had positive impacts on AHD prediction based on a single observation. ML models based on real clinical data can be used to predict AHD.

Propofol is a widely used sedative-hypnotic agent for critically-ill patients requiring invasive mechanical ventilation (IMV). Despite its clinical benefits, propofol is associated with increased risks of hypertriglyceridemia. Early identification of patients at risk for propofol-associated hypertriglyceridemia is crucial for optimizing sedation strategies and preventing adverse outcomes. Machine learning (ML) models offer a promising approach for predicting individualized patient risks of propofol-associated hypertriglyceridemia. We propose the development of a ML model aimed at predicting the risk of propofol-associated hypertriglyceridemia in ICU patients receiving IMV. The study will utilize retrospective data from four Mayo Clinic sites. Nested cross-validation (CV) will be employed, with a 10-fold inner CV loop for model tuning and selection as well as an outer loop using leave-one-site-out CV for external validation. Feature selection will be conducted using Boruta and LASSO-penalized logistic regression. Data preprocessing steps include missing data imputation, feature scaling, and dimensionality reduction techniques. Six ML algorithms will be tuned and evaluated. Bayesian optimization will be used for hyperparameter selection. Global model explainability will be assessed using permutation importance, and local model explainability will be assessed using SHapley Additive exPlanations (SHAP). The proposed ML model aims to provide a reliable and interpretable tool for clinicians to predict the risk of propofol-associated hypertriglyceridemia in ICU patients. The final model will be deployed in a web-based clinical risk calculator. The model development process and performance measures obtained during nested cross-validation will be described in a study publication to be disseminated in a peer-reviewed journal. The proposed study has received ethics approval from the Mayo Clinic Institutional Review Board (IRB #23-007416). Robust external validation using a nested cross-validation (CV) framework will help assess the generalizability of models produced from the modeling pipeline across different hospital settings.A diverse set of machine learning (ML) algorithms and advanced hyperparameter tuning techniques will be employed to identify the most optimal model configuration.Integration of feature explainability will enhance the clinical applicability of the ML models by providing transparency in predictions, which can improve clinician trust and encourage adoption.Reliance on retrospective data may introduce biases due to inconsistent or erroneous data collection, and the computational intensity of the validation approach may limit replication and future model expansion in resource-constrained settings.

Background This study aimed to establish and validate a machine learning (ML) model for predicting in-hospital mortality in critically ill patients with chronic kidney disease (CKD). Methods This study collected data on CKD patients from 2008 to 2019 using the Medical Information Mart for Intensive Care IV. Six ML approaches were used to build the model. Accuracy and area under the curve (AUC) were used to choose the best model. In addition, the best model was interpreted using SHapley Additive exPlanations (SHAP) values. Results There were 8527 CKD patients eligible for participation; the median age was 75.1 (interquartile range: 65.0–83.5) years, and 61.7% (5259/8527) were male. We developed six ML models with clinical variables as input factors. Among the six models developed, the eXtreme Gradient Boosting (XGBoost) model had the highest AUC, at 0.860. According to the SHAP values, the sequential organ failure assessment score, urine output, respiratory rate, and simplified acute physiology score II were the four most influential variables in the XGBoost model. Conclusions In conclusion, we successfully developed and validated ML models for predicting mortality in critically ill patients with CKD. Among all ML models, the XGBoost model is the most effective ML model that can help clinicians accurately manage and implement early interventions, which may reduce mortality in critically ill CKD patients with a high risk of death.

Current prediction tools for venous thromboembolism (VTE) after colorectal cancer (CRC) surgery lack accuracy for individualized care. To address this critical gap, we developed machine learning (ML) models to dynamically predict postoperative VTE in Chinese CRC surgery patients. We developed ML models using data from 1,836 CRC surgery patients across 46 Chinese centers in the CRC-VTE trial. The cohort was divided into a modeling set (n=1,515) and an external test set (n=321), with the modeling set further split into training and validation subsets (4:1 ratio). Preoperative and postoperative clinical features were selected using recursive feature elimination and Boruta. A total of 162 ML models were developed and evaluated across both preoperative and postoperative datasets, with model performance primarily assessed using the area under the receiver operating characteristic curve (AUROC). The final model underwent rigorous validation in an additional multicenter retrospective cohort and incorporated SHapley Additive exPlanations (SHAP) for risk calculation and decision interpretation. The CatBoost model achieved optimal performance, with AUROCs of 0.950 (preoperative data) and 0.971 (postoperative data) in the validation set, and 0.686±0.037 (preoperative) and 0.715±0.036 (postoperative) in the test set. Models incorporating postoperative data consistently outperformed preoperative-only models. Simplified models based on key predictors (7 preoperative and 9 postoperative features selected by SHAP values) maintained comparable performance, with AUROCs of 0.933 (preoperative data) and 0.949 (postoperative data) in the validation set, and 0.640±0.030 (preoperative data) and 0.695±0.030 (postoperative data) in the test set, respectively. Our study demonstrates the feasibility of a ML-based approach for predicting VTE following CRC surgery. The integration of ML with SHAP methodology provides a clinically actionable tool for individualized risk assessment and optimized VTE prevention strategies.

The allure of aesthetic appeal in images captivates our senses, yet the underlying intricacies of aesthetic preferences remain elusive. In this study, we pioneer a novel perspective by utilizing several different machine learning (ML) models that focus on aesthetic attributes known to influence preferences. Our models process these attributes as inputs to predict the aesthetic scores of images. Moreover, to delve deeper and obtain interpretable explanations regarding the factors driving aesthetic preferences, we utilize the popular Explainable AI (XAI) technique known as SHapley Additive exPlanations (SHAP). Our methodology compares the performance of various ML models, including Random Forest, XGBoost, Support Vector Regression, and Multilayer Perceptron, in accurately predicting aesthetic scores, and consistently observing results in conjunction with SHAP. We conduct experiments on three image aesthetic benchmarks, namely Aesthetics with Attributes Database (AADB), Explainable Visual Aesthetics (EVA), and Personalized image Aesthetics database with Rich Attributes (PARA), providing insights into the roles of attributes and their interactions. Finally, our study presents ML models for aesthetics research, alongside the introduction of XAI. Our aim is to shed light on the complex nature of aesthetic preferences in images through ML and to provide a deeper understanding of the attributes that influence aesthetic judgements.

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

Sudden Sensorineural Hearing Loss (SSNHL) is routinely encountered in otolaryngology clinics. The prognosis of SSNHL varies dramatically and depends on multiple influence factors. This study aimed to develop an explainable Machine Learning (ML) model with easily accessible features to predict the prognosis of SSNHL. This bi-center retrospective study included 534 patients with SSNHL. We randomly split the data into training and validation sets. Univariate analysis and Least Absolute Shrinkage and Selection Operator (LASSO) regression were employed to select predominant features, including demographic, disease-specific characteristics, and laboratory items. We evaluated the performance of five ML models constructed with six crucial variables using the Area Under the receiver operating characteristic Curve (AUC), accuracy, specificity, sensitivity, and F1 scores. These models were further calibrated by calibration curve and Brier score. Clinical utility was evaluated by Decision Curve Analysis (DCA). The Shapley Additive Explanations (SHAP) method was applied to interpret feature contribution and explain the ML models. The Random Forest (RF) model reached the highest AUC of 0.998. Its accuracy, sensitivity, specificity, and F1 score were 0.981, 0.963, 0.989, and 0.967, respectively. DCA curve analysis revealed a comparable net benefit of the models. The SHAP method revealed that the primary features were degree of hearing loss, audiogram type, age, Mean Corpuscular Volume (MCV), onset to treatment, and serum Albumin (ALB) accordingly. The explainable ML model was superb in predicting hearing outcome and providing information on feature contributions. Clinicians can better understand the contributors to hearing recovery and guide aural rehabilitation using the SHAP method. Level 3.

Extracorporeal membrane oxygenation-induced coagulopathy (ECMO-IC) represents a frequent and severe complication, contributing to oxygenator replacement and unfavorable outcomes. Currently, no reliable machine learning (ML) model exists for early identification. This study comprehensively assesses routine clinical characteristics to develop a reliable, accurate, and explainable ML model for estimating ECMO-IC risk and to identify modifiable factors. This study included two center cohorts with 266 patients undergoing ECMO from 2015 to 2024. Feature selection utilized the Boruta algorithm, followed by the implementation of a distinctive ML framework incorporating 12 ML algorithms to establish a consensus prediction model (ECMO-IC index). Model and feature variable assessment employed multiple analytical methods: Bootstrapping and fivefold cross-validation, subgroup and interaction analysis, restricted cubic spline (RCS) regression, and threshold effect analysis. Model interpretation and feature quantification relied on the Shapley Additive Explanations (SHAP) methodology for visualization purposes. Through Boruta algorithm selection, 17 characteristics were identified and incorporated into 12 ML methodologies, generating 105 permutations and an optimal algorithm for identifying ECMO-IC. The ECMO-IC index comprising 9 modifiable or nonmodifiable variables, namely platelet (PLT), lactate, systemic immune-inflammation index (SII), K, total protein (TP), shock index (SI), red blood cell volume distribution width (RDWCV), acute physiology and chronic health evaluation II (APACHE II), and Ca, demonstrated strong diagnostic capabilities, achieving a mean area under the curve (AUC) of 0.815 across derivation (AUC = 0.817) and validation (AUC = 0.813) cohorts, along with notable discriminatory power, model fit, and clinical utility. SHAP elucidates the importance of ranking features (PLT, lactate, K, Ca and APACHE II) and visualises global and individual ECMO-IC risk prediction. RCS regression and threshold effect analysis suggested a nonlinear link between model features (PLT: P for nonlinearity = 0.002, SII: P for nonlinearity = 0.001, K: P for nonlinearity = 0.006, Ca: P for nonlinearity = 0.008, and lactate: P for nonlinearity = 0.004) and ECMO-IC, and generated an inflection point for features (PLT = 95 × 109/L, lactate = 5.7mmol/L, SII = 200, K = 4.4mmol/L, TP = 45.6g/L, SI = 0.8, RDWCV = 14%, APACHE II = 15, Ca = 1.03mmol/L). To provide a more flexible predictive tool, the ECMO-IC model was constructed using a free, publicly available web-based calculator ( https://genglongliu.shinyapps.io/DynNomapp/ ). An optimised explainable ML model (ECMO-IC index) incorporating several modifiable parameters was established and internally validated to deliver an readily available and accurate diagnostic tool for ECMO-IC, with potential applications in ECMO clinical management.