Determining the Major Contributing Features to Predict Breast Cancer Imposing ML Algorithms with LIME and SHAP

Blood pressure abnormality detection and Interpretation utilizing Explainable Artificial Intelligence

Background Predicting the prognosis of patients with acute myocardial infarction (AMI) combined with diabetes mellitus (DM) is crucial due to high in-hospital mortality rates. This study aims to develop and validate a mortality risk prediction model for these patients by interpretable machine learning (ML) methods. Methods Data were sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV, version 2.2). Predictors were selected by Least absolute shrinkage and selection operator (LASSO) regression and checked for multicollinearity with Spearman’s correlation. Patients were randomly assigned to training and validation sets in an 8:2 ratio. Seven ML algorithms were used to construct models in the training set. Model performance was evaluated in the validation set using metrics such as area under the curve (AUC) with 95% confidence interval (CI), calibration curves, precision, recall, F1 score, accuracy, negative predictive value (NPV), and positive predictive value (PPV). The significance of differences in predictive performance among models was assessed utilising the permutation test, and 10-fold cross-validation further validated the model’s performance. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) were applied to interpret the models. Results The study included 2,828 patients with AMI combined with DM. Nineteen predictors were identified through LASSO regression and Spearman’s correlation. The Random Forest (RF) model was demonstrated the best performance, with an AUC of 0.823 (95% CI: 0.774–0.872), high precision (0.867), accuracy (0.873), and PPV (0.867). The RF model showed significant differences (p < 0.05) compared to the K-Nearest Neighbours and Decision Tree models. Calibration curves indicated that the RF model’s predicted risk aligned well with actual outcomes. 10-fold cross-validation confirmed the superior performance of RF model, with an average AUC of 0.828 (95% CI: 0.800–0.842). Significant Variables in RF model indicated that the top eight significant predictors were urine output, maximum anion gap, maximum urea nitrogen, age, minimum pH, maximum international normalised ratio (INR), mean respiratory rate, and mean systolic blood pressure. Conclusion This study demonstrates the potential of ML methods, particularly the RF model, in predicting in-hospital mortality risk for AMI patients with DM. The SHAP and LIME methods enhance the interpretability of ML models.

Prediction of a machine’s Remaining Useful Life (RUL) is one of the key tasks in predictive maintenance. The task is treated as a regression problem where Machine Learning (ML) algorithms are used to predict the RUL of machine components. These ML algorithms are generally used as a black box with a total focus on the performance without identifying the potential causes behind the algorithms’ decisions and their working mechanism. We believe, the performance (in terms of Mean Squared Error (MSE), etc.,) alone is not enough to build the stakeholders’ trust in ML prediction rather more insights on the causes behind the predictions are needed. To this aim, in this paper, we explore the potential of Explainable AI (XAI) techniques by proposing an explainable regression framework for the prediction of machines’ RUL. We also evaluate several ML algorithms including classical and Neural Networks (NNs) based solutions for the task. For the explanations, we rely on two model agnostic XAI methods namely Local Interpretable Model-Agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP). We believe, this work will provide a baseline for future research in the domain.

Student dropout is a significant challenge in Bangladesh, with serious impacts on both educational and socio-economic outcomes. This study investigates the factors influencing school dropout among students aged 6–24 years, employing data from the 2019 Multiple Indicator Cluster Survey (MICS). The research integrates statistical analysis with machine learning (ML) techniques and explainable AI (XAI) to identify key predictors and enhance model interpretability. Initially, descriptive and inferential statistical analyses were applied to identify significant predictors and guide feature selection. The hybrid feature selection strategy, combining statistical significance and model-based importance measures, revealed key features. Logistic regression was applied to identify statistically significant predictors of school dropout. ML algorithms, Random Forest (RF) and Extreme Gradient Boosting (XGB), were used to build predictive models. Model performance was evaluated using accuracy, precision, recall, and an F1 score. The XGB model achieved the best performance with an accuracy of 94.4%, followed by the RF model. To interpret model predictions and ensure transparency, SHAP (Shapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations) were employed in tandem with the statistical analyses. Key factors influencing student dropout included age, sex, completed grade, last education grade, division, wealth index, father’s and mother’s education. These insights offer a data-driven foundation for policymakers to develop targeted intervention strategies aimed at reducing student dropout rates and improving educational outcomes in Bangladesh.

Explainable machine learning models for defects detection in industrial processes

This study presents a comprehensive methodology for developing a Fault Detection and Diagnosis (FDD) system for an industrial distillation column using advanced machine learning algorithms. Steady state and dynamic simulations in Aspen Plus® generate extensive datasets under normal and faulty conditions. Feature engineering, using the Minimum Redundancy Maximum Relevance (MRMR) algorithm, selects the most relevant features for fault detection. Various machine learning models, including Decision Trees, Support Vector Machines, k-nearest Neighbours, and Neural Networks, were trained and evaluated based on performance metrics such as accuracy, recall, precision, and F1 score. The top models were integrated into a stacked classifier system with a voting mechanism to enhance fault detection reliability. Explainable Artificial Intelligence (XAI) techniques, such as Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP), were incorporated to improve model interpretability, allowing engineers to understand and validate the FDD system's decision-making process. Simulation results confirm that the proposed methodology accurately identifies and classifies faults. By integrating dynamic simulations, advanced machine learning, and XAI techniques, a robust and scalable solution is achieved for fault detection in distillation columns, improving operational reliability, safety, and reducing downtime. Future work could extend this approach to other industrial processes and explore additional machine learning algorithms to further enhance performance.

Rising shipping emissions greatly affect greenhouse gas (GHG) levels, so precise fuel consumption forecasting is essential to reduce environmental effects. Precision forecasts using machine learning (ML) could offer sophisticated solutions that increase the fuel efficiency and lower emissions. Indeed, five ML techniques, linear regression (LR), decision tree (DT), random forest (RF), XGBoost, and AdaBoost, were used to develop ship fuel consumption models in this study. It was found that, with an R² of 1, zero mean squared error (MSE), and a negligible mean absolute percentage error (MAPE), the DT model suited the training set perfectly, while R² was 0.8657, the MSE was 56.80, and the MAPE was 16.37% for the DT model testing. More importantly, this study provided Taylor diagrams and violin plots that helped in the identification of the best-performing models. Generally, the employed ML approaches efficiently predicted the data; however, they are black-box methods. Hence, explainable machine learning methods like Shapley additive explanations, the DT structure, and local interpretable model-agnostic explanations (LIME) were employed to comprehend the models and perform feature analysis. LIME offered insights, demonstrating that the major variables impacting predictions were distance (≤450.88 nm) and time (40.70 &lt; hr ≤ 58.05). By stressing the most important aspects, LIME can help one to comprehend the models with ease.

In the field of network security, botnet attacks pose a significant challenge, exploiting networks of infected devices to launch sophisticated threats. As these attacks evolve, the need for effective detection methods becomes increasingly critical. This study proposes an explainable Machine Learning (ML) model that aims to identify and categorize botnet attacks, and it investigates the efficiency of different Explainable Artificial Intelligence (XAI) techniques for Intrusion Detection Systems (IDS). To train our models, we employed a subset of the NCC-2 dataset, which includes a mix of normal traffic and seven different types of botnet attacks across three sensors. The ML techniques selected for this research are Random Forest (RF), Extra Trees (ET), Decision Tree (DT), and K-Nearest Neighbors (KNN), with a GridSearch cross-validation approach for optimal hyperparameter tuning. We also explored the effects of class balance through Synthetic Minority Oversampling (SMOTE) and Random Undersampling. The models’ performance was rigorously tested using the Matthews Correlation Coefficient (MCC) and Macro F1-score, with the ET model demonstrating superior results of 99% MCC and 97% F1-score, respectively. To enhance the interpretability of the ET model’s decision-making process, we integrated three XAI techniques: SHapley Additive exPlanations (SHAP), Dalex, and Local Interpretable Model-agnostic Explanations (LIME), and evaluated their efficiency. The LIME and Dalex techniques showed efficient construction times for IDS.

Heart disease remains a major global health challenge, underscoring the need for predictive models that are both accurate and interpretable to support early diagnosis and clinical decision-making. Using Shapley Additive Explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) in this study allowed for a deeper level of understanding of model predictions as well as provided viable alternative ways to overcome the challenges associated with black-box classifiers. Using a soft-voting ensemble technique that combined representations for Random Forest (RF), Decision Tree (DT), and Support Vector Machines (SVM) was applied to evaluate a publicly available heart disease dataset, containing 5110 records with each record containing 76 attributes. A total of 14 clinically relevant features were identified and evaluated using an 80/20 train-test split for the experimental setup, with results further validated by 5-fold cross-validation. The proposed soft-voting ensemble model achieved an accuracy of 89.10%, precision of 83.90%, recall of 95.20%, and F1-score of 89.90%, out-performing all individual baseline models and many of the recent XAI-enhanced deep learning models. The combination of SHAP and LIME allowed for both global and local interpretations of the model’s outputs, providing interpretable insights as to features like cp, thal, ca, and oldpeak contributed to the predictions and aligning these features with known cardiovascular risk factors. The results of this research demonstrate how interpretable (or explainable) ensemble-learning mechanisms can confer highly reliable, clinically useful heart disease risk predictions.

Air pollution in Macau has become a serious problem following the Pearl River Delta’s (PRD) rapid industrialization that began in the 1990s. With this in mind, Macau needs an air quality forecast system that accurately predicts pollutant concentration during the occurrence of pollution episodes to warn the public ahead of time. Five different state-of-the-art machine learning (ML) algorithms were applied to create predictive models to forecast PM2.5, PM10, and CO concentrations for the next 24 and 48 h, which included artificial neural networks (ANN), random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and multiple linear regression (MLR), to determine the best ML algorithms for the respective pollutants and time scale. The diurnal measurements of air quality data in Macau from 2016 to 2021 were obtained for this work. The 2020 and 2021 datasets were used for model testing, while the four-year data before 2020 and 2021 were used to build and train the ML models. Results show that the ANN, RF, XGBoost, SVM, and MLR models were able to provide good performance in building up a 24-h forecast with a higher coefficient of determination (R2) and lower root mean square error (RMSE), mean absolute error (MAE), and biases (BIAS). Meanwhile, all the ML models in the 48-h forecasting performance were satisfactory enough to be accepted as a two-day continuous forecast even if the R2 value was lower than the 24-h forecast. The 48-h forecasting model could be further improved by proper feature selection based on the 24-h dataset, using the Shapley Additive Explanations (SHAP) value test and the adjusted R2 value of the 48-h forecasting model. In conclusion, the above five ML algorithms were able to successfully forecast the 24 and 48 h of pollutant concentration in Macau, with the RF and SVM models performing the best in the prediction of PM2.5 and PM10, and CO in both 24 and 48-h forecasts.

PurposeEarly identification of children at risk for accelerated axial elongation is essential for implementing timely myopia control strategies. Quantitative parameters derived from color fundus photography (CFP) may capture subtle structural and microvascular features relevant to axial length (AL) progression, yet their predictive value remains insufficiently characterized. To develop and validate a machine learning–based model integrating CFP-derived quantitative biomarkers and clinical characteristics to predict 1-year AL progression in school-aged children.MethodsThis cohort study included 693 children aged 6–10 years from Tianjin, China. AL progression >0.2 mm over 1 year was defined as significant elongation. Baseline clinical variables and 144 quantitative CFP metrics were evaluated. Feature selection was performed using Least Absolute Shrinkage and Selection Operator (LASSO) regression, logistic regression screening, and expert ophthalmologic assessment. Seven machine learning algorithms were developed using fivefold cross-validation, with hyperparameters optimized by grid search. Model performance was evaluated on an independent validation set using the area under the receiver operating characteristic (ROC) curve (AUC), F1 score, and other metrics. The best-performing model was interpreted using Shapley Additive Explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (lime).ResultsOf the 693 included children, 457 (65.9%) exhibited AL progression >0.2 mm. LASSO regression selected 39 candidate variables, and 12 predictors were ultimately incorporated into the model construction. Among all algorithms, the Random Forest (RF) model achieved the best discrimination, with an AUC of 0.961 (95% CI: 0.933–0.984) and the highest F1 score. Decision curve analysis (DCA) demonstrated a favorable net benefit across clinically relevant thresholds. SHAP analysis indicated that retinal venous density, venous fractal dimension, presence of leopard-spot lesions, vascular fractal dimension, and inferior-region vascular density were among the most influential predictors of AL progression.ConclusionThe RF model, which combines clinical characteristics with CFP-derived quantitative biomarkers, accurately predicts short-term AL progression in children. Retinal microvascular and fundus structural parameters significantly contributed to model performance, underscoring their potential as early indicators of myopic AL elongation.

In recent times, it has been observed that social media exerts a favorable influence on consumer purchasing behavior. Many organizations are adopting the utilization of social media platforms as a means to promote products and services. Hence, it is crucial for enterprises to understand the consumer buying behavior in order to thrive. This article presents a novel approach that combines the theory of planned behavior (TPB) with machine learning techniques to develop accurate predictive models for consumer purchase behavior. This study examines three distinct factors of the theory of planned behavior (attitude, social norm, and perceived behavioral control) that provide insights into the primary determinants influencing online purchasing behavior. A total of eight machine learning algorithms, namely K-nearest neighbor, Decision Tree, Random Forest, Logistic Regression, Naive Bayes, Support Vector Machine, AdaBoost, and Gradient Boosting, were utilized in order to forecast consumer purchasing behavior. Empirical findings indicate that gradient boosting demonstrates superior performance in predicting customer buying behavior, with an accuracy rate of 0.91 and a macro F1 score of 0.91. This holds true when all factors, namely attitude (ATTD), social norm (SN), and perceived behavioral control (PBC), are included in the analysis. Furthermore, we incorporated Explainable AI (XAI), specifically LIME (Local Interpretable Model-Agnostic Explanations), to elucidate how the best machine learning model (i.e. gradient boosting) makes its prediction. The findings indicate that LIME has demonstrated a high level of confidence in accurately predicting the influence of low and high behavior. The outcome presented in this article has several implications. For instance, this article presents a novel way to combine the theory of planned behavior with machine learning techniques in order to predict consumer purchase behavior. This integration allows for a comprehensive analysis of factors influencing online purchasing decisions. Also, the incorporation of Explainable AI enhances the transparency and interpretability of the model. This feature is valuable for organizations seeking insights into factors driving predictions and the reasons behind certain outcomes. Moreover, these observations have the potential to offer valuable insights for businesses in customizing their marketing strategies to align with these influential factors.

XAI is critical for establishing trust and enabling the appropriate development of machine learning models. By offering transparency into how these models make judgements, XAI enables researchers and users to uncover potential biases, admit limits, and eventually enhance the fairness and dependability of AI systems. In this paper, we demonstrates two techniques, LIME and SHAP, used to improve the interpretability of machine learning models. Assessing Explainable AI (XAI) approaches is critical in searching for transparent and interpretable artificial intelligence (AI) models. Explainable AI (XAI) approaches are designed to provide insight into how complex models make decisions. This paper thoroughly analyzes two prominent XAI methods: Shapley Additive explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME). This study aims to understand the decision made by a machine learning model and how the model came to that decision. We discuss the approaches and framework of both LIME and SHAP and assess their behavior in predicting the model's outcome.

To construct machine learning prediction model for sepsis-associated encephalopathy (SAE), and analyze the application value of the model on early identification of SAE risk in elderly septic patients. Patients aged over 60 years with a primary diagnosis of sepsis admitted to intensive care unit (ICU) from 2008 to 2023 were selected from Medical Information Mart for Intensive Care-IV 2.2 (MIMIC-IV 2.2). Demographic variables, disease severity scores, comorbidities, interventions, laboratory indicators, and hospitalization details were collected. Key factors associated with SAE were identified using univariate Logistic regression analysis. The data were randomly divided into training and validation sets in a 7 : 3 ratio. Multivariable Logistic regression analysis was conducted in the training set and visualized using a nomogram model for prediction of SAE. The discrimination of the model was evaluated in the validation set using the receiver operator characteristic curve (ROC curve), and its calibration was assessed using calibration curve. Furthermore, multiple machine learning algorithms, including multi-layer perceptron (MLP), support vector machine (SVM), naive bayes (NB), gradient boosting machine (GBM), random forest (RF), and extreme gradient boosting (XGB), were constructed in the training set. Their predictive performance was subsequently evaluated on the validation set. Taking the XGB model as an example, the interpretability of the model through the SHapley Additive exPlanations (SHAP) algorithm was enhanced to identify the key predictive factors and their contributions. A total of 2 204 septic patients were finally enrolled, of whom 840 developed SAE (38.1%). A total of 21 variables associated with SAE were screened through univariate Logistic regression analysis. Multivariable Logistic regression analysis showed that endotracheal intubation [odds ratio (OR) = 0.40, 95% confidence interval (95%CI) was 0.19-0.88, P < 0.001], oxygen therapy (OR = 0.76, 95%CI was 0.53-0.95, P = 0.023), tracheotomy (OR = 0.20, 95%CI was 0.07-0.53, P < 0.001), continuous renal replacement therapy (CRRT; OR = 0.32, 95%CI was 0.15-0.70, P < 0.001), cerebrovascular disease (OR = 0.31, 95%CI was 0.16-0.60, P < 0.001), rheumatic disease (OR = 0.44, 95%CI was 0.19-0.99, P < 0.001), male (OR = 0.68, 95%CI was 0.54-0.86, P = 0.001), and maximum anion gap (AG; OR = 0.95, 95%CI was 0.93-0.97, P < 0.001) were associated with an decreased probability of SAE, and age (OR = 1.05, 95%CI was 1.03-1.06, P < 0.001), acute physiology score III (APSIII; OR = 1.02, 95%CI was 1.01-1.02, P < 0.001), Oxford acute severity of illness score (OASIS; OR = 1.04, 95%CI was 1.03-1.06, P < 0.001), and length of hospital stay (OR = 1.01, 95%CI was 1.01-1.02, P < 0.001) were associated with an increased probability of SAE. A nomogram model was constructed based on these variables. In the validation set, ROC curve analysis showed that the model achieved an area under the ROC curve (AUC) of 0.723, and the calibration curve showed good consistency between the predicted probability of the model and the observed probability. Among the machine learning algorithms, including MLP, SVM, NB, GBM, RF, and XGB, the SVM model and RF model demonstrated relatively good predictive performance, with AUC of 0.748 and 0.739, respectively, and the sensitivity was both exceeding 85%. The predictive performance of the XGB model was explained through SHAP analysis, and the results indicated that APSIII score (SHAP value was 0.871), age (SHAP value was 0.521), and OASIS score (SHAP value was 0.443) were important factors affecting the predictive performance of the model. The machine learning-based SAE prediction model exhibits good predictive capability and holds significant application value for the early identification of SAE risk in elderly septic patients.

Objective:The aim of this study was to develop and validate a machine learning model to predict the risk of comorbid depression in asthma patients.Methods:We conducted a retrospective study of 2464 asthma patients with comorbid depression using National Health and Nutrition Examination Survey (NHANES) data. Feature selection was conducted using the Boruta algorithm and the Least Absolute Shrinkage and Selection Operator (LASSO). Eight machine learning algorithms, namely Decision Tree (DT), k-Nearest Neighbors (KNN), Light Gradient Booster Machine (LGBM), Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), eXtreme Gradient Boosting (XGBoost), and Multilayer Perceptron (MLP), were trained using 5-fold cross-validation methodology. Model performance was evaluated through various metrics such as area under the curve (AUC), accuracy, sensitivity, specificity, F1 score, and decision curve analysis (DCA). Interpretation was conducted using SHapley Additive exPlanations (SHAP) analysis, highlighting feature importance.Results:The training set comprised 1724 participants, while the validation set included 740 participants, with a depression prevalence of 14.45%. Significant predictors identified included hypertension, chronic obstructive pulmonary disease (COPD), stroke, sleep questionnaire (SLQ) scores, smoking status, Poverty Index Ratio (PIR), and educational level. The XGBoost model demonstrated superior performance compared with alternative machine learning (ML) algorithms, achieving an AUC of 0.750, an accuracy of 69.1%, a sensitivity of 68.2%, a specificity of 73.8%, and an F1 score of 79%. The SHAP method identified SLQ, PIR, and education level as the primary decision factors influencing the ML model’s predictions.Conclusion:The XGBoost model effectively predicts the risk of depression in asthma patients, serving as a valuable reference for early clinical identification and intervention.