Interpretable Dropout Prediction: Towards XAI-Based Personalized Intervention

Demand forecasting is a crucial element of effective supply chain management, retail planning, and production scheduling. Although models of machine learning (ML) have been reinventing the way demands are predicted with consistently greater accuracy, the complexity can sometimes be a burden to interpret. Such a transparency deficit may impede trust, choices, and adoption in critical business circumstances. XAI methods are applicable in solving this problem because they help in rendering the process of decision-making of complex ML models comprehensible to human stakeholders. In the given article, the authors consider the use of XAI when demand is forecasted, presenting multiple explainability methods: SHAP, LIME, permutation importance, partial dependence plots, and counterfactual explanations. The paper highlights the role of these tools to reveal the motivators behind predictions, enhance user trust, and help users make well-informed decisions

Interpretability in machine learning (ML) has become a critical focus, especially in the context of regulatory compliance and model risk management. As ML models are increasingly employed in high-stakes industries such as finance, healthcare, and insurance, understanding the decisions made by these models is paramount to ensuring compliance with regulatory requirements and managing associated risks. This paper explores the role of interpretability in improving the transparency, accountability, and fairness of machine learning models, highlighting its importance in meeting the standards set by regulatory bodies. It examines the various techniques used to enhance interpretability, including feature importance, surrogate models, and explainable AI (XAI) methods, and discusses their effectiveness in providing insights into model behavior. Furthermore, it emphasizes how interpretability aids in model risk management by enabling organizations to identify and mitigate potential biases, errors, and compliance violations. The paper also considers the challenges associated with interpretability, such as the trade-off between model complexity and transparency, and the need for consistent frameworks that align with evolving regulatory landscapes. In conclusion, the integration of interpretability within ML workflows not only fosters regulatory compliance but also enhances stakeholder trust and supports better decision-making in high-risk environments, thus playing a pivotal role in the responsible deployment of machine learning models.

Many applications are driven by Machine Learning (ML) today. While complex ML models lead to an accurate prediction, their inner decision-making is obfuscated. However, especially for high-stakes decisions, interpretability and explainability of the model are necessary. Therefore, we develop a holistic interpretability and explainability framework (HIEF) to objectively describe and evaluate an intelligent system’s explainable AI (XAI) capacities. This guides data scientists to create more transparent models. To evaluate our framework, we analyse 50 real estate appraisal papers to ensure the robustness of HIEF. Additionally, we identify six typical types of intelligent systems, so-called archetypes, which range from explanatory to predictive, and demonstrate how researchers can use the framework to identify blind-spot topics in their domain. Finally, regarding comprehensiveness, we used a random sample of six intelligent systems and conducted an applicability check to provide external validity.

Abstract. This study investigates the prediction of bending times in a sheet metal workshop using geometric information of parts without a bending sequence computation. Logs from two different sizes of bending machines from an industrial workshop are used, and data-centric machine-learning approaches are applied to achieve accurate predictions. Furthermore, the reduction in bending time with increasing production volume is analyzed as the learning effect in human-operated air bending. Explainable AI techniques, such as Partial Dependence Plots (PDP) and SHAP values, are used to interpret how model predictions are made. The learning effect is examined alongside geometric factors representing bending complexity: the number of bends and the total cutting length. The performances of XGBoost, random forest, and support vector machines (SVM) are compared in terms of bending time prediction, thereby demonstrating the approach's effectiveness.

Artificial intelligence is revolutionizing healthcare. However, its adoption is hindered by the technical-proficiency gap, the disconnect between AI complexity and clinical interpretability. Although Explainable AI (XAI) and Interpretable Machine Learning (IML) techniques aim to address this gap, fragmented research limits the development of generalizable and practical guidance. In response, we introduce FIXAIH, a dynamic, evidence-based framework that consolidates fragmented research in XAI and IML into a coherent structure to guide implementation in healthcare. It is designed to enhance the transparency, accountability, and efficacy of AI systems -- characteristics essential to meet trust and legal requirements, e.g., in the EU AI Act. FIXAIH uses a machine learning-based methodology to holistically analyze literature to identify key parameters shaping the integration of XAI and IML in clinical contexts. We develop a BERT-based tool and examine 5,083 articles, identifying 13 key parameters across three macro-parameters: Research Methods, Health Disorders, and Disease Prevention. This categorization, refined through review of 200+ studies, clarifies applications, trends, and challenges in healthcare XAI. These parameters serve as design elements for developing and aligning AI systems. Drawing on them, FIXAIH supports diagnostic accuracy, treatment efficacy, and risk prediction through, e.g., feature relevance, bias detection, model refinement, data preparation, and expert-guided adjustments. FIXAIH embeds this methodology and adaptable guidelines at its core, evolving as new evidence becomes available, with the intention to expand through integration of social media, clinical insights, regulatory data, and other sources. This adaptive design supports continuous learning and practical implementation, promoting a trustworthy approach to medical innovation.

Soil erosion often leads to land degradation, agricultural production reduction, and environmental deterioration, which seriously restricts the sustainable development of regions. Clarifying the driving factors of soil erosion is the premise of preventing soil erosion. Given the lack of current research on the driving factors/force changes of soil erosion in different regions or under different erosion intensity grades, this paper pioneered to use machine learning methods to address this problem. Firstly, the widely used (Revised) Universal Soil Loss Equation ((R)USLE) framework was applied to simulate the spatial distribution of soil erosion. Then, the K-fold algorithm was used to evaluate the accuracy and stability of five machine learning algorithms for fitting soil erosion. The random forest (RF) method performed best, with average accuracy reaching 86.35%. Then, the Permutation Importance (PI) and the Partial Dependence Plot (PDP) methods based on RF were introduced to quantitatively analyze the main driving factors under different geological conditions and the driving force changes of each factor under different erosion intensity grades, respectively. Results showed that the main drivers of soil erosion in Chongqing and Guizhou were cover management factors (PI: 0.4672, 0.4788), while that in Sichuan was slope length and slope factor (PI: 0.6165). Under different erosion intensity grades, the driving force of each factor shows nonlinear and complex inhibitory or promoting effects with factor value changing. These findings can provide scientific guidance for the refined management of soil erosion, which is significant for halting or reversing land degradation and achieving sustainable use of land resources.

Ultra-high-performance concrete (UHPC) bonded to normal concrete (NC) can significantly enhance the mechanical performance of UHPC–NC composite structures, and the interface shear strength is a crucial indicator for assessing the bonding performance. In this study, interpretable machine learning (ML) methods were used to analyse the effects of different parameters on interface shear strength. A database consisting of 305 UHPC–NC shear tests was created, and the isolation forest algorithm was applied to filter outliers. Subsequently, four ML models were trained to predict the interface shear strength of UHPC–NC composite structures. Among them, the extreme gradient boosting (XGBoost) model demonstrated the highest prediction accuracy, achieving an R2 value of 0.95. Shapley additive explanations (SHAP), partial dependence plots (PDP) and individual conditional expectation (ICE) were used for feature importance analysis, aiding in the interpretation of the ‘black box’ nature of the ML models. The results demonstrate that the normal compressive stress at the interface is the most influential factor affecting interfacial shear strength. Finally, a physically meaningful predictive equation for the interface shear strength of UHPC–NC composite structures was proposed based on the XGBoost model combined with curve fitting. This equation enhances the prediction accuracy of interface shear strength for UHPC–NC structures and offers deeper insights into the model’s decision making process.

Fiber-reinforced polymer (FRP)–concrete bonding is widely adopted for structural strengthening, yet its durability is highly vulnerable to freeze–thaw (FT) degradation. This study combines experimental testing with interpretable machine learning (ML) to reveal the degradation mechanism and predict the interfacial behavior of FRP–concrete systems under FT exposure. Single-lap shear tests showed that all specimens failed through interfacial debonding accompanied by partial concrete peeling. The ultimate bond strength decreased by 6.0–18.5%, and the peak shear stress dropped by 53–80%, indicating a pronounced loss of ductility and adhesion. To extend the analysis, experimental data were integrated with literature datasets, and three ensemble ML algorithms—AdaBoost, Random Forest (RF), and Extreme Gradient Boosting (XGBoost)—were employed to predict key bond–slip parameters including ultimate bond strength, local maximum bond stress, slip values, and interfacial fracture energy. Among them, XGBoost achieved the highest predictive accuracy, with R2 values exceeding 0.94 for most output parameters and consistently low RMSE values. Shapley Additive exPlanations (SHAP) and Partial Dependence Plots (PDPs) further identified adhesive tensile strength, fiber modulus, FRP thickness, and concrete strength as dominant factors and defined their optimal ranges. The findings offer a scientific foundation for evaluating and predicting the long-term bond durability of FRP–concrete systems and support the development of reliable reinforcement strategies for infrastructure in cold and severe environments.

BackgroundAcute respiratory distress syndrome (ARDS) is a prevalent clinical complication among patients with sepsis, characterized by high incidence and mortality rates. The definition of ARDS has evolved over time, with the new global definition introducing significant updates to its diagnosis and treatment. Our objective is to develop and validate an interpretable prediction model for the prognosis of sepsis patients complicated by ARDS, utilizing machine learning techniques in accordance with the new global definition.MethodsThis study extracted data from the MIMIC database (version MIMIC-IV 2.2) to create the training set for our model. For external validation, this study used data from sepsis patients complicated by ARDS who met the new global definition of ARDS, sourced from the Affiliated Hospital of Xuzhou Medical University. Lasso regression with cross-validation was used to identify key predictors of patient prognosis. Subsequently, this study established models to predict the 28-day prognosis following ICU admission using various machine learning algorithms, including logistic regression, random forest, decision tree, support vector machine classifier, LightGBM, XGBoost, AdaBoost, and multi-layer perceptron (MLP). Model performance was assessed using ROC curves, clinical decision curves (DCA), and calibration curves, while SHAP values were utilized to interpret the machine learning models.ResultsA total of 905 patients with sepsis complicated by ARDS were included in our analysis, leading to the selection of 15 key variables for model development. Based on the AUC of the ROC curve, as well as DCA and calibration curve results from the training set, the support vector classifier (SVC) model demonstrated strong performance, achieving an average AUC of 0.792 in the internal validation set and 0.816 in the external validation set.ConclusionThe application of machine learning methodologies to construct prognostic prediction models for sepsis patients complicated by ARDS, informed by the new global definition, proves to be reliable. This approach can assist clinicians in developing personalized treatment strategies for affected patients.

The adoption of complex machine learning (ML) models in recent years has brought along a new challenge related to how to interpret, understand, and explain the reasoning behind these complex models' predictions. Treating complex ML systems as trustworthy black boxes without domain knowledge checking has led to some disastrous outcomes. In this context, interpretability and explainability are often used unintelligibly, and fairness, on the other hand, has become lately popular due to some discrimination problems in ML. While closely related, interpretability and explainability denote different features of prediction. In this sight, the aim of this paper is to give an overview of the interpretability, explainability and the fairness concepts in the literature and to evaluate the performance of the Patient Rule Induction Method (PRIM) concerning these aspects.

This paper presents details of studies conducted to investigate interpretable and explainable machine learning and AI models for cardiovascular disease detection based on the publicly available Cleveland dataset. The study involves evaluating the interpretability and explainability capabilities of tradition shallow machine learning models and their potential for implementation under low resource settings, with limited training data available for model building, as compared to high performing deep learning models, requiring massive training datasets.

With rapid developments in big data technology and the prevalence of large-scale datasets from diverse sources, the healthcare predictive analytics (HPA) field is witnessing a dramatic surge in interest. In healthcare, it is not only important to provide accurate predictions, but also critical to provide reliable explanations to the underlying black-box models making the predictions. Such explanations can play a crucial role in not only supporting clinical decision-making but also facilitating user engagement and patient safety. If users and decision makers do not have faith in the HPA model, it is highly likely that they will reject its use. Furthermore, it is extremely risky to blindly accept and apply the results derived from black-box models, which might lead to undesirable consequences or life-threatening outcomes in domains with high stakes such as healthcare. As machine learning and artificial intelligence systems are becoming more capable and ubiquitous, explainable artificial intelligence and machine learning interpretability are garnering significant attention among practitioners and researchers. The introduction of policies such as the General Data Protection Regulation (GDPR), has amplified the need for ensuring human interpretability of prediction models. In this talk I will discuss methods and applications for developing local as well as global explanations from machine learning and the value they can provide for healthcare prediction.

Hydrothermal carbonization (HTC) is a promising process for biomass valorization; however, optimizing HTC conditions and characterizing hydrochar through experimental methods remain costly and time-consuming. Artificial intelligence (AI) and its related machine learning (ML) techniques provide an efficient and cost-effective alternative, enabling efficient optimization and predictive analysis without extensive experimental tests. In this study, an analysis-ready database comprising 544 data points was constructed using the authors' previous research (41) and scattered data compiled from the literature (503). An ensemble of eight diversified machine learning (ML) models was developed using biomass-agnostic properties and process conditions to predict hydrochar properties including elemental analysis, proximate analysis, and hydrochar yield. Tailor-made decision fusion models were developed for each target by merging the outputs of the best-performing models. Furthermore, a set of interpretable machine learning (IML) and explainable AI (XAI) techniques, leveraging feature importance analysis and SHapley Additive exPlanations (SHAP) values, indicated that biomass fixed carbon (FC) content and process temperature are the most influencing features, which were then considered as inputs for the ensemble models. The decision fusion models achieved high accuracy for target prediction, surpassing the currently used models in the literature, with adjusted R² values ranging from 0.98 to 1.0. For outputs that are typically difficult to predict, such as hydrochar yield, the model achieved an adjusted R² of 0.98, representing over a 5% improvement compared to the best-performing model reported in the literature. All predictions were derived from white-box modeling by incorporating XAI into the ensemble-based learning, ensuring greater model transparency and interpretability. • Human-centric AI framework developed to model HTC and predict hydrochar properties. • Eight diverse ML models fused via decision fusion to boost accuracy and robustness. • Models achieved high predictive power (Test R² = 0.98–1.00) for studied properties • SHAP explainability enabled transparent, white-box modeling for HTC process

This chapter examines the domain of interpretable machine learning with a focus on methods and tools that enhance the transparency and interpretability of machine learning models, specifically within the context of official statistics. We explore various post hoc interpretation methods that provide insights into model behavior, such as loss-based feature importance methods, where we give guidance on how to choose between different such methods. Additionally, we discuss counterfactual and semi-factual explanations (SFEs) and explicitly describe interpretable regional descriptors as an example of a method generating SFEs. Furthermore, we highlight innovative contributions such as the R package mlr3summary, which facilitates model diagnostics and performance evaluation for black-box machine learning models through resampling strategies. This package provides a unified, model-agnostic summary of machine learning models, which facilitates model selection and comparison. Finally, we discuss the importance of ethical considerations and fairness in machine learning applications, emphasizing the need for methods that can address historical biases in data. The objective of this chapter is to provide practitioners and researchers with the practical tools and comprehensive knowledge required to implement interpretable and fair machine learning models, in a variety of domains—but especially in official statistics.

Machine learning-based design of target property-oriented fuels using explainable artificial intelligence