Explanations based on Item Response Theory (eXirt): A model-specific method to explain tree-ensemble model in trust perspective

Background: Water is fundamental to the survival of all life forms, yet access to clean and safe water remains a critical challenge worldwide. Contaminated water is a significant contributor to waterborne diseases, highlighting the need for effective water quality monitoring. The Water Quality Index (WQI) is a standard tool for assessing water quality; however, traditional WQI methods are often constrained by inconsistencies, laboratory inaccuracies, and human error. Methods: This study aimed to overcome these limitations by integrating advanced machine learning (ML) techniques into WQI prediction. Physicochemical parameters, including pH, chloride (Cl?), sulfate (SO4²?), sodium (Na?), potassium (K?), calcium (Ca²?), magnesium (Mg²?), total hardness, and total dissolved solids, were collected from diverse water sources to form a robust dataset. ML algorithms such as Gradient Boosting, Random Forest, and XGBoost, augmented with explainable AI (XAI), were employed to enhance prediction accuracy. The dataset was split into training (70%), testing (15%), and validation (15%) subsets, and model performance was assessed using RMSE, MSE, MAE, and R² metrics. Results: Gradient Boosting outperformed other models, achieving 96% accuracy on the test dataset after fine-tuning. It demonstrated superior predictive capabilities, as evidenced by its performance metrics. These results indicate the potential for ML techniques to address the limitations of traditional WQI methods. Conclusion: This study demonstrates the effectiveness of ML-driven approaches in improving water quality assessments. The integration of Gradient Boosting and explainable AI provides a reliable framework for WQI prediction, enabling better decision-making in environmental health policies and water resource management. This approach offers a pathway to more efficient and accurate water quality monitoring systems.

Opening the Black Box: The Promise and Limitations of Explainable Machine Learning in Cardiology.

In this paper, we suggest an explainable machine learning approach to model the claim frequency of a telematics car dataset. In fact, we use a data-driven method based on tree ensembles, namely, the random forest, to create a claim frequency model. Then, we present a method to build a tree that faithfully synthesizes the predictions of a tree ensemble model such as those derived from the random forest or gradient boosting. This tree serves as a global explanation of the predictions of the black-box. Thanks to this surrogate model, we can extract knowledge from a black-box tree ensemble model. Then, we provide an application to improve the performance of a generalized linear model. Indeed, we integrate this new knowledge into a generalized linear model to increase the predictive power.

Explainable Artificial Intelligence (XAI) aims to introduce transparency and intelligibility into the decision-making process of AI systems. Most often, its application concentrates on supervised machine learning problems such as classification and regression. Nevertheless, in the case of unsupervised algorithms like clustering, XAI can also bring satisfactory results. In most cases, such application is based on the transformation of an unsupervised clustering task into a supervised one and providing generalised global explanations or local explanations based on cluster centroids. However, in many cases, the global explanations are too coarse, while the centroid-based local explanations lose information about cluster shape and distribution. In this paper, we present a novel approach called CLAMP (Cluster Analysis with Multidimensional Prototypes) that aids experts in cluster analysis with human-readable rule-based explanations. The developed state-of-the-art explanation mechanism is based on cluster prototypes represented by multidimensional bounding boxes. This allows representing of arbitrary shaped clusters and combines the strengths of local explanations with the generality of global ones.We demonstrate and evaluate the use of our approach in a real-life industrial case study from the domain of steel manufacturing as well as on the benchmark datasets. The explanations generated with CLAMP were more precise than either centroid-based or global ones.

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the global behavior of the agent, describing the actions it takes in different states. Other approaches devised local explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a set of world-states that are likely to appear during gameplay. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps, in the form of raw heat maps, did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on during its decision-making, suggesting avenues for future work that can further improve explanations of RL agents.

In recent times, it has been observed that social media exerts a favorable influence on consumer purchasing behavior. Many organizations are adopting the utilization of social media platforms as a means to promote products and services. Hence, it is crucial for enterprises to understand the consumer buying behavior in order to thrive. This article presents a novel approach that combines the theory of planned behavior (TPB) with machine learning techniques to develop accurate predictive models for consumer purchase behavior. This study examines three distinct factors of the theory of planned behavior (attitude, social norm, and perceived behavioral control) that provide insights into the primary determinants influencing online purchasing behavior. A total of eight machine learning algorithms, namely K-nearest neighbor, Decision Tree, Random Forest, Logistic Regression, Naive Bayes, Support Vector Machine, AdaBoost, and Gradient Boosting, were utilized in order to forecast consumer purchasing behavior. Empirical findings indicate that gradient boosting demonstrates superior performance in predicting customer buying behavior, with an accuracy rate of 0.91 and a macro F1 score of 0.91. This holds true when all factors, namely attitude (ATTD), social norm (SN), and perceived behavioral control (PBC), are included in the analysis. Furthermore, we incorporated Explainable AI (XAI), specifically LIME (Local Interpretable Model-Agnostic Explanations), to elucidate how the best machine learning model (i.e. gradient boosting) makes its prediction. The findings indicate that LIME has demonstrated a high level of confidence in accurately predicting the influence of low and high behavior. The outcome presented in this article has several implications. For instance, this article presents a novel way to combine the theory of planned behavior with machine learning techniques in order to predict consumer purchase behavior. This integration allows for a comprehensive analysis of factors influencing online purchasing decisions. Also, the incorporation of Explainable AI enhances the transparency and interpretability of the model. This feature is valuable for organizations seeking insights into factors driving predictions and the reasons behind certain outcomes. Moreover, these observations have the potential to offer valuable insights for businesses in customizing their marketing strategies to align with these influential factors.

With rapid developments in big data technology and the prevalence of large-scale datasets from diverse sources, the healthcare predictive analytics (HPA) field is witnessing a dramatic surge in interest. In healthcare, it is not only important to provide accurate predictions, but also critical to provide reliable explanations to the underlying black-box models making the predictions. Such explanations can play a crucial role in not only supporting clinical decision-making but also facilitating user engagement and patient safety. If users and decision makers do not have faith in the HPA model, it is highly likely that they will reject its use. Furthermore, it is extremely risky to blindly accept and apply the results derived from black-box models, which might lead to undesirable consequences or life-threatening outcomes in domains with high stakes such as healthcare. As machine learning and artificial intelligence systems are becoming more capable and ubiquitous, explainable artificial intelligence and machine learning interpretability are garnering significant attention among practitioners and researchers. The introduction of policies such as the General Data Protection Regulation (GDPR), has amplified the need for ensuring human interpretability of prediction models. In this talk I will discuss methods and applications for developing local as well as global explanations from machine learning and the value they can provide for healthcare prediction.

Ensemble models such as gradient boosting and random forests are proven to offer the best predictive performance on a wide variety of supervised learning problems. The high performance of these black box models, however, comes at a cost of model interpretability. They are also inadequate to meet regulatory demands and explainability needs of organizations. The model interpretability in high performance black-box models is achieved with the help of post-hoc explainable models such as Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). This paper presents an alternate intrinsic classifier model that extracts a class of higher order patterns and embeds them into an interpretable learning model. More specifically, the proposed model extracts novel High Utility Gain (HUG) patterns that capture higher order interactions, transforms the model input data into a new space, and applies interpretable classifier methods on the transformed space. We conduct rigorous experiments on forty benchmark binary and multi-class classification datasets to evaluate the proposed model against the state-of-the-art ensemble and interpretable classifier models. The proposed model was comprehensively assessed on three key dimensions: 1) quality of predictions using classifier measures such as accuracy, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F_{1}$ </tex-math></inline-formula>, AUC, H-measure, and logistic loss, 2) computational performance on large and high-dimensional data, and 3) interpretability aspects. The HUG-based learning model was found to deliver performance comparable to that of the state-of-the-art ensemble models. Our model was also found to achieve 2-40% (45%) prediction quality (interpretability) improvements with significantly lower computational requirements over other interpretable classifier models. Furthermore, we present case studies in finance and healthcare domains and generate one- and two-dimensional HUG profiles to illustrate the interpretability aspects of our HUG models. The proposed solution offers an alternate approach to build high performance and transparent machine learning classifier models. We hope that our ML solution help organizations meet their growing regulatory and explainability needs.

AI as a concept has been around since the 1950's. With the recent advancements in machine learning technology, and the availability of big data and large computing processing power, the scene is set for AI to be used in many more systems and applications which will profoundly impact society. The current deep learning based AI systems are mostly in black box form and are often non-explainable. Though it has high performance, it is also known to make occasional fatal mistakes. This has limited the applications of AI, especially in mission critical problems such as decision support, command and control, and other life-critical operations. This talk focuses on explainable AI, which holds promise in helping humans to better understand and interpret the decisions made by black-box AI models. Current research efforts towards explainable multimedia AI center on two parts of solution. The first part focuses on better understanding of multimedia content, especially video. This includes dense annotation of video content from not just object recognition, but also relation inference. The relation includes both correlation and causality relations, as well as common sense knowledge. The dense annotation enables us to transform the level of representation of video towards that of language, in the form of relation triplets and relation graphs, and permits in-depth research on flexible descriptions, question-answering and knowledge inference of video content. A large scale video dataset has been created to support this line of research. The second direction focuses on the development of explainable AI models, which are just beginning. Existing works focus on either the intrinsic approach, which designs self-explanatory models, or post-hoc approach, which constructs a second model to interpret the target model. Both approaches have limitations on trade-offs between interpretability and accuracy, and the lack of guarantees about the explanation quality. In addition, there are issues of quality, fairness, robustness and privacy in model interpretation. In this talk, I present current state-of-the arts approaches in explainable multimedia AI, along with our preliminary research on relation inference in videos, as well as leveraging prior domain knowledge, information theoretic principles, and adversarial algorithms to achieving interpretability. I will also discuss future research towards quality, fairness and robustness of interpretable AI.

Accurate prediction of diabetes based on clinical and demographic indicators is essential, as early prediction of this chronic metabolic disorder plays a critical role in preventing long-term organ complications. However, existing research continues to face significant challenges, including pronounced class imbalance, the scarcity of large and diverse datasets, and limited integration of explainable artificial intelligence. This research compares several ensemble learning methods (Decision Tree, Random Forest, AdaBoost, Gradient Boosting, Histogram based Gradient Boosting, extremely randomized Trees, and XGBoost) on a large imbalanced dataset (87,664 negative vs. 8,482 positive samples). To mitigate imbalance, we evaluate six resampling approaches including (Random Over Sampling, Random Under Sampling, Synthetic Minority Over-sampling, Adaptive Synthetic Sampling, Tomek Links Removal Sampling and (Synthetic Minority Over-sampling with Edited Nearest Neighbors). We assess models using metrics robust to class imbalance (precision, recall, F1, AUC-ROC, and AUC-PR) and calibration measures. The Extra Trees classifier achieved the highest measured accuracy (0.994); with Random Over Sampling for balancing dataset. also, these results were compared with several previous works and number of machine learning algorithms, and the results showed superiority. Explainability is performed at both global and local levels: permutation and SHAP for global feature importance, and (Local Interpretable Model-Agnostic Explanations) force plots for instance-level reasoning. however, we analyzed this result using sensitivity, specificity, PR-AUC and calibration, we report detailed experiments showing how resampling method, hyperparameter tuning, and stratified validation influence performance. Finally, we provide clinical-relevant insights from SHAP analyses and discuss limitations and future directions for deploying interpretable models in screening workflows.

This research proposes a novel strategy for accurate breast lesion classification that combines explainable artificial intelligence (XAI), machine learning (ML) classifiers, and customized weakly dependent features from ultrasound (BU) images. Two new weakly dependent feature classes are proposed to improve the diagnostic accuracy and diversify the training data. These are based on image intensity variations and the area of bounded partitions and provide complementary rather than overlapping information. ML classifiers such as Random Forest (RF), Extreme Gradient Boosting (XGB), Gradient Boosting Classifiers (GBC), and LASSO regression were trained with both customized feature classes. To validate the reliability of our study and the results obtained, we conducted a statistical analysis using the McNemar test. Later, an XAI model was combined with ML to tackle the influence of certain features, the constraints of feature selection, and the interpretability capabilities across various ML models. LIME (Local Interpretable Model-Agnostic Explanations) and SHAP (SHapley Additive exPlanations) models were used in the XAI process to enhance the transparency and interpretation in clinical decision-making. The results revealed common relevant features for the malignant class, consistently identified by all of the classifiers, and for the benign class. However, we observed variations in the feature importance rankings across the different classifiers. Furthermore, our study demonstrates that the correlation between dependent features does not impact explainability.

The use of artificial intelligence (AI) systems is significantly increased in the past few years. AI system is expected to provide accurate predictions and it is also crucial that the decisions made by the AI systems are humanly interpretable i.e. anyone must be able to understand and comprehend the results produced by the AI system. AI systems are being implemented even for simple decision support and are easily accessible to the common man on the tip of their fingers. The increase in usage of AI has come with its own limitation, i.e. its interpretability. This work contributes towards the use of explainability methods such as local interpretable model-agnostic explanations (LIME) to interpret the results of various black box models. The conclusion is that, the bidirectional long short-term memory (LSTM) model is superior for sentiment analysis. The operations of a random forest classifier, a black box model, using explainable artificial intelligence (XAI) techniques like LIME is used in this work. The features used by the random forest model for classification are not entirely correct. The use of LIME made this possible. The proposed model can be used to enhance performance, which raises the trustworthiness and legitimacy of AI systems.

Introduction: In the current educational landscape, universities provide a vast array of diverse and obscure courses. However, the primary challenge students face is decision overload, as the abundance of options can make it difficult to choose courses that align with their interests, career aspirations, and strengths. The development and application of proper course suggestion systems might assist students in overcoming the challenges associated with choosing the right courses. However, these recommendation systems still require improvement to address explainability issues, security issues, and cold start. Our analysis reveals that there is limited research addressing how course recommendation systems can assist 12th-grade students in selecting courses for higher education. Therefore, this study presents a novel personalized recommendation system, namely "Interpretability-Driven Course Recommendation: A Random Forest Approach with Explainable AI", that chooses the top-3 courses for students based on their intermediate class grades/- marks. This research provides accurate and interpretable recommendations using the Random Forest algorithm with Explainable AI that ensures transparency by explaining why a particular course is recommended. Method: The system uses a Random Forest classifier with SHAP (SHapley Additive exPlanations) to forecast the top-3 courses with the greatest expected scores for appropriateness. The system uses SHAP (SHapley Additive exPlanations) values to integrate Explainable AI (XAI) to guarantee openness and trust. The proposed model solves the cold start problem and provides data security. Results: In terms of precision (0.90), recall (0.92), and F1-score (0.92), Random Forest fared better than any other classifier. By combining predictions from several decision trees, its ensemble learning technique increases stability, decreases overfitting, and improves generalization across a broad range of input data patterns. Discussions: A major drawback of conventional black-box machine learning models is their lack of transparency, which is addressed by the incorporation of SHAP into the course recommendation system. In this model, students get suggestions in addition to information on which topic scores had the biggest influence on their choice. Conclusion: This study presents a practical and interpretable course recommendation system designed for 12th-grade students transitioning into higher education. Through the use of Explainable AI-enhanced Random Forest, the model provides precise, transparent, and customized recommendations. It addresses the main drawbacks of conventional systems, such as their lack of explainability and their cold start problems. The model';s excellent precision, recall, and F1-score indicate its superior performance, which makes it a useful tool for student guidance and academic advice. For even more thorough recommendations, future research may consider combining long-term academic objectives, aptitude test results, and student interests.

RuleXAI—A package for rule-based explanations of machine learning model

Post-hoc explainability is essential for understanding black-box machine learning models. Surrogate-based techniques are widely used for local and global model-agnostic explanations but have significant limitations. Local surrogates capture non-linearities but are computationally expensive and sensitive to parameters, while global surrogates are more efficient but struggle with complex local behaviors. In this paper, we present ILLUME, a flexible and interpretable framework grounded in representation learning, that can be integrated with various surrogate models to provide explanations for any black-box classifier. Specifically, our approach combines a globally trained surrogate with instance-specific linear transformations learned with a meta-encoder to generate both local and global explanations. Through extensive empirical evaluations, we demonstrate the effectiveness of ILLUME in producing feature attributions and decision rules that are not only accurate but also robust and computationally efficient, thus providing a unified explanation framework that effectively addresses the limitations of traditional surrogate methods.