Explainable Multi-Modal Medical Image Analysis Through Dual-Stream Multi-Feature Fusion and Class-Specific Selection

Understanding why a machine learning model makes a certain prediction is just as critical as how accurately it predicts, especially when it comes to diagnosing and treating cardiovascular disease.In this study, we applied explainable artificial intelligence (XAI) techniques to improve both the predictive power and interpretability of heart disease detection models.A dataset of 1,025 patient records was thoroughly preprocessed, including the handling of missing values, the encoding of categorical features, and the binarization of the outcome variable.We evaluated several machine learning models LSTM networks, Random Forest, Gradient Boosting, XGBoost, and Logistic Regression using performance metrics such as accuracy, precision, recall, F1-score, and AUC-ROC.While XGBoost emerged as the most accurate model, we moved beyond accuracy to explore the root causes of such strong performance.To accomplish this, we employed three key XAI techniques: SHAP (SHapley Additive exPlanations) to quantify how individual features influenced predictions, LIME (Local Interpretable Model-Agnostic Explanations) to provide intuitive, local-level explanations for individual predictions, and Permutation Feature Importance to assess which features most affected model performance when altered.These methods transformed XGBoost from a high-performing black-box into a transparent and trustworthy diagnostic tool.Our findings underscore how integrating explainability into AI pipelines leads to not just accurate predictions, but also clinically actionable insights highlighting the most important risk factors for cardiovascular disease and supporting more informed, responsible medical decisions.

It is well known that there exist great differences in human brain anatomy and functions between males and females. With the development of noninvasive neuroimaging techniques, the sex differences in adult human brain have been well studied in some researches. However, the sexual dimorphism of human brain anatomy and functions has not been sufficiently described during the developmental period of early childhood brain when the sex differences emerge in behavior. This study was to identify specific patterns of the sexual dimorphism in developing brain structure of early childhood using multimodal brain image analysis. We have performed a multivariate and data-driven analysis by combining multiple neuroimaging technologies including the 3D T1-weighted structural MR images (sMRI) and diffusion tensor imaging (DTI) in a prospective cohort of 188 children (128 males and 60 females) between the ages of 0 and 15. First, the brain images were segmented into 90 regions of interest (ROIs) based on the AAL template to extract the ROI volume and connectivity features. Then, the individual multimodal imaging biomarkers were identified associated with the sex differences. Finally, the selected features from multi-modality neuroimages were combined using multi-kernel support vector machine for classifications of male or female. The method achieved sex classification accuracy 72% for the children between the ages of 0 and 15. And the volumes of right precuneus and right postcentral were more related to sexual dimorphism of developing brain structure than those of other regions in sMRI (p < 0.002). In DTI, the connections between right middle occipital gyrus and right inferior occipital gyrus, between left superior marginal gyrus and left middle temporal gyrus, and between right triangle inferior frontal gyrus and right putamen are also more relevant to the sexual dimorphism than others (p < 0.005). The brain regions and connections related to the sexual differences were identified with sMRI and DTI in early developing brain structure by using the multimodal image analysis. And these sexually dimorphic patterns of brain may be related to the observed sex differences in developing childhood brain structure and connectivity.

In recent times, it has been observed that social media exerts a favorable influence on consumer purchasing behavior. Many organizations are adopting the utilization of social media platforms as a means to promote products and services. Hence, it is crucial for enterprises to understand the consumer buying behavior in order to thrive. This article presents a novel approach that combines the theory of planned behavior (TPB) with machine learning techniques to develop accurate predictive models for consumer purchase behavior. This study examines three distinct factors of the theory of planned behavior (attitude, social norm, and perceived behavioral control) that provide insights into the primary determinants influencing online purchasing behavior. A total of eight machine learning algorithms, namely K-nearest neighbor, Decision Tree, Random Forest, Logistic Regression, Naive Bayes, Support Vector Machine, AdaBoost, and Gradient Boosting, were utilized in order to forecast consumer purchasing behavior. Empirical findings indicate that gradient boosting demonstrates superior performance in predicting customer buying behavior, with an accuracy rate of 0.91 and a macro F1 score of 0.91. This holds true when all factors, namely attitude (ATTD), social norm (SN), and perceived behavioral control (PBC), are included in the analysis. Furthermore, we incorporated Explainable AI (XAI), specifically LIME (Local Interpretable Model-Agnostic Explanations), to elucidate how the best machine learning model (i.e. gradient boosting) makes its prediction. The findings indicate that LIME has demonstrated a high level of confidence in accurately predicting the influence of low and high behavior. The outcome presented in this article has several implications. For instance, this article presents a novel way to combine the theory of planned behavior with machine learning techniques in order to predict consumer purchase behavior. This integration allows for a comprehensive analysis of factors influencing online purchasing decisions. Also, the incorporation of Explainable AI enhances the transparency and interpretability of the model. This feature is valuable for organizations seeking insights into factors driving predictions and the reasons behind certain outcomes. Moreover, these observations have the potential to offer valuable insights for businesses in customizing their marketing strategies to align with these influential factors.

Results The CNN trained using OCT alone showed a diagnostic accuracy of 94%, whilst the OCT-A trained CNN resulted in an accuracy of 91%. When multiple modalities were combined, the CNN accuracy increased to 96% in the AMD cohort. Conclusions Here we demonstrate that superior diagnostic accuracy can be achieved when deep learning is combined with multimodal image analysis.

The value of cross-data set analysis for automobile insurance fraud detection

Deep learning hyperparameter optimization: Application to electricity and heat demand prediction for buildings

Credit scoring models serve as pivotal instruments for lenders and financial institutions, facilitating the assessment of creditworthiness. Traditional models, while instrumental, grapple with challenges related to efficiency and subjectivity. The advent of machine learning heralds a transformative era, offering data-driven solutions that transcend these limitations. This research delves into a comprehensive analysis of various machine learning algorithms, emphasizing their mathematical underpinnings and their applicability in credit score classification. A comprehensive evaluation is conducted on a range of algorithms, including logistic regression, decision trees, support vector machines, and neural networks, using publicly available credit datasets. Within the research, a unified mathematical framework is introduced, which encompasses preprocessing techniques and critical algorithms such as Particle Swarm Optimization (PSO), the Light Gradient Boosting Model, and Extreme Gradient Boosting (XGB), among others. The focal point of the investigation is the LIME (Local Interpretable Model-agnostic Explanations) explainer. This study offers a comprehensive mathematical model using the LIME explainer, shedding light on its pivotal role in elucidating the intricacies of complex machine learning models. This study’s empirical findings offer compelling evidence of the efficacy of these methodologies in credit scoring, with notable accuracies of 88.84%, 78.30%, and 77.80% for the Australian, German, and South German datasets, respectively. In summation, this research not only amplifies the significance of machine learning in credit scoring but also accentuates the importance of mathematical modeling and the LIME explainer, providing a roadmap for practitioners to navigate the evolving landscape of credit assessment.

Ensemble models such as gradient boosting and random forests are proven to offer the best predictive performance on a wide variety of supervised learning problems. The high performance of these black box models, however, comes at a cost of model interpretability. They are also inadequate to meet regulatory demands and explainability needs of organizations. The model interpretability in high performance black-box models is achieved with the help of post-hoc explainable models such as Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). This paper presents an alternate intrinsic classifier model that extracts a class of higher order patterns and embeds them into an interpretable learning model. More specifically, the proposed model extracts novel High Utility Gain (HUG) patterns that capture higher order interactions, transforms the model input data into a new space, and applies interpretable classifier methods on the transformed space. We conduct rigorous experiments on forty benchmark binary and multi-class classification datasets to evaluate the proposed model against the state-of-the-art ensemble and interpretable classifier models. The proposed model was comprehensively assessed on three key dimensions: 1) quality of predictions using classifier measures such as accuracy, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F_{1}$ </tex-math></inline-formula>, AUC, H-measure, and logistic loss, 2) computational performance on large and high-dimensional data, and 3) interpretability aspects. The HUG-based learning model was found to deliver performance comparable to that of the state-of-the-art ensemble models. Our model was also found to achieve 2-40% (45%) prediction quality (interpretability) improvements with significantly lower computational requirements over other interpretable classifier models. Furthermore, we present case studies in finance and healthcare domains and generate one- and two-dimensional HUG profiles to illustrate the interpretability aspects of our HUG models. The proposed solution offers an alternate approach to build high performance and transparent machine learning classifier models. We hope that our ML solution help organizations meet their growing regulatory and explainability needs.

Accurate prediction of diabetes based on clinical and demographic indicators is essential, as early prediction of this chronic metabolic disorder plays a critical role in preventing long-term organ complications. However, existing research continues to face significant challenges, including pronounced class imbalance, the scarcity of large and diverse datasets, and limited integration of explainable artificial intelligence. This research compares several ensemble learning methods (Decision Tree, Random Forest, AdaBoost, Gradient Boosting, Histogram based Gradient Boosting, extremely randomized Trees, and XGBoost) on a large imbalanced dataset (87,664 negative vs. 8,482 positive samples). To mitigate imbalance, we evaluate six resampling approaches including (Random Over Sampling, Random Under Sampling, Synthetic Minority Over-sampling, Adaptive Synthetic Sampling, Tomek Links Removal Sampling and (Synthetic Minority Over-sampling with Edited Nearest Neighbors). We assess models using metrics robust to class imbalance (precision, recall, F1, AUC-ROC, and AUC-PR) and calibration measures. The Extra Trees classifier achieved the highest measured accuracy (0.994); with Random Over Sampling for balancing dataset. also, these results were compared with several previous works and number of machine learning algorithms, and the results showed superiority. Explainability is performed at both global and local levels: permutation and SHAP for global feature importance, and (Local Interpretable Model-Agnostic Explanations) force plots for instance-level reasoning. however, we analyzed this result using sensitivity, specificity, PR-AUC and calibration, we report detailed experiments showing how resampling method, hyperparameter tuning, and stratified validation influence performance. Finally, we provide clinical-relevant insights from SHAP analyses and discuss limitations and future directions for deploying interpretable models in screening workflows.

This research proposes a novel strategy for accurate breast lesion classification that combines explainable artificial intelligence (XAI), machine learning (ML) classifiers, and customized weakly dependent features from ultrasound (BU) images. Two new weakly dependent feature classes are proposed to improve the diagnostic accuracy and diversify the training data. These are based on image intensity variations and the area of bounded partitions and provide complementary rather than overlapping information. ML classifiers such as Random Forest (RF), Extreme Gradient Boosting (XGB), Gradient Boosting Classifiers (GBC), and LASSO regression were trained with both customized feature classes. To validate the reliability of our study and the results obtained, we conducted a statistical analysis using the McNemar test. Later, an XAI model was combined with ML to tackle the influence of certain features, the constraints of feature selection, and the interpretability capabilities across various ML models. LIME (Local Interpretable Model-Agnostic Explanations) and SHAP (SHapley Additive exPlanations) models were used in the XAI process to enhance the transparency and interpretation in clinical decision-making. The results revealed common relevant features for the malignant class, consistently identified by all of the classifiers, and for the benign class. However, we observed variations in the feature importance rankings across the different classifiers. Furthermore, our study demonstrates that the correlation between dependent features does not impact explainability.

This paper explores the use of explainable artificial intelligence (XAI) techniques in actuarial science to address the opacity of advanced machine learning models in financial contexts. While technological advancements have enhanced actuarial models, their black box nature poses challenges in highly regulated environments. This study proposes a protocol for selecting and applying XAI techniques to improve interpretability, transparency, and regulatory compliance. It categorizes techniques based on origin, target, and interpretative capacity, and introduces a protocol to identify the most suitable method for actuarial models. The proposed protocol is tested in a case study involving two classification algorithms, gradient boosting and random forest, with accuracy of 0.80 and 0.79, focusing on two explainability objectives. Several XAI techniques are analyzed, with results highlighting partial dependency variance (PDV) and local interpretable model-agnostic explanations (LIME) as effective tools for identifying key variables. The findings demonstrate that the protocol aids in model selection, internal audits, regulatory compliance, and enhanced decision-making transparency. These advantages make it particularly valuable for improving model governance in the financial sector.

Climate change is causing permafrost in the Qinghai–Tibet Plateau to degrade, triggering thermokarst hazards and impacting the environment. Despite their ecological importance, the distribution and risks of thermokarst lakes are not well understood due to complex influencing factors. In this study, we introduced a new interpretable ensemble learning method designed to improve the global and local interpretation of susceptibility assessments for thermokarst lakes. Our primary aim was to offer scientific support for precisely evaluating areas prone to thermokarst lake formation. In the thermokarst lake susceptibility assessment, we identified ten conditioning factors related to the formation and distribution of thermokarst lakes. In this highly accurate stacking model, the primary learning units were the random forest (RF), extremely randomized trees (EXTs), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost) algorithms. Meanwhile, gradient boosted decision trees (GBDTs) were employed as the secondary learning unit. Based on the stacking model, we assessed thermokarst lake susceptibility and validated accuracy through six evaluation indices. We examined the interpretability of the stacking model using three interpretation methods: accumulated local effects (ALE), local interpretable model-agnostic explanations (LIME), and Shapley additive explanations (SHAP). The results showed that the ensemble learning stacking model demonstrated superior performance and the highest prediction accuracy. Approximately 91.20% of the total thermokarst hazard points fell within the high and very high susceptible areas, encompassing 20.08% of the permafrost expanse in the QTP. The conclusive findings revealed that slope, elevation, the topographic wetness index (TWI), and precipitation were the primary factors influencing the assessment of thermokarst lake susceptibility. This comprehensive analysis extends to the broader impacts of thermokarst hazards, with the identified high and very high susceptibility zones affecting significant stretches of railway and highway infrastructure, substantial soil organic carbon reserves, and vast alpine grasslands. This interpretable ensemble learning model, which exhibits high accuracy, offers substantial practical significance for project route selection, construction, and operation in the QTP.

The emergence of the COVID-19 pandemic over a relatively brief interval illustrates the need for rapid data-driven approaches to facilitate clinical decision making. We examined a machine learning process to predict inpatient mortality among COVID-19 patients using clinical and chest radiographic data. Modeling was performed with a de-identified dataset of encounters prior to widespread vaccine availability. Non-imaging predictors included demographics, pre-admission clinical history, and past medical history variables. Imaging features were extracted from chest radiographs by applying a deep convolutional neural network with transfer learning. A multi-layer perceptron combining 64 deep learning features from chest radiographs with 98 patient clinical features was trained to predict mortality. The Local Interpretable Model-Agnostic Explanations (LIME) method was used to explain model predictions. Non-imaging data alone predicted mortality with an ROC-AUC of 0.87 ± 0.03 (mean ± SD), while the addition of imaging data improved prediction slightly (ROC-AUC: 0.91 ± 0.02). The application of LIME to the combined imaging and clinical model found HbA1c values to contribute the most to model prediction (17.1 ± 1.7%), while imaging contributed 8.8 ± 2.8%. Age, gender, and BMI contributed 8.7%, 8.2%, and 7.1%, respectively. Our findings demonstrate a viable explainable AI approach to quantify the contributions of imaging and clinical data to COVID mortality predictions.

Skin-CAD: Explainable deep learning classification of skin cancer from dermoscopic images by feature selection of dual high-level CNNs features and transfer learning

Living organisms are striking in their complexity at all levels of organization. No one single analytical method can capture all the information necessary to provide comprehensive knowledge about the cellular processes. The rapid development of new technologies for biological imaging and single cell analysis gives an opportunity to measure different aspects of the living systems at single-cell resolution. Combining these diverse data types can lead to an insight that would not have been possible if each modality were to be considered separately. However, there is no general recipe for a successful multimodal analysis project and it remains challenging to both keep high data quality and establish a reliable way to find correspondence between the two data types. In this thesis I show two different examples of combining different imaging modalities. Firstly, I demonstrate how, making use of the stereotypical development of the marine worm Platynereis dumerilii, I developed fully automated deep-learning based registration pipeline that allowed to map multiple 3D smFISH datasets to the EM image stack of a whole body 6-days post fertilization larva of the animal with near single-cell accuracy. Automated registration enables systematic study of the connection between cell-type specific gene expression and cell phenotype. In the second project I combined imaging mass spectrometry for spatially-resolved detection of 13C6-glucose-derived fatty acids in cellular lipids with microscopy and computational methods for data integration and analysis. I validated this method on a spatially-heterogeneous normoxia-hypoxia model of liver cancer cells. I demonstrated the single-cell heterogeneity of acetyl-CoA pool labelling degree upon ACLY knockdown that would be impossible to detect with bulk analysis. Segmentation of matching objects in different modalities is a crucial step in multimodal image analysis. Transferable and easy to train segmentation of large biological images with neural networks remains challenging. In the final part of my thesis I show how feature normalization inside the neural network can lead to tiling artifacts or suboptimal performance, and propose a normalization strategy for successfully eliminating the artifacts while keeping high segmentation accuracy.