A novel explainable AI framework for medical image classification integrating statistical, visual, and rule-based methods.

Skin diseases impact millions of people around the world and pose a severe risk to public health. These diseases have a wide range of effects on the skin’s structure, functionality, and appearance. Identifying and predicting skin diseases are laborious processes that require a complete physical examination, a review of the patient’s medical history, and proper laboratory diagnostic testing. Additionally, it necessitates a significant number of histological and clinical characteristics for examination and subsequent treatment. As a disease’s complexity and quantity of features grow, identifying and predicting it becomes more challenging. This research proposes a deep learning (DL) model utilizing transfer learning (TL) to quickly identify skin diseases like chickenpox, measles, and monkeypox. A pre-trained VGG16 is used for transfer learning. The VGG16 can identify and predict diseases more quickly by learning symptom patterns. Images of the skin from the four classes of chickenpox, measles, monkeypox, and normal are included in the dataset. The dataset is separated into training and testing. The experimental results performed on the dataset demonstrate that the VGG16 model can identify and predict skin diseases with 93.29% testing accuracy. However, the VGG16 model does not explain why and how the system operates because deep learning models are black boxes. Deep learning models’ opacity stands in the way of their widespread application in the healthcare sector. In order to make this a valuable system for the health sector, this article employs layer-wise relevance propagation (LRP) to determine the relevance scores of each input. The identified symptoms provide valuable insights that could support timely diagnosis and treatment decisions for skin diseases.

Background: Artificial intelligence (AI) has recently made unprecedented contributions in every walk of life, but it has not been able to work its way into diagnostic medicine and standard clinical practice yet. Although data scientists, researchers, and medical experts have been working in the direction of designing and developing computer aided diagnosis (CAD) tools to serve as assistants to doctors, their large-scale adoption and integration into the healthcare system still seems far-fetched. Diagnostic radiology is no exception. Imagining techniques like magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET) scans have been widely and very effectively employed by radiologists and neurologists for the differential diagnoses of neurological disorders for decades, yet no AI-powered systems to analyze such scans have been incorporated into the standard operating procedures of healthcare systems. Why? It is absolutely understandable that in diagnostic medicine, precious human lives are on the line, and hence there is no room even for the tiniest of mistakes. Nevertheless, with the advent of explainable artificial intelligence (XAI), the old-school black boxes of deep learning (DL) systems have been unraveled. Would XAI be the turning point for medical experts to finally embrace AI in diagnostic radiology? This review is a humble endeavor to find the answers to these questions. Methods: In this review, we present the journey and contributions of AI in developing systems to recognize, preprocess, and analyze brain MRI scans for differential diagnoses of various neurological disorders, with special emphasis on CAD systems embedded with explainability. A comprehensive review of the literature from 2017 to 2024 was conducted using host databases. We also present medical domain experts' opinions and summarize the challenges up ahead that need to be addressed in order to fully exploit the tremendous potential of XAI in its application to medical diagnostics and serve humanity. Results: Forty-seven studies were summarized and tabulated with information about the XAI technology and datasets employed, along with performance accuracies. The strengths and weaknesses of the studies have also been discussed. In addition, the opinions of seven medical experts from around the world have been presented to guide engineers and data scientists in developing such CAD tools. Conclusions: Current CAD research was observed to be focused on the enhancement of the performance accuracies of the DL regimens, with less attention being paid to the authenticity and usefulness of explanations. A shortage of ground truth data for explainability was also observed. Visual explanation methods were found to dominate; however, they might not be enough, and more thorough and human professor-like explanations would be required to build the trust of healthcare professionals. Special attention to these factors along with the legal, ethical, safety, and security issues can bridge the current gap between XAI and routine clinical practice.

Artificial intelligence (AI) and machine learning (ML) are increasingly being applied in sports science to predict performance, assess injury risk, and support tactical decision-making. However, the opacity of many machine learning (ML) models has led to a growing interest in explainable AI (XAI) methods, which aim to make the decisions of "black box" algorithms more transparent and interpretable. The result of this is an increase in trust, facilitation of model validation, and enablement of more informed decision-making for practitioners and analysts. While XAI is well established in domains like healthcare or finance, its application in sports science remains fragmented and underexplored. This scoping review follows the PRISMA-Sc approach and systematically examines how XAI and interpretable ML techniques have been applied in the context of sports science. Between 2014 and June 2024, a total of 19 studies were identified through a multi-database search and analyzed in terms of sport type, ML model, XAI method, dependent variable, limitations, challenges and future research. The results reveal a clear dominance of SHapley Additive Explanations across multiple disciplines, while other tools such as Gradient-weighted Class Activation Mapping, Individual Conditional Expectation Plots, or domain-specific approaches are rarely used. Visual and rule-based explanation methods were largely absent. The findings highlight a lack of methodological diversity and limited validation of explanations with domain experts or practitioners. To increase real-world impact, future work should explore comparative evaluations of XAI techniques, domain-specific explanation frameworks, and user-centered approaches that align with the needs of coaches, athletes, and analysts.

Abstract: Deep learning models have demonstrated remarkable capabilities across various domains, but their inherent complexity often leads to challenges in understanding and interpreting their decisions. The demand for transparent and interpretable artificial intelligence (AI) systems is particularly crucial in fields such as healthcare, finance, and autonomous systems. This research paper presents a comprehensive study on the application of Explainable AI (XAI) techniques to enhance transparency and interpretability in deep learning models. Keywords: Explainable AI (XAI), artificial intelligence (AI).

Detection of a brain tumor in the early stages is critical for clinical practice and survival rate. Brain tumors arise in multiple shapes, sizes, and features with various treatment options. Tumor detection manually is challenging, time-consuming, and prone to error. Magnetic resonance imaging (MRI) scans are mostly used for tumor detection due to their non-invasive properties and also avoid painful biopsy. MRI scanning of one patient’s brain generates many 3D images from multiple directions, making the manual detection of tumors very difficult, error-prone, and time-consuming. Therefore, there is a considerable need for autonomous diagnostics tools to detect brain tumors accurately. In this research, we have presented a novel TumorResnet deep learning (DL) model for brain detection, i.e., binary classification. The TumorResNet model employs 20 convolution layers with a leaky ReLU (LReLU) activation function for feature map activation to compute the most distinctive deep features. Finally, three fully connected classification layers are used to classify brain tumors MRI into normal and tumorous. The performance of the proposed TumorResNet architecture is evaluated on a standard Kaggle brain tumor MRI dataset for brain tumor detection (BTD), which contains brain tumor and normal MR images. The proposed model achieved a good accuracy of 99.33% for BTD. These experimental results, including the cross-dataset setting, validate the superiority of the TumorResNet model over the contemporary frameworks. This study offers an automated BTD method that aids in the early diagnosis of brain cancers. This procedure has a substantial impact on improving treatment options and patient survival.

Explainable Artificial Intelligence (XAI) is one of the most advanced research areas of Artificial Intelligence (AI). To explain the deep learning (DL) model is the main objective of XAI. It deals with artificial models which are understandable to humans, including the users, developers, policymakers, etc. XAI is very important in some critical domains like security, healthcare, etc. The purpose of XAI is only to provide a clear answer to the question of how the model made its decision. The explanation is very important before any system decision-making. As an example, if a system responds to a decision, it is necessary to have inside knowledge of the model about that decision. The decision can be positive or negative, but it is more important to know the decision based on characteristics. The decision of the model should be trusted when we know the internal structure of the DL model. Generally, DL models come under the black box models. So for security purposes, it is very necessary to explain a system internally for any decision-making. Security is very crucial in healthcare as well as in any other domain. The objective of this research is to provide a decision about security based on XAI which is a big challenge. We can improve security systems based on XAI for the next level. For medical/healthcare security, when we recognize human action using transfer learning techniques, one pre-trained model is considered good for action and the same action is not good in terms of accuracy using another pre-trained model. This is called the black-box model problem, and it needs to know what is the internal mechanism of both models for the same action. Why one model considers good for action and why the same action is not very well using another model? Here need a model-specific approach of post-hoc interpretability to know the internal structure and characteristics of both models for the same action.

In this work we propose a workflow to deal with overlaid images—images with superimposed text and company logos—, which is very common in underwater monitoring videos and surveillance camera footage. It is demonstrated that it is possible to use Explaining Artificial Intelligence to improve deep learning models performance for image classification tasks in general. A deep learning model trained to classify metal surface defect, which previously had a low performance, is then evaluated with Layer-wise relevance propagation—an Explaining Artificial Intelligence technique—to identify problems in a dataset that hinder the training of deep learning models in a wide range of applications. Thereafter, it is possible to remove this unwanted information from the dataset—using different approaches: from cutting part of the images to training a Generative Inpainting neural network model—and retrain the model with the new preprocessed images. This proposed methodology improved F1 score in 20% when compared to the original trained dataset, validating the proposed workflow.

Deep learning algorithms have the potential to automate the examination of medical images obtained in clinical practice. Using digitized medical images, convolution neural networks (CNNs) have demonstrated their ability and promise to discriminate among different image classes. As an initial step towards explainability in clinical diagnosis, deep learning models must be exceedingly precise, offering a measure of uncertainty for their predictions. Such uncertainty-aware models can help medical professionals in detecting complicated and corrupted samples for re-annotation or exclusion. This paper proposes a new model and data-agnostic mechanism, called Actionable Uncertainty Quantification Optimization (AUQantO) to improve the performance of deep learning architectures for medical image classification. This is achieved by optimizing the hyperparameters of the proposed entropy-based and Monte Carlo (MC) dropout uncertainty quantification techniques escorted by single- and multi-objective optimization methods, abstaining from the classification of images with a high level of uncertainty. This helps in improving the overall accuracy and reliability of deep learning models. To support the above claim, AUQantO has been validated with four deep learning architectures on four medical image datasets and using various performance metric measures such as precision, recall, Area Under the Receiver Operating Characteristic (ROC) Curve score (AUC), and accuracy. The study demonstrated notable enhancements in deep learning performance, with average accuracy improvements of 1.76% and 2.02% for breast cancer histology and 5.67% and 4.24% for skin cancer datasets, utilizing two uncertainty quantification techniques, and AUQantO further improved accuracy by 1.41% and 1.31% for brain tumor and 4.73% and 1.83% for chest cancer datasets while allowing exclusion of images based on confidence levels.

Industrial robots are widely used in critical tasks such as assembly, welding, and material handling as core components of modern manufacturing systems. For the reliable operation of these systems, early and accurate detection of execution failures is crucial. In this study, a comprehensive comparison of machine learning and deep learning methods is conducted for the classification of robot execution failures using data acquired from force-torque sensors. Three different feature engineering approaches are proposed. The first is a Baseline approach that includes 90 raw time-series features. The second is the Domain-6 approach, which consists of 6 basic statistical features per sensor (36 in total). The third is the Domain-12 approach, which comprises 12 comprehensive statistical features per sensor (72 in total). The domain features include the mean, standard deviation, minimum, maximum, range, slope, median, skewness, kurtosis, RMS, energy, and IQR. In total, ten classification algorithms are evaluated, including eight machine learning methods and two deep learning models: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (KNN), Artificial Neural Network (ANN), Naive Bayes (NB), Decision Trees (DT), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM-LGBM), as well as a One-Dimensional Convolutional Neural Network (CNN-1D) and Long Short-Term Memory (LSTM). For traditional machine learning algorithms, 5 × 5 nested cross-validation is used, whereas for deep learning models, 5-fold cross-validation with a 20% validation split is employed. To ensure statistical reliability, all experiments are repeated over 30 independent runs. The experimental results demonstrate that feature engineering has a decisive impact on classification performance. In addition, regardless of the feature set, the highest accuracy (93.85% ± 0.90) is achieved by the Naive Bayes classifier using the Baseline features. The Domain-12 feature set provides consistent improvements across many algorithms, with substantial performance gains. The results are reported using accuracy, precision, recall, and F1-score metrics and are supported by confusion matrices. Finally, permutation feature importance analysis indicates that the skewness features of the Fx and Fy sensors are the most critical variables for failure detection. Overall, these findings show that time-domain statistical features offer an effective approach for robot failure classification.

Statistical Parameters for Glaucoma Detection from Color Fundus Images

ObjectivePneumothorax is an acute thoracic disease caused by abnormal air collection between the lungs and chest wall. Recently, artificial intelligence (AI), especially deep learning (DL), has been increasingly employed for automating the diagnostic process of pneumothorax. To address the opaqueness often associated with DL models, explainable artificial intelligence (XAI) methods have been introduced to outline regions related to pneumothorax. However, these explanations sometimes diverge from actual lesion areas, highlighting the need for further improvement. MethodWe propose a template-guided approach to incorporate the clinical knowledge of pneumothorax into model explanations generated by XAI methods, thereby enhancing the quality of the explanations. Utilizing one lesion delineation created by radiologists, our approach first generates a template that represents potential areas of pneumothorax occurrence. This template is then superimposed on model explanations to filter out extraneous explanations that fall outside the template’s boundaries. To validate its efficacy, we carried out a comparative analysis of three XAI methods (Saliency Map, Grad-CAM, and Integrated Gradients) with and without our template guidance when explaining two DL models (VGG-19 and ResNet-50) in two real-world datasets (SIIM-ACR and ChestX-Det). ResultsThe proposed approach consistently improved baseline XAI methods across twelve benchmark scenarios built on three XAI methods, two DL models, and two datasets. The average incremental percentages, calculated by the performance improvements over the baseline performance, were 97.8% in Intersection over Union (IoU) and 94.1% in Dice Similarity Coefficient (DSC) when comparing model explanations and ground-truth lesion areas. We further visualized baseline and template-guided model explanations on radiographs to showcase the performance of our approach. ConclusionsIn the context of pneumothorax diagnoses, we proposed a template-guided approach for improving model explanations. Our approach not only aligns model explanations more closely with clinical insights but also exhibits extensibility to other thoracic diseases. We anticipate that our template guidance will forge a novel approach to elucidating AI models by integrating clinical domain expertise.

The remarkable success of deep learning has prompted interest in its application to medical imaging diagnosis. Even though state-of-the-art deep learning models have achieved human-level accuracy on the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box nature of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical imaging diagnosis, including visual, textual, example-based and concept-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations. In addition, we include a performance comparison among a set of report generation–based methods. Finally, the major challenges in applying XAI to medical imaging and the future research directions on the topic are discussed.

Explainable Artificial Intelligence (XAI) has emerged as a critical component in modern real-time decision support systems, where transparency, trust, and accountability are essential. Traditional AI models, particularly deep learning techniques, often function as “black boxes,” making it difficult for users to understand how decisions are derived. This research paper explores the design, implementation, and evaluation of explainable AI models tailored for real-time decision support environments. The study focuses on integrating interpretable machine learning techniques, such as feature importance analysis, rule-based explanations, and model-agnostic explanation methods, into time-sensitive applications. It also examines the trade-off between model accuracy and interpretability in dynamic decision-making scenarios such as healthcare monitoring, financial risk assessment, and smart industrial systems. Furthermore, the paper proposes a hybrid explainable framework that combines high-performance predictive models with post-hoc explanation tools to ensure both efficiency and transparency. Experimental analysis demonstrates that the proposed approach enhances user trust, improves decision quality, and enables faster human–AI collaboration without significantly compromising system performance. The findings highlight the importance of explainability as a foundational requirement for deploying AI-driven real-time decision support systems in critical domains.

Deep Learning has emerged as a strategic area in medical image diagnosis, with numerous studies exploring automation in this field. However, many existing approaches attempt to fully replace medical professionals by relying exclusively on model predictions. This thesis presents an alternative strategy, proposing Deep Learning architectures designed to support, rather than replace, physicians in diagnostic tasks. Two distinct methodologies are explored to achieve this goal: Content-Based Image Retrieval and eXplainable Artificial Intelligence. Content-Based Image Retrieval system enables physicians to compare complex cases with similar ones from a curated database, enhancing decision-making through feature-based similarities. The eXplainable Artificial Intelligence approach provides visual explanations through heatmaps, indicating the regions of interest detected by the model. The thesis introduces two novel architecturesMulti-Output Classification Autoencoder (MOC-AE) and Multi-Output Classification Variational Autoencoder (MOC-VAE). These models employ multi-objective training to generate rich image descriptors, capturing both pathological and anatomical information. These descriptors are then used to identify visually similar cases in a low-dimensional space. To address the challenge of quantifying visual similarity in Content-Based Image Retrieval, the Sliced Wasserstein distance is proposed, offering a robust metric for comparative diagnosis. Furthermore, the thesis demonstrates the utility of these architectures in generating precise, non-linear explanations for diagnoses, leveraging the factorization process inherent to the models. The simplicity and enhanced performance of the proposed approaches, compared to prior methods, offer a foundation for future research and real-world applications in medical diagnostics using Deep Learning. RESUMEN El Deep Learning ha surgido como un área estratégica en el diagnóstico de imágenes médicas, con numerosos estudios que exploran la automatización en este campo. Sin embargo, muchos enfoques actuales buscan reemplazar por completo a los profesionales médicos basándose únicamente en las predicciones de los modelos. Esta tesis presenta una estrategia alternativa, proponiendo arquitecturas de Deep Learning diseñadas para apoyar, en lugar de reemplazar, a los médicos en tareas diagnósticas. Se exploran dos metodologías distintas para alcanzar este objetivo: Sistemas de Recuperación de Imágenes Basada en Contenido e Inteligencia Artificial Explicable. Los Sistemas de Recuperación de Imágenes Basada en Contenido permiten a los médicos comparar casos complejos con otros similares de una base de datos curada, mejorando la toma de decisiones mediante similitudes basadas en características. El enfoque de la Inteligencia Artificial Explicable proporciona explicaciones visuales a través de mapas de calor, que indican las regiones de interés detectadas por el modelo. La tesis introduce dos arquitecturas novedosas: el Autoencoder de Clasificación Multi-Salida (MOC-AE) y el Autoencoder Variacional de Clasificación Multi-Salida (MOC-VAE). Estos modelos emplean un entrenamiento multiobjetivo para generar descriptores de imágenes enriquecidos que capturan tanto información patológica como anatómica. Estos descriptores se utilizan para identificar casos visualmente similares en un espacio de baja dimensionalidad. Para abordar el desafío de cuantificar la similitud visual en Sistemas de Recuperación de Imágenes Basada en Contenido, se propone la distancia de Wasserstein por secciones, ofreciendo una métrica robusta para el diagnóstico comparativo. Además, la tesis demuestra la utilidad de estas arquitecturas para generar explicaciones precisas y no lineales de los diagnósticos, aprovechando el proceso de factorización inherente a los modelos. La simplicidad y el rendimiento mejorado de los enfoques propuestos, en comparación con métodos anteriores, ofrecen una base sólida para futuras investigaciones y aplicaciones reales en diagnósticos médicos utilizando Deep Learning.

This work evaluates machine learning (ML) and deep learning (DL) models for Photovoltaic (PV) power output prediction based on two real-world datasets. Eight models – Support Vector Regression (SVR), K-Nearest Neighbors (KNN), Linear Regression (LR), Random Forest Regression (RFR), Gradient Boosting Regression (GBR), Decision Tree (DT), one Dimensional Convolutional Neural Network (1DCNN) and Artificial Neural Network (ANN) – were evaluated using Root Mean Squared Error (RMSE), R-squared (R²), Mean Absolute Error (MAE), and Mean Squared Error (MSE). For Dataset1 (region of Cacak) 1DCNN achieved the optimal performance, with the reduced errors and the largest R² values at three spatial locations, and it was closely followed by RFR and GBR. Likewise, in Dataset2 (region of Kraljevo), 1DCNN, ANN, and RFR give the best results. Conventional models, such as LR and DT, performed poorly in both data tribes. The results highlight the ability of state-of-the-art ensemble and DL models in learning nonlinear patterns in solar energy data and thus, the importance of choosing the right prediction tools to provide accurate and reliable PV power forecasts.