Explainable AI: Methods, Challenges, and Future Directions

The rapid proliferation of artificial intelligence (AI) technologies across various industries and decision-making processes has undeniably transformed the way of approaching complex problems and tasks. AI systems have proven their prowess in areas such as healthcare, finance, and autonomous systems, revolutionizing how decisions are made. Nevertheless, this proliferation of AI has raised critical concerns regarding the transparency, accountability, and fairness of these systems, as many of the state-of-the-art AI models often resemble complex black boxes. These intricate models, particularly deep learning neural networks, harbor non-linear relationships that are difficult for human users to decipher, thereby raising concerns about bias, fairness, and overall trustworthiness in AI-driven decisions. The urgency of this issue is underscored by the realization that AI should not merely be accurate; it should also be interpretable. Explainable AI (XAI) has emerged as a vital field of research, emphasizing the development of models and techniques that render AI systems comprehensible and transparent in their decision-making processes. This paper investigates into the relevance and significance of XAI across various domains, including healthcare, finance, and autonomous systems, where the ability to understand the rationale behind AI decisions is paramount. In healthcare, where AI assists in diagnosis and treatment, the interpretability of AI models is crucial for clinicians to make informed decisions. In finance, applications like credit scoring and investment analysis demand transparent AI to ensure fairness and accountability. In the realm of autonomous systems, transparency is indispensable to guarantee safety and compliance with regulations. Moreover, government agencies in areas such as law enforcement and social services require interpretable AI to maintain ethical standards and accountability. This paper also highlights the diverse array of research efforts in the XAI domain, spanning from model-specific interpretability methods to more general approaches aimed at unveiling complex AI models. Interpretable models like decision trees and rule-based systems have gained attention for their inherent transparency, while integrating explanation layers into deep neural networks strives to balance accuracy with interpretability. The study emphasizes the significance of this burgeoning field in bridging the gap between AI's advanced capabilities and human users' need for comprehensible AI systems. It seeks to contribute to this field by exploring the design, development, and practical applications of interpretable AI models and techniques, with the ultimate goal of enhancing the trust and understanding of AI-driven decisions.

Real-World Surveillance of FDA-Cleared Artificial Intelligence Models: Rationale and Logistics.

Artificial Intelligence (AI) is changing the way healthcare, finance & law enforcement work by making them more efficient & creative and making it easier to make these decisions based on their information. As AI models become more complicated, their decision-making processes become less clear, which hurts trust, accountability & ethical use. Explainable AI (XAI) has come up to help with these problems by making AI systems easier to understand and more open, which helps people understand why certain actions were taken. XAI makes AI's decision-making processes easier to understand by employing these approaches including feature significance analysis, model-agnostic methodologies, interpretable models & visualization tools. These methods make sure that important tasks like medical diagnosis, approving loans & finding bias in law enforcement algorithms are all accurate, fair & easy to understand. XAI gives consumers clear, useful information that helps them make informed decisions with confidence, even if they aren't tech-savvy.  This research looks at the main ideas of XAI, including its most important methods, uses & the problems it has to solve to live up to its potential. XAI builds trust in AI systems by making AI models easier to understand. This leads to more widespread use & more responsibility in important areas. As AI becomes better, it will be important to explain things in order to deal with moral difficulties, reduce bias & make sure that people follow the rules. This will help people utilize AI technology in a more responsible & sustainable way

In recent years, there has been a surge of interest in Artificial Intelligence (AI) systems that can provide human-centric explanations for decisions or predictions. No matter how good and efficient an AI model is, users or practitioners find it difficult to trust it if they cannot understand the AI model or its behaviours. Incorporating explainability that is human-centric in event detection systems is significant for building a decision-making process that is more trustworthy and sustainable. Human-centric and semantics-based explainable event detection will achieve trustworthiness, explainability, and reliability, which are currently lacking in AI systems. This paper provides a survey on human-centric explainable AI, explainable event detection, and semantics-based explainable event detection by answering some research questions that bother on the characteristics of human-centric explanations, the state of explainable AI, methods for human-centric explanations, the essence of human-centricity in explainable event detection, research efforts in explainable event solutions, and the benefits of integrating semantics into explainable event detection. The findings from the survey show the current state of human-centric explainability, the potential of integrating semantics into explainable AI, the open problems, and the future directions which can guide researchers in the explainable AI domain.

Artificial Intelligence: The Future of Maxillofacial Prognosis and Diagnosis?

Artificial intelligence (AI) is revolutionizing the world we live in. It holds significant promise for improving performance across various sectors. Researchers worldwide have been working on developing solutions that leverage this technology for application by diverse user groups. A key category of AI is healthcare AI; smart solutions designed to address the needs of various stakeholders in the sector, from medical practitioners to patients. Crucial for any AI deployment are good adoption rates from the intended users and support from key stakeholders. Therefore, understanding the factors that influence its adoption is essential. This paper aims to provide insights through a systematic literature review of research related to healthcare AI adoption. Specifically, it reports on research works from ASEAN countries published over the past decade, between 2014 and 2024, identified through a systematic search of the Lens.org database. The search string focused on terms such as “Artificial Intelligence,” “Medical,” “Healthcare,” “Explainable,” “Transparency,” “Trust,” and “Adoption.” The motivation for this search stems from developments in Explainable Artificial Intelligence (XAI). XAI is vital for fostering trust and facilitating the widespread adoption of AI-driven clinical decision support systems (CDSS) in healthcare. While AI has the potential to augment clinical decision-making, the lack of transparency in traditional “black box” models can undermine trust and hinder adoption. XAI techniques aim to make AI systems more interpretable by providing explanations and visualizations of the decision-making process. This transparency is essential for overcoming skepticism and encouraging the adoption of CDSS in clinical practice. The findings of this review will offer insights into the current research landscape in the region and inform future research on the adoption and use of healthcare AI. These issues are particularly relevant in the ASEAN context, where varying levels of digital literacy and healthcare infrastructure necessitate explainable and transparent AI to build user trust and support effective integration.

Text classification plays an essential role in the medical domain by organizing and categorizing vast amounts of textual data through machine learning (ML) and deep learning (DL). The adoption of Artificial Intelligence (AI) technologies in healthcare has raised concerns about the interpretability of AI models, often perceived as "black boxes." Explainable AI (XAI) techniques aim to mitigate this issue by elucidating AI model decision-making process. In this paper, we present a scoping review exploring the application of different XAI techniques in medical text classification, identifying two main types: model-specific and model-agnostic methods. Despite some positive feedback from developers, formal evaluations with medical end users of these techniques remain limited. The review highlights the necessity for further research in XAI to enhance trust and transparency in AI-driven decision-making processes in healthcare.

Internet of things is projected to make its way into all spheres of human life in the near future. This has been compounded with the growing demand for contactless solutions in the wake of the recent pandemic. A potential solution could involve a privacy-preserving gesture-based control system that could control a wide range of appliances. Implementing such gesture-based control systems is mainly conducted using opaque box Artificial Intelligence (AI) models. Systems based on such opaque box AI models have shown high-performance metrics on in-distribution data in a lab environment. However, they are prone to failure when exposed to real-world out-of-distribution data where they cannot be tuned or calibrated due to their complexity and opaqueness. Interval Type-2 Fuzzy Logic-based explainable AI models offer an alternative to opaque box models showing comparable performance on lab in-distribution data. In contrast, in the real world, out-of-distribution data, the type-2 fuzzy models could be easily calibrated and tuned (thanks for their explainability) to provide similar performance to those achieved on the lab in-distribution data.

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

A comprehensive review of explainable AI for disease diagnosis

A new method to promptly evaluate spatial earthquake probability mapping using an explainable artificial intelligence (XAI) model

Building-damage mapping using remote sensing images plays a critical role in providing quick and accurate information for the first responders after major earthquakes. In recent years, there has been an increasing interest in generating post-earthquake building-damage maps automatically using different artificial intelligence (AI)-based frameworks. These frameworks in this domain are promising, yet not reliable for several reasons, including but not limited to the site-specific design of the methods, the lack of transparency in the AI-model, the lack of quality in the labelled image, and the use of irrelevant descriptor features in building the AI-model. Using explainable AI (XAI) can lead us to gain insight into identifying these limitations and therefore, to modify the training dataset and the model accordingly. This paper proposes the use of SHAP (Shapley additive explanation) to interpret the outputs of a multilayer perceptron (MLP)—a machine learning model—and analyse the impact of each feature descriptor included in the model for building-damage assessment to examine the reliability of the model. In this study, a post-event satellite image from the 2018 Palu earthquake was used. The results show that MLP can classify the collapsed and non-collapsed buildings with an overall accuracy of 84% after removing the redundant features. Further, spectral features are found to be more important than texture features in distinguishing the collapsed and non-collapsed buildings. Finally, we argue that constructing an explainable model would help to understand the model’s decision to classify the buildings as collapsed and non-collapsed and open avenues to build a transferable AI model.

Drug discovery is a complex, lengthy, and costly process with a high failure rate, especially during clinical trials. The integration of Artificial Intelligence (AI) has revolutionized various stages of drug discovery by enabling faster and more accurate analysis of biological and chemical data. However, most AI models in this field operate as “black boxes,” where their decision-making processes are opaque and difficult to interpret. This lack of transparency poses significant challenges in terms of trust, validation, and adoption of AI-generated predictions in both clinical and regulatory settings. To address this issue, Explainable Artificial Intelligence (XAI) has emerged as a promising approach to improve the interpretability of AI models without compromising their predictive power. This study aims to systematically review the opportunities, challenges, and future directions of XAI implementation in drug discovery. Using a qualitative method with a systematic literature review approach, data were collected from reputable databases including Scopus, PubMed, IEEE Xplore, SpringerLink, and ScienceDirect, focusing on publications from 2018 to 2024. The analysis identified five main themes: the role of XAI in molecular target identification, application of XAI in compound screening and molecular structure optimization, interpretation of drug toxicity predictions, challenges in XAI implementation, and future research directions. XAI techniques such as SHAP and LIME have proven useful in explaining AI model predictions, improving biological validation, and enabling more informed decision-making by scientists. However, significant challenges remain, including the trade-off between interpretability and accuracy, lack of universal standards, and the complexity of modeling biological systems. This study highlights the critical need for developing standardized interpretability frameworks, user-friendly interfaces, and collaborative environments between data scientists and healthcare professionals to foster XAI adoption in real-world drug discovery processes. Ultimately, XAI has the potential to increase transparency, trust, and efficiency, paving the way for safer and more effective therapeutic developments.

Background:Brain tumors represent a diverse group of central nervous system neoplasms that require precise diagnosis and characterization for effective treatment. Advances in neuroimaging have improved the detection of these tumors; however, conventional interpretation methods remain time-intensive and subject to variability. Artificial intelligence (AI), particularly through machine learning and deep learning, has emerged as a powerful tool in neuro-oncological imaging, offering automation, consistency, and enhanced diagnostic performance. Objective:This review aims to provide a comprehensive overview of AI-assisted imaging tools used in the detection and characterization of brain tumors. It highlights key AI technologies, their clinical applications, performance compared to human experts, and the emerging trends shaping the future of AI in neuro-oncology. Methods:A narrative review was conducted of recent peer-reviewed studies focused on the application of AI in brain tumor imaging. Emphasis was placed on machine learning and deep learning models used for tumor segmentation, histopathologic and molecular subtype prediction, prognostic modeling, and treatment response monitoring. Technical challenges, ethical concerns, and regulatory considerations were also examined. Results:AI models have demonstrated high accuracy in tasks such as tumor segmentation, classification of tumor types and grades, non-invasive prediction of molecular markers (e.g., IDH mutation, MGMT methylation), and survival prediction. Emerging techniques such as federated learning, multimodal data integration, and explainable AI are addressing key limitations, including data privacy, generalizability, and clinical trust. Conclusion:AI-assisted imaging holds considerable promise in improving the accuracy, speed, and personalization of brain tumor diagnosis and management. For widespread clinical adoption, future efforts should focus on multi-institutional collaboration, prospective validation, regulatory alignment, and clinician education. With continued advancement, AI can become a valuable adjunct in the neuro-oncology diagnostic arsenal, ultimately contributing to better patient outcomes.

The increasing sophistication and frequency of cyberattacks necessitate advanced and transparent decision-making frameworks for threat intelligence and risk assessment. While artificial intelligence (AI) has demonstrated significant potential in automating threat detection, predicting attack patterns, and prioritizing incident response, the opaque nature of many AI models particularly deep learning poses challenges in trust, interpretability, and regulatory compliance. This paper presents an Explainable AI (XAI)-driven framework for enhancing Cyber Threat Intelligence (CTI) and risk assessment, enabling security analysts to understand, validate, and act upon AI-generated insights with confidence. The proposed approach integrates state-of-the-art machine learning algorithms with model-agnostic and model-specific interpretability techniques, such as SHapley Additive exPlanations (SHAP), Local Interpretable Model-agnostic Explanations (LIME), Layer-wise Relevance Propagation (LRP), and attention-based visualization. The framework processes diverse CTI data sources including network traffic logs, vulnerability feeds, malware signatures, and open-source intelligence (OSINT) to identify malicious patterns, quantify risks, and generate actionable intelligence. By coupling predictive accuracy with transparency, the system not only improves detection and classification of advanced persistent threats (APTs), phishing campaigns, and zero-day exploits, but also facilitates compliance with emerging AI governance frameworks, such as the EU AI Act and NIST AI Risk Management Framework. Experimental evaluations on benchmark cybersecurity datasets demonstrate that the XAI-enabled models maintain high precision and recall while offering interpretable outputs that significantly reduce analyst decision latency and improve collaborative threat response. The paper further discusses the role of XAI in mitigating bias, enhancing accountability, and fostering human–machine teaming in security operations centers (SOCs). Practical implementation considerations including scalability, computational efficiency, and integration with existing Security Information and Event Management (SIEM) platforms are addressed, along with recommendations for future research in multi-modal XAI for cybersecurity. This research underscores the critical value of explainability in AI-driven CTI, ensuring that automated systems are not only effective but also transparent, trustworthy, and aligned with organizational and regulatory requirements for responsible AI deployment in cyber defense.