Explainability and transparency of machine learning in ADM systems

The convergence of AI and cultural representation raises critical ethical concerns in the digital age, particularly in culturally diverse and complex societies like Nigeria. This paper takes a conceptual methodological stance to explore the intersection of artificial intelligence (AI) technologies, such as language models and translation systems, in relation to Nigerian cultural and linguistic realities by highlighting the power dynamics of digital language preservation and the risk of reinforcing stereotypes or erasing nuance. Using a bipartite framework involving Floridi’s Information Ethics and the Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) model, this study examines how artificial intelligence (AI) systems, including ChatGPT and Google Translate, may misrepresent or distort Nigerian languages and cultures, such as Yoruba, Hausa, Igbo, and Pidgin. The study advocates for the development of AI that is culturally aware and ethically grounded to promote inclusive frameworks that prioritize linguistic equity and cultural sustainability in the digital representation of Nigerian identities. This research contributes to the development of responsible AI practices that respect and promote cultural diversity in the digital age by exploring the global relevance of these issues.

Regulation of AI and AI Crimes

Artificial intelligence (AI) systems are now prevalent in our daily lives and hold promise for transforming high-stakes fields such as healthcare. Medical AI systems are showing significant potential to support diagnostics and treatment recommendations. As these systems play an increasingly significant role in clinical decision-making, ensuring transparency in their design, operation, and outcomes is essential for building trust among key stakeholders, including patients, providers, developers, and regulators. However, many systems still function as "black boxes," making it challenging for users-such as clinicians, patients, and other stakeholders-to interpret and verify their inner workings. Here, we examine the current state of transparency in medical AIs, identifying key challenges and risks these opaque systems pose. After motivating the need for transparency in all aspects of the machine learning pipeline, from training data to model development to model deployment, we explore a range of techniques that promote explainability throughout the pipeline while highlighting the importance of continual monitoring and system updates to ensure that AI systems remain reliable over time. Finally, we address the need to overcome barriers that inhibit the integration of transparency tools into clinical settings and review regulatory frameworks that prioritize transparency in emerging AI systems. Through this survey, we aim to increase awareness of current challenges and offer actionable insights for stakeholders, such as researchers, clinicians, and regulators, on how to build trustworthy and ethically responsible AI healthcare solutions.

Chapter 11 - Challenges and future work directions in artificial intelligence with human-computer interaction

Decision making, a fundamental human process, has been more and more supported by computer systems in the second part of the last century. In the 21st century, intelligent decision support systems utilize Artificial Intelligence (AI) techniques to enhance and improve support for decision makers. Often decisions suggested by an AI system are based on personal data, such as credit scoring in financial institutions or purchase behavior in online shops and the like. Beyond the protection or personal data by the General Data Protection Regulation (GDPR), developers and operators of decisional AI systems need to ensure ethical standards are met. In respect to individuals, arguably the most relevant ethical aspect is the fairness principle, to ensure individuals are treated fairly. In this paper we present an evaluation model for decision ethicality of AI systems in respect to the fairness principle. The presented model treats any AI system as a “black-box”. It separates sensitive from general attributes in the input matrix. The model measures the distance between predicted values on altering inputs for sensitive attributes. The variance of the outputs is interpreted as individual fairness, that is treating similar individuals similarly. In addition, the model also informs about the group fairness. The validation model helps to determine to what extent an AI System, decides fairly in general for individuals and groups, thus can be used as a test tool in development and operation of AI Systems using personal data.

Artificial intelligence and radiologists in prostate cancer detection on MRI (PI-CAI): an international, paired, non-inferiority, confirmatory study

BackgroundThe surgical difficulty of laparoscopic cholecystectomy (LC) for acute cholecystitis (AC) and the risk of bile duct injury (BDI) depend on the degree of fibrosis and scarring caused by inflammation; therefore, understanding these intraoperative findings is crucial to preventing BDI. Scarring makes it particularly difficult to perform safely and increases the BDI risk. This study aimed to develop an artificial intelligence (AI) system to indicate intraoperative findings of scarring in LC for AC.Materials and methodsAn AI system was developed to detect scarred areas using an algorithm for semantic segmentation based on deep learning. The training dataset consisted of 2025 images extracted from LC videos of 21 cases with AC. External evaluation committees (EEC) evaluated the AI system on 20 cases of untrained data from other centers. EECs evaluated the accuracy in identifying the scarred area and the usefulness of the AI system, which were assessed based on annotation and a 5-point Likert-scale questionnaire.ResultsThe average DICE coefficient for scarred areas between AI detection and EEC annotation was 0.612. The EEC’s average detection accuracy on the Likert scale was 3.98 ± 0.76. AI systems were rated as relatively useful for both clinical and educational applications.ConclusionWe developed an AI system to detect scarred areas in LC for AC. Since scarring increases the surgical difficulty, this AI system has the potential to reduce BDI.

The use of artificial intelligence (AI) systems is significantly increased in the past few years. AI system is expected to provide accurate predictions and it is also crucial that the decisions made by the AI systems are humanly interpretable i.e. anyone must be able to understand and comprehend the results produced by the AI system. AI systems are being implemented even for simple decision support and are easily accessible to the common man on the tip of their fingers. The increase in usage of AI has come with its own limitation, i.e. its interpretability. This work contributes towards the use of explainability methods such as local interpretable model-agnostic explanations (LIME) to interpret the results of various black box models. The conclusion is that, the bidirectional long short-term memory (LSTM) model is superior for sentiment analysis. The operations of a random forest classifier, a black box model, using explainable artificial intelligence (XAI) techniques like LIME is used in this work. The features used by the random forest model for classification are not entirely correct. The use of LIME made this possible. The proposed model can be used to enhance performance, which raises the trustworthiness and legitimacy of AI systems.

Evaluation of physician performance using a concurrent-read artificial intelligence system to support breast ultrasound interpretation

Physicians are required to work with rapidly growing amounts of medical data. Approximately 62% of time per patient is devoted to reviewing electronic health records (EHRs), with clinical data review being the most time-consuming portion. To determine whether an artificial intelligence (AI) system developed to organize and display new patient referral records would improve a clinician's ability to extract patient information compared with the current standard of care. In this prognostic study, an AI system was created to organize patient records and improve data retrieval. To evaluate the system on time and accuracy, a nonblinded, prospective study was conducted at a single academic medical center. Recruitment emails were sent to all physicians in the gastroenterology division, and 12 clinicians agreed to participate. Each of the clinicians participating in the study received 2 referral records: 1 AI-optimized patient record and 1 standard (non-AI-optimized) patient record. For each record, clinicians were asked 22 questions requiring them to search the assigned record for clinically relevant information. Clinicians reviewed records from June 1 to August 30, 2020. The time required to answer each question, along with accuracy, was measured for both records, with and without AI optimization. Participants were asked to assess overall satisfaction with the AI system, their preferred review method (AI-optimized vs standard), and other topics to assess clinical utility. Twelve gastroenterology physicians/fellows completed the study. Compared with standard (non-AI-optimized) patient record review, the AI system saved first-time physician users 18% of the time used to answer the clinical questions (10.5 [95% CI, 8.5-12.6] vs 12.8 [95% CI, 9.4-16.2] minutes; P = .02). There was no significant decrease in accuracy when physicians retrieved important patient information (83.7% [95% CI, 79.3%-88.2%] with the AI-optimized vs 86.0% [95% CI, 81.8%-90.2%] without the AI-optimized record; P = .81). Survey responses from physicians were generally positive across all questions. Eleven of 12 physicians (92%) preferred the AI-optimized record review to standard review. Despite a learning curve pointed out by respondents, 11 of 12 physicians believed that the technology would save them time to assess new patient records and were interested in using this technology in their clinic. In this prognostic study, an AI system helped physicians extract relevant patient information in a shorter time while maintaining high accuracy. This finding is particularly germane to the ever-increasing amounts of medical data and increased stressors on clinicians. Increased user familiarity with the AI system, along with further enhancements in the system itself, hold promise to further improve physician data extraction from large quantities of patient health records.

We discuss how over the last 30 to 50 years, Artificial Intelligence (AI)\nsystems that focused only on data have been handicapped, and how knowledge has\nbeen critical in developing smarter, intelligent, and more effective systems.\nIn fact, the vast progress in AI can be viewed in terms of the three waves of\nAI as identified by DARPA. During the first wave, handcrafted knowledge has\nbeen at the center-piece, while during the second wave, the data-driven\napproaches supplanted knowledge. Now we see a strong role and resurgence of\nknowledge fueling major breakthroughs in the third wave of AI underpinning\nfuture intelligent systems as they attempt human-like decision making, and seek\nto become trusted assistants and companions for humans. We find a wider\navailability of knowledge created from diverse sources, using manual to\nautomated means both by repurposing as well as by extraction. Using knowledge\nwith statistical learning is becoming increasingly indispensable to help make\nAI systems more transparent and auditable. We will draw a parallel with the\nrole of knowledge and experience in human intelligence based on cognitive\nscience, and discuss emerging neuro-symbolic or hybrid AI systems in which\nknowledge is the critical enabler for combining capabilities of the\ndata-intensive statistical AI systems with those of symbolic AI systems,\nresulting in more capable AI systems that support more human-like intelligence.\n

As a typical unit of intelligent manufacturing system, intelligent processing machine is also a microcosm of Intelligent Manufacturing System (IMS), and the realization of intelligent manufacturing system will be firstly embodied in the intelligent processing machine as the core of intelligence. Therefore, it is inevitable to do a comprehensive and systematic study of intelligent processing machines and will get more and more attentions. In the study of intelligent machines, perceptual technology is an important part. Without effective perceptual technology, intelligent machines cannot interact with the environment, and thus the intelligence level cannot be improved. At present, the quite popular approach is to study the connection between perception and behavior from various simple behaviors and realize the higher intelligence through the combination of these behaviors on the basis of simple behaviors. The basis of autonomy perception of intelligent machines is to make it have the capability of multi-physical domain information fusion of manufacturing process and environment. An improved data processing and information fusion method based on GMM-HMM is proposed, which provides a self-perception of intelligent processing machines and a multi-physical domain information fusion method [1]. The simulation results show that the method proposed in this paper can realize the self - identification of intelligent processing machines in the intelligent manufacturing environment, such as the state of manufacturing, health and fault.

Ethical issues occupy a special place when conducting clinical trials of medicines, where it is impossible to predict in advance and accurately the effectiveness and safety of a new drug on the human body. Currently, during clinical trial examination, attention is paid to patients quality of life, and issues of compliance with patients rights, as well as compliance with the rules of good clinical practice and current legislation. Thanks to technological development, the number medical studies and devices using, among other things, specialized medical technologies and software is increasing.
 Automation, development, improvement, and structuring of the processes involved cause an increase in the use of technical devices that use not only programs but also systems in their work. Artificial intelligence system software have a special place in medical science development.
 Artificial intelligence, which was the field of science fiction 5080 years ago, is now firmly embedded in our everyday life. By introducing artificial intelligences capabilities into medical software, using it as part of medical equipment, and developing medical devices with artificial intelligence systems, we get a product that requires careful study and further development, which includes a complex of works on conducting scientific research, and registration and maintenance of such systems and complexes. All work is regulated by legislation in the field of circulation of medical devices and requires a deep systematic and scientific approach, including involving ethics to monitor compliance with the rights and safety of study participants and their medical data.
 The Ethics Committee is an independent body that monitors compliance with the rights and requirements of legislation and conducts ethical and scientific examination of research documentation. Ethical issues when planning any research involving a person or his/her data should be discussed and considered in detail. The ethics committees should be contacted not only at the stage of approving research materials, but also when planning the design and developing research documentation and materials for patients, as well as regularly at all stages of the study.

The implementation of Artificial Intelligence (AI) into transportation systems signifies a fundamental change that holds the potential to improve overall sustainability, safety, and efficiency. However, this transformation is not without its challenges, particularly concerning human factors and ergonomics. Firstly, the ergonomic integration of AI systems in transportation requires careful consideration of human-AI interaction. Unlike traditional systems, AI-driven technologies often involve complex algorithms that are not inherently intuitive to human operators. This complexity can lead to a disconnect between how the system operates and how users understand it. For instance, in autonomous vehicles, drivers may be required to take control in certain scenarios. The ergonomic challenge here lies in ensuring that the transition between AI and human control is seamless and that the user is adequately informed and prepared to take over. Another significant challenge is addressing the variability in human behavior and expectations. AI systems, designed based on standard models of behavior, may not adequately account for the wide range of human responses and interactions. This gap can lead to scenarios where AI systems behave in unexpected or counterintuitive ways to human users, potentially causing confusion and reducing the overall efficiency and safety of the transportation system. CAE Advanced Air Mobility (AAM) research case study focuses on the implementation and certification of AAM in the FAA/EASA environment.The reliability and trustworthiness of AI systems also pose a major ergonomic challenge. To fully integrate AI into transportation, users must trust these systems. This trust depends on the transparency and explainability of AI decision-making processes. Nevertheless, numerous AI algorithms, especially those rooted in deep learning, are frequently regarded as 'black boxes' owing to their intricacy and absence of interpretability. Developing AI systems that are both advanced and transparent is a significant hurdle that needs to be addressed.Data privacy and security are also paramount concerns. AI systems in transportation rely heavily on data, including personal and sensitive information. Ensuring the privacy and security of this data while utilizing it for AI processes is a complex challenge, requiring robust encryption methods and strict data handling policies. Moreover, the ergonomic aspect involves designing systems that not only protect data but also respect user privacy preferences and norms.The implementation of AI in transportation is a multifaceted challenge, requiring a holistic approach that considers human factors and ergonomics. As we move towards more AI-integrated transportation systems, it is essential to address these challenges through interdisciplinary research, collaboration, and a user-centered design approach.

Artificial intelligence (AI) systems are increasingly being used in consumer as well and business applications. Companies are using them, Governments are using them, public security, medical researchers, you and I – everybody seems to be somehow touched. As it becomes more advanced and ubiquitous in our lives, it is critical that AI developers and users understand the ethical implications and challenges posed by this powerful technology. This paper explores the key ethical skills and considerations that are essential for navigating the new world of AI. Artificial Intelligence (AI) presents significant ethical dimensions across various sectors, including research, business, and daily life. It is important to appreciate the major ethical dimensions of AI and how they impact AI systems. Policies and guidelines needs to be laid out for development and usage of such systems. Automated Decisions/Responsibility: No matter how intelligent a AI system might perceived to be, it is always prone to making errors. So, AI system that make automated decisions will always be dangerous. Therefore, it is important to ensure that there is always "human intervention" or "human review" for outcomes/decisions made by AI systems. Determining accountability for AI-generated actions or content remains a significant challenge, emphasizing the need for ethical governance frameworks. There has to be human accountability.Fairness/Bias: AI systems can perpetuate existing biases present in training data, leading to discriminatory outcomes.Transparency: The opaque nature of many AI algorithms complicates accountability, making it difficult to understand decision-making processes.Privacy: AI's capability to process vast datasets can infringe on individual privacy rights, necessitating stringent data protection measuresHuman Dignity: The risk of AI-generated content resembling existing works raises ethical issues regarding originality and authorship.