Identifiability of Causal-based ML Fairness Notions

BackgroundDespite the interest in machine learning (ML) algorithms for analyzing real-world data (RWD) in healthcare, the use of ML in predicting time-to-event data, a common scenario in clinical practice, is less explored. ML models are capable of algorithmically learning from large, complex datasets and can offer advantages in predicting time-to-event data. We reviewed the recent applications of ML for survival analysis using RWD in healthcare.MethodsPUBMED and EMBASE were searched from database inception through March 2023 to identify peer-reviewed English-language studies of ML models for predicting time-to-event outcomes using the RWD. Two reviewers extracted information on the data source, patient population, survival outcome, ML algorithms, and the Area Under the Curve (AUC).ResultsOf 257 citations, 28 publications were included. Random survival forests (N = 16, 57%) and neural networks (N = 11, 39%) were the most popular ML algorithms. There was variability across AUC for these ML models (median 0.789, range 0.6–0.950). ML algorithms were predominately considered for predicting overall survival in oncology (N = 12, 43%). ML survival models were often used to predict disease prognosis or clinical events (N = 27, 96%) in the oncology, while less were used for treatment outcomes (N = 1, 4%).ConclusionsThe ML algorithms, random survival forests and neural networks, are mainly used for RWD to predict survival outcomes such as disease prognosis or clinical events in the oncology. This review shows that more opportunities remain to apply these ML algorithms to inform treatment decision-making in clinical practice. More methodological work is also needed to ensure the utility and applicability of ML models in survival outcomes.

Optimizing operational decisions, routine actions within some business or operational process, is a key challenge across a variety of domains and application areas. The increasing availability of data, computational power, and advanced machine learning (ML) algorithms offers exciting opportunities for data-driven decision support. To advance the potential of ML for optimizing operational decision-making, we explore two research directions, aiming to develop ML models that are decision-focused and causal. This dissertation presents several developments in machine learning in these two areas. ML is effective at making predictions from historical data: for example, estimating a transaction's fraud probability by comparing it to past cases. However, decision-makers not only need to consider these predictions, but also the operational context. For example, the decision-maker uses predicted fraud probabilities to determine which transactions to investigate, while aiming to minimize monetary losses due to fraud and considering the available capacity of the fraud investigations team. Predictions can help reduce uncertainty (e.g., by predicting the fraud probability), but standard ML models are prediction-focused, instead of decision-focused. This distinction involves two challenges for data-driven decision-making. First, prediction-focused models prioritize predictive accuracy instead of the resulting decision quality (e.g., fraud losses recovered by the bank). Second, these models fail to account for operational constraints, such as the available investigation capacity. Decision-focused learning aims to improve data-driven decision-making by addressing these issues and incorporating the operational context into the optimization of ML models. In this dissertation, we analyze cost-sensitive learning within this prediction-optimization framework and evaluate general strategies for making cost-optimal decisions with ML. Additionally, we propose a novel ML method for optimal decision-making under capacity constraints based on learning to rank. To make effective decisions, a decision-maker has to estimate the causal effect of possible interventions in order to choose actions that achieve the desired outcome. Unfortunately, standard ML models identify correlations in the data instead of causal relationships. Because of this, these models cannot guarantee the effectiveness of decisions made based on their predictions. Causal inference provides a formal framework for reasoning about causality and identifying causal effects from data. This dissertation explores the intersection of causality and ML. First, we illustrate the potential of causal ML for optimizing preventive maintenance. Next, we propose novel causal ML methods for predicting causal effect distributions and for addressing informative sampling when predicting treatment outcomes over time. We also argue for a practical, end-to-end perspective for building ML pipelines for causal inference and propose an automated framework doing so. Finally, we combine decision-focused learning with causal inference by introducing ranking metalearners to optimize treatment decisions under capacity constraints.

Hardware Trojans (HTs), maliciously inserted in an integrated circuit during untrusted design or fabrication process pose critical threat to the system security. With the ever increasing capabilities of an adversary to subvert the system during run-time, it is imperative to detect the manifested Trojans in order to reinforce the trust in hardware. In this regard, Machine Learning (ML) algorithms, with their intrinsic capability to execute feature engineering at high learning rates, are emerging as promising candidates to be utilized by system defenders. In this paper, we explore Trojan detection mechanisms that are based on ML, and thereby investigate the prowess of the ML algorithms in bolstering system security. Furthermore, we analyze the efficiency of each proposed Trojan detection strategy based on the underlying ML algorithm. Finally, we underline some problems with existing Trojan detection approaches and discuss future research in the interest of improved performance of the employed ML algorithms, thus aiding in enhancing the intended hardware security.

The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow. To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle. PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores. The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%). The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

The purpose of this analysis is to assess the use of machine learning (ML) algorithms in the prediction of postoperative outcomes, including complications, recurrence, and death in transsphenoidal surgery. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically reviewed all papers that used at least one ML algorithm to predict outcomes after transsphenoidal surgery. We searched Scopus, PubMed, and Web of Science databases for studies published prior to May 12, 2021. We identified 13 studies enrolling 5,048 patients. We extracted the general characteristics of each study; the sensitivity, specificity, area under the curve (AUC) of the ML models developed as well as the features identified as important by the ML models. We identified 12 studies with 5,048 patients that included ML algorithms for adenomas, three with 1807 patients specifically for acromegaly, and five with 2105 patients specifically for Cushing's disease. Nearly all were single-institution studies. The studies used a heterogeneous mix of ML algorithms and features to build predictive models. All papers reported an AUC greater than 0.7, which indicates clinical utility. ML algorithms have the potential to predict postoperative outcomes of transsphenoidal surgery and can improve patient care. Ensemble algorithms and neural networks were often top performers when compared with other ML algorithms. Biochemical and preoperative features were most likely to be selected as important by ML models. Inexplicability remains a challenge, but algorithms such as local interpretable model–agnostic explanation or Shapley value can increase explainability of ML algorithms. Our analysis shows that ML algorithms have the potential to greatly assist surgeons in clinical decision making.

The integration of machine learning (ML) algorithms into audit processes represents a significant advancement in the field of auditing, offering substantial benefits in terms of efficiency, accuracy, and risk management. This review examines the transformative potential of ML in auditing, highlighting its key benefits and the challenges that must be addressed to fully leverage its capabilities. Machine learning algorithms, with their ability to analyze large datasets and identify patterns, enhance the accuracy and thoroughness of audits. Traditional auditing methods often rely on sampling and manual checks, which can miss anomalies and fraudulent activities. In contrast, ML algorithms can process entire datasets, uncovering subtle patterns and irregularities that may indicate fraud or errors. This comprehensive analysis reduces the risk of oversight and improves the reliability of audit findings. One of the primary benefits of ML in auditing is its capacity for anomaly detection. ML models can be trained on historical data to understand normal financial behavior and flag deviations that might signify irregularities. This ability to detect anomalies in real-time enables auditors to identify potential issues promptly, reducing the time lag between occurrence and detection of fraud. Predictive analytics, powered by ML, further enhances audit processes by forecasting future risks based on historical data. This proactive approach allows auditors to anticipate and mitigate risks before they materialize, contributing to more robust risk management strategies. Despite these advantages, integrating ML into audit processes presents several challenges. Ensuring data quality and integrity is crucial, as ML algorithms are only as good as the data they analyze. Poor-quality data can lead to inaccurate predictions and conclusions. Additionally, the "black box" nature of some ML algorithms can pose transparency issues, making it difficult for auditors to explain how specific conclusions were reached, which is critical for stakeholder trust and regulatory compliance. Another significant challenge is the potential for algorithmic bias. ML models can inadvertently perpetuate existing biases in the data, leading to unfair or skewed audit outcomes. Continuous monitoring and validation of ML algorithms are necessary to detect and mitigate such biases. In conclusion, while integrating machine learning algorithms into audit processes offers substantial benefits in terms of accuracy, efficiency, and risk management, it also necessitates careful attention to data quality, transparency, and bias mitigation. Addressing these challenges is essential to fully realize the potential of ML in enhancing audit practices. Keywords: Benefits, Challenges, Audit Processes, Algorithms, ML.

This study presents a comparative analysis of the classification performance of facial and emotion recognition systems using Machine Learning (ML) and Deep Learning (DL) algorithms. The primary objective of this work is to evaluate the applicability of emotion recognition in fields such as psychotherapy and crime analysis, using the FER-2013 dataset.The study was conducted with ML algorithms such as Support Vector Machines (SVM), Random Forest (RF), K-Nearest Neighbor (KNN), Decision Trees, and Gradient Boosting, as well as the DL algorithm, Convolutional Neural Network (CNN). Supported by the application of feature selection, preprocessing, and feature extraction techniques, the model’s performance was measured using standard metrics such as accuracy, precision, F1-score, and AUC-ROC.The experimental results showed that the highest classification accuracy (46.61%) was achieved with the CNN model. While the ML models generally offered lower accuracy, they provided advantages in terms of computational efficiency in specific scenarios.This study aims to contribute to the literature by providing a comparative analysis of ML and DL algorithms and by highlighting the effect of data preprocessing on performance. The findings set targets for future work, such as real-time system integration and hybrid model development.

6 - Machine learning and deep learning algorithms for fault diagnosis of photovoltaic systems

A good deal of humility: Cochran on observational studies

Constructing the outer crust of the neutron stars requires the knowledge of the Binding Energy (BE) of the atomic nuclei. Although the BE of a lot of the nuclei is experimentally determined and can be obtained from the AME data table, for the others we need to depend on theoretical models. There exist a lot of physical theories to predict the BE, each with its own strengths and weaknesses. In this paper we apply Machine Learning (ML) algorithms on AME2016 data set to predict the Binding Energy of atomic nuclei. The novel feature of our work is that it is model independent. We do not assume or use any nuclear physics model but use only ML algorithms directly on the AME2016 data set. Our results are further refined by using another ML algorithm to train the errors of the first algorithm, and repeating this process iteratively. Our best algorithm gives σrms ∼ 0.58 MeV for Binding Energy on randomized testing sets. This is comparable to all physics models or ML improved physics models studied in literature till date. Using the predictions of our Machine Learning algorithm, we construct the outer crust equation of state (EoS) of a neutron star and show that our model is comparable to existing models. This work also demonstrates the use of various ML algorithms and a detailed analysis on how we arrived at our best algorithm. It will help the physics community in understanding how to choose an ML algorithm which would be suited for their data set. Our algorithms and best fit model is also made publicly available for the use of the community.

Surgical site infection (SSI) following a neurosurgical operation is a complication that impacts morbidity, mortality, and economics. Currently, machine learning (ML) algorithms are used for outcome prediction in various neurosurgical aspects. The implementation of ML algorithms to learn from medical data may help in obtaining prognostic information on diseases, especially SSIs. The purpose of this study was to compare the performance of various ML models for predicting surgical infection after neurosurgical operations. A retrospective cohort study was conducted on patients who had undergone neurosurgical operations at tertiary care hospitals between 2010 and 2017. Supervised ML algorithms, which included decision tree, naive Bayes with Laplace correction, k-nearest neighbors, and artificial neural networks, were trained and tested as binary classifiers (infection or no infection). To evaluate the ML models from the testing data set, their sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), as well as their accuracy, receiver operating characteristic curve, and area under the receiver operating characteristic curve (AUC) were analyzed. Data were available for 1471 patients in the study period. The SSI rate was 4.6%, and the type of SSI was superficial, deep, and organ/space in 1.2%, 0.8%, and 2.6% of cases, respectively. Using the backward stepwise method, the authors determined that the significant predictors of SSI in the multivariable Cox regression analysis were postoperative CSF leakage/subgaleal collection (HR 4.24, p < 0.001) and postoperative fever (HR 1.67, p = 0.04). Compared with other ML algorithms, the naive Bayes had the highest performance with sensitivity at 63%, specificity at 87%, PPV at 29%, NPV at 96%, and AUC at 76%. The naive Bayes algorithm is highlighted as an accurate ML method for predicting SSI after neurosurgical operations because of its reasonable accuracy. Thus, it can be used to effectively predict SSI in individual neurosurgical patients. Therefore, close monitoring and allocation of treatment strategies can be informed by ML predictions in general practice.

In this chapter, a brief overview of the role and applications of machine learning (ML) algorithms in future wireless cellular networks is presented, more specifically, in the context of self-organizing networks (SONs). SON is a promising and innovative concept, in which future networks are expected to analyze and use historical data in order to improve and adapt themselves to the network conditions. For this to be possible, however, algorithms that are capable of extracting patterns from data and learn from previous actions are necessary. This chapter highlights the utilization and possible applications of ML algorithms in future cellular networks. A brief introduction of ML and SON is presented, followed by an analysis of current state of the art solutions involving ML in SON. Lastly, guidelines on the utilization of intelligent algorithms in SON and future research trends in the area are highlighted and conclusions are drawn.

Identification of damage in its early stage can have a great contribution in decreasing the maintenance costs and prolonging the life of valuable structures. Although conventional damage detection techniques have a mature background, their widespread application in industrial practice is still missing. In recent years the application of Machine Learning (ML) algorithms have been more and more exploited in structural health monitoring systems (SHM). Because of the superior capabilities of ML approaches in recognizing and classifying available patterns in a dataset, they have demonstrated a significant improvement in traditional damage identification algorithms. This review study focuses on the use of machine learning (ML) approaches in Ultrasonic Guided Wave (UGW)-based SHM, in which a structure is continually monitored using permanent sensors. Accordingly, multiple steps required for performing damage detection through UGWs are stated. Moreover, it is outlined that the employment of ML techniques for UGW-based damage detection can be subtended into two main phases: (1) extracting features from the data set, and reducing the dimension of the data space, (2) processing the patterns for revealing patterns, and classification of instances. With this regard, the most frequent techniques for the realization of those two phases are elaborated. This study shows the great potential of ML algorithms to assist and enhance UGW-based damage detection algorithms.

Application of machine learning in carbon capture and storage: An in-depth insight from the perspective of geoscience

Background and Context Computational thinking (CT) has been increasingly added to K-12 curricula, prompting teachers to grade more and more CT artifacts. This has led to a rise in automated CT assessment tools. Objective This study examines the scope and characteristics of publications that use machine learning (ML) approaches to assess students’ CT competencies from four perspectives: the educational context in which the assessments were implemented, the data used to train and validate ML algorithms, the specific ML algorithms used, and the aspects of CT assessed. Method The PRISMA approach and Arksey and O’Malley’s methodological framework for scoping reviews were adopted to search and screen studies. Findings ML algorithms have been increasingly used to assess CT competencies. However, this study identified several research gaps in the literature: existing studies were mostly conducted in the context of programming or other learning activities related to computing science; datasets used by the ML algorithms were generally small; the most frequently used algorithms were regression techniques, naive Bayes, neural networks, clustering, and natural language processing, whereas no studies used reinforcement learning; and CT competencies were not comprehensively assessed. Implications The applications of ML in CT assessments have the potential to enable personalized learning, improve assessment validity, reduce the workload of graders, and gain insights from large datasets by uncovering complex and subtle patterns.