Evidence-based clinical engineering: Machine learning algorithms for prediction of defibrillator performance

The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow. To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle. PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores. The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%). The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

P68. Machine learning algorithms for predicting patient-controlled analgesia use after minimally invasive spine surgery

Category:Ankle Arthritis; AnkleIntroduction/Purpose:Ankle arthrodesis and total ankle replacement are the most commonly performed procedures for surgical management of ankle arthritis. Arthrodesis provides effective pain relief but the rate of complications after arthrodesis is higher as it is more commonly performed in patients with comorbidities that preclude ankle replacement. Accurately risk- stratifying patients who undergo ankle arthrodesis would be of great utility, given the significant cost and morbidity associated with developing major perioperative complications. There is a paucity of accurate prediction models that can be used to pre- operatively risk-stratify patients for ankle arthrodesis. We aim to develop a machine learning (ML) algorithm for prediction of major perioperative complication after ankle arthrodesis as well as compare its performance against traditional predictive models based on logistic regression.Methods:This is a retrospective cohort study of adult patients who underwent ankle arthrodesis at any non-federal California hospital between 2015 and 2017. The primary outcome was readmission within 30 days or major perioperative complication - venous thromboembolism within 30 days, myocardial infarction within 7 days, pneumonia within 7 days, systemic infection within 7 days, surgical site bleeding within 90 days, and wound complications within 90 days. We build ML and logistic regression models that span different classes of modeling approaches: XGBoost, AdaBoost, Gradient Boosting, and Random Forest. Discrimination and calibration were assessed using area under the receiver operating characteristic curve (AUROC) and Brier score, respectively. We utilize a partial dependence function to measure the importance of an individual feature by assessing the average effect in predicted risks when its value is altered. We rank the contribution of the included variables to the prediction of adverse outcomes.Results:A total of 1,084 patients met inclusion criteria for this study. There were 131 major complications or readmission (12.1%). The optimized XGBoost algorithm demonstrates higher discrimination (AUROC: 0.707 + 0.052) compared to LR (0.691 + 0.055). The receiver operating characteristic curves for the XGBoost and logistic regression models are visualized in Figure 1. XGBoost also outperforms the three other ML models. This model was well calibrated (Brier score: 0.103 + 0.001). The variables most important for the XGBoost model include diabetes, chronic kidney disease, implant complication, and major fracture. Five of the ten most important features for XGBoost were markedly less important for the traditional logistic regression model: male sex, prior hip fracture, cardiorespiratory failure, acute renal failure, and dialysis status.Conclusion:We report a ML algorithm for prediction of major perioperative complications after ankle arthrodesis. The optimized XGBoost model is well-calibrated and demonstrates superior risk prediction to logistic regression. This tool may identify and address potentially modifiable risk factors, helping to accurately risk-stratify patients and decrease likelihood of major complications. Notably, the predictors most important for XGBoost are different from those for logistic regression. This suggests that the superior discriminative capability of ML methods stems from their ability to capture complex non-linear relationships between variables that logistic regression is unable to detect.

BackgroundDespite the interest in machine learning (ML) algorithms for analyzing real-world data (RWD) in healthcare, the use of ML in predicting time-to-event data, a common scenario in clinical practice, is less explored. ML models are capable of algorithmically learning from large, complex datasets and can offer advantages in predicting time-to-event data. We reviewed the recent applications of ML for survival analysis using RWD in healthcare.MethodsPUBMED and EMBASE were searched from database inception through March 2023 to identify peer-reviewed English-language studies of ML models for predicting time-to-event outcomes using the RWD. Two reviewers extracted information on the data source, patient population, survival outcome, ML algorithms, and the Area Under the Curve (AUC).ResultsOf 257 citations, 28 publications were included. Random survival forests (N = 16, 57%) and neural networks (N = 11, 39%) were the most popular ML algorithms. There was variability across AUC for these ML models (median 0.789, range 0.6–0.950). ML algorithms were predominately considered for predicting overall survival in oncology (N = 12, 43%). ML survival models were often used to predict disease prognosis or clinical events (N = 27, 96%) in the oncology, while less were used for treatment outcomes (N = 1, 4%).ConclusionsThe ML algorithms, random survival forests and neural networks, are mainly used for RWD to predict survival outcomes such as disease prognosis or clinical events in the oncology. This review shows that more opportunities remain to apply these ML algorithms to inform treatment decision-making in clinical practice. More methodological work is also needed to ensure the utility and applicability of ML models in survival outcomes.

P125. Development of a novel ensemble machine learning algorithm for prediction of complications and readmission after anterior cervical spinal fusion

BackgroundCervical cancer is a common malignant tumor of the female reproductive system and is considered a leading cause of mortality in women worldwide. The analysis of time to event, which is crucial for any clinical research, can be well done with the method of survival prediction. This study aims to systematically investigate the use of machine learning to predict survival in patients with cervical cancer.MethodAn electronic search of the PubMed, Scopus, and Web of Science databases was performed on October 1, 2022. All articles extracted from the databases were collected in an Excel file and duplicate articles were removed. The articles were screened twice based on the title and the abstract and checked again with the inclusion and exclusion criteria. The main inclusion criterion was machine learning algorithms for predicting cervical cancer survival. The information extracted from the articles included authors, publication year, dataset details, survival type, evaluation criteria, machine learning models, and the algorithm execution method.ResultsA total of 13 articles were included in this study, most of which were published from 2018 onwards. The most common machine learning models were random forest (6 articles, 46%), logistic regression (4 articles, 30%), support vector machines (3 articles, 23%), ensemble and hybrid learning (3 articles, 23%), and Deep Learning (3 articles, 23%). The number of sample datasets in the study varied between 85 and 14946 patients, and the models were internally validated except for two articles. The area under the curve (AUC) range for overall survival (0.40 to 0.99), disease-free survival (0.56 to 0.88), and progression-free survival (0.67 to 0.81), respectively from (lowest to highest) received. Finally, 15 variables with an effective role in predicting cervical cancer survival were identified.ConclusionCombining heterogeneous multidimensional data with machine learning techniques can play a very influential role in predicting cervical cancer survival. Despite the benefits of machine learning, the problem of interpretability, explainability, and imbalanced datasets is still one of the biggest challenges. Providing machine learning algorithms for survival prediction as a standard requires further studies.

Performances of machine learning algorithms for individual thermal comfort prediction based on data from professional and practical settings

Nowadays machine learning (ML) is used in all sorts of fields like health care, retail, travel, finance, social media, etc. ML system is used to learn from input data to construct a suitable model by continuously estimating, optimizing, and tuning parameters of the model. To attain the stated, Python programming language is one of the most flexible languages, and it does contain special libraries for ML applications, namely SciKit-Learn, TensorFlow, PyTorch, Keras, Theano, etc., which is great for linear algebra and getting to know kernel methods of machine learning. The Python programming language is great to use when working with ML algorithms and has easy syntax relatively. When taking the deep-dive into ML, choosing a framework can be daunting. The most common concern is to understand which of these libraries has the most momentum in ML system modeling and development. The major objective of this paper is to provide extensive knowledge on various Python libraries and different ML algorithms in comparison with meet multiple application requirements. This paper also reviewed various ML algorithms and application domains.KeywordsMachine LearningLibrariesPython

Aim: Machine learning (ML) and deep learning (DL) predictive models have been employed widely in clinical settings. Their potential support and aid to the clinician of providing an objective measure that can be shared among different centers enables the possibility of building more robust multicentric studies. This study aimed to propose a user-friendly and low-cost tool for COVID-19 mortality prediction using both an ML and a DL approach. Method: We enrolled 2348 patients from several hospitals in the Province of Reggio Emilia. Overall, 19 clinical features were provided by the Radiology Units of Azienda USL-IRCCS of Reggio Emilia, and 5892 radiomic features were extracted from each COVID-19 patient’s high-resolution computed tomography. We built and trained two classifiers to predict COVID-19 mortality: a machine learning algorithm, or support vector machine (SVM), and a deep learning model, or feedforward neural network (FNN). In order to evaluate the impact of the different feature sets on the final performance of the classifiers, we repeated the training session three times, first using only clinical features, then employing only radiomic features, and finally combining both information. Results: We obtained similar performances for both the machine learning and deep learning algorithms, with the best area under the receiver operating characteristic (ROC) curve, or AUC, obtained exploiting both clinical and radiomic information: 0.803 for the machine learning model and 0.864 for the deep learning model. Conclusions: Our work, performed on large and heterogeneous datasets (i.e., data from different CT scanners), confirms the results obtained in the recent literature. Such algorithms have the potential to be included in a clinical practice framework since they can not only be applied to COVID-19 mortality prediction but also to other classification problems such as diabetic prediction, asthma prediction, and cancer metastases prediction. Our study proves that the lesion’s inhomogeneity depicted by radiomic features combined with clinical information is relevant for COVID-19 mortality prediction.

Raman spectroscopy (RS) is a promising tool for cancer diagnosis. In particular, in the last years several studies have demonstrated how the diagnostic performances of RS can be significantly improved by employing machine learning (ML) algorithms for the interpretation of Raman-based data. Recently, it has been demonstrated that RS can perform an accurate classification of chondrosarcoma tissues. Chondrosarcoma is a cancer of bones, that can occur in the soft tissues near the bones. It is normally characterized by three different malignant degrees and a benign counterpart, knows as enchondroma. In line with these findings, in this paper, we exploited ML algorithms to distinguish, as well as possible, between the three grades of chondrosarcoma and to distinguish between chondrosarcoma and enchondroma. We obtained a high level of accuracy of classification by analyzing a dataset composed of a relatively small number of Raman spectra, collected in a previous study by one of the authors of this paper. Such spectra were acquired from micrometric tissue sections with a confocal Raman microscope. We tested the classification performances of a support vector machine (SVM) and a random forest classifier (RFC), as representatives of ML algorithms, and two versions of the multi-layer perceptron (MLPC) as representatives of deep learning (DL). These models, especially RFC and MLPC, showed excellent classification performances, with accuracy reaching 99.7%. This outcome makes the aforementioned models a promising route for future improvements of diagnostic devices focused on detecting cancerous bone tissues. Alongside the diagnostic purpose, the aforementioned approach allowed us to identify characteristic molecules, i.e., amino acids, nucleic acids, and bioapatites, relevant for obtaining the final diagnostic response, through the use of a tool named by us Raman Band Identification (RBI). The method to evaluate RBI is the most important contribution of this paper, because RBI could represent a relevant parameter for the identification of biochemical processes on the basis of the tumor progression and associated malignant degree. In turn, the spectral bands highlighted by RBI could provide precious indicators in an attempt to restrict the spectral acquisition to specific Raman bands. This last objective could help to reduce the amount of experimental data needed to obtain an accurate final grading outcome, with a consequent reduction in the computational cost.

Machine Learning Algorithm Prospectively Predicts Survival for High-Risk Patients Undergoing Radiotherapy: A Survival Analysis of SHIELD-RT

Constructing the outer crust of the neutron stars requires the knowledge of the Binding Energy (BE) of the atomic nuclei. Although the BE of a lot of the nuclei is experimentally determined and can be obtained from the AME data table, for the others we need to depend on theoretical models. There exist a lot of physical theories to predict the BE, each with its own strengths and weaknesses. In this paper we apply Machine Learning (ML) algorithms on AME2016 data set to predict the Binding Energy of atomic nuclei. The novel feature of our work is that it is model independent. We do not assume or use any nuclear physics model but use only ML algorithms directly on the AME2016 data set. Our results are further refined by using another ML algorithm to train the errors of the first algorithm, and repeating this process iteratively. Our best algorithm gives σrms ∼ 0.58 MeV for Binding Energy on randomized testing sets. This is comparable to all physics models or ML improved physics models studied in literature till date. Using the predictions of our Machine Learning algorithm, we construct the outer crust equation of state (EoS) of a neutron star and show that our model is comparable to existing models. This work also demonstrates the use of various ML algorithms and a detailed analysis on how we arrived at our best algorithm. It will help the physics community in understanding how to choose an ML algorithm which would be suited for their data set. Our algorithms and best fit model is also made publicly available for the use of the community.

Hardware Trojans (HTs), maliciously inserted in an integrated circuit during untrusted design or fabrication process pose critical threat to the system security. With the ever increasing capabilities of an adversary to subvert the system during run-time, it is imperative to detect the manifested Trojans in order to reinforce the trust in hardware. In this regard, Machine Learning (ML) algorithms, with their intrinsic capability to execute feature engineering at high learning rates, are emerging as promising candidates to be utilized by system defenders. In this paper, we explore Trojan detection mechanisms that are based on ML, and thereby investigate the prowess of the ML algorithms in bolstering system security. Furthermore, we analyze the efficiency of each proposed Trojan detection strategy based on the underlying ML algorithm. Finally, we underline some problems with existing Trojan detection approaches and discuss future research in the interest of improved performance of the employed ML algorithms, thus aiding in enhancing the intended hardware security.

Machine learning (ML) algorithms due to their outstanding performances are being extensively used in applications covering several different domains. Recently, the increased growth of cloud services provided training infrastructures for complex ML models able to deal with big data, resulting in the enhancement of ML as a Service (MLaaS). Toward this end, ML applications have been deployed in systems, production models, and businesses. ML algorithms involve accessing data, which is often privacy sensitive. The latter may result in security and privacy risks. Toward this end, this work examines MLaaS and its incorporation into businesses, covering a wide range of different sectors. Companies that develop ML applications are reviewed, and trends in ML-related jobs are reported. Moreover, data protection privacy is discussed and the evolution of graphics processing units (GPUs) as a necessary supporting technology for ML applications is also considered.

Recent advances in machine learning techniques have significantly influenced the development of early diabetes detection systems. In this paper, different machine learning (ML) algorithms are employed for diabetes detection based on hand movement. The spatial and temporal information for horizontal and vertical hand movement deviations for both diabetic and non-diabetic subjects are obtained using a data acquisition system. A dataset is constructed based on the obtained hand movement information along with other related attributes such as age, gender, weight and vision health. Binary classification of diabetic and non-diabetic subjects is achieved using several machine learning algorithms. These algorithms include Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminate Analysis (LDA), Classification and Regression Tree (CART), k-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Multi-layer Perceptron (MLP). The considered ML algorithms are evaluated and ranked according to a set of key performance metrics that include accuracy, precision, recall and F-1 score. Algorithm ranking results demonstrate that LR, CART, KNN and MLP algorithms have achieved higher ranking positions as compared to other algorithms. The best ranked ML algorithms are then validated and thoroughly investigated to prove their ability to discriminate between diabetic and non-diabetic subjects. Validation results show that the best-ranked ML algorithms have achieved high classification accuracy, precision, recall and F-1 scores. Therefore, the considered machine learning algorithms can serve as efficient tools for diabetes detection based on the constructed dataset.