Machine Learning-Based Predictive Techno-Economic Analysis of Power System

Background and purpose — External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Material and methods — We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting. Results — We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43–89), with 6 items being reported in less than 4/18 of the studies. Interpretation — Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.

Machine learning (ML) algorithms are increasingly used to predict outcomes in orthopaedic surgery, but their utility in hip arthroscopy remains unclear. This study aimed to (1) evaluate ML models for predicting outcomes, complications, and resource use following hip arthroscopy, (2) compare ML performance to traditional statistical methods, and (3) critically appraise the quality of existing literature. A systematic search of PUBMED, MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials was conducted on 4 April 2023. Included studies used ML models for clinical prediction in hip arthroscopy. Data on study design, outcomes, model development, validation and adherence to TRIPOD guidelines were extracted. Seventeen studies involving 37,549 patients (52.8% female; mean age 34.4 ± 6.1 years) were included. ML was used to predict clinical outcomes (47.1%), adverse events (47.1%) and resource utilisation (5.9%). The median area under the curve was 0.66 (range 0.5-0.94) for clinical outcomes and 0.7 (range 0.6-0.9) for adverse events. Resource utilisation was reported with a root-mean-square error of $3800, a logarithmic error of 0.2, and R² of 0.8. Only four studies compared ML to traditional regression; two found no difference, while two favoured ML. On average, studies met 12.3 of the 22 TRIPOD criteria, with only 29.4% meeting more than two-thirds. ML models in hip arthroscopy show variable performance in predicting outcomes and complications. Furthermore, most studies comparing ML models to traditional regression did not report any significant difference. Lastly, variability in adherence to TRIPOD guidelines highlights the need for future studies to improve transparency and reporting in the development of ML algorithms for hip arthroscopy. Given the potential of ML models in enhancing clinical decision-making for hip arthroscopy, along with the findings of the current study, their application should be used with caution. Level IV, systematic review of level II-IV studies.

Machine learning (ML) models provide more choices to patients with diabetes mellitus (DM) to more properly manage blood glucose (BG) levels. However, because of numerous types of ML algorithms, choosing an appropriate model is vitally important. In a systematic review and network meta-analysis, this study aimed to comprehensively assess the performance of ML models in predicting BG levels. In addition, we assessed ML models used to detect and predict adverse BG (hypoglycemia) events by calculating pooled estimates of sensitivity and specificity. PubMed, Embase, Web of Science, and Institute of Electrical and Electronics Engineers Explore databases were systematically searched for studies on predicting BG levels and predicting or detecting adverse BG events using ML models, from inception to November 2022. Studies that assessed the performance of different ML models in predicting or detecting BG levels or adverse BG events of patients with DM were included. Studies with no derivation or performance metrics of ML models were excluded. The Quality Assessment of Diagnostic Accuracy Studies tool was applied to assess the quality of included studies. Primary outcomes were the relative ranking of ML models for predicting BG levels in different prediction horizons (PHs) and pooled estimates of the sensitivity and specificity of ML models in detecting or predicting adverse BG events. In total, 46 eligible studies were included for meta-analysis. Regarding ML models for predicting BG levels, the means of the absolute root mean square error (RMSE) in a PH of 15, 30, 45, and 60 minutes were 18.88 (SD 19.71), 21.40 (SD 12.56), 21.27 (SD 5.17), and 30.01 (SD 7.23) mg/dL, respectively. The neural network model (NNM) showed the highest relative performance in different PHs. Furthermore, the pooled estimates of the positive likelihood ratio and the negative likelihood ratio of ML models were 8.3 (95% CI 5.7-12.0) and 0.31 (95% CI 0.22-0.44), respectively, for predicting hypoglycemia and 2.4 (95% CI 1.6-3.7) and 0.37 (95% CI 0.29-0.46), respectively, for detecting hypoglycemia. Statistically significant high heterogeneity was detected in all subgroups, with different sources of heterogeneity. For predicting precise BG levels, the RMSE increases with a rise in the PH, and the NNM shows the highest relative performance among all the ML models. Meanwhile, current ML models have sufficient ability to predict adverse BG events, while their ability to detect adverse BG events needs to be enhanced. PROSPERO CRD42022375250; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=375250.

12074 Background: Skeletal-related events (SREs) are detrimental clinical events in bone metastasis from advanced cancer, defined by pathologic fracture, spinal cord compression, and inevitable surgical or radiational intervention to the bone. Given their negative impact on quality of life and prognosis and interpatient heterogeneity in the SRE risk, accurate identification of patients with high SRE risk is critical. Methods: The patient-level data from three randomized clinical trials that administered zoledronic acid (ZA) to patients with bone-metastatic breast cancer, castration-resistant prostate cancer (CRPC), and other types of cancer were analyzed. (N = 460, 452, and 315, respectively) Machine learning (ML) models to predict SREs within 18 months (breast cancer), 12 months (CRPC), and 9 months (other cancers) were developed based on more than 40 baseline clinical and laboratory data. Seven ML algorithms and five feature selection methods were utilized to develop multiple models. The ML model with the best performance was identified based on the F1 score and the area under the receiver operating characteristic curve (AUC-ROC) for each cancer type and interrogated for important features with Shapley additive explanations. Lastly, the ML models’ ability to stratify patients by the cumulative SRE risk was evaluated by calculating hazard ratio (HR) with Cox-proportional hazards models. Results: Among the multiple ML models developed with different algorithms and feature selection methods, the model developed utilizing the random forests algorithm and the Boruta method for selecting features demonstrated the best performance in all types of cancer (F1 0.70, 0.67, and 0.67, and AUC-ROC 0.72, 0.68, and 0.73 for breast cancer, CRPC, and other cancers, respectively). In the ML model for breast cancer, performance status (PS), history of SRE, serum alkaline phosphatase (ALP), history of anti-neoplastic surgery, radiation therapy, and pathologic fracture were included as important features. Serum ALP, albumin, sodium, Gleason scores, and geographic regions were shown to be relevant in the CRPC model. For other cancers, serum ALP, albumin, total protein, phosphorus, red blood cell count, white blood cell count, visceral metastases, and history of arthritis were incorporated in the ML model. The ML model prediction successfully stratified the patients for cumulative SRE risk in all three cohorts. (HR with 95% confidence interval: 2.43 [1.86 – 3.18], 1.92 [1.51 – 2.45], and 3.06 [2.29 – 4.09] for breast cancer, CRPC, and other types of cancer, respectively). Conclusions: ML models incorporating baseline clinical and laboratory data can identify patients with bone metastasis on ZA harboring a high SRE risk.

PD27-01 DEVELOPMENT OF A MACHINE LEARNING (ML) MODEL TO AUTOMATICALLY AND PRECISELY IDENTIFY KIDNEY STONES FROM URETEROSCOPY VIDEO RECORDINGS

BackgroundAcute coronary syndromes (ACS) are the leading cause of global death. Optimizing mortality risk prediction and early identification of high-risk patients is essential for developing targeted prevention strategies. Many researchers have built machine learning (ML) models to predict the mortality risk in ACS patients. Our meta-analysis aimed to evaluate the predictive value of various ML models in predicting death in ACS patients at different times.MethodsPubMed, Embase, Web of Science, and Cochrane Library were searched systematically from database establishment to March 12, 2022 for studies developing or validating at least one ML predictive model for death in ACS patients. We used PROBAST to assess the risk of bias in the reported predictive models and a random-effects model to assess the pooled C-index and accuracy of these models.ResultsFifty papers were included, involving 216 ML prediction models, 119 of which were externally validated. The combined C-index of the ML models in the validation cohort predicting the in-hospital mortality, 30-day mortality, 3- or 6-month mortality, and 1 year or above mortality in ACS patients were 0.8633 (95% CI 0.8467–0.8802), 0.8296 (95% CI 0.8134–0.8462), 0.8205 (95% CI 0.7881–0.8541), and 0.8197 (95% CI 0.8042–0.8354), respectively, with the corresponding combined accuracy of 0.8569 (95% CI 0.8411–0.8715), 0.8282 (95% CI 0.7922–0.8591), 0.7303 (95% CI 0.7184–0.7418), and 0.7837 (95% CI 0.7455–0.8175), indicating that the ML models were relatively excellent in predicting ACS mortality at different times. Furthermore, common predictors of death in ML models included age, sex, systolic blood pressure, serum creatinine, Killip class, heart rate, diastolic blood pressure, blood glucose, and hemoglobin.ConclusionsThe ML models had excellent predictive power for mortality in ACS, and the methodologies may need to be addressed before they can be used in clinical practice.

Understanding the Performance of Machine Learning Models to Predict Credit Default: A Novel Approach for Supervisory Evaluation

Recently, great progress has been achieved in diabetes management through minimally invasive continuous glucose monitoring, which involves piercing the skin to identify changes in glucose levels in the interstitial fluid. Nevertheless, the development of accurate fully noninvasive (NI) glucose monitors remains a topic of great interest. This work examines the processing of data collected from a multisensor system, which combines millimeter-wave (mm-wave), near-infrared (NIR), and other auxiliary sensors’ technology, coupled with a random forest machine learning (ML) model for prediction. We investigated possible correlations between mm-wave, NIR, skin temperature, pressure data, and blood glucose values and deployed an ML model for prediction. The model is based on the random forest algorithm that was trained and tested on data collected from a study involving five healthy human subjects undergoing an intravenous glucose tolerance test. Our model achieves a root mean square error of 21.06 mg/dl and a mean absolute relative difference of 7.31% for glucose prediction in the collected clinical subjects dataset. In addition, 96.1% of the model’s predictions fall within the clinically acceptable zones according to the Clarke error grid analysis. The preliminary results suggest that combining multiple sensor data with ML techniques is promising for the development of NI glucose monitoring.

Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosis.

To develop a machine-learning (ML) model for prediction of shoulder dystocia (ShD) and to externally validate the model's predictive accuracy and potential clinical efficacy in optimizing the use of Cesarean delivery in the context of suspected macrosomia. We used electronic health records (EHR) from the Sheba Medical Center in Israel to develop the model (derivation cohort) and EHR from the University of California San Francisco Medical Center to validate the model's accuracy and clinical efficacy (validation cohort). Subsequent to application of inclusion and exclusion criteria, the derivation cohort included 686 singleton vaginal deliveries, of which 131 were complicated by ShD, and the validation cohort included 2584 deliveries, of which 31 were complicated by ShD. For each of these deliveries, we collected maternal and neonatal delivery outcomes coupled with maternal demographics, obstetric clinical data and sonographic fetal biometry. Biometric measurements and their derived estimated fetal weight were adjusted (aEFW) according to gestational age at delivery. A ML pipeline was utilized to develop the model. In the derivation cohort, the ML model provided significantly better prediction than did the current clinical paradigm based on fetal weight and maternal diabetes: using nested cross-validation, the area under the receiver-operating-characteristics curve (AUC) of the model was 0.793 ± 0.041, outperforming aEFW combined with diabetes (AUC = 0.745 ± 0.044, P = 1e-16 ). The following risk modifiers had a positive beta that was > 0.02, i.e. they increased the risk of ShD: aEFW (beta = 0.164), pregestational diabetes (beta = 0.047), prior ShD (beta = 0.04), female fetal sex (beta = 0.04) and adjusted abdominal circumference (beta = 0.03). The following risk modifiers had a negative beta that was < -0.02, i.e. they were protective of ShD: adjusted biparietal diameter (beta = -0.08) and maternal height (beta = -0.03). In the validation cohort, the model outperformed aEFW combined with diabetes (AUC = 0.866 vs 0.784, P = 0.00007). Additionally, in the validation cohort, among the subgroup of 273 women carrying a fetus with aEFW ≥ 4000 g, the aEFW had no predictive power (AUC = 0.548), and the model performed significantly better (0.775, P = 0.0002). A risk-score threshold of 0.5 stratified 42.9% of deliveries to the high-risk group, which included 90.9% of ShD cases and all cases accompanied by maternal or newborn complications. A more specific threshold of 0.7 stratified only 27.5% of the deliveries to the high-risk group, which included 63.6% of ShD cases and all those accompanied by newborn complications. We developed a ML model for prediction of ShD and, in a different cohort, externally validated its performance. The model predicted ShD better than did estimated fetal weight either alone or combined with maternal diabetes, and was able to stratify the risk of ShD and neonatal injury in the context of suspected macrosomia. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

BackgroundIn patients with suspected obstructive coronary artery disease (CAD), evaluation using a pre-test probability model is the key element for diagnosis; however, its accuracy is controversial. This study aimed to develop machine learning (ML) models using clinically relevant biomarkers to predict the presence of stable obstructive CAD and to compare ML models with an established pre-test probability of CAD models.MethodsEight machine learning models for prediction of obstructive CAD were trained on a cohort of 1,312 patients [randomly split into the training (80%) and internal validation sets (20%)]. Twelve clinical and blood biomarker features assessed on admission were used to inform the models. We compared the best-performing ML model and established the pre-test probability of CAD (updated Diamond-Forrester and CAD consortium) models.ResultsThe CatBoost algorithm model showed the best performance (area under the receiver operating characteristics, AUROC, 0.796, and 95% confidence interval, CI, 0.740–0.853; Matthews correlation coefficient, MCC, 0.448) compared to the seven other algorithms. The CatBoost algorithm model improved risk prediction compared with the CAD consortium clinical model (AUROC 0.727; 95% CI 0.664–0.789; MCC 0.313). The accuracy of the ML model was 74.6%. Age, sex, hypertension, high-sensitivity cardiac troponin T, hemoglobin A1c, triglyceride, and high-density lipoprotein cholesterol levels contributed most to obstructive CAD prediction.ConclusionThe ML models using clinically relevant biomarkers provided high accuracy for stable obstructive CAD prediction. In real-world practice, employing such an approach could improve discrimination of patients with suspected obstructive CAD and help select appropriate non-invasive testing for ischemia.

Lung adenocarcinoma (LUAD) is the leading cause of cancer-related deaths. High-resolution computed tomography (HRCT) has improved the detection of ground glass nodules (GGNs), which are early indicators of lung cancer. Accurate assessment of GGN invasiveness is crucial for determining the appropriate surgical approach. Dual-layer spectral detector computed tomography (DLCT) offers advanced imaging capabilities, including electron density and iodine density, which enhance the evaluation of GGN invasiveness. This study aims to develop a machine learning (ML) model that integrates DLCT parameters and clinical features to predict the invasiveness of GGNs in LUAD, aiding in surgical decision-making and prognosis improvement. The retrospective study encompassed 272 patients who were diagnosed with LUAD, comprising154 cases of invasive adenocarcinomas(IA) and118 cases of pre-invasive minimally invasive adenocarcinoma (MIA) which were then randomly allocated into atraining setand atest set. Six ML models were developed based on five DLCT parameters (conventional, iodine density, virtual noncontrast, electron density, and effective atomic number). Subsequently, a nomogram was constructed using multi-factor logistic regression, incorporating radiomic characteristics and clinicopathological risk factors. The ML model based on conventional plus electron density performed better than the models with other DLCT parameters,with the area under the curves (AUCs) of0.945 and 0.964 in the training and test sets, respectively. The clinical model and radiomics score (Rad-score) were combinedin the logistic regression to construct a joint model,of which the AUCs were0.974 in the training sets and 0.949 in the test sets. The ML model effectively differentiated between IA and pre-invasive MIA, and further classified patients into high and medium risk categories for invasion using waterfall plots. The ML model based on DLCT parameters helpspredict the invasiveness of GGNs and classifiesthe GGNs intodifferent risk grades.

Background/AimsPrediction of short-term mortality in patients with acute-on-chronic liver failure (ACLF) admitted to the intensive care unit (ICU) may enhance effective management.MethodsTo develop, explain, and validate a predictive machine learning (ML) model for short-term mortality in patients with ACLF with two or more organ failures (OFs). Utilizing a large ICU cohort with detailed clinical information, we identified ACLF patients with two or more OFs according to the EASL-CLIF and NACSELD definitions. ML model was developed for each definition to predict 30-day mortality. The Shapley value was estimated to explain the models. Validation and calibration of these models were performed.ResultsOf 5,994 patients with cirrhosis admitted to ICU, 1,511 met NACSELD criteria, and 1,692 met EASL-CLIF grade II or higher criteria. The CatBoost ACLF (CBA) model had the greatest accuracy in the NACSELD cohort (area under curve [AUC] of 0.87), while the Random Forest ACLF (RFA) model performed best in the EASL-CLIF cohort (AUC of 0.83). Both models showed robust calibration. The models were explained by SHAP score analysis, yielding a rank list, and the top twelve predictors were selected. Both simplified models demonstrated similar performance (CBA model: AUC 0.89, RFA model: AUC 0.81) and significantly outperformed contemporary scoring systems, including CLIF-C ACLF and MELD 3.0. The models were validated in both internal and external cohorts. A simple-to-use online tool was created to predict mortality rates.ConclusionsWe presented explainable, well-validated, and calibrated predictive models for ACLF patients with two or more OFs, which outperformed existing predictive scores.

Applications of Machine Learning Model for Prediction of Outcomes in Primary Pontine Hemorrhage

Background There is a surging development of digital technologies for screening and diagnosis of atrial fibrillation (AF), nonetheless, how these digital technologies impact AF care remains unclear. Objectives The present study aimed to investigate the impact on AF episodes and arrhythmia burden of patient-centered self-management with using smart technologies, incorporating artificial intelligence (AI), wearable and mobile health (mHealth) technology for patient-centred self-management. Methods In this cohort study, we applied AI machine-learning (ML) model predicting AF prior to 4 hours of AF occurrence, while the timely monitoring of AF episodes with photoplethysmography (PPG) technology, and further confirmation of the diagnosis of AF with single-lead ECG, was based on wearable devices. These have been developed in AF screening study (stage 1 pre-mAFA) of Mobile Health technology for improved screening and optimizing integrated care in atrial fibrillation (mAFA II programme), while mHealth supported integrated care of AF was validated to reduce clinical events (stage 2 mAFA II cluster randomized trial). In this observational cohort, the subjects were in 2 groups: (i) subjects with monitored AF but without using mAFA (Group 1); (ii) subjects with monitored AF and using smart devices and mAFA for patient-centered self-management (Group 2). Adult subjects freely downloaded the mAFA AF App, with compatible devices, and were included into the study from across China between October 26, 2018 and Dec 1, 2021. Results From 3499461 subjects involving in AF population screening, there were 5904 subjects in Group 1 (mean age 57 years, SD,15 years; 80.7%, male), while 2667 subjects in Group 2 (51, 15; 89.7% male). The diagnosis of AF episodes was confirmed by single-lead ECG recordings, and decreased among those using mAFA in Group 2 (Confirmed AF: 602 in 1st quarter, 224 in 2nd quarter, 81 in 3rd quarter, 52 in 4th quarter) (all p value for trend&amp;lt;0.001). AF burden was significantly reduced over time (PPG: 26% on 1st month, 19% on 12th month; ECG: 3%, 0.7%, all p&amp;lt;0.001), while the average probability of AF occurrences by AI ML model prediction was decreased among those using mAFA in Group 2 (65%, 57%, p&amp;lt;0.001). The decreasing trend of AF burden over time was not seen in subjects with detected AF without mAFA (Group 1). On multivariate analysis, AI ML model driven mAFA-based upstream risk factor control significantly reduced the changes in AF burden of both Δ 6 month (adjusted OR, 95% CI, 0.51,0.43–0.59) and Δ 12 month (0.39,0.32–0.47, all p&amp;lt;0.001). Conclusion mHealth-AF care increased the adherence of using smart devices for AF detection. The incorporation of smart devices and AI tools into an AF clinical care pathway, effectively reduced AF burden and detected AF episodes, through appropriate rhythm control management and upstream risk factor control. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): National Natural Science Foundation of China