Clinical performance of automated machine learning: A systematic review

Introduction: Automated machine learning (autoML) removes technical and technological barriers to building artificial intelligence models. We aimed to summarise the clinical applications of autoML, assess the capabilities of utilised platforms, evaluate the quality of the evidence trialling autoML, and gauge the performance of autoML platforms relative to conventionally developed models, as well as each other. Method: This review adhered to a prospectively registered protocol (PROSPERO identifier CRD42022344427). The Cochrane Library, Embase, MEDLINE and Scopus were searched from inception to 11 July 2022. Two researchers screened abstracts and full texts, extracted data and conducted quality assessment. Disagreement was resolved through discussion and as if required, arbitration by a third researcher. Results: There were 26 distinct autoML platforms featured in 82 studies. Brain and lung disease were the most common fields of study of 22 specialties. AutoML exhibited variable performance: area under the receiver operator characteristic curve (AUCROC) 0.35–1.00, F1-score 0.16–0.99, area under the precision-recall curve (AUPRC) 0.51–1.00. AutoML exhibited the highest AUCROC in 75.6% trials; the highest F1-score in 42.3% trials; and the highest AUPRC in 83.3% trials. In autoML platform comparisons, AutoPrognosis and Amazon Rekognition performed strongest with unstructured and structured data, respectively. Quality of reporting was poor, with a median DECIDE-AI score of 14 of 27. Conclusion: A myriad of autoML platforms have been applied in a variety of clinical contexts. The performance of autoML compares well to bespoke computational and clinical benchmarks. Further work is required to improve the quality of validation studies. AutoML may facilitate a transition to data-centric development, and integration with large language models may enable AI to build itself to fulfil user-defined goals.

Electronic health records (EHRs) with clinical decision support tools are now ubiquitous in healthcare organizations. Clinical foundation models (CFMs) pretrained on large-scale, heterogeneous structured EHR data have emerged as a powerful approach to improve predictive performance and generalizability. Meanwhile, large language models (LLMs) pretrained on broad data sources are being applied to an expanding range of healthcare tasks. However, it remains unclear whether generalist LLMs can match specialized CFMs for disease risk prediction using structured clinical data. We compared CFMs (Med-BERT, CLMBR) against fine-tuned generalist LLMs (Mistral, LLaMA-2/3/3.1), a clinical LLM (Me-LLaMA), and LLM-generated embeddings paired with simple classifiers (using DeepSeek, Qwen3, and GPT-OSS) on two disease risk prediction tasks: heart failure risk among diabetic patients (DHF) and pancreatic cancer diagnosis (PaCa). Evaluations spanned multi-site EHR data, claims data, and an open-source single-institution benchmark (EHRSHOT). Performance was assessed using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). On larger EHR and claims cohorts (>30,000 patients), fine-tuned CFMs outperformed fine-tuned LLMs by a small but statistically significant margin (<1% AUROC). The clinical LLM performed comparably to generalist LLMs despite being smaller. On the open-source PaCa cohort (3,810 patients, 199 cases), LLMs achieved slightly higher AUROCs that were not statistically significant (LLaMA-3.1-70B 86.1% vs. Med-BERT 85.3%, p=0.27), but CFMs achieved significantly higher AUPRC (Med-BERT 55.9% vs. LLaMA-3.1-70B 41.1%, p=0.001). Notably, LLM-generated trajectory embeddings paired with logistic regression or a simple MLP, without any LLM fine-tuning, achieved the best overall performance, with AUROC exceeding 90% (Qwen3) and AUPRC reaching 66% (GPT-OSS 20B). LLM-generated embeddings with lightweight classifiers outperformed both fine-tuned CFMs and fine-tuned LLMs on AUROC and AUPRC. While these results demonstrate the potential of generalist models to match or surpass specialized CFMs, their substantially greater computational cost and variable AUPRC performance in the fine-tuning setting warrant caution. We provide a reproducible evaluation framework and codebase to support continued benchmarking.

BackgroundIn the pediatric intensive care unit (PICU), quantifying illness severity can be guided by risk models to enable timely identification and appropriate intervention. Logistic regression models, including the pediatric index of mortality 2 (PIM-2) and pediatric risk of mortality III (PRISM-III), produce a mortality risk score using data that are routinely available at PICU admission. Artificial neural networks (ANNs) outperform regression models in some medical fields.ObjectiveIn light of this potential, we aim to examine ANN performance, compared to that of logistic regression, for mortality risk estimation in the PICU.MethodsThe analyzed data set included patients from North American PICUs whose discharge diagnostic codes indicated evidence of infection and included the data used for the PIM-2 and PRISM-III calculations and their corresponding scores. We stratified the data set into training and test sets, with approximately equal mortality rates, in an effort to replicate real-world data. Data preprocessing included imputing missing data through simple substitution and normalizing data into binary variables using PRISM-III thresholds. A 2-layer ANN model was built to predict pediatric mortality, along with a simple logistic regression model for comparison. Both models used the same features required by PIM-2 and PRISM-III. Alternative ANN models using single-layer or unnormalized data were also evaluated. Model performance was compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC) and their empirical 95% CIs.ResultsData from 102,945 patients (including 4068 deaths) were included in the analysis. The highest performing ANN (AUROC 0.871, 95% CI 0.862-0.880; AUPRC 0.372, 95% CI 0.345-0.396) that used normalized data performed better than PIM-2 (AUROC 0.805, 95% CI 0.801-0.816; AUPRC 0.234, 95% CI 0.213-0.255) and PRISM-III (AUROC 0.844, 95% CI 0.841-0.855; AUPRC 0.348, 95% CI 0.322-0.367). The performance of this ANN was also significantly better than that of the logistic regression model (AUROC 0.862, 95% CI 0.852-0.872; AUPRC 0.329, 95% CI 0.304-0.351). The performance of the ANN that used unnormalized data (AUROC 0.865, 95% CI 0.856-0.874) was slightly inferior to our highest performing ANN; the single-layer ANN architecture performed poorly and was not investigated further.ConclusionsA simple ANN model performed slightly better than the benchmark PIM-2 and PRISM-III scores and a traditional logistic regression model trained on the same data set. The small performance gains achieved by this two-layer ANN model may not offer clinically significant improvement; however, further research with other or more sophisticated model designs and better imputation of missing data may be warranted.

Coronary artery disease (CAD) results in substantial morbidity and mortality. The purpose of this study was to develop a deep learning model to detect CAD defined using diagnostic codes ("ECG2CAD") and identify people at risk for adverse events using electrocardiograms (ECGs) in a primary care setting. ECG2CAD was trained on 764,670 ECGs representing 137,199 individuals at Massachusetts General Hospital (MGH). Model performance for discrimination of prevalent CAD was measured using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), and compared against model of age and sex, and Pooled Cohort Equations, in 3 test sets: MGH, Brigham and Women's Hospital (BWH), and UK Biobank. Subgroups were assessed for incident CAD-related events in a BWH primary care cohort. ECG2CAD was evaluated in MGH (N = 18,706 [6,051 cases], age 57 ± 16 years), BWH (N = 88,270 [27,898 cases], age 57 ± 16 years), and UK Biobank (N = 42,147 [1,509 cases], age 65 ± 8 years). ECG2CAD consistently discriminated prevalent CAD (MGH AUROC: 0.782; AUPRC: 0.639; BWH: AUROC: 0.747; AUPRC: 0.588; UK Biobank AUROC: 0.760; AUPRC: 0.155) and incrementally vs models based on age and sex or Pooled Cohort Equations (P < 0.01) in MGH and BWH. In the BWH primary care subset, model performance was consistent across subgroups. Being in the highest quintile of ECG2CAD risk was associated with higher risk for adverse events compared with low-risk group (myocardial infarction HR: 5.59; 95% CI: 4.76-6.56, heart failure 10.49; 95% CI: 7.96-13.84, all-cause mortality 2.68; 95% CI: 2.32-3.10). Artificial intelligence-enabled analysis of the ECG may facilitate identification of individuals with possible undiagnosed CAD and inform downstream testing and preventive measures.

BackgroundTreatment of sepsis requires prompt initiation of appropriate empiric antibiotics. Coverage for Gram-negative bacilli (GNB) is decided based on risk factors for resistant organisms, as documented in both structured and unstructured data in the electronic health record. We hypothesized that adding large language models (LLMs) that can decode clinical narratives would improve performance of deep learning models in predicting antimicrobial resistance in GNB sepsis.MethodsThis retrospective cohort analysis included all adult patients who met CDC sepsis criteria admitted to 10 BJC hospitals from 1/2018 to 12/2023. Sepsis episodes were classified as community-onset (< 48hrs from hospitalization) or hospital-onset. GNB were stratified as ceftriaxone-susceptible (SS), ceftriaxone-resistant but cefepime-susceptible (RS), and ceftriaxone- and cefepime-resistant (RR). Culture negative or non-GNB sepsis were labeled SS. Deep learning models (XGBoost, DeepAMR) were developed using all available structured data (e.g., demographics, vitals, labs). LLMs (BioGPT, Medical-LLaMa3-8B) were developed on full or partial history and physical exam notes. Models were assessed using area under the receiver operating characteristic curve (AUROC) and area under precision-recall curve (AUPRC).ResultsA total of 29,517 patients were included, for 23,949 community-onset and 25,101 hospital-onset sepsis episodes with RS+RR GNB in 9% and 17%, respectively (Table 1). Relevant features of structured data models included history of prior infections and resistant organisms. DeepAMR was the best performing deep learning model with AUROC 0.85/AUPRC 0.40 and AUROC 0.81/AUPRC 0.50 for community- and hospital-onset sepsis, respectively (Table 2). The best performing LLM was Medical-LLaMa3-8B on full notes (Figure 1), with AUROC 0.74/AUPRC 0.21 and AUROC 0.67/AUPRC 0.30 for community- and hospital-onset sepsis, respectively. Combining deep learning and LLMs did not significantly improve the prediction of resistant infections compared to deep learning alone.ConclusionStructured medical data provided the highest prediction of resistant GNB infections in sepsis. More analysis is needed to understand why the unstructured data did not improve model predictive performance.DisclosuresMaria Cristina Vazquez Guillamet, MD, Bionano: Stocks/Bonds (Public Company)|Charisma: Stocks/Bonds (Public Company)|Ocugen: Stocks/Bonds (Public Company)

Purpose: Lung cancer screening (LCS) lowers cancer mortality if individuals adhere to annual scans. Minoritized communities have significantly lower adherence to LCS than non-Hispanic whites. The influence of social determinants of health (SDOH) on these disparate adherence rates are rarely evaluated. The aim of this study is to develop a machine learning (ML) model that integrates individual and neighborhood level SDOH to predict which minoritized individuals are at risk of LCS non-adherence. Methods: A total of 168 participants with high-risk smoking histories who underwent their first low dose computed tomography (LDCT) between 2017 and 2021 were enrolled. Surveys on SDOH and perceived discrimination were completed. Participants’ addresses were joined to neighborhood level SDOH data from various publicly available sources. These indices captured neighborhood socioeconomic status and segregation. Adherence rates were calculated by assessing electronic medical records to determine if participants had a second LDCT within 15 months. Participants were temporally split into training (N=125) and testing (N=43) sets with a cut-off date of June 30, 2021. A gradient boosting classifier, XGBoost, was trained to predict LCS non-adherence, incorporating variables such as sociodemographic status, perceived discrimination, previous screenings, smoking status, and geospatial indices. The SHapley Additive exPlanations (SHAP) method was used to determine the relative importance of each feature in the model’s predictions. Results: The participants represented a diverse group: Asian (26%), Black/African American (32%), and Hispanic/Latino (32%). The LCS non-adherence rates were 66% in the training set and 63% in the testing set. The XGBoost classifier achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.79 and an Area Under the Precision-Recall Curve (AUPRC) of 0.90, outperforming a simple logistic regression model (AUROC=0.63 and AUPRC=0.79). The XGBoost classifier’s performance metrics were Accuracy=0.77, Recall=0.85, Specificity=0.62, Positive Predictive Value=0.79, and Negative Predictive Value=0.71. The SHAP analysis indicated that participants with no prior history of colorectal cancer screening, who lived in areas with lower quality school systems, and with a higher percentage of individuals below the poverty line, were at a higher risk of non-adherence to LCS. Conclusions: LCS reduces lung cancer survival, but LCS adherence is significantly lower in minoritized groups. Our ML model strongly predicted LCS non-adherence based on SDOH and geospatial factors. Specific neighborhood level SDOHs predicted LCS non-adherence more strongly than individual level factors. These findings support interventions that address both individual and neighborhood-level barriers to increase LCS adherence. Citation Format: Kun-Han Lu, Yi Xiao, Aamna Akhtar, Jazma Tapia, Calvin Tribby, Jay Yang, Marta Jankowska, Chi Wah Wong, Loretta Erhunmwunsee. Predicting lung cancer screening adherence among underserved and minoritized communities using social determinants and neighborhood factors [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 4971.

Hospital-onset bacteremia and fungemia (HOB) are common and potentially preventable complications of hospital care. To assess whether nonindividual patient features, which summarize interactions with other patients and health care workers (HCWs), can contribute to predictive and causal machine learning models for HOB. This prognostic study included adult patients admitted to Barnes-Jewish Hospital, an academic hospital in St Louis, Missouri, in 2021. Analyses were developed between October 2023 and August 2024 and in April 2025. Individual patient features were extracted from electronic health records and used to engineer nonpatient features, including interactions with HCWs and direct or indirect (consecutive room occupancy) patient contact. HOB was defined as a positive blood culture after the third day of hospitalization. Patients who were hospitalized for more than 3 days were considered at risk for the outcome. We developed 3 gradient boosting models: 2 predictive (with patient features only and with both patient and nonpatient features to predict the occurrence of HOB) and 1 causal to test the association of nonpatient features and HOB. Predictive performance is reported using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), and the results of the causal model are reported as difference in average effects. Sensitivity analyses separated intensive care unit-onset and ward-onset HOB and included a methicillin-resistant Staphylococcus aureus-specific model to adjust for colonization pressure. Among the 52 442 patients, 34 855 (66.5%) had admissions longer than 72 hours and were included for analysis; of these, 556 (1.6%) developed HOB. The median age for the included patients was 60 (IQR, 44-70) years, 50.5% were female, and obesity was the most frequent comorbidity (25.0%). Nonpatient features, such as a prior occupant of the same room receiving antipseudomonal beta-lactams and the mean number of HCWs per day for the 7 days preceding HOB, improved the model's performance (AUROC, 0.88 [95% CI, 0.88-0.89]; AUPRC, 0.20 [95% CI, 0.20-0.22]) compared with the patient-only model (AUROC, 0.85 [95% CI, 0.85-0.86]; AUPRC, 0.13 [95% CI, 0.12-0.14]) (P < .001). These 2 features were also associated with a higher likelihood of HOB in the causal gradient boosting model. These findings suggest that nonindividual patient features may contribute to a comprehensive analysis of HOB when integrated with individual patient features in a machine learning model.

Older adults with diabetes are at high risk of severe hypoglycemia (SH). Many machine-learning (ML) models predict short-term hypoglycemia are not specific for older adults and show poor precision-recall. We aimed to develop a multidimensional, electronic health record (EHR)-based ML model to predict one-year risk of SH requiring hospitalization in older adults with diabetes. We adopted a case-control design for a retrospective territory-wide cohort of 1,456,618 records from 364,863 unique older adults (age ≥65 years) with diabetes and at least 1 Hong Kong Hospital Authority attendance from 2013 to 2018. We used 258 predictors including demographics, admissions, diagnoses, medications, and routine laboratory tests in a one-year period to predict SH events requiring hospitalization in the following 12 months. The cohort was randomly split into training, testing, and internal validation sets in a 7:2:1 ratio. Six ML algorithms were evaluated including logistic-regression, random forest, gradient boost machine, deep neural network (DNN), XGBoost, and Rulefit. We tested our model in a temporal validation cohort in the Hong Kong Diabetes Register with predictors defined in 2018 and outcome events defined in 2019. Predictive performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC) statistics, and positive predictive value (PPV). We identified 11,128 SH events requiring hospitalization during the observation periods. The XGBoost model yielded the best performance (AUROC = 0.978 [95% CI 0.972 to 0.984]; AUPRC = 0.670 [95% CI 0.652 to 0.688]; PPV = 0.721 [95% CI 0.703 to 0.739]). This was superior to an 11-variable conventional logistic-regression model comprised of age, sex, history of SH, hypertension, blood glucose, kidney function measurements, and use of oral glucose-lowering drugs (GLDs) (AUROC = 0.906; AUPRC = 0.085; PPV = 0.468). Top impactful predictors included non-use of lipid-regulating drugs, in-patient admission, urgent emergency triage, insulin use, and history of SH. External validation in the HKDR cohort yielded AUROC of 0.856 [95% CI 0.838 to 0.873]. Main limitations of this study included limited transportability of the model and lack of geographically independent validation. Our novel-ML model demonstrated good discrimination and high precision in predicting one-year risk of SH requiring hospitalization. This may be integrated into EHR decision support systems for preemptive intervention in older adults at highest risk.

BackgroundMachine learning (ML) models for clinical prediction are often developed independently of downstream value, interpretability, and decision-making needs. As a result, model outputs may be difficult to act upon in practice, especially when trade-offs between various clinical aspects are not explicitly addressed.PurposeThis paper presents a cost-aware prediction (CAP) framework designed to support 1-year mortality prediction for heart failure (HF) patients to identify those eligible for home care based on high predicted mortality risk. By leveraging cost-benefit analysis and large language model (LLM) agents, the framework facilitates clinical interpretability and helps stakeholders navigate real-world trade-offs in ML-based decision support.MethodsFirst, we developed an ML model predicting 1-year mortality using electronic health records including all patients with a first in-hospital HF diagnosis between 2017-2023 (N=30,021, 22% mortality) from Region Västra Götaland, Sweden (total population 1.8 million people). Second, we introduced clinical impact projection (CIP) curves to visualise important cost dimensions – quality of life and healthcare provider expenses. These costs were further divided into treatment and error costs, to assess the consequences of both correct and incorrect predictions. Finally, we developed four LLM agents to generate individual patient-specific descriptions explaining the certainty of the prediction, cost-benefit considerations, and risk reduction strategies. The system was evaluated by clinicians for decision support value.ResultsFor the ML predictive modelling, the eXtreme gradient boosting model achieved the best performance, with an area under the receiver operating characteristic curve (AUROC) of 0.804 (95%; confidence interval (CI) 0.792-0.816), area under the precision-recall curve (AUPRC) of 0.529 (95% CI 0.502-0.558) and a Brier score of 0.135 (95% CI 0.130-0.140) shown in Figure 1. The CIP cost curves provided a population-level overview of cost composition across decision thresholds. LLM-generated cost-benefit descriptions enabled prediction analysis at individual patient-levels. An illustrative example can be found in Figure 2. The system was generally well received according to the evaluation with clinicians. However, clinical feedback emphasised the need to strengthen the technical accuracy for speculative tasks.ConclusionsAn ML algorithm was developed to predict 1-year mortality risks. Further, the CAP framework utilises LLM agents to integrate this ML classifier outcomes and cost-benefit analysis, to support clinical decision-making by more transparent and interpretable communication. While receiving positive clinical feedback, future work is needed for fine-tuning of LLM narratives for optimal accuracy and usefulness.

Delirium is an acute syndrome characterized by fluctuating attention, cognitive impairment, and severe disorganization of behavior, which has been shown to affect up to 31% of patients in the intensive care unit (ICU). Early detection can enable timely interventions and improved health outcomes. While artificial intelligence (AI) models have shown great potential for ICU delirium prediction using structured electronic health records (EHR), most studies have either not leveraged state-of-the-art AI models, been limited to single-center cohorts, or relied on small datasets for development and validation. In this study, we introduce DeLLiriuM, a novel LLM-based delirium prediction model that utilizes EHR data from the first 24 hours of ICU admission to estimate a patient’s risk of developing delirium for the remainder of their ICU stay. We developed and validated DeLLiriuM using ICU admissions from 104,303 patients across 195 hospitals in three large databases: the eICU Collaborative Research Database, the Medical Information Mart for Intensive Care (MIMIC)-IV, and the University of Florida’s Integrated Data Repository. Our DeLLiriuM model achieved superior performance compared to all baseline models on the external validation set, measured by the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) metric. DeLLiriuM attained AUROC 82.4 (95% confidence interval 81.8–83.0) and AUPRC 11.8 (95% confidence interval 11.3–12.4) across 77,543 patients spanning 194 hospitals. Our approach of transforming structured EHR data into an unstructured text format, the primary data modality for LLMs, enables our DeLLiriuM model to capture clinical contextual information, resulting in improved predictive performance. To the best of our knowledge, DeLLiriuM is the first LLM-based delirium prediction tool for the ICU that utilizes structured EHR data with LLMs rather than clinical notes with LLMs or traditional structured feature representations used in AI models.

Fog computing extends cloud services to the network edge, enabling low-latency processing for Internet of Things (IoT) applications. However, this distributed approach is vulnerable to a wide range of attacks, necessitating advanced intrusion detection systems (IDSs) that operate under resource constraints. This study proposes integrating self-awareness (online learning and concept drift adaptation) into a lightweight RL (reinforcement learning)-based IDS for fog networks and quantitatively comparing it with non-RL static thresholds and bandit-based approaches in real time. Novel self-aware reinforcement learning (RL) models, the Hierarchical Adaptive Thompson Sampling–Reinforcement Learning (HATS-RL) model, and the Federated Hierarchical Adaptive Thompson Sampling–Reinforcement Learning (F-HATS-RL), were proposed for real-time intrusion detection in a fog network. These self-aware RL policies integrated online uncertainty estimation and concept-drift detection to adapt to evolving attacks. The RL models were benchmarked against the static threshold (ST) model and a widely adopted linear bandit (Linear Upper Confidence Bound/LinUCB). A realistic fog network simulator with heterogeneous nodes and streaming traffic, including multi-type attack bursts and gradual concept drift, was established. The models’ detection performance was compared using metrics including latency, energy consumption, detection accuracy, and the area under the precision–recall curve (AUPR) and the area under the receiver operating characteristic curve (AUROC). Notably, the federated self-aware agent (F-HATS-RL) achieved the best AUROC (0.933) and AUPR (0.857), with a latency of 0.27 ms and the lowest energy consumption of 0.0137 mJ, indicating its ability to detect intrusions in fog networks with minimal energy. The findings suggest that self-aware RL agents can detect traffic–dynamic attack methods and adapt accordingly, resulting in more stable long-term performance. By contrast, a static model’s accuracy degrades under drift.

Background/Introduction Most artificial intelligence-enabled electrocardiography (AI-ECG) research on single-lead ECG has primarily utilized Lead I data from the standard 12-lead ECG (S-Lead I) for model training. However, it remains uncertain whether the results validated using only S-Lead I data in conventional studies can be directly applied to wearable devices such as smartwatches. S-Lead I and the Lead I that acquired via a smartwatch (W-Lead I) differ substantially in sensor characteristics, noise patterns, and measurement methods. These discrepancies in data properties suggest that a model trained solely on S-Lead I may experience a decline in performance when applied directly to W-Lead I. Despite these concerns, there is limited evidence quantifying this performance gap. Purpose The primary goal of this study was to evaluate the generalizability of an AI-enabled left ventricular systolic dysfunction (LVSD) prediction model, trained on S-Lead I, when applied to W-Lead I. We examined the correlation between S-Lead I and W-Lead I performance throughout model training to determine whether improvements on S-Lead I were consistently reflected in W-Lead I, ensuring the model's generalizability across different ECG acquisition methods. Method We retrospectively collected 151,065 S-Lead I ECGs from 151,137 patients who underwent echocardiography within 14 days at Hospital A between 2017 and 2024, forming an ECG cohort. Additionally, we collected 2,441 W-Lead I data recorded within 24 hours of S-Lead I. The paired S-Lead I and W-Lead I data were used for model evaluation, while the S-Lead I data alone were used for model training. Predictive performance was assessed using the mean of the area under the receiver operating characteristic curve (AUROC) and the area under the precision–recall curve (AUPRC). In addition, correlation values were calculated across every training iteration to quantify how well performance trends on S-Lead I translated to W-Lead I. Results As shown in Figure 1.B , both the training dataset and the S-Lead I test set demonstrated a steady improvement in AUROC and AUPRC throughout the iterative training process, maintaining a strong performance correlation of 0.89 or higher. In contrast, the Apple Watch and Galaxy W-Lead I test sets displayed more pronounced fluctuations in performance and did not follow the same upward trend, with notably lower correlation values (Apple Watch: 0.54; Galaxy Watch: 0.69; see Figure 1.C). These results indicate that a model trained exclusively on S- Lead I may not sufficiently capture the sensor-specific and physiological nuances of W-Lead I. Conclusion The results indicate that an LVSD prediction model trained exclusively on S-Lead I has limited generalizability when applied to real-world W-Lead I data. Future efforts should focus on developing algorithms that incorporate the watch signal characteristics of smartwatch ECG, thereby enhancing clinical utility.

Multi-modal fusion model for Time-Varying medical Data: Addressing Long-Term dependencies and memory challenges in sequence fusion.

Patients with non-small cell lung cancer (NSCLC) and patients with NSCLC combined with chronic obstructive pulmonary disease (COPD) have similar physiological conditions in early stages, and the latter have shorter survival times and higher mortality rates. The purpose of this study was to develop and compare machine learning models to identify future diagnoses of COPD combined with NSCLC patients based on the patient's disease and routine clinical data. Data were obtained from 237 patients with COPD combined with NSCLC as well as NSCLC admitted to Ningxia Hui Autonomous Region People's Hospital from October 2013 to July 2022. Six machine learning algorithms (K-nearest neighbor, logistic regression, eXtreme gradient boosting, support vector machine, naïve Bayes, and artificial neural network) were used to develop prediction models for NSCLC combined with COPD. Sensitivity, specificity, positive predictive value, negative predictive value, accuracy, F1 score, Mathews correlation coefficient (MCC), Kappa, area under the receiver operating characteristic curve (AUROC)and area under the precision-recall curve (AUPRC) were used as performance indicators to evaluate the performance of the models. 135 patients with NSCLC combined with COPD, 102 patients with NSCLC were included in the study. The results showed that pulmonary function and emphysema were important risk factors and that the support vector machine-based identification model showed optimal performance with accuracy:0.946, recall:0.940, specificity:0.955, precision:0.972, npv:0.920, F1 score:0.954, MCC:0.893, Kappa:0.888, AUROC:0.975, AUPRC:0.987. The use of machine learning tools combining clinical symptoms and routine examination data features is suitable for identifying the risk of concurrent NSCLC in COPD patients.

To compare the performance of the ACS NSQIP "universal" risk calculator (N-RC) to operation-specific RCs. Resources have been directed toward building operation-specific RCs because of an implicit belief that they would provide more accurate risk estimates than the N-RC. However, operation-specific calculators may not provide sufficient improvements in accuracy to justify the costs in development, maintenance, and access. For the N-RC, a cohort of 5,020,713 NSQIP patient records were randomly divided into 80% for machine learning algorithm training and 20% for validation. Operation-specific risk calculators (OS-RC) and OS-RCs with operation-specific predictors (OSP-RC) were independently developed for each of 6 operative groups (colectomy, whipple pancreatectomy, thyroidectomy, abdominal aortic aneurysm (open), hysterectomy/myomectomy, and total knee arthroplasty) and 14 outcomes using the same 80%/20% rule applied to the appropriate subsets of the 5M records. Predictive accuracy was evaluated using the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and Hosmer-Lemeshow (H-L) P values, for 13 binary outcomes, and mean squared error for the length of stay outcome. The N-RC was found to have greater AUROC (P = 0.002) and greater AUPRC (P < 0.001) compared to the OS-RC. No other statistically significant differences in accuracy, across the 3 risk calculator types, were found. There was an inverse relationship between the operation group sample size and magnitude of the difference in AUROC (r = -0.278; P = 0.014) and in AUPRC (r = -0.425; P < 0.001) between N-RC and OS-RC. The smaller the sample size, the greater the superiority of the N-RC. While operation-specific RCs might be assumed to have advantages over a universal RC, their reliance on smaller datasets may reduce their ability to accurately estimate predictor effects. In the present study, this tradeoff between operation specificity and accuracy, in estimating the effects of predictor variables, favors the N-R, though the clinical impact is likely to be negligible.