Machine Learning Algorithm Prospectively Predicts Survival for High-Risk Patients Undergoing Radiotherapy: A Survival Analysis of SHIELD-RT

The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow. To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle. PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores. The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%). The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

Introduction: Cataract is a prevalent eye condition that affects millions of individuals worldwide, leading to visual impairment and reduced quality of life. Cataract surgery is the most effective treatment, but post-surgical complications can arise, impacting the success of the intervention. Optical coherence tomography (OCT) has emerged as a valuable imaging technique for evaluating these complications, but the manual interpretation of OCT images is time-consuming and subjective.Objective: In this study, we aimed to assess the performance of a machine learning (ML) tool specifically developed for detecting post-surgical complications in cataract patients.Methods: We employed a support vector machine (SVM) algorithm to analyze a comprehensive dataset of OCT images. The dataset comprised 700 OCT images obtained post-surgery, including patients with cystoid macular oedema (CMO), retinal detachment (RD), and healthy individuals. The ML tool utilized pre-processed OCT images with annotations provided by expert ophthalmologists, undergoing retinal layer segmentation using intensity-based features.Results: The SVM algorithm demonstrated high sensitivity and specificity in detecting and classifying post-surgical complications. It achieved a sensitivity of 92.5% in detecting CMO and 90.9% in identifying RD. The specificity of the algorithm was 90.9% and 96.2% for these complications, respectively. The overall accuracy of the ML tool in correctly identifying and classifying post-surgical complications was 92%.Conclusions: The integration of ML algorithms in OCT imaging shows promise in enhancing the accuracy and efficiency of assessing post-surgical complications in cataract patients. The ML tool developed in this study provides reliable and objective assessments, reducing the subjectivity and variability associated with the manual interpretation of OCT images

Category:Ankle Arthritis; AnkleIntroduction/Purpose:Ankle arthrodesis and total ankle replacement are the most commonly performed procedures for surgical management of ankle arthritis. Arthrodesis provides effective pain relief but the rate of complications after arthrodesis is higher as it is more commonly performed in patients with comorbidities that preclude ankle replacement. Accurately risk- stratifying patients who undergo ankle arthrodesis would be of great utility, given the significant cost and morbidity associated with developing major perioperative complications. There is a paucity of accurate prediction models that can be used to pre- operatively risk-stratify patients for ankle arthrodesis. We aim to develop a machine learning (ML) algorithm for prediction of major perioperative complication after ankle arthrodesis as well as compare its performance against traditional predictive models based on logistic regression.Methods:This is a retrospective cohort study of adult patients who underwent ankle arthrodesis at any non-federal California hospital between 2015 and 2017. The primary outcome was readmission within 30 days or major perioperative complication - venous thromboembolism within 30 days, myocardial infarction within 7 days, pneumonia within 7 days, systemic infection within 7 days, surgical site bleeding within 90 days, and wound complications within 90 days. We build ML and logistic regression models that span different classes of modeling approaches: XGBoost, AdaBoost, Gradient Boosting, and Random Forest. Discrimination and calibration were assessed using area under the receiver operating characteristic curve (AUROC) and Brier score, respectively. We utilize a partial dependence function to measure the importance of an individual feature by assessing the average effect in predicted risks when its value is altered. We rank the contribution of the included variables to the prediction of adverse outcomes.Results:A total of 1,084 patients met inclusion criteria for this study. There were 131 major complications or readmission (12.1%). The optimized XGBoost algorithm demonstrates higher discrimination (AUROC: 0.707 + 0.052) compared to LR (0.691 + 0.055). The receiver operating characteristic curves for the XGBoost and logistic regression models are visualized in Figure 1. XGBoost also outperforms the three other ML models. This model was well calibrated (Brier score: 0.103 + 0.001). The variables most important for the XGBoost model include diabetes, chronic kidney disease, implant complication, and major fracture. Five of the ten most important features for XGBoost were markedly less important for the traditional logistic regression model: male sex, prior hip fracture, cardiorespiratory failure, acute renal failure, and dialysis status.Conclusion:We report a ML algorithm for prediction of major perioperative complications after ankle arthrodesis. The optimized XGBoost model is well-calibrated and demonstrates superior risk prediction to logistic regression. This tool may identify and address potentially modifiable risk factors, helping to accurately risk-stratify patients and decrease likelihood of major complications. Notably, the predictors most important for XGBoost are different from those for logistic regression. This suggests that the superior discriminative capability of ML methods stems from their ability to capture complex non-linear relationships between variables that logistic regression is unable to detect.

Patients undergoing outpatient radiotherapy (RT) or chemoradiation (CRT) frequently require acute care (emergency department evaluation or hospitalization). Machine learning (ML) may guide interventions to reduce this risk. There are limited prospective studies investigating the clinical impact of ML in health care. The objective of this study was to determine whether ML can identify high-risk patients and direct mandatory twice-weekly clinical evaluation to reduce acute care visits during treatment. During this single-institution randomized quality improvement study (ClinicalTrials.gov identifier: NCT04277650), 963 outpatient adult courses of RT and CRT started from January 7 to June 30, 2019, were evaluated by an ML algorithm. Among these, 311 courses identified by ML as high risk (> 10% risk of acute care during treatment) were randomized to standard once-weekly clinical evaluation (n = 157) or mandatory twice-weekly evaluation (n = 154). Both arms allowed additional evaluations on the basis of clinician discretion. The primary end point was the rate of acute care visits during RT. Model performance was evaluated using receiver operating characteristic area under the curve (AUC) and decile calibration plots. Twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (difference, -10.0%; 95% CI, -18.3 to -1.6; relative risk, 0.556; 95% CI, 0.332 to 0.924; P = .02). Low-risk patients had a 2.7% acute care rate. Model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC, 0.851). In this prospective randomized study, ML accurately triaged patients undergoing RT and CRT, directing clinical management with reduced acute care rates versus standard of care. This prospective study demonstrates the potential benefit of ML in health care and offers opportunities to enhance care quality and reduce health care costs.

Predictive Modeling of Hematopoietic Stem Cell Transplant Outcomes: Discovering the Role of Immunogenetic Features through Machine Learning Models

P125. Development of a novel ensemble machine learning algorithm for prediction of complications and readmission after anterior cervical spinal fusion

Aim: Machine learning (ML) and deep learning (DL) predictive models have been employed widely in clinical settings. Their potential support and aid to the clinician of providing an objective measure that can be shared among different centers enables the possibility of building more robust multicentric studies. This study aimed to propose a user-friendly and low-cost tool for COVID-19 mortality prediction using both an ML and a DL approach. Method: We enrolled 2348 patients from several hospitals in the Province of Reggio Emilia. Overall, 19 clinical features were provided by the Radiology Units of Azienda USL-IRCCS of Reggio Emilia, and 5892 radiomic features were extracted from each COVID-19 patient’s high-resolution computed tomography. We built and trained two classifiers to predict COVID-19 mortality: a machine learning algorithm, or support vector machine (SVM), and a deep learning model, or feedforward neural network (FNN). In order to evaluate the impact of the different feature sets on the final performance of the classifiers, we repeated the training session three times, first using only clinical features, then employing only radiomic features, and finally combining both information. Results: We obtained similar performances for both the machine learning and deep learning algorithms, with the best area under the receiver operating characteristic (ROC) curve, or AUC, obtained exploiting both clinical and radiomic information: 0.803 for the machine learning model and 0.864 for the deep learning model. Conclusions: Our work, performed on large and heterogeneous datasets (i.e., data from different CT scanners), confirms the results obtained in the recent literature. Such algorithms have the potential to be included in a clinical practice framework since they can not only be applied to COVID-19 mortality prediction but also to other classification problems such as diabetic prediction, asthma prediction, and cancer metastases prediction. Our study proves that the lesion’s inhomogeneity depicted by radiomic features combined with clinical information is relevant for COVID-19 mortality prediction.

The aim of this research is maximizing differentiation quality of skin neoplasms by Raman Spectroscopy (RS) and Autofluorescence (AF) using conventional Machine Learning (ML) algorithms (which means excluding Neural Networks). Thus, a basic task of this research consists of making and testing ML algorithm ensembles on Raman spectra, that were obtained in vivo in Samara Oncology Clinical Center. The data (spectra) have been obtained in a form of text files containing the identifier of a patient, as well as a Raman spectrum in the form of pair values – a wavelength and an appropriate value. All data (964 spectra) has been divided for two classes: Tumor and Skin. Further preprocessing of the input data and the analysis of models of machine learning for a problem of classification have been carried out. We used the following ML tools, namely Python 3.7.3, an open source ML libraries Scikit-learn v0.21.2, NumPy 1.17.0, and Pandas 1.0.1, IDE Anaconda Enterprise 5.3 and cloud service Google Colaboratory with an interactive environment Jupyter Notebook. Machine learning models that show a high accuracy result include Classification and Regression Tree (CART), Support Vector Classification (SVC), Logistic Regression (LR), K-nearest neighbors algorithm (KNN). These classifiers show high quality classification on standard parameters already. The Soft Voting Classification module was selected as an ensemble, that allows us to use several models of classifiers that are not similar to each other at once, combine them into one classifier. Results of this ensemble testing from these ML algorithms showed that the classification accuracy, unlike the best qualifier, has not been increased. However, the metrics of classification quality show that the model has become stabler and steady. Results: specificity - 93%, sensitivity - 88, harmonic mean between precision and recall (F1 score) - 90%. An analysis of validation and training curves indicated a small size of training data and, for some cases, a high complexity of the model, which led to a decrease in the classification accuracy.

Pharmacometrics is a multidisciplinary field utilizing mathematical models of physiology, pharmacology, and disease to describe and quantify the interactions between medication and patient. As these models become more and more advanced, the need for advanced data analysis tools grows. Recently, there has been much interest in the adoption of machine learning (ML) algorithms. These algorithms offer strong function approximation capabilities and might reduce the time spent on model development. However, ML tools are not yet an integral part of the pharmacometrics workflow. The goal of this work is to discuss how ML algorithms have been applied in four stages of the pharmacometrics pipeline: data preparation, hypothesis generation, predictive modelling, and model validation. We will also discuss considerations before the use of ML algorithms with respect to each topic. We conclude by summarizing applications that hold potential for adoption by pharmacometricians.

ADRD caregivers increasingly use social media to meet their health information wants (HIW). Machine learning (ML) tools may help understand caregivers’ HIW as expressed via social media. This pilot study explored a collaborative, iterative process between domain experts and ML tools to identify ADRD caregivers’ HIW from social media data. The HIW-ADRD framework was adapted from an existing HIW framework. Through multiple rounds of iteration between the experts and the ML tools, the framework was expanded to include 11 types of health information. Each type included corresponding keywords developed through a hybrid approach that included keywords from both the theoretical constructs (top-down) and caregivers’ posts (bottom-up). These keywords were then used to enhance the ML tools’ ability to code 106 recent posts extracted from an ADRD social media group in March 2020. When compared with expert coding results, ML tools accurately predicted 56% of HIW. Further work is underway.

Machine learning (ML) constitutes a division of artificial intelligence (AI) that aims to train computers how to perform specific tasks without explicit programming. Traditional ML tools are widely used for classification and identification of animals. However, these methods have some drawbacks because of the extensive manual reliance and the delay in data interpretation. To overcome this, Applied Deep Learning algorithms are used with Artificial Neural Networks (ANN) and Convolution Neural Network (CNN) models introduced to address species classification, characteristics detection, and pattern recognition tasks helping in accurate identification and classification of animals. In this paper, we have tried to compile and deliver a recent comprehensive information on latest available investigations in the field of life sciences particularly used for animal identification. We have also accentuated the diverse applications of machine learning models including other parameters like, features, accuracy gained, database used and their limitations. The red flour beetle, Tribolium castaneum (Coleoptera; Tenebrionidae) is a prevailing and detrimental secondary insect pest of stored grains along with derived products causing 7% to 35% annual loss. Despite of that, nowdays it is also extensively considered as a model organism for genetic disease investigation. While using it in scientific research, exact sex identification of these insects becomes a crucial preliminary step. Generally, pupal stage is used to sort these insects according to their sex and needs expert humans. It is crucial to employ image processing and ML algorithms to quickly identify gender of this insect which is not done yet. We have used a CNN-based smart technique to recognize and categorize gender differences in T. castaneum using microscopic images in order to build an intelligent system for applied research. For this study, a dataset is created by taking 116 microscopic images of both the dorsal and ventral sides of pupae of two different sexes. In this algorithm, a 2D matrix of feature map is selected sequentially and the maximum value in the matrix is selected to generate a pooled feature map. The Rectified Linear Unit (ReLU) activation function is used for the CNN. The classification model has an accuracy between 97 and 98% with an F-score of 0.67. These results demonstrate the robustness of the classification model, which does not rely heavily on manual intervention compared to traditional machine learning (ML) tools and automates the processes of feature extraction and gender classification regardless of the position of the pupae in the images.

BackgroundCervical cancer is a common malignant tumor of the female reproductive system and is considered a leading cause of mortality in women worldwide. The analysis of time to event, which is crucial for any clinical research, can be well done with the method of survival prediction. This study aims to systematically investigate the use of machine learning to predict survival in patients with cervical cancer.MethodAn electronic search of the PubMed, Scopus, and Web of Science databases was performed on October 1, 2022. All articles extracted from the databases were collected in an Excel file and duplicate articles were removed. The articles were screened twice based on the title and the abstract and checked again with the inclusion and exclusion criteria. The main inclusion criterion was machine learning algorithms for predicting cervical cancer survival. The information extracted from the articles included authors, publication year, dataset details, survival type, evaluation criteria, machine learning models, and the algorithm execution method.ResultsA total of 13 articles were included in this study, most of which were published from 2018 onwards. The most common machine learning models were random forest (6 articles, 46%), logistic regression (4 articles, 30%), support vector machines (3 articles, 23%), ensemble and hybrid learning (3 articles, 23%), and Deep Learning (3 articles, 23%). The number of sample datasets in the study varied between 85 and 14946 patients, and the models were internally validated except for two articles. The area under the curve (AUC) range for overall survival (0.40 to 0.99), disease-free survival (0.56 to 0.88), and progression-free survival (0.67 to 0.81), respectively from (lowest to highest) received. Finally, 15 variables with an effective role in predicting cervical cancer survival were identified.ConclusionCombining heterogeneous multidimensional data with machine learning techniques can play a very influential role in predicting cervical cancer survival. Despite the benefits of machine learning, the problem of interpretability, explainability, and imbalanced datasets is still one of the biggest challenges. Providing machine learning algorithms for survival prediction as a standard requires further studies.

12074 Background: Skeletal-related events (SREs) are detrimental clinical events in bone metastasis from advanced cancer, defined by pathologic fracture, spinal cord compression, and inevitable surgical or radiational intervention to the bone. Given their negative impact on quality of life and prognosis and interpatient heterogeneity in the SRE risk, accurate identification of patients with high SRE risk is critical. Methods: The patient-level data from three randomized clinical trials that administered zoledronic acid (ZA) to patients with bone-metastatic breast cancer, castration-resistant prostate cancer (CRPC), and other types of cancer were analyzed. (N = 460, 452, and 315, respectively) Machine learning (ML) models to predict SREs within 18 months (breast cancer), 12 months (CRPC), and 9 months (other cancers) were developed based on more than 40 baseline clinical and laboratory data. Seven ML algorithms and five feature selection methods were utilized to develop multiple models. The ML model with the best performance was identified based on the F1 score and the area under the receiver operating characteristic curve (AUC-ROC) for each cancer type and interrogated for important features with Shapley additive explanations. Lastly, the ML models’ ability to stratify patients by the cumulative SRE risk was evaluated by calculating hazard ratio (HR) with Cox-proportional hazards models. Results: Among the multiple ML models developed with different algorithms and feature selection methods, the model developed utilizing the random forests algorithm and the Boruta method for selecting features demonstrated the best performance in all types of cancer (F1 0.70, 0.67, and 0.67, and AUC-ROC 0.72, 0.68, and 0.73 for breast cancer, CRPC, and other cancers, respectively). In the ML model for breast cancer, performance status (PS), history of SRE, serum alkaline phosphatase (ALP), history of anti-neoplastic surgery, radiation therapy, and pathologic fracture were included as important features. Serum ALP, albumin, sodium, Gleason scores, and geographic regions were shown to be relevant in the CRPC model. For other cancers, serum ALP, albumin, total protein, phosphorus, red blood cell count, white blood cell count, visceral metastases, and history of arthritis were incorporated in the ML model. The ML model prediction successfully stratified the patients for cumulative SRE risk in all three cohorts. (HR with 95% confidence interval: 2.43 [1.86 – 3.18], 1.92 [1.51 – 2.45], and 3.06 [2.29 – 4.09] for breast cancer, CRPC, and other types of cancer, respectively). Conclusions: ML models incorporating baseline clinical and laboratory data can identify patients with bone metastasis on ZA harboring a high SRE risk.

Performances of machine learning algorithms for individual thermal comfort prediction based on data from professional and practical settings

The important roles of machine learning and ferroptosis in bladder cancer (BCa) are still poorly understood. In this study, a comprehensive analysis of 19 ferroptosis-related genes (FRGs) was performed in 1322 patients with BCa from four independent patient cohorts and a pan-cancer cohort of 9824 patients. Twelve FRGs were selected through machine learning algorithm to construct the prognosis model. Significantly differential survival outcomes (hazard ratio (HR) = 2.09, 95% confidence interval (CI): 1.55–2.82, p < 0.0001) were observed between patients with high and low ferroptosis scores in the TCGA cohort, which was also verified in the E-MTAB-4321 cohort (HR = 4.71, 95% CI: 1.58–14.03, p < 0.0001), the GSE31684 cohort (HR = 1.76, 95% CI: 1.08–2.87, p = 0.02), and the pan-cancer cohort (HR = 1.15, 95% CI: 1.07–1.24, p < 0.0001). Tumor immunity-related pathways, including the IL-17 signaling pathway and JAK-STAT signaling pathway, were found to be associated with the ferroptosis score in BCa through a functional enrichment analysis. Further verification in the IMvigor210 cohort revealed the BCa patients with high ferroptosis scores tended to have worse survival outcome after receiving tumor immunotherapy. Significantly different ferroptosis scores could also be found between BCa patients with different reactions to treatment with immune checkpoint inhibitors.