Natural Language Processing Application on Commit Messages: A Case Study on HEP Software

Background and purpose — External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Material and methods — We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting. Results — We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43–89), with 6 items being reported in less than 4/18 of the studies. Interpretation — Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.

- Version Control is a system that keeps track of the historical changes and usage of a resource within an integrated storage media. Version Control (also widely known as a Source Code Management System) is a crucial component for any team working on software development. Version control will keep a record of every single source code change to a file or document, such as historical data, comparisons between different versions, and even access rights to that file or document. A wise choice on which Version Control system to use will have a massive effect on the performance of the Software Developers. Hence many factors must be acknowledged in the process of making the decision of which product to use. The objective of this research is to find which features/attributes of a Version Control System are most influential in the aforementioned decision making process. The research explores the features/attributes inherent in three well known Version Control systems – Subversion, Mercurial and Git. This is done using ISO 9126 as a criteria. Influential factors in the decision making process are gauged using a descriptive analysis technique, and the instrument used in this case is the Analytical Hierarchy Process (AHP). Extensive questionnaire data was 'fed' into the AHP technique, and based on the results obtained, it was calculated that Subversion is the more favorable Version Control system compared to Mercurial and Git. It can thus be concluded that Subversion is the best Version Control system to support the productivity of software developers. Keywords: Version Control, Subversion, Mercurial, Git, Analytical Hierarchy Process.

PM2.5 pollution is increasing, causing health problems. The objective of this study was to model the behavior of PM2.5AQI (air quality index) using machine learning (ML) predictive models of linear regression, lasso, ridge, and elastic net. A total of 16,543 records from the Huachac, Junin area in Peru were used with regressors of humidity in % and temperature in °C. The focus of this study is PM2.5AQI and environmental variables. Methods: Exploratory data analysis (EDA) and machine learning predictive models were applied. Results: PM2.5AQI has high values in winter and spring, with averages of 52.6 and 36.9, respectively, and low values in summer, with a maximum value in September (spring) and a minimum in February (summer). The use of regression models produced precise metrics to choose the best model for the prediction of PM2.5AQI. Comparison with other research highlights the robustness of the chosen ML models, underlining the potential of ML in PM2.5AQI. Conclusions: The predictive model found was α = 0.1111111 and a Lambda value λ = 0.150025, represented by PM2.5AQI = 83.0846522 − 10.302222000 (Humidity) − 0.1268124 (Temperature). The model has an adjusted R2 of 0.1483206 and an RMSE of 25.36203, and it allows decision making in the care of the environment.

BackgroundIdentifying patients at high risk of stroke-associated pneumonia (SAP) may permit targeting potential interventions to reduce its incidence. We aimed to explore the functionality of machine learning (ML) and natural language processing techniques on structured data and unstructured clinical text to predict SAP by comparing it to conventional risk scores.MethodsLinked data between a hospital stroke registry and a deidentified research-based database including electronic health records and administrative claims data was used. Natural language processing was applied to extract textual features from clinical notes. The random forest algorithm was used to build ML models. The predictive performance of ML models was compared with the A2DS2, ISAN, PNA, and ACDD4 scores using the area under the receiver operating characteristic curve (AUC).ResultsAmong 5,913 acute stroke patients hospitalized between Oct 2010 and Sep 2021, 450 (7.6%) developed SAP within the first 7 days after stroke onset. The ML model based on both textual features and structured variables had the highest AUC [0.840, 95% confidence interval (CI) 0.806–0.875], significantly higher than those of the ML model based on structured variables alone (0.828, 95% CI 0.793–0.863, P = 0.040), ACDD4 (0.807, 95% CI 0.766–0.849, P = 0.041), A2DS2 (0.803, 95% CI 0.762–0.845, P = 0.013), ISAN (0.795, 95% CI 0.752–0.837, P = 0.009), and PNA (0.778, 95% CI 0.735–0.822, P < 0.001). All models demonstrated adequate calibration except for the A2DS2 score.ConclusionsThe ML model based on both textural features and structured variables performed better than conventional risk scores in predicting SAP. The workflow used to generate ML prediction models can be disseminated for local adaptation by individual healthcare organizations.

Machine learning (ML) algorithms are increasingly used to predict outcomes in orthopaedic surgery, but their utility in hip arthroscopy remains unclear. This study aimed to (1) evaluate ML models for predicting outcomes, complications, and resource use following hip arthroscopy, (2) compare ML performance to traditional statistical methods, and (3) critically appraise the quality of existing literature. A systematic search of PUBMED, MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials was conducted on 4 April 2023. Included studies used ML models for clinical prediction in hip arthroscopy. Data on study design, outcomes, model development, validation and adherence to TRIPOD guidelines were extracted. Seventeen studies involving 37,549 patients (52.8% female; mean age 34.4 ± 6.1 years) were included. ML was used to predict clinical outcomes (47.1%), adverse events (47.1%) and resource utilisation (5.9%). The median area under the curve was 0.66 (range 0.5-0.94) for clinical outcomes and 0.7 (range 0.6-0.9) for adverse events. Resource utilisation was reported with a root-mean-square error of $3800, a logarithmic error of 0.2, and R² of 0.8. Only four studies compared ML to traditional regression; two found no difference, while two favoured ML. On average, studies met 12.3 of the 22 TRIPOD criteria, with only 29.4% meeting more than two-thirds. ML models in hip arthroscopy show variable performance in predicting outcomes and complications. Furthermore, most studies comparing ML models to traditional regression did not report any significant difference. Lastly, variability in adherence to TRIPOD guidelines highlights the need for future studies to improve transparency and reporting in the development of ML algorithms for hip arthroscopy. Given the potential of ML models in enhancing clinical decision-making for hip arthroscopy, along with the findings of the current study, their application should be used with caution. Level IV, systematic review of level II-IV studies.

Background/AimsPrediction of short-term mortality in patients with acute-on-chronic liver failure (ACLF) admitted to the intensive care unit (ICU) may enhance effective management.MethodsTo develop, explain, and validate a predictive machine learning (ML) model for short-term mortality in patients with ACLF with two or more organ failures (OFs). Utilizing a large ICU cohort with detailed clinical information, we identified ACLF patients with two or more OFs according to the EASL-CLIF and NACSELD definitions. ML model was developed for each definition to predict 30-day mortality. The Shapley value was estimated to explain the models. Validation and calibration of these models were performed.ResultsOf 5,994 patients with cirrhosis admitted to ICU, 1,511 met NACSELD criteria, and 1,692 met EASL-CLIF grade II or higher criteria. The CatBoost ACLF (CBA) model had the greatest accuracy in the NACSELD cohort (area under curve [AUC] of 0.87), while the Random Forest ACLF (RFA) model performed best in the EASL-CLIF cohort (AUC of 0.83). Both models showed robust calibration. The models were explained by SHAP score analysis, yielding a rank list, and the top twelve predictors were selected. Both simplified models demonstrated similar performance (CBA model: AUC 0.89, RFA model: AUC 0.81) and significantly outperformed contemporary scoring systems, including CLIF-C ACLF and MELD 3.0. The models were validated in both internal and external cohorts. A simple-to-use online tool was created to predict mortality rates.ConclusionsWe presented explainable, well-validated, and calibrated predictive models for ACLF patients with two or more OFs, which outperformed existing predictive scores.

In recent years, machine learning models for clinical prediction have become increasingly prevalent in the neurosurgical literature. However, little is known about the quality of these models, and their translation to clinical care has been limited. The aim of this systematic review was to empirically determine the adherence of machine learning models in neurosurgery with standard reporting guidelines specific to clinical prediction models. Studies describing the development or validation of machine learning predictive models published between January 1, 2020, and January 10, 2023, across five neurosurgery journals (Journal of Neurosurgery, Journal of Neurosurgery: Spine, Journal of Neurosurgery: Pediatrics, Neurosurgery, and World Neurosurgery) were included. Studies where the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were not applicable, radiomic studies, and natural language processing studies were excluded. Forty-seven studies featuring a machine learning-based predictive model in neurosurgery were included. The majority (53%) of studies were single-center studies, and only 15% of studies externally validated the model in an independent cohort of patients. The median compliance across all 47 studies was 82.1% (IQR 75.9%-85.7%). Giving details of treatment (n = 17 [36%]), including the number of patients with missing data (n = 11 [23%]), and explaining the use of the prediction model (n = 23 [49%]) were identified as the TRIPOD criteria with the lowest rates of compliance. Improved adherence to TRIPOD guidelines will increase transparency in neurosurgical machine learning predictive models and streamline their translation into clinical care.

MACHINE LEARNING PREDICTION OF IN-HOSPITAL DISEASE PROGRESSION IN COMMUNITY-ACQUIRED PNEUMONIA: DERIVATION AND VALIDATION OF CLINICAL DETERIORATION IN PNEUMONIA (CDIP)

BackgroundAcute coronary syndromes (ACS) are the leading cause of global death. Optimizing mortality risk prediction and early identification of high-risk patients is essential for developing targeted prevention strategies. Many researchers have built machine learning (ML) models to predict the mortality risk in ACS patients. Our meta-analysis aimed to evaluate the predictive value of various ML models in predicting death in ACS patients at different times.MethodsPubMed, Embase, Web of Science, and Cochrane Library were searched systematically from database establishment to March 12, 2022 for studies developing or validating at least one ML predictive model for death in ACS patients. We used PROBAST to assess the risk of bias in the reported predictive models and a random-effects model to assess the pooled C-index and accuracy of these models.ResultsFifty papers were included, involving 216 ML prediction models, 119 of which were externally validated. The combined C-index of the ML models in the validation cohort predicting the in-hospital mortality, 30-day mortality, 3- or 6-month mortality, and 1 year or above mortality in ACS patients were 0.8633 (95% CI 0.8467–0.8802), 0.8296 (95% CI 0.8134–0.8462), 0.8205 (95% CI 0.7881–0.8541), and 0.8197 (95% CI 0.8042–0.8354), respectively, with the corresponding combined accuracy of 0.8569 (95% CI 0.8411–0.8715), 0.8282 (95% CI 0.7922–0.8591), 0.7303 (95% CI 0.7184–0.7418), and 0.7837 (95% CI 0.7455–0.8175), indicating that the ML models were relatively excellent in predicting ACS mortality at different times. Furthermore, common predictors of death in ML models included age, sex, systolic blood pressure, serum creatinine, Killip class, heart rate, diastolic blood pressure, blood glucose, and hemoglobin.ConclusionsThe ML models had excellent predictive power for mortality in ACS, and the methodologies may need to be addressed before they can be used in clinical practice.

In recent years, cardiovascular disease (CVD) has emerged as a leading cause of mortality worldwide, highlighting the urgent need for reliable predictive models to support early diagnosis and preventive care. This study conducts a comparative analysis of various machine learning models to assess CVD risk, focusing on their ability to accurately identify individuals at high risk based on clinical and demographic data. We evaluate multiple supervised learning algorithms, including logistic regression, random forests, support vector classifiers, K-nearest neighbors, gradient boosting, and AdaBoost, comparing their predictive performance in terms of accuracy, precision, recall, and F1-score. Additionally, we propose a novel hybrid model that combines Random Forest (RF) for feature selection and Deep Neural Networks (DNN) for classification, aiming to leverage the strengths of both approaches for more accurate CVD prediction. Our findings demonstrate that ensemble models such as random forests and gradient boosting achieve superior performance, with high accuracy (0.99) and balanced precision and recall values, outperforming simpler models like logistic regression and support vector classifiers. The hybrid model further enhances prediction accuracy, achieving 92.4% accuracy, 91.7% precision, 93.0% recall, and an AUC-ROC score of 96.0%. The analysis also underscores the importance of data preprocessing techniques, including normalization and handling of missing values, in optimizing model accuracy and stability. Notably, K-nearest neighbors also performed exceptionally well with a high F1-score across classes, highlighting its robustness for this task. This study provides a detailed examination of each model's strengths and limitations, including the proposed hybrid model, offering valuable insights for healthcare practitioners and data scientists in selecting effective machine learning models for CVD risk prediction. By integrating these models into healthcare systems, real-time risk prediction can be enhanced, ultimately supporting clinical decision-making and advancing personalized care in cardiovascular health. Keywords— cardiovascular disease, machine learning, risk assessment, predictive models, supervised learning, healthcare

e13596 Background: Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer (BC) and neoadjuvant therapy (NAT) has become an essential treatment strategy. Recent studies have showed tertiary lymphoid structures (TLSs) is associated with better treatment response to NAT in BC. However, there is still a lack of non-invasive biomarkers to predict the presence of TLSs in TNBC. This study aimed to develop a machine learning model for early identification and prediction of the presence of TLSs and treatment response to NAT in TNBC, which were essential for timely adjustments in treatment strategies for TNBC. Methods: In this study, 698 patients from multicenter were divided into a training cohort (n = 137), the TLSs validation cohort (n = 63) and the NAT response validation cohorts (n = 561). A total of 4788 radiomic features per patient were extracted from the intratumoral and peritumoral regions of DCE-MR. After extracting the optimal features, multiple machine learning models were developed to predict the presence of TLSs and were subsequently applied to predict the treatment response to NAT in NAT response validation cohorts. The performance of the models was assessed by area under curve values (AUC), and prognostic analysis were performed to evaluate its predictive value. Finally, the correlation between key radiomic features and key pathomic features was further analyzed to reveal the tumor heterogeneity of TNBC from a pathological perspective. Results: The XGBoost model, which outperformed all other predictive models, was selected as the radiomics-based TLS (rTLS) predictive model. It achieved AUCs of 0.922 in the training cohort, 0.852 in the TLS validation cohort, 0.724 in the Chinese-TNBC cohort, 0.919 in the DUKE cohort, and 0.883 in the I-SPY2 cohort. The rTLS predictive model demonstrated robust predictive performance, including across various patient subgroups defined by age, menopausal status, Ki67 level, cT stage, cN stage, and MammaPrint gene status. In YNCH cohort and DUKE cohort, prognostic analysis showed that low rTLS predictive score was significantly correlated with better disease-free survival in TNBC receiving NAT, and Cox regression analysis also confirmed the rTLS predictive score was a strong independent prognostic factor. Pathomic features further explained the pathological heterogeneity of TNBC with different responses to NAT. Conclusions: The rTLS predictive model, which accurately predicted the presence of TLSs and treatment response to NAT in TNBC, held promise for future clinical application in formulating personalized and effective treatment strategies for TNBC, ultimately improving prognosis.

This paper proposes a predictive techno-economic analysis in terms of voltage stability and cost using regression-based machine learning (ML) models and effectiveness of the analysis is validated. Predictive analysis of a power system is proposed to address the need for faster and accurate analyses that would aid in the operation and control of modern power system. Several methods of analyses including metaheuristic optimization algorithms, artificial intelligence techniques and machine learning algorithms are being developed and used. Predictive ML models for two modified IEEE 14-bus and IEEE-30 bus systems, integrated with renewable energy sources (solar and wind) and reactive power compensative device (STATCOM) are proposed and developed with features that include hour of the day, solar irradiation, wind velocity, dynamic grid price and system load. An hour-wise input database for the model development is generated from monthly average data and hour-wise daily curves with normally distributed standard deviations. The data feasibility tests and output database generation is performed using MATLAB. Linear and higher order polynomial regression models are developed for the 8760hr database using Python 3.0 in JupyterLab and a best-fit predictive ML model is identified by analysing the coefficients of determination. The voltage stability and cost predictive ML models were tested for a 24hr input profile. The results obtained and the comparison with the expected values are furnished. Prediction of the outputs for the test data validate the accuracy of the developed model.

Prediction of seismic acceleration response of precast segmental self-centering concrete filled steel tube single-span bridges based on machine learning method

Recently, great progress has been achieved in diabetes management through minimally invasive continuous glucose monitoring, which involves piercing the skin to identify changes in glucose levels in the interstitial fluid. Nevertheless, the development of accurate fully noninvasive (NI) glucose monitors remains a topic of great interest. This work examines the processing of data collected from a multisensor system, which combines millimeter-wave (mm-wave), near-infrared (NIR), and other auxiliary sensors’ technology, coupled with a random forest machine learning (ML) model for prediction. We investigated possible correlations between mm-wave, NIR, skin temperature, pressure data, and blood glucose values and deployed an ML model for prediction. The model is based on the random forest algorithm that was trained and tested on data collected from a study involving five healthy human subjects undergoing an intravenous glucose tolerance test. Our model achieves a root mean square error of 21.06 mg/dl and a mean absolute relative difference of 7.31% for glucose prediction in the collected clinical subjects dataset. In addition, 96.1% of the model’s predictions fall within the clinically acceptable zones according to the Clarke error grid analysis. The preliminary results suggest that combining multiple sensor data with ML techniques is promising for the development of NI glucose monitoring.

Accurate risk assessment of high-risk patients is essential in clinical practice. However, there is no practical method to predict or monitor the prognosis of patients with ST-segment elevation myocardial infarction (STEMI) complicated by hyperuricemia. We aimed to evaluate the performance of different machine learning models for the prediction of 1-year mortality in STEMI patients with hyperuricemia. We compared five machine learning models (logistic regression, k-nearest neighbor, CatBoost, random forest, and XGBoost) with the traditional global (GRACE) risk score for acute coronary event registrations. We registered patients aged >18 years diagnosed with STEMI and hyperuricemia at the Affiliated Hospital of Zunyi Medical University between January 2016 and January 2020. Overall, 656 patients were enrolled (average age, 62.5 ± 13.6 years; 83.6%, male). All patients underwent emergency percutaneous coronary intervention. We evaluated the performance of five machine learning classifiers and the GRACE risk model in predicting 1-year mortality. The area under the curve (AUC) of the six models, including the GRACE risk model, ranged from 0.75 to 0.88. Among all the models, CatBoost had the highest predictive accuracy (0.89), AUC (0.87), precision (0.84), and F1 value (0.44). After hybrid sampling technique optimization, CatBoost had the highest accuracy (0.96), AUC (0.99), precision (0.95), and F1 value (0.97). Machine learning algorithms, especially the CatBoost model, can accurately predict the mortality associated with STEMI complicated by hyperuricemia after a 1-year follow-up.