Performance Evaluation of Machine Learning Models for HIV/AIDS Classification

In last two decades the growing life expectancy has spiralled the prevalence of neurogenerative disorder such as Dementia among elderly population posing a significant challenge to mental health globally. The dementia is generally characterized by progressive cognitive decline which it affects memory, thinking, behaviour, and the ability to perform everyday activities. Early diagnosis and intervention are crucial for improving patient outcomes and managing the societal burden of dementia. The primary aim of this study is to deploy various machine learning models such as Logistic Regression, K-Nearest Neighbour (KNN), Support Vector Machine (SVM), Decision Tree, Random Forest (RF). Extreme Gradient Boosting (XGBoost) for the classification of Dementia. The suitable features required for the classification is determined through efficient filter-based and wrapper-based feature selection techniques. The experimental evaluation of machine learning models is performed using standard metrics such as accuracy, precision, recall and F1 score. Furthermore, Explainable AI (XAI) techniques such as SHAP and LIME are employed to interpret the black-box nature of these models, offering transparency and insights into the contribution of individual features to the model predictions. The thorough evaluation of machine learning models exhibited that Random Forest outperforms other models with an accuracy of 96%, with CDR identified as a key predictor through XAI analysis.

Student’s mental health problems have been explored previously in higher education literature in various contexts including empirical work involving quantitative and qualitative methods. Nevertheless, comparatively few research could be found, aiming for computational methods that learn information directly from data without relying on set parameters for a predetermined equation as an analytical method. This study aims to investigate the performance of Machine learning (ML) models used in higher education. ML models considered are Naïve Bayes, Support Vector Machine, K-Nearest Neighbor, Logistic Regression, Stochastic Gradient Descent, Decision Tree, Random Forest, XGBoost (Extreme Gradient Boosting Decision Tree), and NGBoost (Natural) algorithm. Considering the factors of mental health illness among students, we follow three phases of data processing: segmentation, feature extraction, and classification. We evaluate these ML models against classification performance metrics such as accuracy, precision, recall, F1 score, and predicted run time. The empirical analysis includes two contributions: 1. It examines the performance of various ML models on a survey-based educational dataset, inferring a significant classification performance by a tree-based XGBoost algorithm; 2. It explores the feature importance [variables] from the datasets to infer the significant importance of social support, learning environment, and childhood adversities on a student’s mental health illness.

Gender classification is one of the major challenges in the domain of speech analysis now-a-days. Identification of gender from acoustic properties of voice such as mean, median, and frequency, is of high importance. Machine learning is used to solve this problem because it gives promising results for classification techniques. There are several algorithms that can be used to predict the gender using acoustic properties. In this work, classifiers are evaluated using 6 different machine learning algorithms. These algorithms include K-Nearest Neighbor (KNN), Decision Tree (DT), Random Forest (RF), and Support Vector Machine (SVM), Support Vector Machine Using Poly kernel and Gradient Boosting (GB). For the same voice sample dataset, the accuracy, recall, precision and F1 Score are computed for all the above-mentioned classifiers. The accuracy obtained from KNN is 96.71, DT is 93.33%, Random Forest is 97.72%, SVM is 98.48%, SVM using Poly Kernel is 97.72%, Gradient Boosting is 96.96%. The predicted values of the considered evaluation metrices on test data for all above classifiers evidenced the supremacy of SVM classifier as it is best fitted model for gender identification of acoustic. SVM attained the accuracy of 98.48% which is the highest among all.

The prediction of wind energy generation is important to enhance the performance and dependability of renewable energy systems due to the rising demand for wind-generated electricity and advancements in wind energy technology competitiveness. This study leverages advanced machine learning (ML) and some other statistical and deep learning based time series forecasting models to enhance the accuracy of wind energy predictions. This comprehensive analysis includes nine ML models—Linear Regression, Random Forests (RF), Gradient Boosting Machines (GBM), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), AdaBoost, XGBoost, Support Vector Regression (SVR), and Neural Networks—as well as Four time-series forecasting models—ARIMA, Temporal Convolutional Networks (TCNs), Long Short-Term Memory (LSTM) networks and GRU. Each ML model underwent rigorous cross-validation to ensure optimal performance. The assessment criteria utilized here comprised the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the R² Score. It was found that among the nine ML models, Random Forests, GBM and KNN consistently provided superior accuracy and robustness, making them the top choices for wind energy prediction whereas the performance of linear regression, SVM and SVR were very poor for the considered dataset. From the experiment, Random Forest, GBM, and KNN showed the best performance with low MSE values of 0.77, 1.95, and 1.51 respectively, while other models had MSEs above 7.5, with AdaBoost reaching 30. Their RMSEs (0.88, 1.40, 1.23) and MAEs (0.093, 0.73, 0.10) also indicate strong predictive accuracy compared to the rest.In this paper, time series forecasting, TCNs, LSTM and GRU networks showed strong capabilities in capturing temporal dependencies and trends within the wind energy data. Visualization techniques were employed to compare model performances comprehensively, providing clear insights into their predictive power. Therefore, this present study offers a robust framework for researchers and practitioners aiming to leverage machine learning and time series forecasting in the realm of renewable energy prediction.

Forests hold vital ecological significance, and the ability to accurately classify tree species is integral to conservation and management practices. This research investigates the application of machine learning techniques to airborne Light Detection and Ranging (LiDAR) data for the multi-class classification of tree species, specifically Alder, Aspen, Birch, Fir, Pine, Spruce, and Tilia. High-density LiDAR data from varied forest landscapes were subjected to a rigorous preprocessing and noise reduction protocol, followed by feature extraction to discern structural characteristics indicative of species identity. We assessed the performance of six machine learning models: Logistic Regression, Decision Tree, Random Forest, Support Vector Classifier (SVC), k-Nearest Neighbors (KNN), and Gradient Boosting. The analysis was based on metrics of accuracy, precision, recall, and F1 score. Logistic Regression and Random Forest models outperformed others, achieving accuracies of 0.81, precision of 0.80, recall of 0.81, and an F1 score of 0.80. In contrast, the KNN algorithm had the lowest accuracy of 0.60, precision and recall of 0.60, and an F1 score of 0.59. These results demonstrate the robustness of Logistic Regression and Random Forest for classifying complex LiDAR datasets. The study underscores the potential of these models to support ecological monitoring, enhance forest management, and aid in biodiversity conservation. Future research directions include the fusion of LiDAR data with other environmental variables, application of deep learning for improved feature extraction, and validation of the models across broader species and geographical ranges. This research marks a significant step towards leveraging advanced machine learning to interpret and utilize LiDAR data for environmental and ecological applications.

Forecasting Hematological Activity in Antiphospholipid Syndrome Using Predictive Models

Sepsis is a leading cause of mortality in the intensive care unit. Early prediction of sepsis can reduce the overall mortality rate and cost of sepsis treatment. Some studies have predicted mortality and development of sepsis using machine learning models. However, there is a gap between the creation of different machine learning algorithms and their implementation in clinical practice.This study utilized data from the Medical Information Mart for Intensive Care III. We established and compared the gradient boosting decision tree (GBDT), logistic regression (LR), k-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM).A total of 3937 sepsis patients were included, with 34.3% mortality in the Medical Information Mart for Intensive Care III group. In our comparison of 5 machine learning models (GBDT, LR, KNN, RF, and SVM), the GBDT model showed the best performance with the highest area under the receiver operating characteristic curve (0.992), recall (94.8%), accuracy (95.4%), and F1 score (0.933). The RF, SVM, and KNN models showed better performance (area under the receiver operating characteristic curve: 0.980, 0.898, and 0.877, respectively) than the LR (0.876).The GBDT model showed better performance than other machine learning models (LR, KNN, RF, and SVM) in predicting the mortality of patients with sepsis in the intensive care unit. This could be used to develop a clinical decision support system in the future.

Background: Heart diseases continue to be among the primary causes of morbidity and mortality worldwide, highlighting the need for novel diagnostic methods to accurately, early, and economically predict and diagnose heart diseases. Current physical explores upon expert knowledge, possibly missing intricate and non-linear associations between heart disease risk factors. This is where machine learning (ML)methods emerge as promising alternatives for prediction and diagnosis of heart diseases by emphasizing data availability. Methods: This paper attempts to find out how six popular machine learning algorithms namely, Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting (GB), K-Nearest Neighbors (KNN), and eXtreme Gradient Boosting (XGB) perform on a publicly available heart disease dataset collected from Kaggle. For better performance and to make them comparable to each other, basic preprocessing techniques have also been implemented. The performance of all these algorithms will be measured by their respective definitions of accuracy, precision, recall, and F1 score. Result: With an F1-score of 98.52%, Random Forest and XGBoost surpassed other models, yielding a superior accuracy in predictions, measuring 98.53%. The models recorded a perfect level of precision, 100%, coupled with high values of recall, exceeding 97%. Gradient Boosting recorded impressive results, receiving an accuracy score of 93.17%. Conclusion: The research findings revealed that the Random Forest and XGBoost models are very effective in the prediction and diagnosis of heart disease. These two models have the capacity to support a clinical decision. Novelty: That is to say, the novelty of this research is that it evaluates multiple machine learning models with comparison to each other. The role of machine learning in enhancing cardiovascular disease diagnosis as well as decision support systems in healthcare and educating and aware the people about heart care and health.

Machine Learning (ML) models have been extensively applied in various fields to enhance prediction. For instance, in cybersecurity, they examine large amounts of data, establish trends in the data, and draw insights from previous events, to enhance detection and respond to cyber threats. Random Forest, Logistic Regression, K-Nearest Neighbor and LSTM are some of the popular ML models vastly used for anomaly detection. The accuracy of these models is therefore the cornerstone of the organization’s information systems security since wrong predictions result to false positives and negatives which significantly reduce employees’ output and may result into workers’ frustrations when interacting with the information systems. Among the many factors that affect ML model performance, data pre-processing has been underscored. Using the various publicly available datasets, this paper examines the impact of data preprocessing techniques on selected ML model architectures’ performance. Training time, Accuracy, Precision, Recall and F1 scores are used for evaluating the ML models’ performance.

Multiple sclerosis (MS) is a heterogeneous immune-mediated disorder of the central nervous system, and primary progressive MS (PPMS) is characterized by insidious injury independent of clinical relapses from disease onset. Predicting disease progression remains an unmet need across the MS disease spectrum. We evaluated statistical and machine learning (ML) models for predicting progression in 596 patients with PPMS from ORATORIO, a randomized, placebo (PBO)-controlled, phase 3 clinical study of ocrelizumab (OCR). Using baseline clinical, magnetic resonance imaging, and biomarker (neurofilament light chain [NfL]) data, we trained 6 ML models to predict a progression event (logistic regression, support vector machines, random forest, neural network, gradient boosting, and extreme gradient boosting) and the time to progression (Cox proportional hazards [PH]) on upper limb function, as measured by 20% confirmed worsening on the 9-hole peg test (9HPT). Predictive performance was similar across the six ML models (area under the curve = 0.63 to 0.67) and was modest using the Cox PH model (C-index = 0.76, an improvement over previous studies). In the Cox PH model, treatment arm, baseline 9HPT, neurologic functional system scores, and serum NfL levels were identified as factors associated with progression; model performance was similar across the OCR, PBO, and combined OCR + PBO treatment arms. Accounting for potential interactions with treatment arm resulted in similar predictive performance. To our knowledge, this is the first study to systematically evaluate ML models and statistical methods for predicting progression in PPMS using a well-characterized clinical trial cohort.

BackgroundAcute upper gastrointestinal bleeding (AUGIB) is one of the most common critical diseases encountered in the intensive care unit (ICU), with a mortality rate ranging from 15 to 20%. Accurate stratification of acute gastrointestinal bleeding into acute variceal gastrointestinal bleeding (AVGIB) and acute non-variceal gastrointestinal bleeding (ANGIB) subtypes is clinically essential as distinct entities require markedly different therapeutic approaches and even divergent prognostic implications. AUGIB characterized by hemorrhagic shock, hypotension, multiple organ dysfunction (MODS), and even circulatory failure is life-threatening. Machine learning (ML) prediction model can be an effective tool for mortality prediction, enabling the timely identification of high-risk patients and improving outcomes.MethodsA total of 3,050 acute upper gastrointestinal bleeding (AUGIB) patients were included in our research from the MIMIC-IV database, among which 625 patients were classified as AVGIB and 2,425 patients were categorized as ANGIB. Patients’ clinical features, intervention methods, vital signs, scores, and important laboratory results were collected. The Synthetic Minority Over-sampling Technique-Edited Nearest Neighbors (SMOTE-ENN) and Adaptive Synthetic Sampling (ADASYN) were adopted to address the imbalance of the dataset. As many as 12 machine learning (ML) algorithms, namely, logistic regression (LR), decision tree (DT), random forest (RF), gradient boosting (GB), AdaBoost, XGBoost, Naive Bayes (NB), support vector machine (SVM), light gradient-boosting machine (LightGBM), K-nearest neighbors (KNN), extremely randomized trees (ET), and voting classifier (VC), were performed. The model performance was evaluated using accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC). Shapley Additive exPlanations (SHAP) analysis was conducted to identify the most influential features contributing to mortality prediction.ResultsIn terms of AVGIB patients, extremely randomized trees model demonstrated excellent predictive value among other ML models, with the AUC of 0.996 ± 0.007, accuracy of 0.996 ± 0.009, precision of 0.957 ± 0.024, recall of 0.988 ± 0.012, and F1 score of 0.972 ± 0.007. The top 10 primary feature variables of ET model were whether combined with acute kidney failure, transfusion of albumin, vasoactive drugs, transfusion of plasma, transfusion of platelet, the max of international normalized ratio (INR), the max of prothrombin time (PT), and the max of activated partial thromboplastin time (APTT). In case of ANGIB patients, gradient boosting model proven to be the optimal machine learning models, with the AUC of 0.985 ± 0.002, accuracy of 0.948 ± 0.009, precision of 0.949 ± 0.009, recall of 0.968 ± 0.009, and F1 score of 0.959 ± 0.007. Similarly, the top 10 feature variables of GB model were Glasgow Coma Scale (GCS) score, vasoactive drugs, acute kidney failure, AIMS65 score, APACHE-II score, mechanical ventilation, the minimum of lactate, chronic liver disease, and the minimum and maximum of APTT. The SHAP visualization shows the weights of two ML models feature variables and the average sharp values of variables. Meanwhile, SHAP waterfall outputs the model prediction process with true positive and negative patients. Most importantly, two website prognostic prediction platforms were developed to enhance clinical accessibility: the ET model for AVGIB patients available at https://10zr656do5281.vicp.fun while the GB model for ANGIB patients accessible at http://10zr656do5281.vicp.fun.ConclusionThe ET model provides a reliable prognostic tool for AVGIB patients, while the GB model serves as a robust tool for ANGIB patients in predicting in-hospital mortality. By systematically integrating clinical features, risk stratification scores, vital signs, and invention measures, the ML models may deliver comprehensive predictions that benefit for clinical decision-making and potentially enhance clinical outcomes in the near future.

Background: Coronary Artery Disease (CAD) is one of the biggest causes of mortality worldwide. Risk stratification for early detection is essential for the primary prevention of CAD. QRISK3 is known to overestimate future CAD risk in some populations, resulting in unnecessary preventive treatment and reduced cost-effectiveness and safety. Combining machine learning model with the metaheuristic optimisation approach using the PSO algorithm may outperform QRISK3 in predicting CAD. It improves performance by selecting the best-performing subset of features related to clinical outcomes. Research Question: Does the performance of Machine Learning Models Combined with PSO Algorithm for Feature Selection as a Metaheuristic Optimisation Approach in Predicting Coronary Arterial Disease using the UK Biobank dataset outperform the QRISK3 calculator? Aims: This study is to assess the accuracy of QRISK3 in predicting CAD using the UK Biobank dataset. It aims to evaluate the efficacy of machine learning models on the identical dataset for predicting CAD. The work utilises the PSO algorithm for feature selection to identify the optimal subset of features from the UK Biobank dataset. Methods: This study utilises data from the UK Biobank. The dataset consists of 348,015 participants aged 24-84 with no prior diagnosis of CAD. The performance of both QRISK3 and machine learning models was evaluated separately using ROC analysis. Several machine learning models were employed: Logistic Regression, Decision Tree, Random Forest, Naïve Bayes, and Gradient Boosting. The dataset was split into training and test sets with a ratio of 4:1 for the machine learning models. Each model has been developed by adding a PSO algorithm to enhance the model's classification accuracy. Results: Out of the total 348,015 participants, 23,136 individuals (6.64%) were diagnosed with CAD within 10 years following their first visit, while 324,879 individuals (93.4%) did not develop CAD. The AUC value of the QRISK3 prediction is 0.6113, while the combined machine learning models using the PSO algorithm using the gradient boosting model achieve an AUC of 0.7258, showing better performance. Conclusions: This study shows hybrid machine learning models optimised with the PSO algorithm can better predict CAD than QRISK3. These ML models can effectively identify high-risk CAD patients, allowing for more personalised preventative strategies and supporting policymakers in implementing lifestyle change recommendations.

Breast cancer continues to be a leading cause of death among women in the world. The prediction of survival outcomes based on treatment modalities, i.e., chemotherapy, hormone therapy, surgery, and radiation therapy is an essential step towards personalization in treatment planning. However, Machine Learning (ML) models may improve these predictions by investigating intricate relationships between clinical variables and survival. This study investigates the performance of several ML models to predict survival rate in patients undergoing diverse breast cancer treatments i.e., chemotherapy, hormone therapy, surgery and radiation using multiple clinical parameters. The dataset consisted of 5000 samples and turned into downloaded from Kaggle. The models assessed blanketed Support Vector Machines (SVM), K-Nearest Neighbor (KNN), AdaBoost, Gradient Boosting, Random Forest, Gaussian Naive Bayes, Logistic Regression, Extreme Gradient Boosting (XG boost), and Decision tree. Performance of the models was assessed using parameters such as Accuracy, Precision, Recall, F1-Score and Area under the Receiver Operating Characteristic Curve (AUC-ROC). SHAP (SHapley Additive exPlanations) values analysis was done to provide an explanation for the impact of a feature on model predictions using Waterfall and Beeswarm plots. Anticipated baseline (E(f(x))) were in comparison to the predictions (f(x)) for each therapy group. Performance of Gradient Boosting was shown to be the best with an Accuracy: 0.972, Precision: 0.973, Recall: 0.972, F1-Score: 0.973, and AUC-ROC Score: 0.997. Chemotherapy had a notably bad impact on survival, with an f(x) of -0.274 and an E(f(x)) of -0.025. Hormone therapy showed the maximum detrimental effect on survival, with an f(x) of -0.408. Surgical operation had an extraordinarily impartial impact (f(x) = -0.041), even as radiation therapy positively impacted survival results with an f(x) of 0.22. Gradient Boosting was the most predictive algorithm for breast cancer survival outcomes. This SHAP-primarily based analysis provides a complete knowledge of ways one-of-a-kind treatments have an effect on survival predictions in breast cancer patients. Radiation therapy indicates the most tremendous effect on survival, whilst hormone therapy reveals the maximum poor effect. Future studies need to explore personalized treatment strategies that comprise these insights to enhance patient effects.

647 Background: The primary treatment of most mNSTC is chemotherapy followed by surgery if the residual disease (RD) is >1 cm. However, conventional imaging lacks the specificity to characterize the tissue, often leading to overtreatment. This study hypothesizes that integrating CT-driven radiomics features with plasma miR371 and miR375 will enhance the predictive accuracy of Machine Learning (ML) models to predict teratoma, viable germ cell (vGCT) and fibrosis/necrosis (F/N) in mNSTC patients with RD. Methods: 111 lesions from52 patients, including residual teratoma (n=57), F/N (n=33), vGCT (n=10), and additional seminoma (n=11) for training purposes were included, split into training (N=78) and test cohorts (N=33). Lesions were lymph nodes (n=87), lung (n=21), and brain (n=3) with a median size of 1.6 cm (Q1-Q3 interval=1.2-2.73 cm). 3D Slicer version 5.6.1 was used to segment the RD > 1 cm (short axis) and extract radiomics features. Plasma miRNA levels before resection were measured by RT-PCR. Random Forest (RF), Support Vector Machine (SVM), Gradient Boosting (GB), and CatBoost (CB) ML models were evaluated to define the operating characteristics of radiomics alone (R-only) and in combination with miR371 (371) and/or miR375 (375) levels in predicting teratoma, vGCT and F/N. Results: For predicting teratoma, the best models were RF (R+375 and R+371+375), CB (R+371+375), and GB (R+371 and R+371+375). While adding miR371 or miR375 to R-only slightly improved AUC across models, the best results were achieved with the R+375+371 dataset. CB achieved AUCs ranging from 0.94 to 0.97 in training and 0.81 to 0.93 in test sets, with its highest AUC of 0.93 (95% CI: 0.78-0.97) on the R+375+371 dataset to differentiate all three classes. Similarly, GB demonstrated strong performance, achieving its highest AUC of 0.93 (95% CI: 0.79-0.96) on the R+375+371 dataset (Table). Conclusions: Integration of plasma miR371, miR375 and radiomics improved accuracy of predicting histologies across all ML models. These methods could be used to characterize the histology of RD in mNSTC patients to better inform treatment decisions. Further refinement, including incorporation of histological findings of the primary tumor, will be reported. AUC values of different ML algorithms on training and test sets. TRAINING SET TEST SET Model ±SD R R+375 R+371 R+375+371 Model (95% CI) R R+375 R+371 R+375+371 RF 0.93±0.05 0.95±0.04 0.95±0.03 0.96±0.04 RF 0.8(0.59-0.89) 0.85(0.72-0.93) 0.87(0.76-0.95) 0.91(0.78-0.95) SVM 0.84±0.06 0.84±0.09 0.89±0.11 0.89±0.09 SVM 0.72(0.54-0.80) 0.74(0.56-0.82) 0.83(0.69-0.92) 0.84(0.76-0.94) GB 0.94±0.04 0.91±0.08 0.95±0.05 0.97±0.03 GB 0.84(0.61-0.96) 0.89(0.77-0.97) 0.89(0.79-0.96) 0.93(0.79-0.96) CB 0.95±0.03 0.94±0.03 0.94±0.04 0.97±0.03 CB 0.81(0.6-0.93) 0.86(0.73-0.94) 0.89(0.78-0.97) 0.93(0.78-0.97)

Using speech data, it is difficult to learn through machine learning how to diagnose Parkinson's disease (PD) and evaluate the effects of treatment. For this issue, the article has developed a three-stage PD discovery method. The base classifiers used in the initial stage are logistic regression (LR), K-nearest neighbor (KNN), naive bayes (NB), support vector machine (SVC), and decision tree (DT). The second stage, or stack model, is a meta-model that combines all of the classifiers mentioned earlier. The third stage ensemble model consists of Bagging, AdaBoost, Random Forest (RF), and Gradient Boosting (GBC) components. The RF and GBC classifiers are utilized to estimate the most important features from the PD dataset. The models' validation has been evaluated using the confusion matrix and validation metrics like precision, recall, and F1 score. Out of all the ensemble models, the GBC—the third model—had the highest accuracy with testing data—97.43%. KNN from the base model and stacking from the meta-model, on the other hand, had the highest accuracy, with 94.87% each. Out of all the models mentioned in this manuscript, the GBC is the only ensemble model classifier with the highest accuracy. The proposed classifier appears to be an extremely useful model for the discovery of Parkinson's disease, as demonstrated by the exploratory findings and factual analyses.