A Comparative Study of Improved Ensemble Learning Algorithms for Patient Severity Condition Classification

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

Harmful Algal Blooms (HABs), driven by environmental pollution, pose significant threats to water quality, public health, and aquatic ecosystems. This study aims to enhance the prediction of HABs in Lake Erie, part of the Great Lakes system, by utilizing ensemble machine learning (ML) models coupled with explainable artificial intelligence (XAI) for interpretability. Using water quality data from 2013 to 2020, various physical, chemical, and biological parameters were analyzed to predict chlorophyll-a (Chl-a) concentrations, a proxy for algal blooms. The study employed multiple ensemble ML models, including Random Forest (RF), Deep Forest (DF), Gradient Boosting (GB), and XGBoost, and compared their performance against individual models such as Support Vector Machine (SVM), Decision Tree (DT), and Multi-Layer Perceptron (MLP). The findings reveal that ensemble models, particularly XGBoost and Deep Forest (DF), achieve superior predictive accuracy with R² values of 0.8517 and 0.8544, respectively. The application of SHapley Additive exPlanations (SHAP) provided insights into the relative importance of input features, identifying Particulate Organic Nitrogen (PON), Particulate Organic Carbon (POC), and Total Phosphorus (TP) as critical factors influencing Chl-a concentrations. This research demonstrates the effectiveness of integrating ensemble ML models with XAI to improve HAB prediction accuracy and interpretability. The results support the development of proactive water quality management strategies and highlight the potential of advanced ML techniques in environmental monitoring.

ObjectivesTo develop and identify machine learning (ML) models using pretreatment 2-deoxy-2-[18F]fluoro-D-glucose ([18F]-FDG)-positron emission tomography (PET)-based radiomic features to differentiate benign from malignant parotid gland diseases (PGDs).Materials and methodsThis retrospective study included 62 patients with 63 PGDs who underwent pretreatment [18F]-FDG-PET/computed tomography (CT). The lesions were assigned to the training (n = 44) and testing (n = 19) cohorts. In total, 49 [18F]-FDG-PET-based radiomic features were utilized to differentiate benign from malignant PGDs using five different conventional ML algorithmic models (random forest, neural network, k-nearest neighbors, logistic regression, and support vector machine) and the deep learning (DL)-based ensemble ML model. In the training cohort, each conventional ML model was constructed using the five most important features selected by the recursive feature elimination method with the tenfold cross-validation and synthetic minority oversampling technique. The DL-based ensemble ML model was constructed using the five most important features of the bagging and multilayer stacking methods. The area under the receiver operating characteristic curves (AUCs) and accuracies were used to compare predictive performances.ResultsIn total, 24 benign and 39 malignant PGDs were identified. Metabolic tumor volume and four GLSZM features (GLSZM_ZSE, GLSZM_SZE, GLSZM_GLNU, and GLSZM_ZSNU) were the five most important radiomic features. All five features except GLSZM_SZE were significantly higher in malignant PGDs than in benign ones (each p < 0.05). The DL-based ensemble ML model had the best performing classifier in the training and testing cohorts (AUC = 1.000, accuracy = 1.000 vs AUC = 0.976, accuracy = 0.947).ConclusionsThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can be useful for differentiating benign from malignant PGDs.Second abstractThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can overcome the previously reported limitation of [18F]-FDG-PET/CT scan for differentiating benign from malignant PGDs. The DL-based ensemble ML approach using [18F]-FDG-PET-based radiomic features can provide useful information for managing PGD.

The identification of an effective inhibitor is an important starting step in drug development. Unfortunately, many issues such as the characterization of protein binding sites, the screening library, materials for assays, etc., make drug screening a difficult proposition. As the size of screening libraries increases, more resources will be inefficiently consumed. Thus, new strategies are needed to preprocess and focus a screening library towards a targeted protein. Herein, we report an ensemble machine learning (ML) model to generate a CDK8-focused screening library. The ensemble model consists of six different algorithms optimized for CDK8 inhibitor classification. The models were trained using a CDK8-specific fragment library along with molecules containing CDK8 activity. The optimized ensemble model processed a commercial library containing 1.6 million molecules. This resulted in a CDK8-focused screening library containing 1,672 molecules, a reduction of more than 99.90%. The CDK8-focused library was then subjected to molecular docking, and 25 candidate compounds were selected. Enzymatic assays confirmed six CDK8 inhibitors, with one compound producing an IC50 value of ≤100 nM. Analysis of the ensemble ML model reveals the role of the CDK8 fragment library during training. Structural analysis of molecules reveals the hit compounds to be structurally novel CDK8 inhibitors. Together, the results highlight a pipeline for curating a focused library for a specific protein target, such as CDK8.

Worm gearboxes are popular across various industrial applications since they offer significant gear ratios in small installation spaces. Despite having multiple advantages, worm gearboxes are subjected to higher friction due to sliding design and are prone to damage and increased transmission error over the period of operation. Delayed diagnosis of worm gearbox degradation can lead to low-quality products and/or unnecessary production line downtime. Using vibration characteristics of worm gearbox, it is possible to determine the fault and transmission error at a given period in time. In this paper, an ensemble machine learning model is trained and deployed to monitor the transmission error of worm gearbox and classify between new, operational and old conditions. 1D CNN (one dimensional convolutional neural network) model is used to automatically extract features in vibration signal of X, Y, and Z axes and predict the relevant state of worm gear. The proposed technique uses ensemble machine learning technique fusion of features extracted by multi-layer 1D CNN for three axes vibration data. The proposed method could achieve 96% accuracy and performs significantly better than traditional sequential and ensemble machine learning models on a dataset of with 7,870 samples with 800 samples labeled as new condition, 4,740 samples as operational and 2,330 as old.

Steel corrosion poses a serious threat to the structural performance of reinforced concrete (RC) structures. Thus, this study evaluates the flexural capacity of RC beams through machine learning (ML)-based techniques with six parameters used as input features: beam width, beam effective depth, concrete compressive strength, reinforcement ratio, reinforcement yield strength, and corrosion level. Four single and ensemble ML models are evaluated; namely, decision tree, support vector machine, adaptive boosting, and gradient boosting. Hyperparameters of each model were optimized using grid search and K-fold cross-validation with root mean squared error used as the performance index. The predictive performance of each model was assessed using four statistical performance metrics. The analysis results demonstrated that the decision tree model exhibited overfitting and limited generalization ability. The adaptive boosting model also had a slight overfitting issue. In addition, the support vector machine reported comparable accuracy to that of adaptive boosting. Conversely, the proposed gradient boosting ensemble model achieved the best performance with strong generalization ability, as indicated by its lowest mean absolute error of 2.78 kN.m, mean absolute percent error of 13.40%, and root mean squared error of 3.56 kN.m, and the highest coefficient of determination of 97.30% on the test dataset. The optimized gradient boosting model has been deployed into a graphical user interface, allowing for practical implementation of the model and enabling fast, efficient, and intelligent prediction of the flexural capacity of corroded RC beams.

Child mortality is a big problem around the world, especially in low- and middle-income nations where there are big differences in health care and social conditions. This investigation seeks to create a predictive model for child mortality and pinpoint the key factors that significantly contribute to it, employing machine learning (ML) methodologies. The dataset includes various features such as parental age, maternal education, birth weight, wealth index, and access to healthcare services. Thirteen machine learning classifiers were used, categorized into four model groups: Traditional Models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Naive Bayes), Tree-Based Models (Decision Tree, Random Forest, Extra Trees), Boosting Models (AdaBoost, Gradient Boosting, XGBoost), and Ensemble Learning Models (Soft Voting, Hard Voting, Stacking). The efficacy of each model was assessed using classification metrics, including Accuracy, Precision, Recall, and F1-Score within a 10-fold cross-validation framework to guarantee robustness. Results indicate that ensemble models, particularly AdaBoost, achieved the highest predictive accuracy, with perfect scores across all metrics (1.00). XGBoost and Stacking also demonstrated strong and consistent performance. The findings indicate that ensemble learning methods are effective in predicting child mortality and can assist policymakers and healthcare planners in identifying high-risk populations and implementing targeted interventions to reduce child mortality.

Investigating soil swelling potential is indeed a critical research area in geotechnical engineering, given its significant influence on the stability and longevity of civil structures. This study aims to predict and assess the one-dimensional vertical free swelling potential of soils using ensemble machine learning models. Within the study context, a large dataset encompassing a wide array of soil parameters from 210 soil samples, including moisture content, unit weight, plasticity, and clay content, will be used. These parameters are critical in understanding the swelling behavior of soils under varying environmental and load conditions. The novel approach of this research lies in the application of ensemble machine learning techniques, which offer a robust framework to analyze complex, nonlinear relationships within soil properties. Another key aspect of this research is the parametric assessment, where the influence of individual soil properties on swelling potential is investigated using feature importance and partial dependence analyses. These analyses provide valuable insights into the relative importance of different soil parameters on soil behavior. The outcomes of this study contribute to soil mechanics and machine learning applications in geotechnical engineering and offer practical implications for engineers and practitioners. Besides, the predictive models developed in this study aid in more informed decision-making in the design and construction of civil structures, particularly in swelling-prone areas.

Removal of Ni(II) ions from wastewater by ion exchange resin: Process optimization using response surface methodology and ensemble machine learning techniques

A study on road accident prediction and contributing factors using explainable machine learning models: analysis and performance

Reconstructing the vertical structures of the ocean from sea surface information is of great importance for ocean and climate studies. In this study, an ensemble machine learning (Ens-ML) model is proposed to retrieve ocean subsurface thermal structure (OSTS) by using satellite-derived sea surface data and Argo data in the South China Sea (SCS). The input data include sea surface height (SSH), sea surface temperature (SST), sea surface salinity (SSS), sea surface wind (SSW), and geographic information (including longitude and latitude). We select three stable machine learning models, namely, extreme gradient boosting (XGBoost), RandomForest and light gradient boosting machine (LightGBM) as our benchmark models, and then use an artificial neural network (ANN) technique to combine outputs from the three individual models. The proposed Ens-ML model using sea surface data only by SSH, SST, SSS, and SSW performs less satisfactorily than that considering the contribution of geographical information, indicating that the geographical information is essential to estimate the OSTS accurately. The estimated OSTS from the Ens-ML model are compared with Argo data. The results show that the proposed Ens-ML model can accurately estimate the OSTS (upper 1000 m) in the SCS, which is relatively more accurate and precise than the individual models. The performance of the Ens-ML model also varies with season, and better estimation is obtained in winter, which is probably due to stronger mixing and weaker stratification. This study shows the great potential and advantage of the multi-model ensemble of machine learning algorithm for the ocean’s interior information retrieving, showing great potential in expanding the scope of ocean observations.

Predicting mortality among people living with HIV enables clinicians to implement timely, targeted, and preventive interventions at the start of antiretroviral therapy (ART). However, prognostic models must rely strictly on baseline predictors to avoid look-ahead bias and ensure scientific validity. This study evaluates machine-learning (ML) algorithms for baseline mortality prediction using routine electronic medical record data. This study aims to predict mortality among people living with HIV receiving ART using baseline clinical and sociodemographic characteristics through ML models in public health facilities of Gondar City Administration, Northwest Ethiopia. The retrospective cohort study was conducted using electronic medical record data from 12,871 people living with HIV on ART (2005-2024). Seven base classifiers were evaluated using stratified 10-fold cross-validation. Synthetic minority oversampling technique (SMOTE)-balanced variants were used only for sensitivity analysis. SMOTE oversampling was applied only to training folds; the final evaluation used the original imbalanced test data. Shapley Additive Explanations (SHAP) analysis identified key baseline predictors. Gradient boosting on the original data achieved superior performance (accuracy 87.0%, F1-score 0.619, area under the receiver operating characteristic curve 0.859), outperforming extreme gradient boosting (F1-score 0.609, area under the receiver operating characteristic curve 0.835) and SMOTE variants. The SHAP analysis identified education level, lack of formal education (+0.84), and a low baseline cluster of differentiation 4 (CD4; a type of immune cell count) count of 140 cells/mm³ (+0.54) as substantially increasing predicted mortality risk. Urban residence (-0.35) and working functional status (-0.12) showed protective effects, whereas age (45 y; -0.02) had minimal influence in the illustrated case. Globally, lower CD4 counts and the absence of formal education were consistently associated with increased mortality risk. Ensemble ML models demonstrated moderate-to-strong discrimination for predicting mortality among people living with HIV using strictly baseline routine electronic medical record data. SHAP-based interpretability confirmed that educational attainment and baseline CD4 count were the dominant determinants of predicted mortality risk, underscoring the combined influence of socioeconomic vulnerability and immunological status at ART initiation. These findings support the potential utility of interpretable ML models for early risk stratification and targeted clinical decision-making in resource-limited settings; however, external validation is required before routine clinical implementation.

Neo-adjuvant chemoradiotherapy response prediction using MRI based ensemble learning method in rectal cancer patients

Ensemble machine learning prediction of compressive strength in waste-derived sulfoaluminate cement paste.

Internet traffic prediction is essential for effective network management, resource allocation, and ensuring efficient quality of service. Network resources can be dynamically managed by forecasting future traffic using past traffic patterns. Network traffic prediction enables the dynamic resource allocation to avoid the congestion and conflicts in the network. An Empirical mode decomposition (EMD) based machine learning models were proposed in this paper for the prediction of Software Defined Networks (SDN) traffic. SDN is a modern network architecture which separates the data plane from the control plane to provide centralized control over the network. EMD is an adaptive signal decomposition technique which extracts various frequencies from the collected network traffic at specified sampling intervals which are intrinsic mode frequencies (IMFs). Ensemble machine learning models such as Random Forest (RF), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM), were applied on intrinsic mode frequencies to generate the accurate predictions. SDN traffic traces were generated using CAIDA traffic traces and the experimental results suggests that EMD-XGBOOST outperforms than the other models with low root mean squared error (RMSE) and mean squared error (MSE).