Improving anchorage occupancy forecasting with stacked ensemble learning

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

The identification of an effective inhibitor is an important starting step in drug development. Unfortunately, many issues such as the characterization of protein binding sites, the screening library, materials for assays, etc., make drug screening a difficult proposition. As the size of screening libraries increases, more resources will be inefficiently consumed. Thus, new strategies are needed to preprocess and focus a screening library towards a targeted protein. Herein, we report an ensemble machine learning (ML) model to generate a CDK8-focused screening library. The ensemble model consists of six different algorithms optimized for CDK8 inhibitor classification. The models were trained using a CDK8-specific fragment library along with molecules containing CDK8 activity. The optimized ensemble model processed a commercial library containing 1.6 million molecules. This resulted in a CDK8-focused screening library containing 1,672 molecules, a reduction of more than 99.90%. The CDK8-focused library was then subjected to molecular docking, and 25 candidate compounds were selected. Enzymatic assays confirmed six CDK8 inhibitors, with one compound producing an IC50 value of ≤100 nM. Analysis of the ensemble ML model reveals the role of the CDK8 fragment library during training. Structural analysis of molecules reveals the hit compounds to be structurally novel CDK8 inhibitors. Together, the results highlight a pipeline for curating a focused library for a specific protein target, such as CDK8.

Child mortality is a big problem around the world, especially in low- and middle-income nations where there are big differences in health care and social conditions. This investigation seeks to create a predictive model for child mortality and pinpoint the key factors that significantly contribute to it, employing machine learning (ML) methodologies. The dataset includes various features such as parental age, maternal education, birth weight, wealth index, and access to healthcare services. Thirteen machine learning classifiers were used, categorized into four model groups: Traditional Models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Naive Bayes), Tree-Based Models (Decision Tree, Random Forest, Extra Trees), Boosting Models (AdaBoost, Gradient Boosting, XGBoost), and Ensemble Learning Models (Soft Voting, Hard Voting, Stacking). The efficacy of each model was assessed using classification metrics, including Accuracy, Precision, Recall, and F1-Score within a 10-fold cross-validation framework to guarantee robustness. Results indicate that ensemble models, particularly AdaBoost, achieved the highest predictive accuracy, with perfect scores across all metrics (1.00). XGBoost and Stacking also demonstrated strong and consistent performance. The findings indicate that ensemble learning methods are effective in predicting child mortality and can assist policymakers and healthcare planners in identifying high-risk populations and implementing targeted interventions to reduce child mortality.

Threats in the digital landscape increased as different emerging technologies progress. Despite the various implementations of different defensive measures and tools to monitor and detect traffic anomaly, complexity and cost are the main challenges for small to medium enterprise networks. This study developed a malicious traffic detection system that utilized signature-based and anomaly-based techniques through packet analysis and network flow using machine learning algorithms. The system consists of a Raspberry Pi-based packet capturing tool, website application, implored with Ensemble Learning Model to detect and classify malicious network traffic. The ensemble learning model which includes weak learners like Decision Tree, Naïve Bayes and Support Vector Machine (SVM), in which individual results are combined and boosted using XG Boost algorithm. CTU-13 and CSE-CIC-IDS2018 datasets were used for training and validating the model. The ensemble learning model achieved an accuracy of 96.42%, precision of 98.99%, recall of 93.79%, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{F 1}$</tex>-score of 96.32% for the binary classification using the validation data. Moreover, the model also achieved an accuracy of 96.39%, precision of 95.82%, recall of 96.39%, and F1-score of 95.66% for the multiple classification. The ensemble machine learning model was also evaluated using generated traffic, resulting in a decline in performance: accuracy dropped to 60.94%, precision to 44.44%, recall to 30.77%, and the F1-score to 61.54%.

The reduction of NOx emissions is a paramount endeavor in contemporary engineering and energy production, as these emissions are closely linked to adverse environmental and health impacts. The prediction of NOx emission from gas turbines through several integrated data-driven machine learning methods has been evaluated in study. The study compares the performance of ensemble and conventional machine learning models, demonstrating superior accuracy achieved by the ensemble models. Specifically, the Random Forest model achieved an accuracy rate of 91.68%, XGBoost yielded an accuracy of 91.54%, and CATBoost exhibited the highest accuracy at 92.76%. These findings highlight the capability of data-driven machine learning techniques in enhancing NOx emission predictions in gas turbines. The improved prediction by ensembles can be utilized in the development and implementation of more effective control and mitigation strategies in practical applications. Through the application of these advanced machine learning approaches, the gas turbine industry can play a pivotal role in minimizing its environmental impact while optimizing operational efficiency. This study also provides valuable insights into the effectiveness of ensemble machine learning models, advancing our understanding of their capabilities in addressing the critical issue of NOx emissions from gas turbines.

Background and Objectives: We aimed to apply the ensemble machine learning model to diagnose thyroid cartilage invasion detected in computer tomography (CT) images in laryngeal cancers and evaluate the diagnostic performance of the model. Materials and Methods: A total of 313 patients were divided into two groups: the cartilage invasion group and the no cartilage invasion group. At least four CT slices were randomly selected for each patient, resulting in a total of 1251 images used in the study. A total of 619 axial CT images from the no cartilage invasion group and 632 axial CT images from the cartilage invasion group were used in the study. We reviewed the CT images and histopathological diagnoses in all cases to determine the invasion positive- or negative-status as a ground truth. The ensemble model, comprising ResNet50 and MobileNet deep learning architectures, was applied to CT images. Results: The following were obtained by the ensemble model with the test dataset: area under the curve (AUC) 0.99, and accuracy 96.54%. This model demonstrates a very high level of performance in detecting thyroid cartilage invasion. Conclusions: The ensemble machine learning model is an effective method for detecting neoplastic infiltration of the thyroid cartilage. Moreover, it may be a valuable diagnostic tool for clinicians in assessing disease prognosis and determining appropriate treatment strategies in laryngeal cancers. In conclusion, this model could be integrated into future clinical practice in laryngology and head and neck surgery for the detection of cartilage neoplastic infiltration.

A fire accident is the most tragic incident in human life. Particularly environmental hazards such as forest fires lead loss of wildlife, economy, wealth, human lives and pollution. our research purpose of predict the occurrence of fire incidents using ensemble machine learning models. The goal is to develop an accurate and reliable model that can forecast the occurrence of forest fires based on various environmental factors. The best performance is obtained by the ensemble machine learning model for this work. Comparative study of individual model and ensemble model. If you check all models Decision tree predicts 75.4%, the Random Forest tree predicts 83.2%, the Support Vector Machine predicts 71.8%, and the K nearest neighbour predicts 82.1%. Ensemble models with two combinations of decision tree and random forest tree predicts accuracy is 80.8%. Support vector machine and KNN predicts the accuracy rate is 73.4%. The individual model predicts more accuracy compared to ensemble learning model.

Harmful Algal Blooms (HABs), driven by environmental pollution, pose significant threats to water quality, public health, and aquatic ecosystems. This study aims to enhance the prediction of HABs in Lake Erie, part of the Great Lakes system, by utilizing ensemble machine learning (ML) models coupled with explainable artificial intelligence (XAI) for interpretability. Using water quality data from 2013 to 2020, various physical, chemical, and biological parameters were analyzed to predict chlorophyll-a (Chl-a) concentrations, a proxy for algal blooms. The study employed multiple ensemble ML models, including Random Forest (RF), Deep Forest (DF), Gradient Boosting (GB), and XGBoost, and compared their performance against individual models such as Support Vector Machine (SVM), Decision Tree (DT), and Multi-Layer Perceptron (MLP). The findings reveal that ensemble models, particularly XGBoost and Deep Forest (DF), achieve superior predictive accuracy with R² values of 0.8517 and 0.8544, respectively. The application of SHapley Additive exPlanations (SHAP) provided insights into the relative importance of input features, identifying Particulate Organic Nitrogen (PON), Particulate Organic Carbon (POC), and Total Phosphorus (TP) as critical factors influencing Chl-a concentrations. This research demonstrates the effectiveness of integrating ensemble ML models with XAI to improve HAB prediction accuracy and interpretability. The results support the development of proactive water quality management strategies and highlight the potential of advanced ML techniques in environmental monitoring.

The evolution of Electronic Health Records (EHR) has facilitated comprehensive patient record-keeping, enhancing healthcare delivery and decision-making processes. Despite these advancements, analyzing EHR data using ensemble machine learning methods poses unique challenges. These challenges include data dimensionality, imbalanced class distributions, and the need for effective hyperparameter tuning to optimize model performance. The study conducted a thorough comparative analysis of various ensemble machine learning (EML) models using Electronic Health Record (EHR) datasets. After addressing data imbalance and reducing dimensionality, the accuracy of the EML models showed significant improvement. Notably, the Gradient Boosting Machine (GBM) and CatBoost models exhibited superior performance with an accuracy of 73%, achieved through experiments involving dimensionality reduction and handling of imbalanced data. Furthermore, optimization techniques such as Grid Search and Random Search were employed to enhance the EML models. The results of model optimization revealed that the GBM + Random Search model performed the best, achieving an accuracy of 74%, followed by the XGBoost + Grid Search model with an accuracy of 73%. The GBM model also excelled in distinguishing between positive and negative classes, boasting the highest Area under Curve (AUC) value of 0.78, indicative of its superior classification capabilities compared to other models. This study emphasizes the significance of incorporating cutting-edge EML techniques into clinical workflows and emphasizes the revolutionary potential of GBM in classification modeling for patient severity conditions. Future research should focus on deep learning (DL) applications and the integration of these models.

An ensemble deep learning model for exhaust emissions prediction of heavy oil-fired boiler combustion

Worm gearboxes are popular across various industrial applications since they offer significant gear ratios in small installation spaces. Despite having multiple advantages, worm gearboxes are subjected to higher friction due to sliding design and are prone to damage and increased transmission error over the period of operation. Delayed diagnosis of worm gearbox degradation can lead to low-quality products and/or unnecessary production line downtime. Using vibration characteristics of worm gearbox, it is possible to determine the fault and transmission error at a given period in time. In this paper, an ensemble machine learning model is trained and deployed to monitor the transmission error of worm gearbox and classify between new, operational and old conditions. 1D CNN (one dimensional convolutional neural network) model is used to automatically extract features in vibration signal of X, Y, and Z axes and predict the relevant state of worm gear. The proposed technique uses ensemble machine learning technique fusion of features extracted by multi-layer 1D CNN for three axes vibration data. The proposed method could achieve 96% accuracy and performs significantly better than traditional sequential and ensemble machine learning models on a dataset of with 7,870 samples with 800 samples labeled as new condition, 4,740 samples as operational and 2,330 as old.

Investigating soil swelling potential is indeed a critical research area in geotechnical engineering, given its significant influence on the stability and longevity of civil structures. This study aims to predict and assess the one-dimensional vertical free swelling potential of soils using ensemble machine learning models. Within the study context, a large dataset encompassing a wide array of soil parameters from 210 soil samples, including moisture content, unit weight, plasticity, and clay content, will be used. These parameters are critical in understanding the swelling behavior of soils under varying environmental and load conditions. The novel approach of this research lies in the application of ensemble machine learning techniques, which offer a robust framework to analyze complex, nonlinear relationships within soil properties. Another key aspect of this research is the parametric assessment, where the influence of individual soil properties on swelling potential is investigated using feature importance and partial dependence analyses. These analyses provide valuable insights into the relative importance of different soil parameters on soil behavior. The outcomes of this study contribute to soil mechanics and machine learning applications in geotechnical engineering and offer practical implications for engineers and practitioners. Besides, the predictive models developed in this study aid in more informed decision-making in the design and construction of civil structures, particularly in swelling-prone areas.

The utilization of waste material, such as fly ash, in the concrete industry will provide a valuable alternative solution for creating an eco-friendly environment. However, experimental work is time-consuming; employing soft machine learning techniques can accelerate the process of forecasting the strength properties of concrete. Ensemble machine learning modeling using Python Jupyter Notebook was employed in the forecasting of compressive strength (CS) of high-performance concrete. Multilayer perceptron neuron network (MLPNN) and decision tree (DT) were used as individual learning which then ensembled with bagging and boosting to provide strong correlations. Random forest (RF) and gradient boosting regression (GBR) were also used for prediction. A total of 471 data points with input parameters (e.g., cement, fine aggregate, coarse aggregate, superplasticizer, water, days, and fly ash), and an output parameter of compressive strength (CS), were retrieved to train and test the individual learners. Cross-validation with K-fold and statistical error (i.e., MAE, MSE, RMSE, and RMSLE) analysis was applied to check the accuracy of all models. All models showed the best correlation with an ensemble model rather than an individual one. DT with AdaBoost and random forest gave a strong correlation of R2 = 0.89 with fewer errors. Cross-validation results revealed a good response with an error of less than 10 MPa. Thus, ensemble modeling not only trains the data by employing several weak learners but also produces a robust correlation that can then be used to model and predict the mechanical performance of concrete.

Machine learning algorithms aim to improve the power of predictors over conventional regression models. This study aims to tap the predictive potential of crash mechanism-related variables using ensemble machine learning models. The results demonstrate selected models can predict severity at a high level of accuracy. The stacking model with a linear blender is preferred for the designed ensemble combination. Most bagging, boosting, and stacking algorithms perform well, indicating ensemble models are capable of improving upon individual models.

ObjectivesTo develop and identify machine learning (ML) models using pretreatment 2-deoxy-2-[18F]fluoro-D-glucose ([18F]-FDG)-positron emission tomography (PET)-based radiomic features to differentiate benign from malignant parotid gland diseases (PGDs).Materials and methodsThis retrospective study included 62 patients with 63 PGDs who underwent pretreatment [18F]-FDG-PET/computed tomography (CT). The lesions were assigned to the training (n = 44) and testing (n = 19) cohorts. In total, 49 [18F]-FDG-PET-based radiomic features were utilized to differentiate benign from malignant PGDs using five different conventional ML algorithmic models (random forest, neural network, k-nearest neighbors, logistic regression, and support vector machine) and the deep learning (DL)-based ensemble ML model. In the training cohort, each conventional ML model was constructed using the five most important features selected by the recursive feature elimination method with the tenfold cross-validation and synthetic minority oversampling technique. The DL-based ensemble ML model was constructed using the five most important features of the bagging and multilayer stacking methods. The area under the receiver operating characteristic curves (AUCs) and accuracies were used to compare predictive performances.ResultsIn total, 24 benign and 39 malignant PGDs were identified. Metabolic tumor volume and four GLSZM features (GLSZM_ZSE, GLSZM_SZE, GLSZM_GLNU, and GLSZM_ZSNU) were the five most important radiomic features. All five features except GLSZM_SZE were significantly higher in malignant PGDs than in benign ones (each p < 0.05). The DL-based ensemble ML model had the best performing classifier in the training and testing cohorts (AUC = 1.000, accuracy = 1.000 vs AUC = 0.976, accuracy = 0.947).ConclusionsThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can be useful for differentiating benign from malignant PGDs.Second abstractThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can overcome the previously reported limitation of [18F]-FDG-PET/CT scan for differentiating benign from malignant PGDs. The DL-based ensemble ML approach using [18F]-FDG-PET-based radiomic features can provide useful information for managing PGD.