EnPSO: An AutoML Technique for Generating Ensemble Recommender System

Child mortality is a big problem around the world, especially in low- and middle-income nations where there are big differences in health care and social conditions. This investigation seeks to create a predictive model for child mortality and pinpoint the key factors that significantly contribute to it, employing machine learning (ML) methodologies. The dataset includes various features such as parental age, maternal education, birth weight, wealth index, and access to healthcare services. Thirteen machine learning classifiers were used, categorized into four model groups: Traditional Models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Naive Bayes), Tree-Based Models (Decision Tree, Random Forest, Extra Trees), Boosting Models (AdaBoost, Gradient Boosting, XGBoost), and Ensemble Learning Models (Soft Voting, Hard Voting, Stacking). The efficacy of each model was assessed using classification metrics, including Accuracy, Precision, Recall, and F1-Score within a 10-fold cross-validation framework to guarantee robustness. Results indicate that ensemble models, particularly AdaBoost, achieved the highest predictive accuracy, with perfect scores across all metrics (1.00). XGBoost and Stacking also demonstrated strong and consistent performance. The findings indicate that ensemble learning methods are effective in predicting child mortality and can assist policymakers and healthcare planners in identifying high-risk populations and implementing targeted interventions to reduce child mortality.

The reduction of NOx emissions is a paramount endeavor in contemporary engineering and energy production, as these emissions are closely linked to adverse environmental and health impacts. The prediction of NOx emission from gas turbines through several integrated data-driven machine learning methods has been evaluated in study. The study compares the performance of ensemble and conventional machine learning models, demonstrating superior accuracy achieved by the ensemble models. Specifically, the Random Forest model achieved an accuracy rate of 91.68%, XGBoost yielded an accuracy of 91.54%, and CATBoost exhibited the highest accuracy at 92.76%. These findings highlight the capability of data-driven machine learning techniques in enhancing NOx emission predictions in gas turbines. The improved prediction by ensembles can be utilized in the development and implementation of more effective control and mitigation strategies in practical applications. Through the application of these advanced machine learning approaches, the gas turbine industry can play a pivotal role in minimizing its environmental impact while optimizing operational efficiency. This study also provides valuable insights into the effectiveness of ensemble machine learning models, advancing our understanding of their capabilities in addressing the critical issue of NOx emissions from gas turbines.

Purpose This study aims to propose a new ensemble learning model and compare its performance with other ensemble models to obtain the best model for detecting financial statement fraud during the COVID-19 pandemic. Design/methodology/approach This study uses a quantitative approach, using secondary data from financial reports, annual reports, regulatory reports and other information on the internet. It focuses on all companies listed on the Indonesia Stock Exchange from 2020 to 2023. The independent variables in this study use financial and nonfinancial variables. In contrast, the target variable for fraudulent financial reports is based on sanctions from regulators and the company’s special supervisory status. Findings This study results show that the ensemble blending model performs best in detecting financial statement fraud compared to the ensemble model that construct it. Research limitations/implications This study sets ensemble learning to default settings. Setting certain conditions can further improve the performance of ensemble learning models. Practical implications This study can broaden the insights of practitioners, academics, investors, regulators, stakeholders and corporate finance experts into detecting financial report fraud. Originality/value This study proposes a new ensemble learning model that previous studies have not discussed. This ensemble learning model performs best compared to other ensemble learning models.

The effective operation of water injection pumps is vital for enhancing oil recovery in the oil and gas industry. To ensure optimal pump performance and prevent unplanned downtime, this study focused on implementing predictive maintenance strategies. We began by identifying five critical operational parameters—Seal Pressure 1, Seal Pressure 2, Vibration Data for the Drive End (VIB DE), Vibration Data for the Non-Drive End (VIB NDE), and Ampere. These parameters were monitored and analyzed to evaluate their impact on pump performance and maintenance needs. To achieve this, we applied three machine learning algorithms: Extreme Gradient Boosting (XGBoost), Light Gradient-Boosting Machine (LGBM), and Random Forest. Each algorithm was independently trained and tested on the dataset corresponding to each operational parameter. We assessed their performance using key accuracy metrics, including R squared, Mean Absolute Error (MAE), and Root Mean Square Error (RMSE). Following this, we developed an Ensemble model, combining the predictive outputs of XGBoost, LGBM, and Random Forest. The Ensemble model was then applied to the same parameters to evaluate its ability to address the limitations observed in standalone models. The results demonstrated that the Ensemble model consistently delivered superior performance, achieving lower RMSE and MAE values and higher R squared coefficients across all parameters. This study culminates in the validation of the Ensemble model as a robust and reliable approach for predictive maintenance. By leveraging the strengths of multiple algorithms, the Ensemble model offers significant improvements in accuracy and reliability, contributing to more effective maintenance systems for the oil and gas industry.

Swarm intelligence techniques with incredible success rates are broadly used for various irregular and interdisciplinary topics. However, their impact on ensemble models is considerably unexplored. This study proposes an optimized-ensemble model integrated for smart home energy consumption management based on ensemble learning and particle swarm optimization (PSO). The proposed model exploits PSO in two distinct ways; first, PSO-based feature selection is performed to select the essential features from the raw dataset. Secondly, with larger datasets and comprehensive range problems, it can become a cumbersome task to tune hyper-parameters in a trial-and-error manner manually. Therefore, PSO was used as an optimization technique to fine-tune hyper-parameters of the selected ensemble model. A hybrid ensemble model is built by using combinations of five different baseline models. Hyper-parameters of each combination model were optimized using PSO followed by training on different random samples. We compared our proposed model with our previously proposed ANN-PSO model and a few other state-of-the-art models. The results show that optimized-ensemble learning models outperform individual models and the ANN-PSO model by minimizing RMSE to 6.05 from 9.63 and increasing the prediction accuracy by 95.6%. Moreover, our results show that random sampling can help improve prediction results compared to the ANN-PSO model from 92.3% to around 96%.

To develop and compare different AutoML frameworks and machine learning models to predict premature birth. The study used a large electronic medical record database to include 715,962 participants who had the principal diagnosis code of childbirth. Three Automatic Machine Learning (AutoML) were used to construct machine learning models including tree-based models, ensembled models, and deep neural networks on the training sample (N = 536,971). The area under the curve (AUC) and training times were used to assess the performance of the prediction models, and feature importance was computed via permutation-shuffling. The H2O AutoML framework had the highest median AUC of 0.846, followed by AutoGluon (median AUC: 0.840) and Auto-sklearn (median AUC: 0.820), and the median training time was the lowest for H2O AutoML (0.14 min), followed by AutoGluon (0.16 min) and Auto-sklearn (4.33 min). Among different types of machine learning models, the Gradient Boosting Machines (GBM) or Extreme Gradient Boosting (XGBoost), stacked ensemble, and random forrest models had better predictive performance, with median AUC scores being 0.846, 0.846, and 0.842, respectively. Important features related to preterm birth included premature rupture of membrane (PROM), incompetent cervix, occupation, and preeclampsia. Our study highlights the potential of machine learning models in predicting the risk of preterm birth using readily available electronic medical record data, which have significant implications for improving prenatal care and outcomes.

Background and Objectives: We aimed to apply the ensemble machine learning model to diagnose thyroid cartilage invasion detected in computer tomography (CT) images in laryngeal cancers and evaluate the diagnostic performance of the model. Materials and Methods: A total of 313 patients were divided into two groups: the cartilage invasion group and the no cartilage invasion group. At least four CT slices were randomly selected for each patient, resulting in a total of 1251 images used in the study. A total of 619 axial CT images from the no cartilage invasion group and 632 axial CT images from the cartilage invasion group were used in the study. We reviewed the CT images and histopathological diagnoses in all cases to determine the invasion positive- or negative-status as a ground truth. The ensemble model, comprising ResNet50 and MobileNet deep learning architectures, was applied to CT images. Results: The following were obtained by the ensemble model with the test dataset: area under the curve (AUC) 0.99, and accuracy 96.54%. This model demonstrates a very high level of performance in detecting thyroid cartilage invasion. Conclusions: The ensemble machine learning model is an effective method for detecting neoplastic infiltration of the thyroid cartilage. Moreover, it may be a valuable diagnostic tool for clinicians in assessing disease prognosis and determining appropriate treatment strategies in laryngeal cancers. In conclusion, this model could be integrated into future clinical practice in laryngology and head and neck surgery for the detection of cartilage neoplastic infiltration.

Machine Learning Techniques are gaining a lot of momentum in constant improvement of disease diagnosis. In this study, we have investigated the discriminative performance of ensemble learning model for diabetes prediction at an early stage. We have used different machine learning models and then ensemble it to improve the overall prediction accuracy. The dataset used is NHANES 2013-14 comprising of 10,172 samples and 54 feature variables for diabetes section. The feature variables used are in the form of questionnaire, a set of questions suggested by NHANES (National Health and Nutrition Examination Survey). An Ensemble model using majority voting technique was developed by combining the unweighted prediction probabilities of different machine learning models. Also, the model is evaluated and validated for real user input data for user friendliness. The overall performance was improved by Ensemble Model and had an AUC (Area under Curve) of 0.75 indicating high performance.

Leveraging advanced ensemble models to increase building energy performance prediction accuracy in the residential building sector

Machine learning algorithms aim to improve the power of predictors over conventional regression models. This study aims to tap the predictive potential of crash mechanism-related variables using ensemble machine learning models. The results demonstrate selected models can predict severity at a high level of accuracy. The stacking model with a linear blender is preferred for the designed ensemble combination. Most bagging, boosting, and stacking algorithms perform well, indicating ensemble models are capable of improving upon individual models.

<p>Ensemble learning is one of the most powerful extensions for improving upon individual machine learning models. Rather than a single model being used, several models are trained and the predictions combined to make a more informed decision. Such combinations will ideally overcome the shortcomings of any individual member of the ensemble. Most ma- chine learning competition winners feature an ensemble of some sort, and there is also sound theoretical proof to the performance of certain ensem- bling schemes. The benefits of ensembling are clear in both theory and practice. Despite the great performance, ensemble learning is not a trivial task. One of the main difficulties is designing appropriate ensembles. For exam- ple, how large should an ensemble be? What members should be included in an ensemble? How should these members be weighted? Our first contribution addresses these concerns using a strongly-typed population- based search (genetic programming) to construct well-performing ensem- bles, where the entire ensemble (members, hyperparameters, structure) is automatically learnt. The proposed method was found, in general, to be significantly better than all base members and commonly used compari- son methods trialled. With automatically designed ensembles, there is a range of applica- tions, such as competition entries, forecasting and state-of-the-art predic- tions. However, often these applications also require additional prepro- cessing of the input data. Above the ensemble considers only the original training data, however, in many machine learning scenarios a pipeline is required (for example performing feature selection before classification). For the second contribution, a novel automated machine learning method is proposed based on ensemble learning. This method uses a random population-based search of appropriate tree structures, and as such is em- barrassingly parallel, an important consideration for automated machine learning. The proposed method is able to achieve equivalent or improved results over the current state-of-the-art methods and does so in a fraction of the time (six times as fast). Finally, while complex ensembles offer great performance, one large limitation is the interpretability of such ensembles. For example, why does a forest of 500 trees predict a particular class for a given instance? In an effort to explain the behaviour of complex models (such as ensem- bles), several methods have been proposed. However, these approaches tend to suffer at least one of the following limitations: overly complex in the representation, local in their application, limited to particular fea- ture types (i.e. categorical only), or limited to particular algorithms. For our third contribution, a novel model agnostic method for interpreting complex black-box machine learning models is proposed. The method is based on strongly-typed genetic programming and overcomes the afore- mentioned limitations. Multi-objective optimisation is used to generate a Pareto frontier of simple and explainable models which approximate the behaviour of much more complex methods. We found the resulting rep- resentations are far simpler than existing approaches (an important con- sideration for interpretability) while providing equivalent reconstruction performance. Overall, this thesis addresses two of the major limitations of existing ensemble learning, i.e. the complex construction process and the black- box models that are often difficult to interpret. A novel application of ensemble learning in the field of automated machine learning is also pro- posed. All three methods have shown at least equivalent or improved performance than existing methods.</p>

Background and objectiveThe increasing global prevalence of diabetes has led to a surge in complications, significantly burdening healthcare systems and affecting patient quality of life. Early prediction of these complications is critical for timely intervention, yet existing models often rely heavily on clinical indicators while underutilizing fundamental laboratory test parameters. This study aims to bridge this gap by leveraging the 12 most frequently tested laboratory indicators in diabetic patients to develop an optimized predictive model for diabetes complications.MethodsA comprehensive dataset was established through meticulous data collection from a high-volume tertiary hospital, followed by extensive data cleaning and classification. Various machine learning classifiers, including Random Forest, XGBoost, Support Vector Machine (SVM), and Multilayer Perceptron (MLP), were trained on this dataset to evaluate their predictive performance. We further introduced an ensemble learning model with Bayesian optimization to enhance accuracy and cost-efficiency. Additionally, feature importance analysis was conducted to refine the model by reducing testing costs while maintaining high predictive accuracy.ResultsOur ensemble model with Bayesian optimization demonstrated superior performance, achieving over 90% accuracy in predicting various diabetic complications, with an outstanding 98.50% accuracy and 99.76% AUC for diabetic nephropathy. Feature correlation analysis enabled a refined model that not only improved predictive accuracy but also reduced overall medical costs by 2.5% through strategic feature elimination.ConclusionsThis study makes three key contributions: (1) Development of a high-quality dataset based on fundamental laboratory indicators, (2) Creation of a highly accurate predictive model using ensemble learning and Bayesian optimization, particularly excelling in diabetic nephropathy prediction, and (3) Implementation of a cost-efficient diagnostic approach that reduces testing expenses without compromising accuracy. The proposed model provides a strong foundation for future research and practical clinical applications, demonstrating the potential of integrating machine learning with cost-conscious medical testing.

The identification of an effective inhibitor is an important starting step in drug development. Unfortunately, many issues such as the characterization of protein binding sites, the screening library, materials for assays, etc., make drug screening a difficult proposition. As the size of screening libraries increases, more resources will be inefficiently consumed. Thus, new strategies are needed to preprocess and focus a screening library towards a targeted protein. Herein, we report an ensemble machine learning (ML) model to generate a CDK8-focused screening library. The ensemble model consists of six different algorithms optimized for CDK8 inhibitor classification. The models were trained using a CDK8-specific fragment library along with molecules containing CDK8 activity. The optimized ensemble model processed a commercial library containing 1.6 million molecules. This resulted in a CDK8-focused screening library containing 1,672 molecules, a reduction of more than 99.90%. The CDK8-focused library was then subjected to molecular docking, and 25 candidate compounds were selected. Enzymatic assays confirmed six CDK8 inhibitors, with one compound producing an IC50 value of ≤100 nM. Analysis of the ensemble ML model reveals the role of the CDK8 fragment library during training. Structural analysis of molecules reveals the hit compounds to be structurally novel CDK8 inhibitors. Together, the results highlight a pipeline for curating a focused library for a specific protein target, such as CDK8.

Reinforced concrete shear walls (RCSWs) are complicated to compute their shear capacity due to their large cross-sectional height-to-thickness ratios and the fact that they are subjected to vertical loads. Numerous factors influence RCSWs’ shear strength capacity, and the analytical models find it challenging to fully account for each factor’s impact on RCSWs’ shear-bearing capacity. Machine learning (ML) technology can deeply capture the mapping relationship between each input feature and the target value, and provide a more flexible and effective prediction method for RCSW shear-bearing capacity. To this end, a shear capacity test database containing 583 RCSW specimens was first established and characterized, and then the database was employed to train single, ensemble, and deep learning models for the shear strength of shear walls and combined with hyper-parameter tuning to enhance each model’s prediction performance, after which the prediction performance of each model was compared. Then, the ML models were contrasted with conventional techniques founded on the mechanical premise. Finally, in order to improve the prediction accuracy and reliability of the ML methods, the individually trained models were integrated into a stacking model using the stacking method, and the stacking model’s prediction performance was assessed. The results of this study show that in the single model, the test set R2 of the decision tree (DT) reaches 0.94, showing good trend-capturing ability. Among the ensemble models, Gradient Boosting (GB) performs the best and is comparable to DT in terms of RMSE and R2 and significantly outperforms other ensemble methods, such as Random Forest (RF) and Bagging. Deep Neural Networks (DNNs) show the strongest predictive ability among all models, with the lowest RMSE (263 kN) and a test R2 of 0.95, which is much better than the majority of ensemble models. The ML models show high accuracy and reliability compared to the traditional RC shear wall shear capacity models. The stacking model has an R2 of 0.98 and a CoV of 0.147 in the test set, and it is much better than other independent ML models (R2 = 0.88~0.95, CoV = 0.179~0.651).

The utilization of waste material, such as fly ash, in the concrete industry will provide a valuable alternative solution for creating an eco-friendly environment. However, experimental work is time-consuming; employing soft machine learning techniques can accelerate the process of forecasting the strength properties of concrete. Ensemble machine learning modeling using Python Jupyter Notebook was employed in the forecasting of compressive strength (CS) of high-performance concrete. Multilayer perceptron neuron network (MLPNN) and decision tree (DT) were used as individual learning which then ensembled with bagging and boosting to provide strong correlations. Random forest (RF) and gradient boosting regression (GBR) were also used for prediction. A total of 471 data points with input parameters (e.g., cement, fine aggregate, coarse aggregate, superplasticizer, water, days, and fly ash), and an output parameter of compressive strength (CS), were retrieved to train and test the individual learners. Cross-validation with K-fold and statistical error (i.e., MAE, MSE, RMSE, and RMSLE) analysis was applied to check the accuracy of all models. All models showed the best correlation with an ensemble model rather than an individual one. DT with AdaBoost and random forest gave a strong correlation of R2 = 0.89 with fewer errors. Cross-validation results revealed a good response with an error of less than 10 MPa. Thus, ensemble modeling not only trains the data by employing several weak learners but also produces a robust correlation that can then be used to model and predict the mechanical performance of concrete.