PBIL Ensemble: Many Better than One

Ensemble method has been deeply studied and widely used in the machine learning communities. Its basic idea can be represented as: A ‘weak’ learning algorithm that performs just slightly better than random guessing can be ‘boosted’ into an arbitrarily accurate ‘strong’ learning algorithm. Inspired from the fascinating idea, the paper used ensemble method to improve the performance of genetic algorithm and proposed an efficient hybrid optimization algorithm: GA ensemble. In GA ensemble, a collection of genetic algorithms are designed to solve the same problem and population of each algorithm is sampled from a solutions pool using bagging method. Experiments on combinatorial optimization problem and GA-deceptive problems show that ensemble method improves the performance of genetic algorithm greatly.

Flood susceptibility mapping using a novel ensemble weights-of-evidence and support vector machine models in GIS

Solar radiation prediction plays a crucial role in renewable energy management, impacting various decision-making processes aimed at optimizing the utilization of solar resources and promoting sustainability. Ensemble regression methods, notably VotingRegressor, have emerged as promising tools for accurate solar radiation forecasting. By integrating predictions from multiple base estimators, ensemble methods have the potential to capture intricate patterns inherent in solar radiation data. However, achieving optimal predictive performance with ensemble methods heavily relies on the careful weighting assigned to each base estimator, presenting a significant challenge. In this study, a novel approach is presented to enhance solar radiation prediction by utilizing meta-learning techniques to optimize the weighting mechanism in the VotingRegressor ensemble. Meta-learning, a subfield of machine learning focusing on learning algorithms across different tasks, provides a systematic framework for learning to learn. This enables models to adapt and generalize more effectively to new datasets and tasks. Our proposed methodology demonstrated significant improvements, with the VotingRegressor with meta-learning techniques achieving an RMSE of 8.7343, an MAE of 5.42145, and an R² of 0.991913. These results mitigate the need for manual weight tuning and improve the adaptability of the VotingRegressor to varying solar radiation conditions, ultimately contributing to the sustainability of renewable energy systems. The methodology involves a comprehensive exploration of meta-learning techniques, encompassing gradient-based optimization, reinforcement learning, and Bayesian optimization.

Flood is one of the most commonly occurred natural hazards worldwide. Severe flood occurrences in Kelantan, Malaysia cause damage to both life and property every year. Due to the huge losses in this area, development of appropriate flood modeling is required for the government. Remote sensing and geographic information system techniques can support overall flood management as they can produce rapid data collection and analysis for hydrological studies. The existing models for flood mapping have some weak points that may improve through more sophisticated and ensemble methods. The current research aimed to propose a novel ensemble method by integrating support vector machine (SVM) and frequency ratio (FR) to produce spatial modeling in flood susceptibility assessment. In the literature, mostly statistical and machine learning methods are used individually; however, their integration can enhance the final output. The FR model can perform bivariate statistical analysis and evaluate the correlation between the flooding and classes of each conditioning factors. The weights achieved by FR can be assigned to each conditioning factor and the resulted factors can be used in SVM analysis. In order to examine the efficiency of the proposed ensemble method and to show the proficiency of SVM, another machine learning algorithm such as decision tree (DT) was applied and the results were compared. To perform the methods, the upper catchment of the Kelantan basin in Malaysia was chosen. First, a flood inventory map with a total of 155 flood locations were extracted from various sources over the study area. The flood inventory map was randomly divided into two dataset; 70 % (115 flood locations) for the purpose of training and the remaining 30 % (40 flood locations) was used for validation. The spatial database included digital elevation model, curvature, geology, river, stream power index, rainfall, land use/cover, soil type, topographic wetness index and slope. For model validation, area under curve method was used and both success and prediction rate curves were calculated. The validation results for ensemble method showed 88.71 and 85.21 % for success rate and prediction rate respectively. The DT model showed 87.00 and 82.00 % for the success rate and prediction rate respectively. It is evident that the accuracies were increased using the ensemble method. The acquired results proved the efficiency of the proposed ensemble method as rapid, accurate and reasonable in flood susceptibility assessment.

Multi-model ensemble learning for battery state-of-health estimation: Recent advances and perspectives

This work presents an on-line diagnosis algorithm for dynamic systems that combines model based diagnosis and machine learning techniques. The Possible Conflicts (PCs) method is used to perform consistency based diagnosis, providing fault detection and isolation. Machine learning methods are use to induce time series classifiers, that are applied on line for fault identification. The main contribution of this work is that Possible Conflicts are used to decompose the physical system, defining the input-output structure of an ensemble of classifiers. Experimental results on a simulated pilot plant show that the ensemble created from PCs decomposition has an important potential to increase the accuracy of individual classifiers for several learning algorithms. Without PCs decomposition, the best results were for another ensemble method, Stacking. These results are improved when combining Stacking with PCs decomposition.

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

Material decomposition allows the reconstruction of material-specific images in spectral X-ray imaging, which requires efficient decomposition models. Due to the presence of nonideal effects in X-ray imaging systems, it is difficult to explicitly estimate the imaging systems for material decomposition tasks. As an alternative to previous empirical material decomposition methods, we investigated material decomposition using ensemble learning methods in this paper. Three ensemble methods with two decision trees as the base learning algorithms were investigated to perform material decomposition in both simulation and experiment. The results were quantitatively evaluated for comparison studies. In general, the results demonstrate that the proposed ensemble learning methods often outperform their base learning algorithms, and rarely reduce performance. Compared to the reference methods and its base learning algorithm, the performance of the Boosting method using REPTree with regularization is improved by over 42% and 13%, respectively, in the noiseless simulated scenario of the XCAT phantom with cardiac and respiratory motion, and over 36% and 17%, respectively, in the noisy scenario. Simultaneously, the performance is improved by over 9% and 8%, respectively, in the original torso phantom scenario, and over 13% and 12%, respectively, in the denoising scenario. The results indicate that ensemble learning with gradient descent optimization algorithms is more appropriate for material decomposition tasks. Further studies are encouraged to facilitate future efforts toward clinical applications.

The bankruptcy of manufacturing corporates is an important factor affecting economic stability. Corporate bankruptcy has become a hot research topic mainly through financial data analysis and prediction. With the development of data science and artificial intelligence, machine learning technology helps researchers improve the accuracy and robustness of classification models. Ensemble learning, with its strong predictive power and robustness, plays an important role in machine learning and binary classification prediction. In this study, we proposed a bankruptcy classification model combining feature engineering method and ensemble learning method, Synthetic Minority Oversampling Technique (SMOTE) imbalanced data learning algorithm is applied to generate balanced dataset, multi-interval discretization filter is applied to enhance the interpretability of the features and ensemble learning method is applied to get an accurate and objective prediction. To demonstrate the validity and performance of the proposed model, we conducted comparative experiments with ten other baseline classifiers, proving that SMOTE imbalanced learning algorithm and feature engineering method with multi-interval discretization was effective. The comparative experiment results show that the ensemble learning method has a good effect on improving the performance of the proposed model. The final results show that the proposed model has achieved better performance and robustness than other baseline classifiers in terms of classification accuracy, F-measure and Area under Curve (AUC).

A comparative study of machine learning models for predicting the state of reactive mixing

Traffic speed is an important traffic parameter whose accurate prediction is crucial in traffic management. Different machine learning and ensemble methods have been used for predicting traffic parameters. Proper feature selection, the type of learners, ensemble methods, and appropriate aggregation technique have a significant impact on the prediction accuracy of ensemble methods. ‎To achieve higher prediction accuracy, using the nonlinear learning ability of Support Vector Regression (SVR) model, we propose a stacking ensemble learning model.‎‏‎ We train the base learners using different training subsets based on random subspace selection. In the analysis of empirical and complex data, Empirical Mode Decomposition (EMD) is used for decomposing time series data into seasonal and trend components. We use EMD to create new features. In order to reduce the computation time, we also propose a sample selection method. The results of our experiments show that the proposed method outperforms the existing hybrid EMD-ARIMA and non-ensemble models. In comparison with the baseline models, using the EMD technique leads to a 0.32 to 4% decrease in RMSE for different links. The results show that the proposed model outperforms the ARIMA and ARIMA-EMD models in terms of long-term traffic speed forecasting performance.

The 0/1 Knapsack problem is one of the most popular real-world optimization problems that arise in searching space and finding the most optimum solution. Theoretically, the optimum solution problem of the 0/1 Knapsack requires suitable technique to explore the search space effectively. Practically, as many metaheuristic algorithms, Whale Optimization Algorithm (WOA) may fail in local optimum solution. This paper proposes Opposition-based Whale Optimization Algorithm (OWOA) to optimize solution problem in 0/1 Knapsack. The OWOA has been tested original WOA by using twenty cases of Knapsack problem and against other metaheuristic algorithms such as (CGMA) and HS-Jaya. The experimental results indicate a significant performance of the optimization solution and stabilization with minimal standard deviation value. This shows that the OWOA improved the original version WOA and has promising result in comparison with other existing algorithms.

Hyperspectral inversion of soil organic matter based on improved ensemble learning method.

Ensembles are a well established machine learning paradigm, leading to accurate and robust models, predominantly applied to predictive modeling tasks. Ensemble models comprise a finite set of diverse predictive models whose combined output is expected to yield an improved predictive performance as compared to an individual model. In this paper, we propose a new method for learning ensembles of process-based models of dynamic systems. The process-based modeling paradigm employs domain-specific knowledge to automatically learn models of dynamic systems from time-series observational data. Previous work has shown that ensembles based on sampling observational data (i.e., bagging and boosting), significantly improve predictive performance of process-based models. However, this improvement comes at the cost of a substantial increase of the computational time needed for learning. To address this problem, the paper proposes a method that aims at efficiently learning ensembles of process-based models, while maintaining their accurate long-term predictive performance. This is achieved by constructing ensembles with sampling domain-specific knowledge instead of sampling data. We apply the proposed method to and evaluate its performance on a set of problems of automated predictive modeling in three lake ecosystems using a library of process-based knowledge for modeling population dynamics. The experimental results identify the optimal design decisions regarding the learning algorithm. The results also show that the proposed ensembles yield significantly more accurate predictions of population dynamics as compared to individual process-based models. Finally, while their predictive performance is comparable to the one of ensembles obtained with the state-of-the-art methods of bagging and boosting, they are substantially more efficient.

Ensemble ML methods build predictive models by inducing several different predictors, called base predictors or base learners. Typically, a base predictor is a very simple model that is not meant to work on its own. The predictions of such simple models are normally not much better than random guesses, which is why they are called weak learners. It is through the aggregation of the different answers from the various base learners that ensemble learning builds strong learners that produce accurate predictions.