Using Ensemble Method to Improve the Performance of Genetic Algorithm

A `weak' learning algorithm that performs just slightly better than random guessing can be `boosted' into an arbitrarily accurate `strong' learning algorithm by Schapire, R.E., (1990), Inspired from the `ensemble method' idea, the paper proposes a novel conceptive model of EDA ensemble: a collection of EDAs are used to optimize the same problem, during the evolution process information interaction happens among EDAs, and at last optimum solutions can be obtained more likely than a single `strong' EDA. As an instance, PBIL ensemble model is designed in details. Every PBIL serves as a component in PBIL ensemble and cooperate with others to efficiently accomplish an optimization process. Experiments on knapsack problems and function optimization problems show that PBIL ensemble exhibits better performance than simple GA and PBIL. And amazingly, to the GA-hard problem, e.g. 4-order fully deceptive problem, PBIL ensemble can achieve the optimal solution almost all the time

Although machine learning has been extensively used in various fields, it has only recently been applied to soil erosion pin modeling. To improve upon previous methods of quantifying soil erosion based on erosion pin measurements, this study explored the possible application of ensemble machine learning algorithms to the Shihmen Reservoir watershed in northern Taiwan. Three categories of ensemble methods were considered in this study: (a) Bagging, (b) boosting, and (c) stacking. The bagging method in this study refers to bagged multivariate adaptive regression splines (bagged MARS) and random forest (RF), and the boosting method includes Cubist and gradient boosting machine (GBM). Finally, the stacking method is an ensemble method that uses a meta-model to combine the predictions of base models. This study used RF and GBM as the meta-models, decision tree, linear regression, artificial neural network, and support vector machine as the base models. The dataset used in this study was sampled using stratified random sampling to achieve a 70/30 split for the training and test data, and the process was repeated three times. The performance of six ensemble methods in three categories was analyzed based on the average of three attempts. It was found that GBM performed the best among the ensemble models with the lowest root-mean-square error (RMSE = 1.72 mm/year), the highest Nash-Sutcliffe efficiency (NSE = 0.54), and the highest index of agreement (d = 0.81). This result was confirmed by the spatial comparison of the absolute differences (errors) between model predictions and observations using GBM and RF in the study area. In summary, the results show that as a group, the bagging method and the boosting method performed equally well, and the stacking method was third for the erosion pin dataset considered in this study.

This article focuses on the problem of detecting toxicity in online discussions. Toxicity is currently a serious problem when people are largely influenced by opinions on social networks. We offer a solution based on classification models using machine learning methods to classify short texts on social networks into multiple degrees of toxicity. The classification models used both classic methods of machine learning, such as naïve Bayes and SVM (support vector machine) as well ensemble methods, such as bagging and RF (random forest). The models were created using text data, which we extracted from social networks in the Slovak language. The labelling of our dataset of short texts into multiple classes—the degrees of toxicity—was provided automatically by our method based on the lexicon approach to texts processing. This lexicon method required creating a dictionary of toxic words in the Slovak language, which is another contribution of the work. Finally, an application was created based on the learned machine learning models, which can be used to detect the degree of toxicity of new social network comments as well as for experimentation with various machine learning methods. We achieved the best results using an SVM—average value of accuracy = 0.89 and F1 = 0.79. This model also outperformed the ensemble learning by the RF and Bagging methods; however, the ensemble learning methods achieved better results than the naïve Bayes method.

Solar radiation prediction plays a crucial role in renewable energy management, impacting various decision-making processes aimed at optimizing the utilization of solar resources and promoting sustainability. Ensemble regression methods, notably VotingRegressor, have emerged as promising tools for accurate solar radiation forecasting. By integrating predictions from multiple base estimators, ensemble methods have the potential to capture intricate patterns inherent in solar radiation data. However, achieving optimal predictive performance with ensemble methods heavily relies on the careful weighting assigned to each base estimator, presenting a significant challenge. In this study, a novel approach is presented to enhance solar radiation prediction by utilizing meta-learning techniques to optimize the weighting mechanism in the VotingRegressor ensemble. Meta-learning, a subfield of machine learning focusing on learning algorithms across different tasks, provides a systematic framework for learning to learn. This enables models to adapt and generalize more effectively to new datasets and tasks. Our proposed methodology demonstrated significant improvements, with the VotingRegressor with meta-learning techniques achieving an RMSE of 8.7343, an MAE of 5.42145, and an R² of 0.991913. These results mitigate the need for manual weight tuning and improve the adaptability of the VotingRegressor to varying solar radiation conditions, ultimately contributing to the sustainability of renewable energy systems. The methodology involves a comprehensive exploration of meta-learning techniques, encompassing gradient-based optimization, reinforcement learning, and Bayesian optimization.

Abstract. The shapes of ice crystals play an important role in global precipitation formation and the radiation budget. Classifying ice crystal shapes can improve our understanding of in-cloud conditions and these processes. Existing classification methods rely on features such as the aspect ratio of ice crystals, environmental temperature, and so on, which bring high instability to the classification performance, or employ supervised machine learning algorithms that heavily rely on human labeling. This poses significant challenges, including human subjectivity in classification and a substantial labor cost in manual labeling. In addition, previous deep learning algorithms for ice crystal classification are often trained and evaluated on datasets with varying sizes and classification schemes, each with distinct criteria and a different number of categories, making it difficult to make a fair comparison of algorithm performance. To overcome these limitations, a contrastive semi-supervised learning (CSSL) algorithm for the classification of ice crystals is proposed. The algorithm consists of an upstream unsupervised learning network tasked with extracting meaningful representations from a large number of unlabeled ice crystal images, and a downstream supervised network is fine-tuned with a small subset of labeled images of the entire dataset to perform the classification task. To determine the minimum number of ice crystal images that require human labeling while maintaining the algorithm performance, the algorithm is trained and evaluated on datasets with varying sizes and numbers of categories. The ice crystal data used in this study were collected during the NASCENT campaign at Ny-Ålesund and CLOUDLAB project on the Swiss Plateau using a holographic imager mounted on a tethered balloon system. In general, the CSSL algorithm outperforms a purely supervised algorithm in classifying 19 categories. Approximately 154 h of manual labeling can be avoided using just 11 % (2048 images) of the training set for fine tuning, sacrificing only 3.8 % in overall precision compared to a fully supervised model trained on the entire dataset. In the four-category classification task, the CSSL algorithm also outperforms the purely supervised algorithm. When fine-tuned on just 2048 images (25 % of the dataset), it achieves an overall accuracy of 89.6 %, nearly matching the 91.0 % accuracy of the supervised algorithm trained on 8192 images. When tested on the unseen CLOUDLAB dataset, CSSL shows superior generalization, improving accuracy by an average of 2.19 %. Our analysis also reveals that both CSSL and purely supervised algorithms exhibit inherent instability when trained on small dataset sizes, and the performance difference between them converges as the training set size exceeds 2048 samples. These results highlight the strength and practical effectiveness of CSSL in comparison to purely supervised methods and the potential of the CSSL algorithm to perform well on datasets collected under different conditions.

Flood is one of the most commonly occurred natural hazards worldwide. Severe flood occurrences in Kelantan, Malaysia cause damage to both life and property every year. Due to the huge losses in this area, development of appropriate flood modeling is required for the government. Remote sensing and geographic information system techniques can support overall flood management as they can produce rapid data collection and analysis for hydrological studies. The existing models for flood mapping have some weak points that may improve through more sophisticated and ensemble methods. The current research aimed to propose a novel ensemble method by integrating support vector machine (SVM) and frequency ratio (FR) to produce spatial modeling in flood susceptibility assessment. In the literature, mostly statistical and machine learning methods are used individually; however, their integration can enhance the final output. The FR model can perform bivariate statistical analysis and evaluate the correlation between the flooding and classes of each conditioning factors. The weights achieved by FR can be assigned to each conditioning factor and the resulted factors can be used in SVM analysis. In order to examine the efficiency of the proposed ensemble method and to show the proficiency of SVM, another machine learning algorithm such as decision tree (DT) was applied and the results were compared. To perform the methods, the upper catchment of the Kelantan basin in Malaysia was chosen. First, a flood inventory map with a total of 155 flood locations were extracted from various sources over the study area. The flood inventory map was randomly divided into two dataset; 70 % (115 flood locations) for the purpose of training and the remaining 30 % (40 flood locations) was used for validation. The spatial database included digital elevation model, curvature, geology, river, stream power index, rainfall, land use/cover, soil type, topographic wetness index and slope. For model validation, area under curve method was used and both success and prediction rate curves were calculated. The validation results for ensemble method showed 88.71 and 85.21 % for success rate and prediction rate respectively. The DT model showed 87.00 and 82.00 % for the success rate and prediction rate respectively. It is evident that the accuracies were increased using the ensemble method. The acquired results proved the efficiency of the proposed ensemble method as rapid, accurate and reasonable in flood susceptibility assessment.

Performances of machine learning algorithms for individual thermal comfort prediction based on data from professional and practical settings

An unsupervised feature selection algorithm based on ant colony optimization

Contributors

As robots become more capable in terms of hardware, and more complex tasks are considered, optimality starts playing a more important role in the design of algorithms implemented in these systems. Optimality is a guiding principle that directs the computation of feasible and efficient solutions to different robotics tasks. In control theory, this principle is implemented online as a set of efficient numerical optimization algorithms, that in addition to solving the task, purports to save a suitably defined effort or energy term. This thesis investigates trajectory generation, learning and control for dynamic tasks from the unifying point of view of optimization. As an application, we focus on Table Tennis, a chal- lenging task where robots are yet to outperform humans. We believe that the required dexterity and accuracy for this dynamical task hinges on the developments in online optimization and efficient learning algorithms. We consider trajectory generation for table tennis in the first part of the thesis. In highly dynamic tasks like table tennis that involve moving targets, planning is necessary to figure out when, where and how to intercept the target. Motion planning can be very challenging in robotic table tennis in particular, due to time constraints, dimension of the search space and joint limits. Conventional planning algorithms often rely on a fixed virtual hitting plane to construct robot striking trajectories. These algorithms, however, generate restrictive strokes and can result in unnatural strategies when compared with human playing. In this thesis, we introduce a new trajectory generation framework for robotic table tennis that does not involve a fixed hitting plane. A free-time optimal control approach is used to derive two different trajectory optimizers. The resulting two algorithms, Focused Player and Defensive Player, encode two different play- styles. We evaluate their performance in simulation and in our robot table tennis platform with a high speed cable-driven seven DOF robot arm. The algorithms return the balls with a higher probability to the opponent’s court when compared with a virtual hitting plane based method. Moreover, both can be run online and the trajectories can be corrected with new ball observations. In the second part of the thesis, we look at how such trajectories, computed on the kine- matics level, can be tracked accurately with learning control based approaches. Highly dynamic tasks like table tennis require large accelerations and precise tracking for successful perfor- mance. To track desired trajectories well, such tasks usually rely on accurate models and/or high gain feedback. While kinematic optimization allows for efficient representation and online generation of hitting trajectories, learning to track such dynamic movements with inaccurate models remains an open problem. In particular, stability issues surrounding the learning per- formance, in the iteration domain, can prevent the successful implementation of model based learning approaches. To achieve accurate tracking for these tasks in a stable and efficient way, we propose a new adaptive Iterative Learning Control algorithm that is implemented efficiently using a recursive approach. Moreover, covariance estimates of model matrices are used to ex- ercise caution during learning. We evaluate the performance of the proposed approach in our robotic table tennis platform, where we show how the performance of two Barrett WAMs can be optimized. Our implementation on the table tennis platform compares favorably with two state-of-the-art approaches. Finally, we discuss an alternative learning from demonstrations approach, where we learn sparse representations from demonstrated movements. Learning from demonstrations is an easy and intuitive way to show examples of successful behavior to a robot. However, the fact that humans optimize or take advantage of their body and not of the robot, usually called the embodiment problem in robotics, often prevents industrial robots from executing the task in a straightforward way. The shown movements often do not or cannot utilize the degrees of freedom of the robot efficiently, and typically suffer from excessive execution errors. In the last chapter, we show a new approach that can alleviate some of these difficulties by learning sparse representations of movement. Moreover, the number of learned parameters are independent of the degrees of freedom of the robot. Sparsity is a desirable feature for policy search Rein- forcement Learning algorithms that adapt the parameters of these movement primitives. By ranking the learned parameters on the Elastic Net path in terms of importance, we note that our approach could be potentially useful to combat the curse of dimensionality in robot learning applications. We show preliminary results on the real robot setup, including a successful table tennis serve using our new movement primitive representation. Throughout the thesis, we present and analyze in detail new control and learning algo- rithms. Efficient online optimization approaches are presented that can be used to solve not just table tennis problems, but they can be adapted to solve different dynamic tasks.

A recent innovation in computational learning theory is the statistical query (SQ) model. The advantage of specifying learning algorithms in this model is that SQ algorithms can be simulated in the probably approximately correct (PAC) model, both in the absenceandin the presence of noise. However, simulations of SQ algorithms in the PAC model have non-optimal time and sample complexities. In this paper, we introduce a new method for specifying statistical query algorithms based on a type ofrelative errorand provide simulations in the noise-free and noise-tolerant PAC models which yield more efficient algorithms. Requests for estimates of statistics in this new model take the following form: “Return an estimate of the statistic within a 1±?factor, or return ?, promising that the statistic is less than?.” In addition to showing that this is a very natural language for specifying learning algorithms, we also show that this new specification is polynomially equivalent to standard SQ, and thus, known learnability and hardness results for statistical query learning are preserved. We then give highly efficient PAC simulations of relative error SQ algorithms. We show that the learning algorithms obtained by simulating efficient relative error SQ algorithms both in the absence of noise and in the presence of malicious noise have roughly optimal sample complexity. We also show that the simulation of efficient relative error SQ algorithms in the presence of classification noise yields learning algorithms at least as efficient as those obtained through standard methods, and in some cases improved, roughly optimal results are achieved. The sample complexities for all of these simulations are based on thed?metric, which is a type of relative error metric useful for quantities which are small or even zero. We show that uniform convergence with respect to thed?metric yields “uniform convergence” with respect to (?, ?) accuracy. Finally, while we show that manyspecificlearning algorithms can be written as highly efficient relative error SQ algorithms, we also show, in fact, thatallSQ algorithms can be written efficiently by proving general upper bounds on the complexity of (?, ?) queries as a function of the accuracy parameter?. As a consequence of this result, we give general upper bounds on the complexity of learning algorithms achieved through the use of relative error SQ algorithms and the simulations described above.

<p><span>Healthcare fraud has become a common encounter in the healthcare finance industry. The financial security of healthcare payers and providers is seriously impacted by healthcare fraud. When incorrect or exaggerated medical services are invoiced for reimbursement, fraudulent healthcare claims result. The effective operation of the healthcare system depends on the detection of such fraudulent actions. This paper develops a healthcare claim fraud detection method based on ensemble learning. Stack ensemble learning algorithm performance is compared to that of methods such as multi-layer perceptron (MLP) classifier, support vector classifier (SVC), logistic regression (LR), and decision tree (DT) algorithm. Because of the healthcare data imbalance, the normal transaction is significantly higher than the fraudulent transaction. The machine learning (ML) algorithm performs poorly because imbalanced data causes it to be biased toward the majority class. As a result, the data is unsampled using the synthetic minority oversampling technique (SMOTE) technique to provide balanced data. The experimental results show that for the identification of healthcare claim fraud, the ensemble learning strategy greatly <span>outperforms single learning algorithms. The stack ensemble learning outperforms all the area under the curve for the receiver-operating characteristic (AUC ROC) curves from various algorithms, and the AUC-ROC curve is determined to be producing results that are adequate for all approaches.</span></span></p>

Material decomposition allows the reconstruction of material-specific images in spectral X-ray imaging, which requires efficient decomposition models. Due to the presence of nonideal effects in X-ray imaging systems, it is difficult to explicitly estimate the imaging systems for material decomposition tasks. As an alternative to previous empirical material decomposition methods, we investigated material decomposition using ensemble learning methods in this paper. Three ensemble methods with two decision trees as the base learning algorithms were investigated to perform material decomposition in both simulation and experiment. The results were quantitatively evaluated for comparison studies. In general, the results demonstrate that the proposed ensemble learning methods often outperform their base learning algorithms, and rarely reduce performance. Compared to the reference methods and its base learning algorithm, the performance of the Boosting method using REPTree with regularization is improved by over 42% and 13%, respectively, in the noiseless simulated scenario of the XCAT phantom with cardiac and respiratory motion, and over 36% and 17%, respectively, in the noisy scenario. Simultaneously, the performance is improved by over 9% and 8%, respectively, in the original torso phantom scenario, and over 13% and 12%, respectively, in the denoising scenario. The results indicate that ensemble learning with gradient descent optimization algorithms is more appropriate for material decomposition tasks. Further studies are encouraged to facilitate future efforts toward clinical applications.

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

A review on the studies employing artificial bee colony algorithm to solve combinatorial optimization problems