Concept Drift Adaptation by Exploiting Historical Knowledge.

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

This chapter introduces ensemble learning and provides an overview of ensemble methods for class imbalance learning (CIL). Ensemble methods use a set of classifiers to make predictions. The generalization ability of an ensemble is usually much stronger than that of the individual ensemble members. In CIL, ensemble methods are broadly used to further improve the existing methods or help design brand new ones. According to how the base learners are generated, ensemble methods can be roughly categorized into two paradigms: parallel ensemble methods and sequential ensemble methods. According to what ensemble method is involved, ensemble methods for CIL can be roughly categorized into Bagging-style methods, boosting-based methods, hybrid ensemble methods, and other methods. Most of the ensemble methods for CIL deal with binary class problems. How to use ensemble learning to help multiclass problems is an interesting direction.

Material decomposition allows the reconstruction of material-specific images in spectral X-ray imaging, which requires efficient decomposition models. Due to the presence of nonideal effects in X-ray imaging systems, it is difficult to explicitly estimate the imaging systems for material decomposition tasks. As an alternative to previous empirical material decomposition methods, we investigated material decomposition using ensemble learning methods in this paper. Three ensemble methods with two decision trees as the base learning algorithms were investigated to perform material decomposition in both simulation and experiment. The results were quantitatively evaluated for comparison studies. In general, the results demonstrate that the proposed ensemble learning methods often outperform their base learning algorithms, and rarely reduce performance. Compared to the reference methods and its base learning algorithm, the performance of the Boosting method using REPTree with regularization is improved by over 42% and 13%, respectively, in the noiseless simulated scenario of the XCAT phantom with cardiac and respiratory motion, and over 36% and 17%, respectively, in the noisy scenario. Simultaneously, the performance is improved by over 9% and 8%, respectively, in the original torso phantom scenario, and over 13% and 12%, respectively, in the denoising scenario. The results indicate that ensemble learning with gradient descent optimization algorithms is more appropriate for material decomposition tasks. Further studies are encouraged to facilitate future efforts toward clinical applications.

In data streams, concepts are often not stable but change with time. In this paper, we propose a selective integration algorithm DGASEN (Dynamic GA based Selected ENsemble) for handling concept-drifting data streams. This algorithm selects a near optimal subset of base classifiers based on GA algorithm and the predictive accuracy of each base classifier on validation dataset. This paper chooses SEA(with simulating abrupt concept drift) and Hyperplane (with gradual concept drift) as experimental data sets. The experimental results demonstrate that selective integration of classifiers can be significantly better than majority voting and weighted voting, which are currently the most commonly used integration techniques for handling concept drift in ensemble learning. The experimental results show that DGASEN algorithm improves the classification accuracy of integrated algorithm in handling concept-drifting data streams.

Multi-model ensemble learning for battery state-of-health estimation: Recent advances and perspectives

Estimation of time dependent scour depth around circular bridge piers: Application of ensemble machine learning methods

Extraction of insulators from power corridors is crucial for the construction of smart grids. Recent significant progress has been made in transmission line inspection through the application of Unmanned Aerial Vehicle Light Detection and Ranging (UAV LiDAR) systems, which can efficiently capture three-dimensional structural information. However, accurately extracting insulator components from massive point clouds remains challenging due to their high similarity in shape with surrounding pylons and power lines, as well as the inherent data imbalance issues. After a series of preprocessing steps, numerous multi-scale geometric features are designed and fused to precisely characterize insulators. Considering the issue of data imbalance due to the limited number of insulator samples, an Anomaly-Guided Progressively Balanced Ensemble (APE) Learning method is presented, using anomaly scores to achieve a good balance. Finally, the initial results are refined by incorporating spatial context information. The proposed delicate features and ensemble learning method were verified to be suitable for small insulator target extraction (including tangent insulators and tension insulators of different pylon types), achieving average precision, recall, and F1 scores of up to 95%. The results can promote intelligent inspection and renovation of large-scale power corridors.

Vehicle insurance claim severity modeling requires accurate and interpretable methods that can handle skewed and heterogeneous loss data. This study provides a structured empirical comparison between classical parametric regression models and tree-based ensemble learning approaches for predicting claim size conditional on claim occurrence. The analysis is conducted within a cross-sectional conditional severity framework using real-world motor insurance data. We implement and compare ordinary least squares (OLS), a Tweedie generalized linear model (GLM), and three ensemble methods: bagging, random forests (RFs), and gradient boosting. Model performance is evaluated using out-of-sample root mean square error (RMSE), and variable importance measures assess the relative contribution of predictors. The results indicate that tree-based ensemble methods achieve modest improvements in predictive accuracy relative to classical parametric models. The Tweedie GLM remains a competitive, flexible parametric benchmark for skewed positive claim amounts. Variable importance analysis consistently identifies premium and insured value as key determinants of claim severity. Overall, the findings suggest that ensemble learning methods can complement traditional actuarial models, offering additional flexibility in capturing non-linear effects while maintaining comparable predictive performance in moderate-complexity severity data.

Gene regulatory network GRN inference is an important problem in bioinformatics. Many machine learning methods have been applied to increase the inference accuracy. Ensemble learning methods are shown in DREAM3 and DREAM5 challenges to yield a higher inference accuracy than individual algorithms. However, no ensemble method has been proposed to take advantage of the complementarity among existing algorithms from the perspective of network motifs. We propose an ensemble method based on the principle of Mixture-of-Experts ensemble learning. The method can quantitatively evaluate the accuracy of individual algorithms on predicting each type of the network motifs and assign weights to the algorithms accordingly. The individual predictions are then used to generate the ensemble prediction. By performing controlled experiments and statistical tests, the proposed ensemble method is shown to yield a significantly higher accuracy than the generic average ranking method used in the DREAM5 challenge. In addition, a new type of network motif is found in GRN, the inclusion of which can increase the accuracy of the proposed method significantly.

Spatial and temporal data such as multiple data streams often have concept drift problems, which refers to changes of the data distributions over time. Once concept drift occurs, a stationary machine learning predictor will probably be invalid because the testing data have a different data distribution from that of the training data. Recent studies in data streams aim to address this issue by concept drift adaptation techniques. Concept drift adaptation methods update the predictor by time and have been validated to provide accurate real-time prediction for a single data stream. However, handling multiple relevant data streams is still an unsolved challenge in this area, considering many real-world applications generate multiple data streams that are simultaneously evolving. To fill this gap, we here present a predicting network for multiple data streams, named MuNet. MuNet leverages the dependency between data streams to efficiently lower down the computational cost of real-time predictions for multiple data streams. In MuNet, an online-learned Bayesian neural network (BNN) is designed as a connector between streams. The BNN-connector can continuously use the real-time information from only one base stream to correct the stationary predictor for the other streams, which avoids the high cost caused by repeated adaptation.

Short-term traffic volume prediction by ensemble learning in concept drifting environments

With the development of the smart manufacturing, data-driven fault diagnosis has receiving more and more attentions from both academic and engineering fields. As one of the most important data-driven fault diagnosis method, deep learning (DL) has achieved remarkable applications. However, the DL based fault diagnosis methods still have the following two drawbacks: 1) One of the most major branch of deep learning is to construct the deeper structures, however the deep learning models in fault diagnosis is very shadow. 2) As stated by the no-free-lunch theorem, no single model can perform best on every dataset, and the individual deep learning model still suffers from the generalization ability. In this research, a new negative correlation ensemble transfer learning method (NCTE) is proposed. Firstly, the transfer learning based ResNet-50 is proposed to construct a deep learning structure that has 50 layers. Secondly, several fully-connected layers and softmax classifiers are trained cooperatively using negative correlation learning (NCL). Thirdly, the hyper-parameters of the proposed NCTE are determined by cross validation. The proposed NCTE is conducted on the KAT Bearing Dataset, and the prediction accuracy of NCTE is as high as 98.73%. This results show that NCTE has achieved a good results compared with other machine learning and deep learning method.

In evolving data stream, when its concept undergoes a change it is known as concept drift. Detecting concept drift and handling it is a challenging task in data stream mining. If an algorithm is not adapted to concept drift, then it directly affects its performance. A number of algorithms have been developed to handle concept drift, but they are not suited for both sudden concept drift and gradual concept drift. Thus, there is a demand for an algorithm that can react to both the types of concept drift as well as incur less computational cost. A new approach hybrid early drift detection method (HEDDM) has been proposed for drift detection, which works with an ensemble method to improve the performance.

Using dynamical systems tools to detect concept drift in data streams

The silicon content of the hot metal is not only an important indicator of the quality of the hot metal and blast furnace (BF) operation but also reflects the level of energy utilization and the thermal state within a BF. It is important to develop an accurate prediction model for hot metal silicon content. In present study, two models for predicting hot metal silicon content are developed based on two ensemble learning methods, random forest regression (RFR) and extreme gradient boosting (XGBoost). First, Box plot was used to visualize the collected data and determine extreme outliers in the raw data. Extreme outliers are replaced with null value, and all null value are filled by linear interpolation. Secondly, feature selection is performed using recursive feature elimination. Cross-validation is performed to optimize machine learning hyperparameters and having a robust accuracy measure. Based on this, two hot metal silicon content prediction models are developed. Finally, the prediction results of the two models are compared and evaluated. The results show that both ensemble learning models show good prediction performance in predicting hot metal silicon content, but the prediction performance of the RFR model is better than that of the XGBoost model and reaching 98.77%.