Ensemble of kernel extreme learning machine based elimination optimization for multi-label classification

ANID-SEoKELM: Adaptive network intrusion detection based on selective ensemble of kernel ELMs with random features

Multi-site structural damage identification using a multi-label classification scheme of machine learning

The medical dispute cases are professional and closely related to medicine. Therefore, the mediation of cases in practice depends heavily on similar historical cases. Multi-label classification of legal documents can efficiently filter irrelevant historical cases, which helps to recommend the similar historical cases faster and better. However, the imbalance and label symbiosis of the data set directly affect multi-label classification of legal documents. Therefore, a multi-label classification scheme based on resampling and ensemble learning is presented in this paper The scheme includes two parts: in the first part, in order to reduce the impact of label symbiosis on resampling, a resampling algorithm based on the average sparsity of the label set is proposed improve the imbalance of the data set; in the second one, a multi-label classification algorithm based on ensemble learning is proposed to train multiple base classifiers and combine each base classifier with a voting strategy of one vote. It can effectively improve the effect of multi-label classification. The experimental results show that the scheme proposed in this paper can improve the effect of multi-label classification and is not only suitable for legal documents but also applicable for other text data sets with imbalanced classes and label symbiosis problems.KeywordsClass imbalanceMulti-label classificationEnsemble learningResampling algorithmLabel symbiosis

Improving remote sensing crop classification by argumentation-based conflict resolution in ensemble learning

The existence of fraud in credit card transactions causes many financial losses leading to customers’ loss of trust. Fraud detection methods based on machine learning techniques prevent such losses. Despite the literature on fraud detection, there is a lack of algorithms that detect fraud with acceptable performance in the credit card fraud detection field. Therefore, this study proposed a comprehensive ensemble-based method using an efficient weighted voting strategy for credit card fraud detection that can address the previous algorithms’ weaknesses. First, since the dataset is imbalanced, the proposed method balanced the dataset by stratifying it into three different proportions of normal and fraudulent transactions (1 to 1, 1 to 4 and 1 to 9 ratios). The features in each dataset are ranked by four feature-ranking methods, and the Random Forest classifier is applied to each of them for selecting the essential features. Afterward, using the seven base classifiers and the chosen features, 12 ensembles have been developed. Besides, a weighted voting strategy is proposed, and the fraudulent transactions are detected through voting based on the base classifiers’ and ensembles’ weights, which are calculated by their performance. The computational results indicated that the suggested Eclf10 is the best ensemble and its Logistic Regression classifier also has the best performance among other base classifiers. The Eclf10 leads to 99.97% accuracy, 87.78% precision, 97.70% recall, 92.21% F1-score and 95.634% F2-score, which has a superiority over the previous ensemble-based methods (e.g. majority voting ensemble, stacking classifier, Adaboost, Gradient Boosting).

Ensemble-based methods are among the most widely used techniques for data stream classification. Their popularity is attributable to their good performance in comparison to strong single learners while being relatively easy to deploy in real-world applications. Ensemble algorithms are especially useful for data stream learning as they can be integrated with drift detection algorithms and incorporate dynamic updates, such as selective removal or addition of classifiers. This work proposes a taxonomy for data stream ensemble learning as derived from reviewing over 60 algorithms. Important aspects such as combination, diversity, and dynamic updates, are thoroughly discussed. Additional contributions include a listing of popular open-source tools and a discussion about current data stream research challenges and how they relate to ensemble learning (big data streams, concept evolution, feature drifts, temporal dependencies, and others).

Implementing predictive maintenance is paramount in guaranteeing the dependability and efficiency of intricate industrial systems. Classification approaches have been extensively utilized to detect and anticipate potential malfunctions in acknowledged systems. Numerous machine learning algorithms exhibit effectiveness, yet they encounter interpretability issues, posing a challenge in comprehending the fundamental factors contributing to their predictions. This study investigates the utilization of machine learning algorithms for classification in the domain of predictive maintenance. This research focuses on applying Random Forest, XGBoost, Support Vector Machine (SVM), and K-Nearest Neighbors (KNN) algorithms for Turbofan Engine classification into multiple categories. We use Explainable Artificial Intelligence (XAI) methodologies to mitigate the interpretability challenges commonly associated with black-box models. We aim to utilize XAI algorithms on the most effective models to elucidate the essential characteristics and decision-making procedures underlying the classification outcomes. The findings of our experiment indicate that the Random Forest algorithm based on ensemble learning and the XGBoost algorithm based on gradient boosting exhibit superior classification accuracy. Additionally, the utilization of Explainable Artificial Intelligence (XAI) algorithms amplifies the transparency and interpretability of said models, thereby facilitating a more profound comprehension of the determinants that impact the classification results. The results of this study make a valuable contribution to predictive maintenance by demonstrating the efficacy of machine learning algorithms in multiclass classification and emphasizing the significance of explainable artificial intelligence (XAI) in furnishing practical insights for decision-making.

Remote sensing using unmanned aerial vehicles (UAV) combined with machine learning (ML) has significantly advanced field-scale prediction of aboveground biomass. Although ensemble learning frameworks (ELFs) typically outperform individual ML algorithms in accuracy, systematic evaluations of meta learner selection and the effects of base learner quantity and diversity remain limited. Leveraging vegetation indices and texture features extracted from multi-temporal UAV imagery, ELFs were constructed incorporating nine ML algorithms with three meta learners, Linear model, Random forest and Bayesian model averaging (BMA), to systematically evaluate how base learner configuration affects prediction accuracy. Using the fused feature set of sensitive vegetation indices and texture features, Gaussian process regression (GPR) achieved the highest accuracy among all base learners, with R2 = 0.769 and RMSE = 1.83 t·ha–1. Also, the three meta learners outperformed the best base learner, with the BMA meta learner yielding the superior accuracy (R2 = 0.795, RMSE = 1.73 t·ha–1). However, meta learner performance depended strongly on the composition of the base learners pool, stability was optimal with five base learners, and maximum accuracy was achieved by hybrid ensembles that combined linear-kernel models with GPR. This study highlights the importance of both meta learner selection and base learner composition in ELFs for aboveground biomass prediction in rice. These findings offer methodological guidance for UAV-based high-precision monitoring of crop AGB, with practical implications for precision agriculture and crop management.

The increasing number of space debris and space-active targets makes the space environment more and more complex. Space target recognition, a crucial component of space situational awareness, is of paramount importance to space security. Firstly, this paper elucidates the fundamental principles of ensemble learning, analyzes its characteristics and fusion method, and provides a comprehensive comparison of three common ensemble learning methods. Secondly, this paper analyzes the basic attributes and characteristics of space targets and categorizes the hierarchy of space target recognition. Again, the paper reviews the advances in the application of ensemble learning in space target recognition, focusing on three aspects: space target recognition datasets, the ensemble of traditional machine learning models, and ensemble deep learning. Subsequently, classical machine learning and ensemble learning algorithms are tested on a self-built space target simulation dataset, and we find that Stacking performs well on this dataset. Finally, the paper discusses future research directions.

A novel random fast multi-label deep forest classification algorithm

Multi-label classification (MLC) deals with the task of assigning an instance to all its relevant classes. This task becomes challenging in the presence of the label dependencies. The MLC methods that assume label independence do not use the dependencies among labels. We present a two-stage framework which improves the performance of MLC by using label dependencies. In the first stage, a standard MLC method is used to get the confidence scores for different labels. A conditional random field (CRF) is used in the second stage that improves the performance of the first-stage MLC by using the label dependencies among labels. An optimization-based framework is used to learn the structure and parameters of the CRF. Experiments show that the proposed model performs better than the state-of-the-art methods for MLC.

A three-way selective ensemble model for multi-label classification

Achieving scalability of learning algorithms has become an increasingly critical issue in knowledge discovery from "big data". Sampling techniques can be exploited as one of the approaches to address the issue of scalability. We present in this paper a method to employ a newly developed sampling-based ensemble learning method by boosting for multi-class (non-binary) classification. This current research extends our previous work on multi-class classification with sampling-based ensemble learning method, in which the base classifiers are the most simplistic, such as decision stumps. Here we generalize the sampling-based method to handle more complex base classifiers such as decision trees in building an ensemble, which require sampling a set of instances before building a base classifier. We present experimental results using bench-mark data sets from the UC-Irvine ML data repository that confirm the efficiency and competitive prediction accuracy of the proposed adaptive boosting method for the multi-class classification task.

One of the main problems of water transportation pipelines is leak which can cause water resources loss, possible human injuries, and damages to the environment. There are many studies in the literature focusing on detection and localization of leaks in the water pipeline systems. In this study, we have designed a wireless sensor network-based real-time monitoring system to detect and locate the leaks on multiple positions on water pipelines by using pressure data. At first, the pressure data are collected from wireless pressure sensor nodes. After that, unlike from the previous works in the literature, both the detection and localization of leakages are carried out by using multi-label learning methods. We have used three multi-label classification methods which are RAkELd, BRkNN, and BR with SVM. After the evaluation and comparison of the methods with each other, we observe that the RAkELd method performs best on almost all measures with the accuracy ratio of 98%. As a result, multi-label classification methods can be used on the detection and localization of the leaks in the pipeline systems successfully.

Developing a multi-label learning model to predict major adverse cardiovascular events in patients with unstable angina pectoris: A prospective cohort study.