Random Oracle Ensembles for Imbalanced Data

Improving predictive performance on survival in dairy cattle using an ensemble learning approach

In this paper, ensemble methods for different base classifiers are proposed. An ensemble technique is a supervised learning algorithm that combines a group of classifiers in order to acquire an overall model with more exact decisions. The classifiers that are support vector machine (SVM), naive Bayes (NB), and back propagation neural network (BPNN) are trained and tested on different gene expression datasets using both random selection method and k-fold cross-validation method. Both binary-class and multi-class datasets are used for evaluation of effectiveness of the ensemble method. Various publicly available gene expression datasets have been used for experiments in order to find the accuracy and effectiveness of the ensemble technique. Performance of the different classification methods and ensemble methods has been compared by using the accuracy values. The results have shown that the accuracy for the gene expression datasets has been increased by using the ensemble methods.

In this paper, we present a novel ensemble method random projection random discretization ensembles (RPRDE) to create ensembles of linear multivariate decision trees by using a univariate decision tree algorithm. The present method combines the better computational complexity of a univariate decision tree algorithm with the better representational power of linear multivariate decision trees. We develop random discretization (RD) method that creates random discretized features from continuous features. Random projection (RP) is used to create new features that are linear combinations of original features. A new dataset is created by augmenting discretized features (created by using RD) with features created by using RP. Each decision tree of a RPRD ensemble is trained on one dataset from the pool of these datasets by using a univariate decision tree algorithm. As these multivariate decision trees (because of features created by RP) have more representational power than univariate decision trees, we expect accurate decision trees in the ensemble. Diverse training datasets ensure diverse decision trees in the ensemble. We study the performance of RPRDE against other popular ensemble techniques using C4.5 tree as the base classifier. RPRDE matches or outperforms other popular ensemble methods. Experiments results also suggest that the proposed method is quite robust to the class noise.

Hypertension is a chronic disease and major risk factor for cardiovascular and cerebrovascular diseases that often leads to damage to target organs. The prevention and treatment of hypertension is crucially important for human health. In this paper, a novel ensemble method based on a flexible neural tree (FNT) is proposed to identify hypertension-related active compounds. In the ensemble method, the base classifiers are Multi-Grained Cascade Forest (gcForest), support vector machines (SVM), random forest (RF), AdaBoost, decision tree (DT), Gradient Boosting Decision Tree (GBDT), KNN, logical regression, and naïve Bayes (NB). The classification results of nine classifiers are utilized as the input vector of FNT, which is utilized as a nonlinear ensemble method to identify hypertension-related drug compounds. The experiment data are extracted from hypertension-unrelated and hypertension-related compounds collected from the up-to-date literature. The results reveal that our proposed ensemble method performs better than other single classifiers in terms of ROC curve, AUC, TPR, FRP, Precision, Specificity, and F1. Our proposed method is also compared with the averaged and voting ensemble methods. The results reveal that our method could identify hypertension-related compounds more accurately than two classical ensemble methods.

Multiobjective genetic classifier selection for random oracles fuzzy rule-based classifier ensembles: How beneficial is the additional diversity?

The subject of the research is methods and means of identifying the state of a computer system . The purpose of the article is to improve the quality of computer system state identification by developing a method based on ensemble classifiers. Task: to investigate methods for constructing bagging classifiers based on decision trees, to configure them and develop a method for identifying the state of the computer system. Methods used: artificial intelligence methods, machine learning, ensemble methods. The following results were obtained: the use of bagging classifiers based on meta-algorithms were investigated: Pasting Ensemble, Bootstrap Ensemble, Random Subspace Ensemble, Random Patches Ensemble and Random Forest methods and their accuracy were assessed to identify the state of the computer system. The research of tuning parameters of individual decision trees was carried out and their optimal values were found, including: the maximum number of features used in the construction of the tree; the minimum number of branches when building a tree; minimum number of leaves and maximum tree depth. The optimal number of trees in the ensemble has been determined. A method for identifying the state of the computer system is proposed, which differs from the known ones by the choice of the classification meta-algorithm and the selection of the optimal parameters for its adjustment. An assessment of the accuracy of the developed method for identifying the state of a computer system is carried out. The developed method is implemented in software and investigated when solving the problem of identifying the abnormal state of the computer system functioning. Conclusions. The scientific novelty of the results obtained lies in the development of a method for identifying the state of the computer system by choosing a meta-algorithm for classification and determining the optimal parameters for its configuration.

Using Ensemble StackingC Method and Base Classifiers to Ameliorate Prediction Accuracy of Pedagogical Data

Educational data mining has illustrated an increasing demand for extracting and maneuvering data from academic backdrop, to generate prolific information which is indispensible for decision making. Therefore in this paper, an attempt has been made to deploy various data mining techniques including base and meta learning classifiers across our pedagogical dataset to foretell the performance of students. Among several contemporary ensemble approaches, researchers have practiced widespread learning classifiers viz. boosting to predict the performance of students. As exploitation of ensemble methods is considered to be significant phenomenon in classification and prediction mechanisms, therefore analogous method (boosting) has been applied across our pedagogical dataset. The entire results have been evaluated with 10-fold cross validation, once pedagogical dataset has been subjected to base classifiers including j48, random tree, naive bayes and knn. In addition, techniques such as oversampling (SMOTE) and undersampling (Spread subsampling) have been employed to further draw a comparison among ensemble classifiers and base classifiers. These methods were exploited with the key objective to observe any improvement in prediction accuracy of students.

Functional Trees are one type of multivariate trees. This work studies the performance of different ensemble methods (Bagging, Random Subspaces, AdaBoost, Rotation Forest) using three variants (multivariate internal nodes, multivariate leaves or both) of these trees as base classifiers. The best results, for all the ensemble methods, are obtained using Functional Trees with multivariate leaves and univariate internal nodes. The best overall configuration is obtained with Rotation Forest. Ensembles of Functional Trees are compared to ensembles of univariate Decision Trees, being the results favourable for the variant of Functional Trees with univariate internal nodes and multivariate leaves. Kappa-error diagrams are used to study the diversity and accuracy of the base classifiers.

In recent years, ensemble methods have been widely applied to classify binary imbalanced data. Traditional ensemble rules have manifested performance in dealing with imbalanced data. However, shortage appears that only the results of base classifiers is considered, while these traditional ensemble rules ignore the Euclidean distance between the new data and train data as well as the relations of majority and minority classes in the train data. So we proposed a novel ensemble rule which take Support Vector Classification (SVC) as base classifier. Moreover, the distance between the new data and train data and relations of majority classes and minority classes are taken into account to overcome conventional drawbacks. Simulation results are provided to confirm that the proposed method has better performance than existing ensemble methods.

One of the major developments in machine learning in the past decade is the ensemble method, which finds highly accurate classifier by combining many moderately accurate component classifiers. In this research work, new ensemble classification methods are proposed with homogeneous ensemble classifier using bagging and heterogeneous ensemble classifier using arcing and their performances are analyzed in terms of accuracy. A Classifier ensemble is designed using Radial Basis Function RBF and Support Vector Machine SVM as base classifiers. The feasibility and the benefits of the proposed approaches are demonstrated by the means of standard datasets of direct marketing. The main originality of the proposed approach is based on three main parts: preprocessing phase, classification phase and combining phase. A wide range of comparative experiments are conducted for standard datasets of direct marketing. The performance of the proposed homogeneous and heterogeneous ensemble classifiers are compared to the performance of other standard homogeneous and heterogeneous ensemble methods. The standard homogeneous ensemble methods include Error correcting output codes, dagging and heterogeneous ensemble methods include majority voting, stacking. The proposed ensemble methods provide significant improvement of accuracy compared to individual classifiers and the proposed bagged RBF and SVM performs significantly better than ECOC and Dagging and the proposed hybrid RBF-SVM performs significantly better than voting and stacking. Also heterogeneous models exhibit better results than homogeneous models for standard datasets of direct marketing.

This study evaluates ensemble learning techniques for optimizing heart disease prediction, with a focus on Random Forest due to its robustness in handling complex medical data. The dataset used, "Heart Disease Prediction Dataset" from Kaggle, consists of 270 instances and 13 features like age, cholesterol, and family history. Data preprocessing involved mean imputation for missing values and min-max normalization. The study compares Random Forest with other ensemble classifiers—AdaBoost, Gradient Boosting, and XGBoost—using 10-fold cross-validation and evaluation metrics such as accuracy, precision, recall, and F1 score. Results show that Random Forest outperforms the other models with an accuracy of 87.04%, precision of 85.00%, recall of 80.95%, and F1 score of 82.93%. These findings emphasize Random Forest's ability to maintain prediction stability across various medical attributes and imbalanced data. Although the study highlights Random Forest as a promising method for early heart disease risk prediction, it remains a computational evaluation and requires clinical validation. The results aim to inform the development of predictive tools for enhancing early diagnosis and preventive strategies in healthcare systems.

Financial distress prediction: Regularized sparse-based Random Subspace with ER aggregation rule incorporating textual disclosures

The speedy development of the amount of medical health information available to the healthcare industry is driving demand for more accurate, precise, and reliable tools for exploiting these data. Real-world experience suggests that combining the predictions of individual machine learning classifiers may allow us to meet all three criteria at a low cost, potentially saving scarce funds, and patient’s lives. We, therefore, attempt to go beyond the existing literature on individual machine learning algorithms to discover the effectiveness of combining their predictions, and to explore the potential benefits of such ensemble methods to personalized diagnostics and prognostic medicine. Consideration of a range of classifiers is an essential aspect of getting the best predictive performance for a given dataset. For each of the ten biomedical datasets considered in this paper, a range of classifier ensemble methods performed competitively concerning individual classifiers. Furthermore, it was possible to reliably automate the ensemble methods, making them straightforward to apply to new datasets. Strikingly, we found that merely selecting the best-performing individual model was a powerful strategy. This strategy outperforming most of the model ‘compounding’ approaches leading to best or near-best predictive performance on all ten datasets. We conclude that while it is essential to consider a range of classifiers for a given dataset, simple strategies such as selecting the best-performing base classifier should not be discounted. Also, there may be significant merit in the development of formal methods for selecting between a wide range of different classifiers.

Achieving scalability of learning algorithms has become an increasingly critical issue in knowledge discovery from "big data". Sampling techniques can be exploited as one of the approaches to address the issue of scalability. We present in this paper a method to employ a newly developed sampling-based ensemble learning method by boosting for multi-class (non-binary) classification. This current research extends our previous work on multi-class classification with sampling-based ensemble learning method, in which the base classifiers are the most simplistic, such as decision stumps. Here we generalize the sampling-based method to handle more complex base classifiers such as decision trees in building an ensemble, which require sampling a set of instances before building a base classifier. We present experimental results using bench-mark data sets from the UC-Irvine ML data repository that confirm the efficiency and competitive prediction accuracy of the proposed adaptive boosting method for the multi-class classification task.