Fault Detection of Bearings Using a Rule-based Classifier Ensemble and Genetic Algorithm

To predict potential problems and avoid an unexpected breakdown of rotating machinery, a rule-based classifier ensemble approach is presented. Feature reduction was first implemented on a fault decision table using discernibility matrices and the genetic algorithm. The generated rules of the reducts were used to build the candidate base classifiers. Then, several base classifiers were selected according to their diversity and scale. The weights of the selected base classifiers were also calculated based on the support rate measurements. A classifier ensemble was constructed through an integration of the base classifiers using an improved weighted voting technique. Finally, the proposed classifier ensemble was verified based on the vibration data of bearing types SKF6203 and NU205. The accuracy for the SKF6203 bearing type reached 88.75 %, which is at least 5 % higher than that of the three base classifiers for this type of bearing. In addition, the recognition rate for the latter bearing type was 90 %. The reasoning process was much easier to comprehend owing to the semantic descriptions of the rules. The results show that this is a promising and transparent approach for diagnosing typical faults of rotating machinery.

Diversity is a key characteristic of a classifier ensemble. A classifier ensemble must be composed of base classifiers with different performance in different areas of the problem space. Several works studied different diversity measures by performing extensive numerical experiments. However, up to our knowledge, no method has been proposed to visualize the diversity of a classifier ensemble. In this contribution, we propose a novel approach to visualize and analyze classifier ensembles diversity. Our novel proposal (called DivGrams), allows the visualization of the diversity between base classifiers in the classifier ensemble by using a network representation. The nodes represent base classifiers and its edges represent diversity relations between the base classifiers. We apply DivGrams to fuzzy rule-based classifier ensembles, as in our previous work these machine learning models proved to deal well with complex classification problems. A preliminary experimental study is performed using two UCI datasets in order to show the usefulness of DivGrams. Furthermore, we show a practical application of DivGrams to develop manual classifier selection on eight UCI datasets. The results obtained show the interesting characteristics of our proposal and support this novel approach in the fuzzy classifier ensemble field.

Zelenkov et al. [1] proposed a two-step classification method (TSCM) based on genetic algorithm to predict the bankruptcy of Russian companies. This current study was conducted to do more comprehensive evaluation than Zelenkov et al. [1] did. We involved more datasets, compared to a greater number of competitive methods, and developed the ensemble using more base classifiers. The datasets consisted of nine datasets and then the result of prediction compared to previous studies from 13 papers that were published between 1996 and 2009. In addition, the previous method using five base classifiers, this study involved seven base classifiers. The purpose of this study is to compare the accuracy of prediction from previous studies with this method. In this study, we used k-Nearest Neighbour (k-NN), Logistic Regression (LR), Naïve Bayes (NB), Decision Tree (DT), support vector machine (SVM), random forest (RF), and boosting as the base classifiers of ensemble model. Genetic algorithm is used to find the best predictor variables and the best weight for each base classifier. The result of TSCM shows that the accuracy of prediction can increase about 0-34% for some datasets compared to the previous studies.

Network intrusion behaviour data is the imbalanced data. It includes a large amount of normal behavior data and a small amount of intrusion behavior data. The traditional selective ensemble learning algorithm will lead to high false negative rate. This paper proposes a selective ensemble learning algorithm for imbalanced data based on under sampling (SELAUS). First of all, the algorithm uses Bootstrap method to extract samples equal to the number of samples of a few classes from majority class samples to construct multiple balanced training subsets. Then, in order to ensure that the obtained base classifiers have great differences, several features are randomly selected on the training subset and a decision tree is constructed as the base classifier using CART algorithm. This method can also make some base classifiers have poor performance, so it can select and integrate base classifiers instead of all base classifiers. In order to accurately evaluate the generalization error of the classifier for imbalanced dataset, this paper defines the performance evaluation method in the imbalanced dataset and the difference evaluation method between the base classifiers. Then the generalization error of each base classifier is calculated, and the base classifier is selected according to the generalization error. In the integration of weighted voting, the weight of each base classifier is calculated by the weight calculation method for imbalanced data. Finally, the validity of the algorithm is verified by UCI dataset and applied to network intrusion detection. The simulation results show that the algorithm can improve the detection rate of minority class samples, that is to say, reduce the false negative rate.

Non-parametric classification procedures based on a certainty measure and nearest neighbour rule for motor unit potential classification (MUP) during electromyographic (EMG) signal decomposition were explored. A diversity-based classifier fusion approach is developed and evaluated to achieve improved classification performance. The developed system allows the construction of a set of non-parametric base classifiers and then automatically chooses, from the pool of base classifiers, subsets of classifiers to form candidate classifier ensembles. The system selects the classifier ensemble members by exploiting a diversity measure for selecting classifier teams. The kappa statistic is used as the diversity measure to estimate the level of agreement between base classifier outputs, i.e., to measure the degree of decision similarity between base classifiers. The pool of base classifiers consists of two kinds of classifiers: adaptive certainty-based classifiers (ACCs) and adaptive fuzzy k-NN classifiers (AFNNCs) and both utilize different types of features. Once the patterns are assigned to their classes, by the classifier fusion system, firing pattern consistency statistics for each class are calculated to detect classification errors in an adaptive fashion. Performance of the developed system was evaluated using real and simulated EMG signals and was compared with the performance of the constituent base classifiers and the performance of the fixed ensemble containing the full set of base classifiers. Across the EMG signal data sets used, the diversity-based classifier fusion approach had better average classification performance overall, especially in terms of reducing classification errors.

Multiple Classifier System (MCS) has attracted increasing interest in the field of pattern recognition and machine learning where this technique has also been introduced in remote sensing. The importance of classifier diversity in MCS has been raised recently; nevertheless, only a few of the researches have been studied in land cover classification problem. In this paper, a SPOT IV satellite image covering the Hong Kong Island and Kowloon Peninsula with six land cover classes were classified with four base classifiers: Minimum Distance Classifier, Maximum Likelihood Classifier, Mahalanobis Classifier and K-Nearest Neighbor Classifier. Same training and testing data sets were applied throughout the experiments and five Bayesian decision rules, including product rule, sum rule, max rule, min rule, and median rule, were utilized to construct different ensemble of classifiers. Performance of MCS was measured using the overall accuracy and kappa statistics, and three statistical tests including McNemarpsilas test, Cochranpsilas Q test and F-test were introduced to examine the dependence of the classification results. The experimental comparison reveals that (i) increasing the number of base classifiers may not improve the overall accuracy in MCS, (ii) significant diversity in base classifiers cannot enhance the overall performance and vice versa. These findings are noted with the condition in using the same data set and the same training set.

In this paper, a new method for enhancing the performance of Neural Network ensemble is proposed. The main idea of this method is creating diversity for training artificial neural networks (ANNs) using an interesting method which applies clustering ensemble and genetic algorithm. In combinational classifier systems, the more diversity in results of the base classifiers yields to better final performance. Inspiring from the boosting, the diversity of the base classifiers is provided by different train sets for base classifiers. The different train sets are derived from the original train set by adding some of data samples in train set. Finding near optimal sets is implemented using clustering ensemble technique and genetic algorithm. Finally, the majority vote fuses the outputs of trained MLPs on the new train sets from population of the last generation of GA. Experimental results demonstrate the strength of proposed method on three different datasets.

Detecting harmful content or hate speech on social media is a significant challenge due to the high throughput and large volume of content production on these platforms. Identifying hate speech in a timely manner is crucial in preventing its dissemination. We propose a novel stacked ensemble approach for detecting hate speech in English tweets. The proposed architecture employs an ensemble of three classifiers, namely support vector machine (SVM), logistic regression (LR), and XGBoost classifier (XGB), trained using word2vec and universal encoding features. The meta classifier, LR, combines the outputs of the three base classifiers and the features employed by the base classifiers to produce the final output. It is shown that the proposed architecture improves the performance of the widely used single classifiers as well as the standard stacking and classifier ensemble using majority voting. We also present results on the use of various combinations of machine learning classifiers as base classifiers. The experimental results from the proposed architecture indicated an improvement in the performance on all four datasets compared with the standard stacking, base classifiers, and majority voting. Furthermore, on three of these datasets, the proposed architecture outperformed all state-of-the-art systems.

An integrated TOPSIS crow search based classifier ensemble: In application to stock index price movement prediction

Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms

With the rising demand for integrated and autonomous systems in the field of engineering, efficient frameworks for instant detection of performance anomalies are imperative for improved productivity and cost-effectiveness. This study proposes a systematic predictive maintenance framework based on the hybrid multisensor fusion technique of fuzzy rough set feature selection and stacked ensemble for the efficient classification of fault conditions characterised by uncertainties. First, a feature vector of time-domain features was extracted from 17 multiple sensor signals. Then, a comparative study of six different Fuzzy Rough Set Feature Selection (FRFS) methods was employed to select the various combinations of optimal feature subsets for various faults classification tasks. The determined optimal feature subsets then served as inputs for training the stacked ensemble (ESB(STK)). In the ESB(STK), Support Vector Machine (SVM), Multilayer Perceptron (MLP), k -Nearest Neighbour ( k -NN), C4.5 Decision Tree (C4.5 DT), Logistic Regression (LR), and Linear Discriminant Analysis (LDA) served as the base classifiers while the LR was selected to be the metaclassifier. The proposed hybrid framework (FRFS-ESB(STK)) improved the classification accuracy with the selected combinations of optimal feature subset size whiles reducing the computational cost, overfitting, training runtime, and uncertainty in modelling. Overall analyses showed that the FRFS-ESB(STK) proved to be generalisable and versatile in the classification of all conditions of four monitored hydraulic components (i.e., cooler, valve, accumulator, and internal pump leakage) when compared with the six base classifiers (standalone) and three existing ensemble classifiers (Stochastic Gradient Boosting (SGB), Ada Boost (ADB), and Bagging (BAG)). The proposed FRFS-ESB(STK) showed an average improvement of 11.28% and 0.88% test accuracies when classifying accumulator and pump conditions, respectively, whiles 100% classification rates were obtained for both cooler and valve.

This paper introduces a novel concept for creating an ensemble of classifiers. The concept is based on generating an ensemble of classifiers through clustering of data at multiple layers. The ensemble classifier model generates a set of alternative clustering of a dataset at different layers by randomly initializing the clustering parameters and trains a set of base classifiers on the patterns at different clusters in different layers. A test pattern is classified by first finding the appropriate cluster at each layer and then using the corresponding base classifier. The decisions obtained at different layers are fused into a final verdict using majority voting. As the base classifiers are trained on overlapping patterns at different layers, the proposed approach achieves diversity among the individual classifiers. Identification of difficult-to-classify patterns through clustering as well as achievement of diversity through layering leads to better classification results as evidenced from the experimental results.

A novel ensemble of classifiers that use biological relevant gene sets for microarray classification

Ensembles need their base classifiers do not always agree for any prediction (diverse base classifiers). Disturbing Neighbors ($\mathcal{DN}$) is a method for improving the diversity of the base classifiers of any ensemble algorithm. $\mathcal{DN}$ builds for each base classifier a set of extra features based on a 1-Nearest Neighbors (1-NN) output. These 1-NN are built using a small subset of randomly selected instances from the training dataset. $\mathcal{DN}$ has already been proved successfully on unstable base classifiers (i.e. decision trees). This paper presents an experimental validation on 62 UCI datasets for standard ensemble methods using Support Vector Machines (SVM) with a linear kernel as base classifiers. SVMs are very stable, so it is hard to increase their diversity when they belong to an ensemble. However, experiments will show that $\mathcal{DN}$ usually improves ensemble accuracy and base classifiers diversity.

Decisions of multiple hypotheses are combined in ensemble learning to produce more accurate and less risky results. In this article, we present a novel ensemble machine learning approach for the development of robust thermo stable protein discrimination. But unlike widely used ensemble approaches in which bootstrapped training data are used, we keep the original data unchanged. Instead, we build an ensemble of base classifiers that each of them uses a division of features (called feature group) to predict the class label of each sample. Then, we try to learn the base classifiers' outputs (behavior) for different samples. By testing the proposed method on a well-known dataset, we show that our ensemble method is comparable in precision, recall and f-measure to the state of the art classifiers.