Distribution based ensemble for class imbalance learning

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise. Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, we present a method to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise. We thoroughly evaluated the proposed FSVM-CIL method on ten real-world imbalanced datasets and compared its performance with five existing CIL methods, which are available for normal SVM training. Based on the overall results, we can conclude that the proposed FSVM-CIL method is a very effective method for CIL, especially in the presence of outliers and noise in datasets.

The class imbalance problem has been said recently to hinder the performance of learning systems. Most of traditional learning algorithms are designed with the assumption of well-balanced datasets, and are biased towards the majority class and thus may predict poorly the minority class examples. In this paper, we develop weighted rough sets (WRS) to deal with this problem. In weighted rough sets, weighted entropy is introduced and extended to compute the information content introduced by attributes. A forward greedy weighted attribute reduction algorithm based on the weighted entropy and a weighted rule extraction algorithm are provided. The factors of weighted strength, weighted certainty and weighted cover are employed to evaluate the extracted rules. Finally, a decision algorithm based on the weighted strength factor is constructed. Based on weighted rough sets, a series of experiments on class imbalance learning are conducted on 20 UCI data sets. In the meaning of AUC and minority class accuracy, WRS achieves the better results than classical rough set in class imbalance learning. Moreover, the evaluation of extracted rules has greater influence than the selection of attributes on weighted rough set learning.

Sampling technique for noisy and borderline examples problem in imbalanced classification

The problem of class imbalance in the in-hospital mortality prediction for ICU patients is presented. We propose to build a novel predicting model using the balanced random forest (BRF) algorithm, and tune the hyper parameters using a better performance measure, i.e., adjusted geometric-mean. The performance of the model is evaluated using the data derived from the publicly available Medical Information Mart for Intensive Care (MIMIC-III) database. Our results show that the recall rate of the death class of ICU patients was significantly improved compared with the benchmarking model.

A machine learning framework to predict kidney graft failure with class imbalance using Red Deer algorithm

Online sparse class imbalance learning on big data

Automation in arrhythmia classification helps medical professionals to make accurate decisions upon the patient's health. Classification becomes complicated when class overlapping and class imbalance problem occurs together. The aim of this work is to improve the arrhythmia classification accuracy. Proposed methodology consists of fisher discriminant ratio based feature ranking stage and anomaly detection based training sample selection stage followed by classification using probabilistic neural network classifier. As per the recommendations of the Association for the Advancement of Medical Instrumentation, five arrhythmia classes were classified. The proposed method resulted in average sensitivity, positive predictive value and F Score of 95.37%, 98.35% and 96.72%, respectively. The experimental results revealed that: (1) Selected non-overlapping features were able to better discriminate arrhythmia classes, (2) Mixture of Gaussians based anomaly detection method suited well to handle the class imbalance problem and (3) Minority classes with few training samples were also correctly classified using the proposed method.

RuleCOSI: Combination and simplification of production rules from boosted decision trees for imbalanced classification

Textual stream classification has become a realistic and challenging issue since large-scale, high-dimensional, and non-stationary streams with class imbalance have been widely used in various real-life applications. According to the characters of textual streams, it is technically difficult to deal with the classification of textual stream, especially in imbalanced environment. In this paper, we propose a new ensemble framework, clustering forest, for learning from the textual imbalanced stream with concept drift (CFIM). The CFIM is based on ensemble learning by integrating a set of clustering trees (CTs). An adaptive selection method, which flexibly chooses the useful CTs by the property of the stream, is presented in CFIM. In particular, to deal with the problem of class imbalance, we collect and reuse both rare-class instances and misclassified instances from the historical chunks. Compared to most existing approaches, it is worth pointing out that our approach assumes that both majority class and rareclass may suffer from concept drift. Thus the distribution of resampled instances is similar to the current concept. The effectiveness of CFIM is examined in five real-world textual streams under an imbalanced nonstationary environment. Experimental results demonstrate that CFIM achieves better performance than four state-of-the-art ensemble models.

KNN weighted reduced universum twin SVM for class imbalance learning

The class imbalance problem has been said recently to hinder the performance of learning systems. In fact, many of them are designed with the assumption of well-balance datasets. However, it is very common to find higher presence of one of the classes in real classification problems. The aim of this paper is to make a preliminary analysis on the effect of the class imbalance problem in learning classifier systems. Particularly we focus our study on UCS, a supervised version of XCS classifier system. We analyze UCS's behavior on unbalanced datasets and find that UCS is sensitive to high levels of class imbalance. We study strategies for dealing with class imbalances, acting either at the sampling level or at the classifier system's level.

One of the factors that affect the performance of the classification predictive modelling task is the skewed distribution of the classes in the training data, which is known as the “class imbalance problem”. It hinders the performance of the predictive models. The contribution of this paper is threefold; primarily, this paper discusses the latest techniques used in different domains to handle class imbalance. Then, the taxonomy of data preprocessing, algorithm level and hybrid techniques have been presented. Finally, various evaluation strategies and application domains are presented. This paper covers all the latest and most effective techniques and elaborates on effective techniques such as cost-sensitive learning and ensemble learning, which collectively provide the reader with an extensive overview of the techniques used by researchers to alleviate skewed class distribution of the classes in the training data.KeywordsClass imbalance problemClassification predictive modellingData preprocessingCost-sensitive learningOne-class learningEnsemble learningHybrid learning techniquesEvaluation metrics

The class imbalance problem is a well-known classification challenge in machine learning that has vexed researchers for over a decade. Under-representation of one or more of the target classes (minority class(es)) as compared to others (majority class(es)) can restrict the application of conventional classifiers directly on the data. In addition, emerging challenges such as overlapping classes, make class imbalance even harder to solve. Class overlap is caused due to ambiguous regions in the data where the prior probability of two or more classes are approximately equal. We are motivated to address the challenge of class overlap in the presence of imbalanced classes by a problem in pervasive computing. Specifically, we are designing smart environments that perform health monitoring and assistance. Our solution, ClusBUS, is a clustering-based under sampling technique that identifies data regions where minority class samples are embedded deep inside majority class. By removing majority class samples from these regions, ClusBUS preprocesses the data in order to give more importance to the minority class during classification. Experiments show that ClusBUS achieves improved performance over an existing method for handling class imbalance.

Class imbalance and noisy data widely exist in real-world problems, and the support vector machine (SVM) is hard to construct good classifiers on these data. Fuzzy SVMs (FSVMs), as variants of SVM, use a fuzzy membership function both to reflect the samples’ importance and to remove the impact of noises, and employ cost-sensitive technology to address the class imbalance. They can handle the noise and class imbalance problems in many cases; however, the fuzzy membership functions are often affected by the class imbalance data, leading to inaccurate measures for samples’ performance and affecting the performance of FSVMs. To solve this problem, we design a new fuzzy membership function and combine it with cost-sensitive learning to deal with the class imbalance problem with noisy data, named Slack-Factor-based FSVM (SFFSVM). In SFFSVM, the relative distances between samples and an estimated hyperplane, called slack factors, are used to define the fuzzy membership function. To eliminate the impact of class imbalance on the function and gain more accurate samples’ importance, we rectify the importance according to the positional relationship between the estimated hyperplane and the optimal hyperplane of the problem, and the slack factors of samples. Comprehensive experiments on artificial and real-world datasets demonstrate that SFFSVM outperforms other comparative methods on F1, MCC, and AUC-PR metrics.

Objective.Accurate localization of the epileptogenic zone (EZ) is crucial for epilepsy surgery, but the class imbalance of epileptogenic vs. non-epileptogenic electrode contacts in intracranial electroencephalography (iEEG) data poses significant challenges for automatic localization methods. This review evaluates methodologies for handling the class imbalance in EZ localization studies that use machine learning (ML).Approach.We systematically reviewed studies employing ML to localize the EZ from iEEG data, focusing on strategies for addressing class imbalance in data handling, algorithm design, and evaluation.Results.Out of 2,128 screened studies, 35 fulfilled the inclusion criteria. Across the studies, the iEEG contacts annotated as epileptogenic prior to automatic localization constituted a median of 18.34% of all contacts. However, many of these studies did not adequately address the class imbalance problem. Techniques such as data resampling and cost-sensitive learning were used to mitigate the class imbalance problem, but the chosen evaluation metrics often failed to account for it.Significance.Class imbalance significantly impacts the reliability of EZ localization models. More comprehensive management and innovative approaches are needed to enhance the robustness and clinical utility of these models. Addressing class imbalance in ML models for EZ localization will improve both the predictive performance and reliability of these models.