Slack-Factor-Based Fuzzy Support Vector Machine for Class Imbalance Problems

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise. Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, we present a method to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise. We thoroughly evaluated the proposed FSVM-CIL method on ten real-world imbalanced datasets and compared its performance with five existing CIL methods, which are available for normal SVM training. Based on the overall results, we can conclude that the proposed FSVM-CIL method is a very effective method for CIL, especially in the presence of outliers and noise in datasets.

KNN weighted reduced universum twin SVM for class imbalance learning

BackgroundThe goal of class prediction studies is to develop rules to accurately predict the class membership of new samples. The rules are derived using the values of the variables available for each subject: the main characteristic of high-dimensional data is that the number of variables greatly exceeds the number of samples. Frequently the classifiers are developed using class-imbalanced data, i.e., data sets where the number of samples in each class is not equal. Standard classification methods used on class-imbalanced data often produce classifiers that do not accurately predict the minority class; the prediction is biased towards the majority class. In this paper we investigate if the high-dimensionality poses additional challenges when dealing with class-imbalanced prediction. We evaluate the performance of six types of classifiers on class-imbalanced data, using simulated data and a publicly available data set from a breast cancer gene-expression microarray study. We also investigate the effectiveness of some strategies that are available to overcome the effect of class imbalance.ResultsOur results show that the evaluated classifiers are highly sensitive to class imbalance and that variable selection introduces an additional bias towards classification into the majority class. Most new samples are assigned to the majority class from the training set, unless the difference between the classes is very large. As a consequence, the class-specific predictive accuracies differ considerably. When the class imbalance is not too severe, down-sizing and asymmetric bagging embedding variable selection work well, while over-sampling does not. Variable normalization can further worsen the performance of the classifiers.ConclusionsOur results show that matching the prevalence of the classes in training and test set does not guarantee good performance of classifiers and that the problems related to classification with class-imbalanced data are exacerbated when dealing with high-dimensional data. Researchers using class-imbalanced data should be careful in assessing the predictive accuracy of the classifiers and, unless the class imbalance is mild, they should always use an appropriate method for dealing with the class imbalance problem.

Comparison between error correcting output codes and fuzzy support vector machines

Support Vector Machine (SVM) is a supervised classification technique that uses the regularization parameter and Kernel function in deciding the best hyperplane to classify the data. SVM is sensitive to outliers, and it makes the model weak. To overcome the issue, the Fuzzy Support Vector Machine (FSVM) introduces fuzzy membership weight into its objective function, which focuses on grouping the fuzzy data points accurately. Knowing the importance of the membership weights in FSVM, we have introduced four new expressions to compute the FSVM membership weights in this study. They are determined from the Fuzzy C-means Algorithm's membership values (FCM). The performances of FSVM with three different kernels are assessed in terms of accuracy. The experiments are conducted for various combinations of FSVM parameters, and the best combinations for each kernel are highlighted. Six benchmark datasets are used to demonstrate the performance of FSVM and the proposed models’ performance are compared with the existing models in recent literature.

Support Vector Machines (SVMs) are considered to be one of the most powerful classification tools, widely used in many applications. However, in numerous scenarios the classes are not equally represented and the predictive performance of SVMs on such data can drop dramatically. Different methods have been proposed to address moderate class imbalance issues, but there are few methods that can be successful at detecting the minority class while also keeping high accuracy, especially when applied to datasets with significant level of imbalance. In this paper, we consider SVM ensembles that are built by using a bootstrap-based undersampling technique. We target to reduce the bias induced by class imbalances via multiple undersampling procedures and then decrease the variance using SVM ensembles. For combining the SVMs, we propose a new technique that deals with class imbalance problems of varying levels. Experiments on several datasets demonstrate the performance of the proposed scheme compared to state-of-the-art balancing methods.

The main problem in producing high accuracy software defect prediction is if the data set has an imbalance class and dichotomous characteristics. The imbalanced class problem can be solved using a data level approach, such as resampling methods. While the problem of software defects predicting if the data set has dichotomous characteristics can be approached using the classification method. This study aimed to analyze the performance of the proposed software defect prediction method to identify the best combination of resampling methods with the appropriate classification method to provide the highest accuracy. The combination of the proposed methods first is the resampling process using oversampling, under-sampling, or hybrid methods. The second process uses the classification method, namely the Support Vector Machine (SVM) algorithm and the Logistic Regression (LR) algorithm. The proposed, tested model uses five NASA MDP data sets with the same number attributes of 37. Based on the t-test, the < = 0.0344 < 0.05 and the > = 3.1524 > 2.7765 which indicates that the combination of the proposed methods is suitable for classifying imbalanced class. The performance of the classification algorithm has also improved with the use of the resampling process. The average increase in AUC values using the resampling in the SVM algorithm is 17.19%, and the LR algorithm is at 7.26% compared to without the resampling process. Combining the three resampling methods with the SVM algorithm and the LR algorithm shows that the best combining method is the oversampling method with the SVM algorithm to software defects prediction in imbalanced class with an average accuracy value of 84.02% and AUC 91.65%.

While analyzing health data is important for improving health outcomes, class imbalance in datasets poses major challenges to machine learning classification models. This work, therefore, considers the class imbalance problem in stroke prediction using models such as K‐nearest neighbors, support vector machine, logistic regression, random forest, and decision tree. This work balances the stroke dataset, thereby enhancing model performance, through various oversampling strategies: random oversampling (RO), ADASYN, SMOTE, and SMOTE–Tomek. Compared to the results of the imbalanced dataset, all applied oversampling techniques enhanced the correct classification of stroke events by the ML model. Among these, RO–SVM with RBF kernel was the best in terms of sensitivity, specificity, G‐mean, F1‐score, and accuracy values, offering the highest results with respective values of 89.87%, 94.91%, 92.36%, 89.64%, and 89.87%. After applying oversampling techniques, all the machine learning classifications were good enough to classify stroke status, especially for the minority class. This study has highlighted the importance of class imbalance issues in health datasets. Precise detection of instances of minority classes can be enhanced considerably by employing classification models with the implementation of hybrid strategies to effectively solve class imbalance issues, which, in turn, will help improve healthcare outcomes. Further research in integrating more advanced deep learning techniques into other health datasets with imbalances is encouraged to further validate or refine class imbalance approaches, as effective handling of imbalanced classes can substantially promote predictive model performance in the analysis of healthcare.

Credit risk evaluation using multi-criteria optimization classifier with kernel, fuzzification and penalty factors

The fuzzy support vector machine (FSVM) assigns each sample a fuzzy membership value based on its relevance, making it less sensitive to noise or outliers in the data. Although FSVM has had some success in avoiding the negative effects of noise, it uses hinge loss, which maximizes the shortest distance between two classes and is ineffective in dealing with feature noise near the decision boundary. Furthermore, whereas FSVM concentrates on misclassification errors, it neglects to consider the critical within-class scatter minimization. We present a Fuzzy support vector machine with pinball loss (FPin-SVM), which is a fuzzy extension of a reformulation of a recently proposed support vector machine with pinball loss (Pin-SVM) with several significant improvements, to improve the performance of FSVM. First, because we used the squared L2- norm of errors variables instead of the L1 norm, our FPin-SVM is a strongly convex minimization problem; second, to speed up the training procedure, solutions of the proposed FPin-SVM, as an unconstrained minimization problem, are obtained using the functional iterative and Newton methods. Third, it is proposed to solve the minimization problem directly in primal. Unlike FSVM and Pin-SVM, our FPin-SVM does not require a toolbox for optimization. We dig deeper into the features of FPin-SVM, such as noise insensitivity and within-class scatter minimization. We conducted experiments on synthetic and real-world datasets with various sounds to validate the usefulness of the suggested approach. Compared to the SVM, FSVM, and Pin-SVM, the presented approaches demonstrate equivalent or superior generalization performance in less training time.

Multi-kernel multi-criteria optimization classifier with fuzzification and penalty factors for predicting biological activity

Imbalanced data classification is still remaining thje important topic and during the past decades, plenty of works are devoted to this field of study. More and more real-life based imbalanced class problems inspired researchers to come up with new solutions with better performance. Various techniques are employed such as data handling approaches, algorithm-level approaches, active learning approaches, and kernel-based methods to enumerate only a few. This work aims at applying a novel dynamic selection methods on imbalanced data classification problems. The experiments carried out on several benchmark datasets confirm its pretty high performance.

Imbalanced classification problem is a hot issue in data mining and machine learning. Traditional classification algorithms are proposed based on some form of symmetry hypothesis of class distribution, whose main purpose is to improve the overall classification performance. It is difficult to obtain ideal classification result when handling imbalanced datasets. In order to improve the classification performance of imbalanced datasets, this paper proposes a cluster-based under-sampling algorithm (CUS) according to the important characteristic of support vector machines (SVM) classification relying on support vector. Firstly, majority class is divided into different clusters using improved clustering by fast search and find of density peaks (CFSFDP) algorithm. The improved clustering algorithm can realize automatic selection of clustering centers, which overcomes the limitation of the original algorithm. Then the minority class and each cluster of the majority class are used to construct training set to get the support vector of each cluster by support vector machine. Retaining support vectors for each cluster and deleting non-support vectors are to construct a new majority class sample points to obtain relatively balanced datasets. Finally, the new datasets are classified by support vector machines and the performance is evaluated by cross validation sets. The experimental results show that CUS algorithm is effective.

The class imbalance problem is a well-known classification challenge in machine learning that has vexed researchers for over a decade. Under-representation of one or more of the target classes (minority class(es)) as compared to others (majority class(es)) can restrict the application of conventional classifiers directly on the data. In addition, emerging challenges such as overlapping classes, make class imbalance even harder to solve. Class overlap is caused due to ambiguous regions in the data where the prior probability of two or more classes are approximately equal. We are motivated to address the challenge of class overlap in the presence of imbalanced classes by a problem in pervasive computing. Specifically, we are designing smart environments that perform health monitoring and assistance. Our solution, ClusBUS, is a clustering-based under sampling technique that identifies data regions where minority class samples are embedded deep inside majority class. By removing majority class samples from these regions, ClusBUS preprocesses the data in order to give more importance to the minority class during classification. Experiments show that ClusBUS achieves improved performance over an existing method for handling class imbalance.

In order to reduce the higher training cost of support vector machine (SVM) and its sensitivity towards noise and outliers, two fuzzy based approaches are proposed in this paper. The proposed approaches are based on least squares twin support vector machine (LSTWSVM) and fuzzy support vector machine (FSVM). The effects of noise and outliers are reduced by assigning lower membership values to the data points which are away from the class centers. Further, 2-norm of the slack vectors of the LSTWSVM formulation is taken after multiplying to their respective diagonal matrices of the membership values to effectively utilize the fuzzy membership principle and to make the optimization problem strongly convex. Moreover, the proposed approaches solve linear equations instead of quadratic programming problems which help in training faster. The effectiveness of the proposed approaches are established by comparing the classification accuracies and training time with support vector machine, fuzzy support vector machine, twin support vector machine and least squares twin support vector machine.