Combining Re-Sampling with Twin Support Vector Machine for Imbalanced Data Classification

Kreĭn twin support vector machines for imbalanced data classification

KNN weighted reduced universum twin SVM for class imbalance learning

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Imbalanced data classification is a challenging task in real applications. In this work. A method is proposed for image classification using imbalanced distribution of classes. The proposed method extracts an appropriate feature vector which can be fed into a simple classifier such as support vector machine (SVM) for classification. At first, the clustering-based feature extraction (CBFE) is applied to initially reduce the feature space. Then, the relationships among each considered sample with the minority classes and majority classes are explored through constructing two individual graphs. The Laplacian matrices of these graphs are combined to provide a dedicated projection matrix for each sample under test while preserve the manifold structure of minority and majority classes. The initial feature reduction just needs the first order data statistics and the second graph-based projection is unsupervised. Moreover, SVM is not sensitive to the training set size. So, the proposed method is not only efficient for imbalanced data classification but also it is efficient for data classification in small sample size situations. Different subsets of SVHN and CIFAR-10 datasets with various imbalance ratios are used for experiments. The results show superior performance of the proposed method compared to SVM and convolutional neural network (CNN). No data augmentation or oversampling is done for balancing the data distribution in the proposed method. But, due to weak performance of CNN with imbalanced datasets, the image augmentation is applied for CNN as the competitor. However, the proposed method without data augmentation provides better results than CNN with data augmentation.

A rough margin-based multi-task [formula omitted]-twin support vector machine for pattern classification

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

Event Abstract Back to Event High Performance EEG Signal Classification using Classifiability and the Twin SVM Sumit Soman1* and Jayadeva .1 1 Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, India Classification of Electroencephalogram (EEG) data for imagined motor movements has been a challenge in the design and development of Brain Computer Interfaces (BCIs). There are two principle challenges. The first is the variability in the recorded EEG data, which manifests across trials as well as across individuals. Consequently, features that are more discriminative need to be identified before any pattern recognition technique can be applied. The second challenge is in the pattern recognition domain. The number of data samples in a class of interest, e.g. a specific action, is a small fraction of the total data, which is composed of samples corresponding to all actions of all users. Building a robust classifier when learning from a highly unbalanced dataset is very difficult; minimizing the classification error typically causes the larger class to overwhelm the smaller one. We show that the combination of ‘classifiability’ for selecting the optimal frequency band and the use of the Twin Support Vector Machine (Twin SVM) for classification, yields significantly improved generalization. On benchmark BCI Competition datasets, the proposed approach often yields up to 20% improvement over the state-of-the-art. Figure 1 References Soman, Sumit and Jayadeva. "High performance EEG signal classification using classifiability and the Twin SVM." Applied Soft Computing Vol. 30 (May 2015): pp. 305-318 [http://www.sciencedirect.com/science/article/pii/S1568494615000204] Keywords: Brain Computer Interface, Classifiability, Support Vector Machines, Motor Imagery (MI), multiclass classification, Unbalanced data Conference: German-Japanese Adaptive BCI Workshop, Kyoto, Japan, 28 Oct - 29 Oct, 2015. Presentation Type: Poster presentation Topic: Adaptive BCI Citation: Soman S and . J (2015). High Performance EEG Signal Classification using Classifiability and the Twin SVM. Front. Comput. Neurosci. Conference Abstract: German-Japanese Adaptive BCI Workshop. doi: 10.3389/conf.fncom.2015.56.00020 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 18 Sep 2015; Published Online: 04 Nov 2015. * Correspondence: Mr. Sumit Soman, Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, Delhi, India, sumit.soman@gmail.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Sumit Soman Jayadeva . Google Sumit Soman Jayadeva . Google Scholar Sumit Soman Jayadeva . PubMed Sumit Soman Jayadeva . Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

The condition monitoring and fault diagnosis of power transformers plays a significant role in the safe, stable and reliable operation of the whole power system. Dissolved gas analysis (DGA) methods are widely used for fault diagnosis, however, their accuracy is limited by the selection of DGA features and the performance of fault diagnosis models, for example, the classical support vector machine (SVM), is easily affected by unbalanced training samples. This paper presents a transformer fault diagnosis model based on chemical reaction optimization and a twin support vector machine. Twin support vector machines (TWSVMs) are used as classifiers for solving problems involving unbalanced and insufficient samples. Restricted Boltzmann machines (RBMs) are used for data preprocessing to ensure the effective identification of feature parameters and improve the efficiency and accuracy of fault diagnosis. The chemical reaction optimization (CRO) algorithm is used to optimize TWSVM parameters to select the optimal training parameters. The cross-validation (CV) method is used to ensure the reliability and generalization ability of the diagnostic model. Finally, the validity of the model is verified using real fault samples and random testing.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

Laplacian least squares twin support vector machine for semi-supervised classification

Least squares twin support vector machine (LSTSVM) is a new machine learning method, as opposed to solving two quadratic programming problems in twin support vector machine (TWSVM), which generates two nonparallel hyperplanes by solving a pair of linear system of equations. However, LSTSVM obtains the resultant classifier by giving same importance to all training samples which may be important for classification performance. In this paper, by considering the fuzzy membership value for each sample, we propose an entropy-based fuzzy least squares twin support vector machine where fuzzy membership values are assigned based on the entropy values of all training samples. The proposed method not only retains the superior characteristics of LSTSVM which is simple and fast algorithm, but also implements the structural risk minimization principle to overcome the possible over- fitting problem. Experiments are performed on several synthetic as well as benchmark datasets and the experimental results illustrate the effectiveness of our method.

EEG signal classification using universum support vector machine

An Experimental Assessment of Random Forest Classification Performance Improvisation with Sampling and Stage Wise Success Rate Calculation

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

In this paper a new iterative approach is proposed for solving the Lagrangian formulation of twin support vector machine classifiers. The main advantage of our method is that rather than solving a quadratic programming problem as in the case of the standard support vector machine the inverse of a matrix of size equals to the number of input examples needs to be determined at the very beginning of the algorithm. The convergence of the algorithm is stated. Experiments have been performed on a number of interesting datasets. The predicted results are in good agreement with the observed values clearly demonstrates the applicability of the proposed method.