A rough margin-based multi-task [formula omitted]-twin support vector machine for pattern classification

Weighted Twin Support Vector Machines with Local Information and its application

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

Aiming at the sensitivity of twin support vector machines to noise and outliers, an improved fuzzy twin support vector machine is proposed. Aiming at the shortcomings of fuzzy membership based on the distance between a sample and its class center, this algorithm proposes a new fuzzy membership function that can effectively reflect sample uncertainty— based on intra-class hyperplane and sample affinity. When determining the membership function, the intra-class hyperplane replaces the class center, reducing the dependence on the sample set geometry and improving the model generalization ability. At the same time, considering the relation between each sample and the confusion degree of sample points, the sample affinity is applied to the design of membership function, so as to effectively distinguish the effective sample from noise and outliers and improve the classification accuracy. The experimental results show that compared with the twin support vector machine and the classic fuzzy support vector machine, the improved fuzzy twin support vector machine has better classification performance.

EEG signal classification using universum support vector machine

Event Abstract Back to Event High Performance EEG Signal Classification using Classifiability and the Twin SVM Sumit Soman1* and Jayadeva .1 1 Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, India Classification of Electroencephalogram (EEG) data for imagined motor movements has been a challenge in the design and development of Brain Computer Interfaces (BCIs). There are two principle challenges. The first is the variability in the recorded EEG data, which manifests across trials as well as across individuals. Consequently, features that are more discriminative need to be identified before any pattern recognition technique can be applied. The second challenge is in the pattern recognition domain. The number of data samples in a class of interest, e.g. a specific action, is a small fraction of the total data, which is composed of samples corresponding to all actions of all users. Building a robust classifier when learning from a highly unbalanced dataset is very difficult; minimizing the classification error typically causes the larger class to overwhelm the smaller one. We show that the combination of ‘classifiability’ for selecting the optimal frequency band and the use of the Twin Support Vector Machine (Twin SVM) for classification, yields significantly improved generalization. On benchmark BCI Competition datasets, the proposed approach often yields up to 20% improvement over the state-of-the-art. Figure 1 References Soman, Sumit and Jayadeva. "High performance EEG signal classification using classifiability and the Twin SVM." Applied Soft Computing Vol. 30 (May 2015): pp. 305-318 [http://www.sciencedirect.com/science/article/pii/S1568494615000204] Keywords: Brain Computer Interface, Classifiability, Support Vector Machines, Motor Imagery (MI), multiclass classification, Unbalanced data Conference: German-Japanese Adaptive BCI Workshop, Kyoto, Japan, 28 Oct - 29 Oct, 2015. Presentation Type: Poster presentation Topic: Adaptive BCI Citation: Soman S and . J (2015). High Performance EEG Signal Classification using Classifiability and the Twin SVM. Front. Comput. Neurosci. Conference Abstract: German-Japanese Adaptive BCI Workshop. doi: 10.3389/conf.fncom.2015.56.00020 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 18 Sep 2015; Published Online: 04 Nov 2015. * Correspondence: Mr. Sumit Soman, Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, Delhi, India, sumit.soman@gmail.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Sumit Soman Jayadeva . Google Sumit Soman Jayadeva . Google Scholar Sumit Soman Jayadeva . PubMed Sumit Soman Jayadeva . Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Support vector machine (SVM), as a supervised learning method, has different kinds of varieties with significant performance. In recent years, more research focused on nonparallel SVM, where twin SVM (TWSVM) is the typical one. In order to reduce the influence of outliers, more robust distance measurements are considered in these methods, but the discriminability of the models is neglected. In this article, we propose robust manifold twin bounded SVM (RMTBSVM), which considers both robustness and discriminability. Specifically, a novel norm, that is, capped L₁-norm, is used as the distance metric for robustness, and a robust manifold regularization is added to further improve the robustness and classification performance. In addition, we also use the kernel method to extend the proposed RMTBSVM for nonlinear classification. We introduce the optimization problems of the proposed model. Subsequently, effective algorithms for both linear and nonlinear cases are proposed and proved to be convergent. Moreover, the experiments are conducted to verify the effectiveness of our model. Compared with other methods under the SVM framework, the proposed RMTBSVM shows better classification accuracy and robustness.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

The condition monitoring and fault diagnosis of power transformers plays a significant role in the safe, stable and reliable operation of the whole power system. Dissolved gas analysis (DGA) methods are widely used for fault diagnosis, however, their accuracy is limited by the selection of DGA features and the performance of fault diagnosis models, for example, the classical support vector machine (SVM), is easily affected by unbalanced training samples. This paper presents a transformer fault diagnosis model based on chemical reaction optimization and a twin support vector machine. Twin support vector machines (TWSVMs) are used as classifiers for solving problems involving unbalanced and insufficient samples. Restricted Boltzmann machines (RBMs) are used for data preprocessing to ensure the effective identification of feature parameters and improve the efficiency and accuracy of fault diagnosis. The chemical reaction optimization (CRO) algorithm is used to optimize TWSVM parameters to select the optimal training parameters. The cross-validation (CV) method is used to ensure the reliability and generalization ability of the diagnostic model. Finally, the validity of the model is verified using real fault samples and random testing.

KNN weighted reduced universum twin SVM for class imbalance learning

Imbalanced datasets exist widely in real life. The identification of the minority class in imbalanced datasets tends to be the focus of classification. The twin support vector machine (TWSVM) as a variant of enhanced SVM provides an effective technique for data classification. In the paper, we propose to combine a re-sampling technique, which utilizes over-sampling and under-sampling to balance the training data, with TWSVM to deal with imbalanced data classification. Experimental results show that our proposed approach outperforms other state-of–art methods.

In this paper a new iterative approach is proposed for solving the Lagrangian formulation of twin support vector machine classifiers. The main advantage of our method is that rather than solving a quadratic programming problem as in the case of the standard support vector machine the inverse of a matrix of size equals to the number of input examples needs to be determined at the very beginning of the algorithm. The convergence of the algorithm is stated. Experiments have been performed on a number of interesting datasets. The predicted results are in good agreement with the observed values clearly demonstrates the applicability of the proposed method.

The traditional vector-based algorithms, such as Support Vector Machine (SVM) and Twin Support Vector Machine (TSVM), have many limitations especially when tensor is considered as input matrix. In this paper, we proposed a novel algorithm with a tensor-based classification paradigm to utilize the structural information. The new proposed algorithm is called  -Twin Support Tensor Machine ( -TSTM), which is an extension of  -Twin Support Vector Machine ( -TSVM). Similarly,  -TSTM solves a pair of smaller-sized Quadratic Programming Problems (QPPs). It reduces the computational complexity substantially. Besides, we formulate  -TSTM, which separates samples in the tensor space with two non-parallel hyperplanes, and the pair of parameters ( ) have theoretical interpretation which are used to control the bounds of the fractions of support tensors and the error margins. What’s more, the structure information of data is retained by the direct use of tensor representation. The proposed  -TSTM can preferably overcome overfitting problem and deal with big data while most vector-based algorithms could hardly compare. In addition, it has better performances on high dimensional and small-sample-size (S3) problem. The efficiency and superiority of the proposed method are demonstrated by experiments on various datasets.