Laplacian least squares twin support vector machine for semi-supervised classification

In this paper, we propose two Laplacian nonparallel hyperplane proximal classifiers (LapNPPCs) for semi-supervised and full-supervised classification problem respectively by adding manifold regularization terms. Due to the manifold regularization terms, our LapNPPCs are able to exploit the intrinsic structure of the patterns of the training set. Furthermore, our classifiers only need to solve two systems of linear equations rather than two quadratic programming (QP) problems as needed in Laplacian twin support vector machine (LapTSVM) (Z. Qi, Y. Tian and Y. Shi, Neural Netw.35 (2012) 46–53). Numerical experiments on toy and UCI benchmark datasets show that the accuracy of our LapNPPCs is comparable with other classifiers, such as the standard SVM, TWSVM and LapTSVM, etc. It is also the case that based on our LapNPPCs, some other TWSVM type classifiers with manifold regularization can be constructed by choosing different norms and loss functions to deal with semi-supervised binary and multi-class classification problems.

Fast Laplacian twin support vector machine with active learning for pattern classification

Weighted Twin Support Vector Machines with Local Information and its application

Network security has become one of the well-known concerns in the last decades. Machine learning techniques are robust methods in detecting malicious activities and network threats. Most previous works learn offline supervised classifiers while they require large amounts of labeled examples and also should update models because the data change over time in real world applications. To alleviate these problems, we propose a novel online version of laplacian twin support vector machine classifier, which can exploit the geometry information of the marginal distribution embedded in unlabeled data to construct a more accurate and faster semi-supervised classifier. The results of experiments on large network datasets show that Online Lap-TSVM combined by two nonparallel hyper planes improves the accuracy with the comparable computing time and storage to Lap-TSVM.

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

Kreĭn twin support vector machines for imbalanced data classification

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Sparse pinball twin support vector machines

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

Color image classification and retrieval through ternary decision structure based multi-category TWSVM

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

In this paper, a novel nonparallel hyperplane based classifier termed as “angle-based twin support vector machine” (ATWSVM) has been proposed which is motivated by the concept of twin support vector machine (TWSVM). TWSVM obtains two nonparallel hyperplanes by solving a pair of quadratic programming problems (QPPs). ATWSVM presents a generic classification model, where the first problem can be formulated using a TWSVM-based classifier and the second problem is an unconstrained minimization problem (UMP) which is reduced to solving a system of linear equations. The second hyperplane is determined so that it is proximal to its own class and the angle between the normals to the two hyperplanes is maximized. The notion of angle has been introduced to have maximum separation between the two hyperplanes. In this work, we have presented two versions of ATWSVM: one that solves a QPP and a UMP; second which formulates both the problems as UMPs. The training time of ATWSVM is much less than that of TWSVM because ATWSVM solves the second problem as UMP instead of QPP. To test the efficacy of the proposed classifier, experiments have been conducted on synthetic and benchmark datasets and it is observed that the proposed classifier achieves classification accuracy comparable or better than that of TWSVM. This work also proposes application of ATWSVM for color image segmentation.

The traditional vector-based algorithms, such as Support Vector Machine (SVM) and Twin Support Vector Machine (TSVM), have many limitations especially when tensor is considered as input matrix. In this paper, we proposed a novel algorithm with a tensor-based classification paradigm to utilize the structural information. The new proposed algorithm is called  -Twin Support Tensor Machine ( -TSTM), which is an extension of  -Twin Support Vector Machine ( -TSVM). Similarly,  -TSTM solves a pair of smaller-sized Quadratic Programming Problems (QPPs). It reduces the computational complexity substantially. Besides, we formulate  -TSTM, which separates samples in the tensor space with two non-parallel hyperplanes, and the pair of parameters ( ) have theoretical interpretation which are used to control the bounds of the fractions of support tensors and the error margins. What’s more, the structure information of data is retained by the direct use of tensor representation. The proposed  -TSTM can preferably overcome overfitting problem and deal with big data while most vector-based algorithms could hardly compare. In addition, it has better performances on high dimensional and small-sample-size (S3) problem. The efficiency and superiority of the proposed method are demonstrated by experiments on various datasets.

Least squares twin support vector machine (LSTSVM) is a new machine learning method, as opposed to solving two quadratic programming problems in twin support vector machine (TWSVM), which generates two nonparallel hyperplanes by solving a pair of linear system of equations. However, LSTSVM obtains the resultant classifier by giving same importance to all training samples which may be important for classification performance. In this paper, by considering the fuzzy membership value for each sample, we propose an entropy-based fuzzy least squares twin support vector machine where fuzzy membership values are assigned based on the entropy values of all training samples. The proposed method not only retains the superior characteristics of LSTSVM which is simple and fast algorithm, but also implements the structural risk minimization principle to overcome the possible over- fitting problem. Experiments are performed on several synthetic as well as benchmark datasets and the experimental results illustrate the effectiveness of our method.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.