- Research Article
144
- 10.1016/j.knosys.2013.01.008
Structural twin support vector machine for classification
- Jan 31, 2013
- Knowledge-Based Systems
- Zhiquan Qi + 2 more +2
Structural twin support vector machine for classification
The ν-nonparallel support vector machine (ν-SNPSVM) for classification has the advantage of using a parameter ν on controlling the number of support vectors. However, it ignores the prior structural information in data. In this paper, we propose a novel nonparallel classifier, named ν-Structural Nonparallel Support Vector Machine (ν-SNPSVM), for binary classification. Each model of ν-SNPSVM considers not only the compactness in both classes by the structural information but also the separability between classes, thus it can fully exploit prior knowledge to directly improve the algorithm's generalization capacity and is certainly superior to NPSVMs and other twin SVMs theoretically. Experimental results on lots of data sets show the effectiveness of our method.
Structural twin support vector machine for classification
Structural twin support vector machine for classification
Structural twin parametric-margin support vector machine for binary classification
Structural twin parametric-margin support vector machine for binary classification
Linear lJ-nonparallel support vector machine for pattern classification
In this paper, we propose a novel nonparallel linear hyperplane classifier called linear ν-nonparallel support vector machine (L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -ν-NPSVM) for binary classification. Based on L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -NPSVM (Linear Nonparallel Support Vector Machine), and combining the ν-support vector classification and ν-support vector regression together, the primal problem of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -ν-NPSVM is obtained. Compared to L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -NPSVM, L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -ν-NPSVM has the following advantages: (1) By introducing a new parameter ν to effectively control the number of support vectors, the model's generalization ability and accuracy can be improved; (2) By introducing a new parameter v, we can eliminate one of the other free parameters of the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -NPSVM to reduce the difficulty of selecting parameters. Moreover, experimental results on data sets show the effectiveness of our method.
Read moreNonparallel Support Vector Machines for Pattern Classification
We propose a novel nonparallel classifier, called nonparallel support vector machine (NPSVM), for binary classification. Our NPSVM that is fully different from the existing nonparallel classifiers, such as the generalized eigenvalue proximal support vector machine (GEPSVM) and the twin support vector machine (TWSVM), has several incomparable advantages: 1) two primal problems are constructed implementing the structural risk minimization principle; 2) the dual problems of these two primal problems have the same advantages as that of the standard SVMs, so that the kernel trick can be applied directly, while existing TWSVMs have to construct another two primal problems for nonlinear cases based on the approximate kernel-generated surfaces, furthermore, their nonlinear problems cannot degenerate to the linear case even the linear kernel is used; 3) the dual problems have the same elegant formulation with that of standard SVMs and can certainly be solved efficiently by sequential minimization optimization algorithm, while existing GEPSVM or TWSVMs are not suitable for large scale problems; 4) it has the inherent sparseness as standard SVMs; 5) existing TWSVMs are only the special cases of the NPSVM when the parameters of which are appropriately chosen. Experimental results on lots of datasets show the effectiveness of our method in both sparseness and classification accuracy, and therefore, confirm the above conclusion further. In some sense, our NPSVM is a new starting point of nonparallel classifiers.
Read moreStructural least square twin support vector machine for classification
The least square twin support vector machine (LS-TSVM) obtains two non-parallel hyperplanes by directly solving two systems of linear equations instead of two quadratic programming problems (QPPs) as in the conventional twin support vector machine (TSVM), which makes the computational speed of LS-TSVM faster than that of the TSVM. However, LS-TSVM ignores the structural information of data which may contain some vital prior domain knowledge for training a classifier. In this paper, we apply the prior structural information of data into the LS-TSVM to build a better classifier, called the structural least square twin support vector machine (S-LSTSVM). Since it incorporates the data distribution information into the model, S-LSTSVM has good generalization performance. Furthermore, S-LSTSVM costs less time by solving two systems of linear equations compared with other existing methods based on structural information. Experimental results on twelve benchmark datasets demonstrate that our S-LSTSVM performs well. Finally, we apply it into Alzheimer's disease diagnosis to further demonstrate the advantage of our algorithm.
Read moreStructural nonparallel support vector machine for pattern recognition
Structural nonparallel support vector machine for pattern recognition
A novel projection nonparallel support vector machine for pattern classification
A novel projection nonparallel support vector machine for pattern classification
Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier
Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of "structural granularity" and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities.
Read moreAn Efficient Angle-based Universum Least Squares Twin Support Vector Machine for Classification
Universum-based support vector machine incorporates prior information about the distribution of data in training of the classifier. This leads to better generalization performance but with increased computation cost. Various twin hyperplane-based models are proposed to reduce the computation cost of universum-based algorithms. In this work, we present an efficient angle-based universum least squares twin support vector machine (AULSTSVM) for classification. This is a novel approach of incorporating universum in the formulation of least squares-based twin SVM model. First, the proposed AULSTSVM constructs a universum hyperplane, which is proximal to universum data points. Then, the classifying hyperplane is constructed by minimizing the angle with the universum hyperplane. This gives prior information about data distribution to the classifier. In addition to the quadratic loss, we introduce linear loss in the optimization problem of the proposed AULSTSVM, which leads to lesser computation cost of the model. Numerical experiments are performed on several benchmark synthetic, real-world, and large-scale datasets. The results show that proposed AULSTSVM performs better than existing algorithms w.r.t. generalization performance as well as computation time. Moreover, an application to Alzheimer’s disease is presented, where AULSTSVM obtains accuracy of 95% for classification of healthy and Alzheimers subjects. The results imply that the proposed AULSTSVM is a better alternative for classification of large-scale datasets and biomedical applications.
Read moreSupervised laplacian graph multiple kernel classification
In supervised classification, many previous methods are model selections or parameters learning, not taking consideration of the structural information of data. Therefore in this paper, manifold regularization is used to obtain the structural information of data. Widely used method in computing manifold regularization is laplacian graph and it is usually unsupervised. Considering our problem is supervised classification, we propose a supervised method to compute the laplacian graph and use hellinger distance to measure the similarity of data samples, for hellinger distance can give a comprehensive evaluation of the relations of samples from four aspects, which are similarity, density, dimension and direction. Traditional linear model or support vector machine needs improving when dealing with datasets with many attributes or multi-source. Therefore, adaptable and flexible multiple kernel learning is proposed recently. Our supervised classification model uses manifold regularization to constrain multiple kernel classifier, incorporating the structural information, hoping to obtain a classifier which reaches the minimization of classification error and considers the structural information of data to have a reasonable view of the whole data. In experiments, we use UCI dataset to show the classification performance of the proposed model, and use a synthetic dataset to validate that adding manifold regularization can obtain parts of the structural information.
Read moreSupport Vector Machine and Kernel Classification Algorithms
This chapter introduces the basics of support vector machine (SVM) and other kernel classifiers for pattern recognition and detection. It also introduces the main elements and concept underlying the successful binary SVM. The chapter starts by introducing the main elements and concept underlying the successful binary SVM. Next, it introduces more advanced topics in SVM for classification, including large margin filtering (LMF), SSL, active learning, and large‐scale classification using SVMs. The LMF method performs both signal filtering and classification simultaneously by learning the most appropriate filters. SSL with SVMs exploits the information contained in both labeled and unlabeled examples. The chapter further illustrates the capabilities of the parallel SVM (PSVM) for classification of large‐scale real data problems. PSVM performs parallel IPM to solve the QP optimization problem, while the computation and the memory demands are improved with respect to other decomposition‐based algorithms.
Read moreWS-TWSVM: Weighted Structural Twin Support Vector Machine by local and global information
Recently many researches have published their papers on training a classifier based on the structural information of data. A Structural Twin Support Vector Machine (S-TWSVM) was proposed to introduce and balance all structural information of both intra-class and inter-class into its optimization problems. In fact, this method neither consider the structural information conflicting between clusters of one class nor the noise data points that can influence on the structure of the data distribution. In this paper, we propose a new Weighted Structural Twin Support Vector Machine (WS-TWSVM) by its local and global information. In our proposed method, we use a weighted (rather than a simple) summing of structural information to sufficiently exploit class's distribution information, so that, applying the density information of data points, the effects of the noise points on the data structure can be handled. Thus, our proposed method, WS-TWSVM can fully exploit the prior knowledge more efficiently than S-TWSVM leads to improve the model's generalization capacity. As been shown in the experiments, WS-TWSVM is superior to S-TWSVM in term of classification accuracy.
Read moreStructural Minimax Probability Machine.
Minimax probability machine (MPM) is an interesting discriminative classifier based on generative prior knowledge. It can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification. The structural information of data is an effective way to represent prior knowledge, and has been found to be vital for designing classifiers in real-world problems. However, MPM only considers the prior probability distribution of each class with a given mean and covariance matrix, which does not efficiently exploit the structural information of data. In this paper, we use two finite mixture models to capture the structural information of the data from binary classification. For each subdistribution in a finite mixture model, only its mean and covariance matrix are assumed to be known. Based on the finite mixture models, we propose a structural MPM (SMPM). SMPM can be solved effectively by a sequence of the second-order cone programming problems. Moreover, we extend a linear model of SMPM to a nonlinear model by exploiting kernelization techniques. We also show that the SMPM can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi-min margin machine under certain special conditions. Experimental results on both synthetic and real-world data sets demonstrate the effectiveness of SMPM.
Read moreVehicle Detection Algorithm Using Hypothesis Generation and Verification
In this paper, we present a two-stage vision-based approach to detect front and rear vehicle views in road scene images using eigenspace and a support vector machine for classification. The first stage is hypothesis generation (HG), in which potential vehicles are hypothesized. During the hypothesis generation step, we use a vertical, horizontal edge map to create potential regions where vehicles may be present. In the second stage verification (HV) step, all hypotheses are verified by using a Principle Component Analysis (PCA) for feature extraction and a Support Vector Machine (SVM) for classification, which is robust for both front and rear vehicle view detection problems. Our methods have been tested on different real road images and show very good performance.KeywordsAutonomous VehicleVision-basedHypothesis Generation (HG)Hypothesis Verification (HV)Principle Component Analysis (PCA)Support Vector Machine (SVM)
Read moreFisher-regularized supervised and semi-supervised extreme learning machine
The structural information of data contains useful prior knowledge and thus is important for designing classifiers. Extreme learning machine (ELM) has been a potential technique in handling classification problems. However, it only simply considers the prior class-based structural information and ignores the prior knowledge from statistics and geometry of data. In this paper, to capture more structural information of the data, we first propose a Fisher-regularized extreme learning machine (called Fisher-ELM) by applying Fisher regularization into the ELM learning framework, the main goals of which is to build an optimal hyperplane such that the output weight and within-class scatter are minimized simultaneously. The proposed Fisher-ELM reflects both the global characteristics and local properties of samples. Intuitively, the Fisher-ELM can approximatively fulfill the Fisher criterion and can obtain good statistical separability. Then, we exploit graph structural formulation to obtain semi-supervised Fisher-ELM version (called Lap-FisherELM) by introducing manifold regularization that characterizes the geometric information of the marginal distribution embedded in unlabeled samples. An efficient successive overrelaxation algorithm is used to solve the proposed Fisher-ELM and Lap-FisherELM, which converges linearly to a solution, and can process very large datasets that need not reside in memory. The proposed Fisher-ELM and Lap-FisherELM do not need to deal with the extra matrix and burden the computations related to the variable switching, which makes them more suitable for relatively large-scale problems. Experiments on several datasets verify the effectiveness of the proposed methods.
Read more