- Research Article
39
- 10.1016/j.patcog.2016.04.017
Structural nonparallel support vector machine for pattern recognition
- May 17, 2016
- Pattern Recognition
- Dandan Chen + 2 more +2
Structural nonparallel support vector machine for pattern recognition
Structural twin support vector machine for classification
Structural nonparallel support vector machine for pattern recognition
Structural nonparallel support vector machine for pattern recognition
WS-TWSVM: Weighted Structural Twin Support Vector Machine by local and global information
Recently many researches have published their papers on training a classifier based on the structural information of data. A Structural Twin Support Vector Machine (S-TWSVM) was proposed to introduce and balance all structural information of both intra-class and inter-class into its optimization problems. In fact, this method neither consider the structural information conflicting between clusters of one class nor the noise data points that can influence on the structure of the data distribution. In this paper, we propose a new Weighted Structural Twin Support Vector Machine (WS-TWSVM) by its local and global information. In our proposed method, we use a weighted (rather than a simple) summing of structural information to sufficiently exploit class's distribution information, so that, applying the density information of data points, the effects of the noise points on the data structure can be handled. Thus, our proposed method, WS-TWSVM can fully exploit the prior knowledge more efficiently than S-TWSVM leads to improve the model's generalization capacity. As been shown in the experiments, WS-TWSVM is superior to S-TWSVM in term of classification accuracy.
Read moreStructural least square twin support vector machine for classification
The least square twin support vector machine (LS-TSVM) obtains two non-parallel hyperplanes by directly solving two systems of linear equations instead of two quadratic programming problems (QPPs) as in the conventional twin support vector machine (TSVM), which makes the computational speed of LS-TSVM faster than that of the TSVM. However, LS-TSVM ignores the structural information of data which may contain some vital prior domain knowledge for training a classifier. In this paper, we apply the prior structural information of data into the LS-TSVM to build a better classifier, called the structural least square twin support vector machine (S-LSTSVM). Since it incorporates the data distribution information into the model, S-LSTSVM has good generalization performance. Furthermore, S-LSTSVM costs less time by solving two systems of linear equations compared with other existing methods based on structural information. Experimental results on twelve benchmark datasets demonstrate that our S-LSTSVM performs well. Finally, we apply it into Alzheimer's disease diagnosis to further demonstrate the advantage of our algorithm.
Read moreStructural twin parametric-margin support vector machine for binary classification
Structural twin parametric-margin support vector machine for binary classification
V-Structural Nonparallel Support Vector Machine for Pattern Classification
The ν-nonparallel support vector machine (ν-SNPSVM) for classification has the advantage of using a parameter ν on controlling the number of support vectors. However, it ignores the prior structural information in data. In this paper, we propose a novel nonparallel classifier, named ν-Structural Nonparallel Support Vector Machine (ν-SNPSVM), for binary classification. Each model of ν-SNPSVM considers not only the compactness in both classes by the structural information but also the separability between classes, thus it can fully exploit prior knowledge to directly improve the algorithm's generalization capacity and is certainly superior to NPSVMs and other twin SVMs theoretically. Experimental results on lots of data sets show the effectiveness of our method.
Read moreStructural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier
Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of "structural granularity" and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities.
Read moreFisher-regularized supervised and semi-supervised extreme learning machine
The structural information of data contains useful prior knowledge and thus is important for designing classifiers. Extreme learning machine (ELM) has been a potential technique in handling classification problems. However, it only simply considers the prior class-based structural information and ignores the prior knowledge from statistics and geometry of data. In this paper, to capture more structural information of the data, we first propose a Fisher-regularized extreme learning machine (called Fisher-ELM) by applying Fisher regularization into the ELM learning framework, the main goals of which is to build an optimal hyperplane such that the output weight and within-class scatter are minimized simultaneously. The proposed Fisher-ELM reflects both the global characteristics and local properties of samples. Intuitively, the Fisher-ELM can approximatively fulfill the Fisher criterion and can obtain good statistical separability. Then, we exploit graph structural formulation to obtain semi-supervised Fisher-ELM version (called Lap-FisherELM) by introducing manifold regularization that characterizes the geometric information of the marginal distribution embedded in unlabeled samples. An efficient successive overrelaxation algorithm is used to solve the proposed Fisher-ELM and Lap-FisherELM, which converges linearly to a solution, and can process very large datasets that need not reside in memory. The proposed Fisher-ELM and Lap-FisherELM do not need to deal with the extra matrix and burden the computations related to the variable switching, which makes them more suitable for relatively large-scale problems. Experiments on several datasets verify the effectiveness of the proposed methods.
Read moreStructural regularized projection twin support vector machine for data classification
Structural regularized projection twin support vector machine for data classification
WEIGHTED STRUCTURAL SUPPORT VECTOR MACHINE
In binary classification problems, two classes of data seem to be different from each other. It is expected to be more complicated due to the clusters in each class also tend to be different. Traditional algorithms as Support Vector Machine (SVM) or Twin Support Vector Machine (TWSVM) cannot sufficiently exploit structural information with cluster granularity of the data, cause limitation on the capability of simulation of data trends. Structural Twin Support Vector Machine (S-TWSVM) sufficiently exploits structural information with cluster granularity for learning a represented hyperplane. Therefore, the capability of S-TWSVM’s data simulation is better than that of TWSVM. However, for the datasets where each class consists of clusters of different trends, the S-TWSVM’s data simulation capability seems restricted. Besides, the training time of S-TWSVM has not been improved compared to TWSVM. This paper proposes a new Weighted Structural - Support Vector Machine (called WS-SVM) for binary classification problems with a class-vs-clusters strategy. Experimental results show that WS-SVM could describe the tendency of the distribution of cluster information. Furthermore, both the theory and experiment show that the training time of the WS-SVM for classification problem has significantly improved compared to S-TWSVM.
Read moreK-nearest neighbor based structural twin support vector machine
K-nearest neighbor based structural twin support vector machine
Structural Nonparallel Support Vector Machine Based on LSH for Large-Scale Prediction
Considering the fact that the underlying structural information in the training data within classes is vital for a good classifier in real-world classification problems, Structural Nonparallel Support Vector Machine (or SNPSVM, for short) has been proposed. By combining the structural information with nonparallel support vector machine (NPSVM), SNPSVM can fully exploit prior knowledge to directly improve the algorithm’s generalization capacity, each model of which considers not only the compactness in both classes by the structural information but also the separability between classes. In recent decades, discussions on how to train effective classifiers on huge amount of data has attracted more and more research works across several communities. However, the counterpart scalability issue how to apply big trained models efficiently on huge non annotated media collections has been reported rarely. In this paper, we address the problem of speeding-up the prediction phase of linear SNPSVM via Locality Sensitive Hashing (LSH). We propose building efficient hash-based classifiers that are applied in a first stage in order to approximate the exact results and filter the hypothesis space. Furthermore, SNPSVM is solved efficiently by an alternating optimization method, called the improved alternating direction method of multipliers (ADMM). The proposed SNPSVM based on LSH and the solving algorithm can guarantee that we can deal with large-scale classification problems with a huge number of instances as well as features. Experiments performed with millions of one-against-one classifiers show that the proposed hash-based classifier can be more than two orders of magnitude faster than the exact classifier with minor losses in quality.
Read moreStructural improved regular simplex support vector machine for multiclass classification
Structural improved regular simplex support vector machine for multiclass classification
An efficient regularized K-nearest neighbor structural twin support vector machine
K-nearest neighbor based structural twin support vector machine (KNN-STSVM) performs better than structural twin support vector machine (S-TSVM). It applies the intra-class KNN method, and different weights are given to the samples in one class to strengthen the structural information. For the other class, the redundant constraints are deleted by the inter-class KNN method to speed up the training process. However, the empirical risk minimization principle is implemented in the KNN-STSVM, so it easily leads to over-fitting and reduces the prediction accuracy of the classifier. To enhance the generalization ability of the classifier, we propose an efficient regularized K-nearest neighbor structural twin support vector machine, called RKNN-STSVM, by introducing a regularization term into the objective function. So there are two parts in the objective function, one of which is to maximize the margin between the two parallel hyper-planes, and the other one is to minimize the training errors of two classes of samples. Therefore the structural risk minimization principle is implemented in our RKNN-STSVM. Besides, a fast DCDM algorithm is introduced to handle relatively large-scale problems more efficiently. Comprehensive experimental results on twenty-seven benchmark datasets and two popular image datasets demonstrate the efficiency of our proposed RKNN-STSVM.
Read moreIncomplete multi-modal representation learning for Alzheimer's disease diagnosis.
Incomplete multi-modal representation learning for Alzheimer's disease diagnosis.
Structural Minimax Probability Machine.
Minimax probability machine (MPM) is an interesting discriminative classifier based on generative prior knowledge. It can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification. The structural information of data is an effective way to represent prior knowledge, and has been found to be vital for designing classifiers in real-world problems. However, MPM only considers the prior probability distribution of each class with a given mean and covariance matrix, which does not efficiently exploit the structural information of data. In this paper, we use two finite mixture models to capture the structural information of the data from binary classification. For each subdistribution in a finite mixture model, only its mean and covariance matrix are assumed to be known. Based on the finite mixture models, we propose a structural MPM (SMPM). SMPM can be solved effectively by a sequence of the second-order cone programming problems. Moreover, we extend a linear model of SMPM to a nonlinear model by exploiting kernelization techniques. We also show that the SMPM can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi-min margin machine under certain special conditions. Experimental results on both synthetic and real-world data sets demonstrate the effectiveness of SMPM.
Read more