- Research Article
13
- 10.1016/j.neucom.2016.12.052
PAC-Bayes bounds for twin support vector machines
- Dec 24, 2016
- Neurocomputing
- Xijiong Xie + 1 more +1
PAC-Bayes bounds for twin support vector machines
Twin support vector machines are a powerful learning method for binary classification. Compared to standard support vector machines, they learn two hyperplanes rather than one as in standard support vector machines, and work faster and sometimes perform better than support vector machines. However, relatively little is known about their theoretical performance. As recent tightest bounds for practical applications, PAC-Bayes bounds are based on a prior and posterior over the distribution of classifiers. In this paper, we study twin support vector machines from a theoretical perspective and use the PAC-Bayes bound to measure the generalization error bound of twin support vector machines. Experimental results on real-world datasets show better predictive capabilities of the PAC-Bayes bound for twin support vector machines compared to the PAC-Bayes bound for support vector machines.
PAC-Bayes bounds for twin support vector machines
PAC-Bayes bounds for twin support vector machines
Wavelet twin support vector machines based on glowworm swarm optimization
Wavelet twin support vector machines based on glowworm swarm optimization
Ensemble of classification models with weighted functional link network
Ensemble of classification models with weighted functional link network
Twin support vector machine based on improved artificial fish swarm algorithm with application to flame recognition
In this paper, a twin support vector machine (TWSVM) based on improved artificial fish swarm algorithm (IAFSA) for fire flame recognition is proposed in view of the large computation burden and slow classification speed of the traditional support vector machine (SVM). Twin support vector machine is a machine learning algorithm developing from standard support vector machine. However, twin support vector machine cannot deal with the parameter selection problem well. The difficulty of parameter selection may greatly restrict the application of TWSVM in flame recognition problem. So a novel artificial fish swarm algorithm (AFSA) is used to solve the parameter selection problem of TWSVM. In order to make up for the drawbacks of the basic AFSA, the chaotic transformation is first applied to initialize the position of artificial fish swarm since it may be non-uniformly initialized in the basic artificial fish swarm algorithm. Then, the Cauchy mutation is used to make the fish swarm jump out of the local optimal solution after continuously expanding the visual scope of the artificial fish during the foraging procedure. An adaptively step-size adjusting method is then developed to optimize the moving steps of the swarming and following behaviors in order to accelerate the convergence speed of the developed algorithm. Last, to further improve the efficiency and accuracy of the algorithm, an elimination and regeneration mechanism based on adaptive t-distribution mutation is utilized to update the artificial fish swarm at each iterative procedure. Experimental results show that the TWSVM algorithm based on improved artificial fish swarm algorithm is a more effective method to identify the flame and greatly improves the accuracy and real-time performance of the flame recognition compared with PSO-TWSVM, Grid-TWSVM, GA-TWSVM, FOA-TWSVM, GSO-TWSVM, AFSA-TWSVM and the traditional SVM.
Read moreTwin support vector machines: A survey
Twin support vector machines: A survey
Intrusion detection model with twin support vector machines
Intrusion detection system (IDS) is becoming a critical component of network security. However, the performance of many proposed intelligent intrusion detection models is still not competent to be applied to real network security. This paper aims to explore a novel and effective approach to significantly improve the performance of IDS. An intrusion detection model with twin support vector machines (TWSVMs) is proposed. In this model, an efficient algorithm is also proposed to determine the parameter of TWSVMs. The performance of the proposed intrusion detection model is evaluated with KDD’99 dataset and is compared with those of some recent intrusion detection models. The results demonstrate that the proposed intrusion detection model achieves remarkable improvement in intrusion detection rate and more balanced performance on each type of attacks. Moreover, TWSVMs consume much less training time than standard support vector machines (SVMs).
Read moreWeighted Twin Support Vector Machines with Local Information and its application
Weighted Twin Support Vector Machines with Local Information and its application
A Novel Twin Support Vector Machines with Manifold Regularization
Twin support vector machine (TWSVM), as a variant of the generalized eigenvalue proximal support vector machine (GEPSVM), attempts to improve the generalization of GEPSVM, whose solution follows from solving two quadratic programming problems (QPPs), each of which is smaller than in a standard SVM. Unfortunately, TWSVM fails to fully consider the local geometry structure and the local underlying descriminant information inside the samples that may be important for classification performance and only preserves the global data structure. In this paper, a novel TWSVM with manifold regularization is proposed by introducing the basic idea of the locality preserving within-class scatter matrix (LPWSM) into TWSVM. We termed this method manifold TWSVM (MTWSVM). MTWSVM not only retains the superior characteristics of TWSVM, but also preserves the local geometry structure between samples and shows the local underlying discriminant information. Experimental results confirm the effectiveness of our method.
Read moreA feature selection method for nonparallel plane support vector machine classification
Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.
Read moreTwin Support Vector Machine: A review from 2007 to 2014
Twin Support Vector Machine: A review from 2007 to 2014
P2P traffic identification method based on K-means and twin support vector machine
Most of the P2P traffic identification methods have the problem of high time cost.Therefore,it was proposed to use TWin Support Vector Machine(TWSVM) whose time cost was a quarter of the common Support Vector Machine(SVM) to build classifier.K-means ensemble was used to create labeled sample set and labeled sample set was combined as the training sample of the TWSVM.At last,the constructed classification model was used to identify P2P traffic.The experimental results show that the method based on K-means and TWSVM can significantly decrease time cost of the P2P traffic identification,and has a higher accuracy rate and better stability than the standard SVM.
Read moreA comparison on multi-class classification methods based on least squares twin support vector machine
A comparison on multi-class classification methods based on least squares twin support vector machine
Aircraft Engine Gas Path Fault Diagnosis Based on Hybrid PSO-TWSVM
Twin support vector machine (TWSVM) is a new development of support vector machine (SVM) algorithm. It has the smaller computation scale and the stronger ability to cope with unbalanced problems. In this paper, TWSVM is introduced into aircraft engine gas path fault diagnosis. The generalization capacity of Gauss kernel function usually used in TWSVM is relatively weak. So a mixed kernel function is used to improve performance to ensure that the TWSVM algorithm can better balance a strong generalization ability and a good learning ability. Experimental results prove that the cross validation training accuracy of TWSVM using the mixed kernel function averagely increases 2%. Grid search is usually applied in parameter optimization of TWSVM, but it heavily depends on experience. Therefore, the hybrid particle swarm algorithm is introduced. It can intelligently and rapidly find the global optimum. Experiments prove that its training accuracy is better than that of the classical particle swarm algorithm by 5%.
Read moreTwin support vector machine: theory, algorithm and applications
Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and theory of TWSVM, including the theory of proximal support vector machine, generalized eigenvalue proximal support vector machine, and TWSVM. Then, we focus on the various improvements made to TWSVM, mainly including least squares twin support vector machine, smooth twin support vector machine, regularized twin support vector machine, projection twin support vector machine, and modified TWSVM on the model selection problem. These newly emerging algorithms greatly expand the applications of TWSVM. In recent years, there is a lot of research on application of TWSVM. Next, we list some research on application of TWSVM in detail. Finally, we try to provide a comprehensive view of these advances in TWSVM and discuss the direction of future research.
Read moreSupport Vector Machines for Classification in Nonstandard Situations
The majority of classification algorithms are developed for the standard situation in which it is assumed that the examples in the training set come from the same distribution as that of the target population, and that the cost of misclassification into different classes are the same. However, these assumptions are often violated in real world settings. For some classification methods, this can often be taken care of simply with a change of thresholds for others, additional effort is required. In this paper, we explain why the standard support vector machine is not suitable for the nonstandard situation, and introduce a simple procedure for adapting the support vector machine methodology to the nonstandard situation. Theoretical justification for the procedure is provided. Simulation study illustrates that the modified support vector machine significantly improves upon the standard support vector machine in the nonstandard situation. The computational load of the proposed procedure is the same as that of the standard support vector machine. The procedure reduces to the standard support vector machine in the standard situation.
Read more