Feature Selection Using Sparse Twin Support Vector Machine with Correntropy-Induced Loss

Sparse pinball twin support vector machines

Effective common spatial pattern (CSP) feature extraction for motor-imagery (MI) EEG recordings usually depends on the filter band selection to a large extent. However, the most proper filter band can hardly be determined manually due to its subject-specific property. This study introduces a sparse support vector machine (SSVM) approach to implement simultaneous CSP feature selection and classification for MI-based brain–computer interface (BCI). In SSVM, CSP features are first extracted on multiple signals that are filtered from raw EEG data at a set of overlapping subbands. SVM with l 1-norm regularization is then proposed to classify MI tasks with automatic selection of filter bands giving the significant CSP features. The effectiveness of SSVM for MI classification is demonstrated on the BCI Competition III dataset IVa, in comparison with several other competing methods. Experimental results indicate that the proposed SSVM method is promising for development of an improved MI-based BCI.

Melt index is one of the most important quality variables to determine product specifications in the polymer processing, which cannot be real-time measured by ordinary method. In order to estimate the melt index, a soft sensor method using sparse least square support vector machine based on genetic algorithm is proposed. In the proposed method, genetic algorithm is used to realize the sparseness of least square support vector machine and the optimization of its parameters. Simulation results of industrial data show that the proposed method can reduce the model scale which has better generalization capability than that of the non-sparse model.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Weighted Twin Support Vector Machines with Local Information and its application

The condition monitoring and fault diagnosis of power transformers plays a significant role in the safe, stable and reliable operation of the whole power system. Dissolved gas analysis (DGA) methods are widely used for fault diagnosis, however, their accuracy is limited by the selection of DGA features and the performance of fault diagnosis models, for example, the classical support vector machine (SVM), is easily affected by unbalanced training samples. This paper presents a transformer fault diagnosis model based on chemical reaction optimization and a twin support vector machine. Twin support vector machines (TWSVMs) are used as classifiers for solving problems involving unbalanced and insufficient samples. Restricted Boltzmann machines (RBMs) are used for data preprocessing to ensure the effective identification of feature parameters and improve the efficiency and accuracy of fault diagnosis. The chemical reaction optimization (CRO) algorithm is used to optimize TWSVM parameters to select the optimal training parameters. The cross-validation (CV) method is used to ensure the reliability and generalization ability of the diagnostic model. Finally, the validity of the model is verified using real fault samples and random testing.

Scene classification based on single-layer SAE and SVM

Twin support vector machine (TWSVM) is a classical distance metric learning method for classification problems. The formulation of TWSVM criterion is based on L2-norm distance, which makes TWSVM prone to being influenced by the presence of outliers. In this paper, to develop a robust distance metric learning method, we propose a new objective for TWSVM classifier using L1-norm distance metric, termed as L1-TWSVM. The optimization strategy is to maximize the ratio of the inter-class distance dispersion to the intra-class distance dispersion by using L1-norm distance rather than L2-norm distance. Besides, we design a simple and valid iterative algorithm to solve L1-norm optimal problems, which is easy to actualize and its convergence to an optimum is theoretically ensured. The efficiency and robustness of L1-TWSVM have been validated by experiments on UCI datasets and artificial datasets. The promising experimental results indicate that our proposals outperform relevant state-of-the-art methods in all kinds of experimental settings.

Sparse discriminant twin support vector machine for binary classification

Glaucoma is a group of eye diseases that cause an irreparable decrease in the view field which is the second most common and leading causes of blindness among retinal diseases. Computer aided diagnosis (CAD) system is an emerging field in medical informatics which has high importance for providing prognosis of diseases. Research efforts have reported with increasing confirmation that the twin support vector machines (TWSVM) have greater accurate diagnosis ability. The goal of TWSVM is to construct two non-parallel planes for each class such that each hyper-plane is closer to one of two classes and as far as possible from the other one. In this paper, we propose a new CAD system for glaucoma diagnosis using TWSVM and three heterogeneous families of feature extraction. In this work, we have used 169 images to classify into normal and glaucoma classes. The performance of the method is evaluated using classification accuracy, sensitivity, specificity and receiver operating characteristic (ROC) curves. The results show that the highest classification accuracy (98,53%) is obtained for the TWSVM using Gaussian kernel function, and this is very promising compared to the SVM results.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

In this paper, a novel unconstrained convex minimization problem formulation for the Lagrangian dual of the recently introduced twin support vector machine (TWSVM) in simpler form is proposed for constructing binary classifiers. Since the objective functions of the modified minimization problems contain non-smooth `plus' function, we solve them by Newton iterative method either by considering their generalized Hessian matrices or replacing the `plus' function by a smooth approximation function. Numerical experiments were performed on a number of interesting real-world benchmark data sets. Computational results clearly illustrates the effectiveness and the applicability of the proposed approach as comparable or better generalization performance with faster learning speed is obtained in comparison with SVM, least squares TWSVM (LS-TWSVM) and TWSVM.

This paper addresses the problem of feature selection for the classification of ground penetrating radar signals. We propose a new classification approach based on time-frequency analysis and sparse kernel feature selection. In the proposed approach, a time-frequency or a time-scale transform is first applied to the one-dimensional radar trace. Sparse kernel feature selection is then employed to extract an optimum set of features for classification. The sparse kernel method is formulated as an underdetermined linear system in a high-dimensional space, and the category labels of the training samples are used as measurements to select the most informative features. The proposed approach is evaluated through an industrial application of assessing railway ballast fouling conditions. Experimental results show that the proposed combination of sparse kernel feature selection and support vector machine classification yields very high classification rates using only a small number of features.

Color image classification and retrieval through ternary decision structure based multi-category TWSVM