Incremental Learning Algorithm of Least Square Twin KSVC

Least squares twin multi-class classification support vector machine

Classifiers in Automatic Speech Recognition (ASR) aims to improve the generalization ability of the machine learning and improve the recognition accuracy in noisy environments. This paper discusses the classification performance of Hidden Markov Models (HMM) and Support Vector Machines (SVM) applied to a wavelet front end based ASR. The experiments are performed on speaker independent TIMIT database which are trained in a clean environment and later tested in the presence of Additive White Gaussian Noise (AWGN) for various SNR levels using the HTK toolkit and SVM Light software tool. Experiments indicate that for large vocabulary the wavelet front end and the Multiclass SVM classifier with RBF kernel performs better than the conventional HMM classifier.

Detection and treatment of arrhythmias has become one of the main goals in cardiac care diagnosis provided by general practitioners. Electrocardiogram (ECG) analysis is one of the most commonly used tools to test and diagnose heart problems. Classification of ECG heartbeats enables the identification of specific arrhythmia or other heart conditions. This paper presents and contrasts the results from two effective ECG arrhythmia classification schemes. The first scheme consists of a principal component analysis (PCA) step for feature reduction at the input vector to the classifier, combined with soft independent modelling of class analogy (SIMCA). The second method uses a multi-class support vector machine (MSVM) classifier to differentiate between four different types of arrhythmia from ECG beats. The four types of beats include Normal (N), Premature Ventricular Contraction (PVC), and Atrial premature contraction (APC) and Right Bundle Branch Block Beat (RBBB). The time domain features were obtained from the St Petersburg INCART 12-lead Arrhythmia Database (incartdb). Between 10 and 30 Principal Components (PCs) were selected for reconstructing individual ECG beats and create the input vector to the classifier. The average classification accuracy of the proposed scheme is 76.83% and 98.33% using MSVM and SIMCA classifier respectively. The SIMCA classification algorithm provided better performance than the MSVM classifiers.

Temperature modulation of electronic nose combined with multi-class support vector machine classification for identifying export caraway cultivars

Support vector machine (SVM), which is based on statistical learning theory (SLT), has shown much better performance than most other existing machine learning methods, which are based on the traditional statistics. The original SVM was developed to solve the dichotomy classification problem. Various approaches have been presented to solve multi-class problems. Using multi-class SVM classifier we have obtained high class rate of 95.4% in remote sensing image classification. However for the class number of remote sensing image is much great, manually obtaining of training samples is a much time-consuming work. Hence, we present a multi-class SVM based semi-supervised approach. We choose the initial cluster centers manually first, then label the samples as the training ones automatically with fuzzy C-means clustering algorithm. It is believed that this method upgrades the classification efficiency greatly with practicable class rate.

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Support Vector Machine (SVM) is mainly used to classify the data into two categories. To solve the multi-category problems using SVM, researchers used two approaches. The first approach based on solving multiple SVM binary classifiers, whereas another approach based on solving a single optimization problem. In this paper, we have used the first approach and proposed an Efficient Multiclass Support Vector Machine (ESVM) algorithm using a skewed binary tree. To construct the skewed binary tree, no extra efforts are required as compared to the binary tree approach. The algorithm is tested on the benchmark data sets, and the results are compared with both the multiclass approaches of SVM. The ESVM’s results are compared with five techniques of solving multiple binary SVM classifiers and four techniques of solving a single optimization problem. The comparative experiments prove the efficiency of the ESVM in terms of its accuracy as compared to other contemporary algorithms. Further, ESVM is successfully applied for classification of the email dataset into positive, negative and neutral sentiments.

Laser induced breakdown spectroscopy (LIBS) is an atomic emission based spectroscopy that uses a laser pulse as the source of excitation. The laser is focused to form hot plasma, which atomizes and excites the sample. In the LIBS spectrum each “feature” is the amplitude or intensity detected at different wavelengths in the range of 200–1000 nm. Pattern recognition techniques were applied on samples with similar elemental composition resulting in almost similar LIBS spectra which are visually very difficult to differentiate. It was observed that the classification results obtained from different classifiers were sensitive to data preprocessing. The outlier detection and removal techniques PCA, Dendrogram using Agglomerative Algorithm, Editing by Nearest Neighbour (NN) and Distance Matrix approaches were used in preprocessing step. After removing outlier(s) the resulting training patterns were used to model the k-Nearest Neighbour (k-NN), Principal Component Analysis (PCA), Dendrogram, Multiclass Support Vector Machine (SVM) and Decision Tree classifiers. In k-NN after removing outlier(s) the average classification accuracy was increased by 2% for high energy materials (HEM), but no improvement in non high energy materials (Non HEM) or in top level classification (decide either HEM or Non HEM). But, for other classifiers the classification accuracy gets reduced. Finally instead of removing outlier(s) dimensionality reduction by thresholding was applied and the classification accuracy increased by 4% in k-NN for HEM and 38% in multiclass SVM for HEM and 4% for Non-HEM.

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Weighted Twin Support Vector Machines with Local Information and its application

Twin support vector machine (TWSVM) is a classical distance metric learning method for classification problems. The formulation of TWSVM criterion is based on L2-norm distance, which makes TWSVM prone to being influenced by the presence of outliers. In this paper, to develop a robust distance metric learning method, we propose a new objective for TWSVM classifier using L1-norm distance metric, termed as L1-TWSVM. The optimization strategy is to maximize the ratio of the inter-class distance dispersion to the intra-class distance dispersion by using L1-norm distance rather than L2-norm distance. Besides, we design a simple and valid iterative algorithm to solve L1-norm optimal problems, which is easy to actualize and its convergence to an optimum is theoretically ensured. The efficiency and robustness of L1-TWSVM have been validated by experiments on UCI datasets and artificial datasets. The promising experimental results indicate that our proposals outperform relevant state-of-the-art methods in all kinds of experimental settings.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

Cracks are common defects in stainless steel which often lead to serious industrial accidents. In this paper, magnetic detection without external excitation was proposed. Quantitative defect identification was performed using multiclass classification support vector machine. The magnetic signals of 304 austenitic stainless steel were collected before and after annealing. The width, amplitude and area of magnetic signals were extracted as the input set of support vector machine model, and the prediction accuracy of cracks were compared and analyzed. The results showed that the prediction accuracy of length, width and depth of cracks are 80.70%, 92.71% and 65.63%, respectively. The width and depth of defects are increased by 5.89% and 33.34% respectively. This research provides a potential possibility for quantitative defect identification for nonferromagnetic materials under the natural magnetic field.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

E-quality is a holistic approach to gauge and ascertain product quality in real time with the use of advanced technologies. E-quality manifests a sensor-based, networked, fully automated quality control, through which a reduction of inspection time is attained. Even with the e-quality, a part classification still remains as one of the most challenging tasks because the classification is based on the minute differences among a multitude of dimensional attributes. Part classification entails many steps that complicate the accuracy of final outcome. Achieving 100% classification accuracy is not a trivial matter. In this context, this study focuses on a novel approach for improving part classification accuracy in tune with the notion of e-quality and concurrent engineering. Two approaches are proposed and compared with the traditional multiclass support vector machine classification method. One of the approaches is to modify the data before applying the support vector machine. The other is a completely new Sine methodology using dimensional index values for classifying parts into different categories. Support vector machine is employed due to its higher generalization ability, especially when the data set is small and the class overlap is nonexistent. The data extracted from a machine vision system in a networked robotic inspection cell is used to test the proposed approaches. Experimental results show that the new Sine methodology performs better than the others, displaying near 100% classification accuracy.