One-against-all multi-class SVM classification using reliability measures

Support Vector Machine (SVM) is mainly used to classify the data into two categories. To solve the multi-category problems using SVM, researchers used two approaches. The first approach based on solving multiple SVM binary classifiers, whereas another approach based on solving a single optimization problem. In this paper, we have used the first approach and proposed an Efficient Multiclass Support Vector Machine (ESVM) algorithm using a skewed binary tree. To construct the skewed binary tree, no extra efforts are required as compared to the binary tree approach. The algorithm is tested on the benchmark data sets, and the results are compared with both the multiclass approaches of SVM. The ESVM’s results are compared with five techniques of solving multiple binary SVM classifiers and four techniques of solving a single optimization problem. The comparative experiments prove the efficiency of the ESVM in terms of its accuracy as compared to other contemporary algorithms. Further, ESVM is successfully applied for classification of the email dataset into positive, negative and neutral sentiments.

Classifiers in Automatic Speech Recognition (ASR) aims to improve the generalization ability of the machine learning and improve the recognition accuracy in noisy environments. This paper discusses the classification performance of Hidden Markov Models (HMM) and Support Vector Machines (SVM) applied to a wavelet front end based ASR. The experiments are performed on speaker independent TIMIT database which are trained in a clean environment and later tested in the presence of Additive White Gaussian Noise (AWGN) for various SNR levels using the HTK toolkit and SVM Light software tool. Experiments indicate that for large vocabulary the wavelet front end and the Multiclass SVM classifier with RBF kernel performs better than the conventional HMM classifier.

Detection and treatment of arrhythmias has become one of the main goals in cardiac care diagnosis provided by general practitioners. Electrocardiogram (ECG) analysis is one of the most commonly used tools to test and diagnose heart problems. Classification of ECG heartbeats enables the identification of specific arrhythmia or other heart conditions. This paper presents and contrasts the results from two effective ECG arrhythmia classification schemes. The first scheme consists of a principal component analysis (PCA) step for feature reduction at the input vector to the classifier, combined with soft independent modelling of class analogy (SIMCA). The second method uses a multi-class support vector machine (MSVM) classifier to differentiate between four different types of arrhythmia from ECG beats. The four types of beats include Normal (N), Premature Ventricular Contraction (PVC), and Atrial premature contraction (APC) and Right Bundle Branch Block Beat (RBBB). The time domain features were obtained from the St Petersburg INCART 12-lead Arrhythmia Database (incartdb). Between 10 and 30 Principal Components (PCs) were selected for reconstructing individual ECG beats and create the input vector to the classifier. The average classification accuracy of the proposed scheme is 76.83% and 98.33% using MSVM and SIMCA classifier respectively. The SIMCA classification algorithm provided better performance than the MSVM classifiers.

Support vector machine (SVM), which is based on statistical learning theory (SLT), has shown much better performance than most other existing machine learning methods, which are based on the traditional statistics. The original SVM was developed to solve the dichotomy classification problem. Various approaches have been presented to solve multi-class problems. Using multi-class SVM classifier we have obtained high class rate of 95.4% in remote sensing image classification. However for the class number of remote sensing image is much great, manually obtaining of training samples is a much time-consuming work. Hence, we present a multi-class SVM based semi-supervised approach. We choose the initial cluster centers manually first, then label the samples as the training ones automatically with fuzzy C-means clustering algorithm. It is believed that this method upgrades the classification efficiency greatly with practicable class rate.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Support vector machines (SVM) has achieved great success in multi-class classification. However, with the increase in dimension, the irrelevant or redundant features may degrade the generalization performances of the SVM classifiers, which make dimensionality reduction (DR) become indispensable for high-dimensional data. At present, most of the DR algorithms reduce all data points to the same dimension for multi-class datasets, or search the local latent dimension for each class, but they neglect the fact that different class pairs also have different local latent dimensions. In this paper, we propose an adaptive class pairwise dimensionality reduction algorithm (ACPDR) to improve the generalization performances of the multi-class SVM classifiers. In the proposed algorithm, on the one hand, different class pairs are reduced to different dimensions; on the other hand, a tabu strategy is adopted to select adaptively a suitable embedding dimension. Five popular DR algorithms are employed in our experiment, and the numerical results on some benchmark multi-class datasets show that compared with the traditional DR algorithms, the proposed ACPDR can improve the generalization performances of the multi-class SVM classifiers, and also verify that it is reasonable to consider the different class pairs have different local dimensions.

Image annotation systems aim at automatically annotating images with some predefined keywords. In this paper, we propose an automatic image annotation approach by incorporating word correlations into multi-class support vector machine (SVM). At first, each image is segmented into five fixed-size blocks or tiles and MPEG-7 visual descriptors are applied to represent color and texture features of blocks. Keywords are manually assigned to every block of training images. Then, multi-class SVM classifier is trained for semantic concepts. Word or concept correlations are computed by a co-occurrence matrix. The probability outputs from SVM and word correlations are combined to obtain the final results. The minimal-redundancy-maximum-relevance (mRMR) method is used to reduce feature dimensions. The experiments on Corel 5000 dataset demonstrate our approach is effective and efficient.

Statistical learning theory-based support vector machine (SVM), which is a supervised learning mechanism, can get good class rate in remote sensing image classification. But manual obtaining of labeled training samples is a much time-consuming work because of the much greater class number of remote sensing image. In addition, there are some subjective factors in manual job by different operators. In order to overcome these shortcomings, a semi-supervised approach has been developed and implemented. The training samples are labeled automatically with fuzzy C-means clustering algorithm. Only the initial clustering centroid for each class is chosen manually. Using these automatically labeled samples, multi-class SVM classifier is trained for remote sensing images classification. The results of the experiment show that the method does upgrade the classification efficiency greatly with practicable class rate.

With the popularization of network applications and the great changes in the international political, economic and military situations, network security is becoming more and more important. As an important part of network security, network intrusion detection (NID) is still facing the problem of low detection rate and difficulty to meet the real-time demand with the rapid increase of network traffic. Therefore, for the requirement of fast and accurate detection in real-time applications, this paper proposes a NID method based on optimized multiclass support vector machine (SVM). Firstly, the ReliefF feature selection algorithm is introduced to extract features with heuristic search rules based on variable similarity, which reduces the complexity of features and the amount of calculation; Secondly, a SVM training method based on data block method is proposed to improve the training speed; Finally, a multiclass SVM classifier is designed for typical attack types. Experimental results show that the proposed optimization method can achieve a detection rate of 96.9% and shorten the training time by 13.2% on average.KeywordsNetwork intrusion detectionSupport vector machineData blockMulticlass

Temperature modulation of electronic nose combined with multi-class support vector machine classification for identifying export caraway cultivars

Metric-based detection of robot kidnapping with an SVM classifier

In this paper, surface electromyographic signal is analyzed by wavelet transform. The feature vectors are built by extracting the singular value of the wavelet coefficients. The multi-class support vector machine classifier is designed by using four kinds of multi-class classification approaches, and completed the eight class surface EMG pattern classification. The SVM classifier is applied to the classification of eight movements with recording of the surface EMG. Experimental results show that the average recognition rate is over 90%. The classification accuracy of SVM classifier is significantly better than RBF neural network classifier.

Support vector machine (SVM) is originally proposed to solve binary classification problem. Multi-class classification is solved by combining multiple binary classifiers, which leads to high computation cost by introducing many quadratic programming (QP) problems. To decrease computation cost, hyper-sphere SVM is put forward to compute class-specific hyper-sphere for each class. If all resulting hyper-spheres are independent, all training and test samples can be correctly classified. When some of hyper-spheres intersect, new decision rules should be adopted. To solve this problem, a multiple sub-hyper-sphere SVM is put forward in this paper. New algorithm computed hyper-spheres by SMO algorithm for all classes first, and then obtained position relationships between hyper-spheres. If hyper-spheres belong to the intersection set, overlap coefficient is computed based on map of key value index and mother hyper-spheres are partitioned into a series of sub-hyper-spheres. For the new intersecting hyper-spheres, one similarity function or same error sub-hyper-sphere or different error sub-hyper-sphere are used as decision rule. If hyper-spheres belong to the inclusion set, the hyper-sphere with larger radius is partitioned into sub-hyper-spheres. If hyper-spheres belong to the independence set, a decision function is defined for classification. With experimental results compared to other hyper-sphere SVMs, our new proposed algorithm improves the performance of the resulting classifier and decreases computation complexity for decision on both artificial and benchmark data set.

In view of the batch implementations of standard support vector machine must be retrained from scratch every time when the training set is incremental modified, an incremental learning algorithm based on least squares twin multi-class classification support vector machine (ILST-KSVC) is proposed by solving two inverse matrix. The method will be applied on online environment to update initial data, which avoided cumbersome double counting. ILST-KSVC inherited the advantages of the basic algorithm and has some merits of Least square twin support vector machine for excellent performance on training speed and support vector classification regression for K-class’s well classification accuracy. The result will be confirmed no matter in low dimension or in high dimension in UCI datasets.

BackgroundTo harness the full potential of social media for epidemiological surveillance of drug abuse trends, the field needs a greater level of automation in processing and analyzing social media content.ObjectivesThe objective of the study is to describe the development of supervised machine-learning techniques for the eDrugTrends platform to automatically classify tweets by type/source of communication (personal, official/media, retail) and sentiment (positive, negative, neutral) expressed in cannabis- and synthetic cannabinoid–related tweets.MethodsTweets were collected using Twitter streaming Application Programming Interface and filtered through the eDrugTrends platform using keywords related to cannabis, marijuana edibles, marijuana concentrates, and synthetic cannabinoids. After creating coding rules and assessing intercoder reliability, a manually labeled data set (N=4000) was developed by coding several batches of randomly selected subsets of tweets extracted from the pool of 15,623,869 collected by eDrugTrends (May-November 2015). Out of 4000 tweets, 25% (1000/4000) were used to build source classifiers and 75% (3000/4000) were used for sentiment classifiers. Logistic Regression (LR), Naive Bayes (NB), and Support Vector Machines (SVM) were used to train the classifiers. Source classification (n=1000) tested Approach 1 that used short URLs, and Approach 2 where URLs were expanded and included into the bag-of-words analysis. For sentiment classification, Approach 1 used all tweets, regardless of their source/type (n=3000), while Approach 2 applied sentiment classification to personal communication tweets only (2633/3000, 88%). Multiclass and binary classification tasks were examined, and machine-learning sentiment classifier performance was compared with Valence Aware Dictionary for sEntiment Reasoning (VADER), a lexicon and rule-based method. The performance of each classifier was assessed using 5-fold cross validation that calculated average F-scores. One-tailed t test was used to determine if differences in F-scores were statistically significant.ResultsIn multiclass source classification, the use of expanded URLs did not contribute to significant improvement in classifier performance (0.7972 vs 0.8102 for SVM, P=.19). In binary classification, the identification of all source categories improved significantly when unshortened URLs were used, with personal communication tweets benefiting the most (0.8736 vs 0.8200, P<.001). In multiclass sentiment classification Approach 1, SVM (0.6723) performed similarly to NB (0.6683) and LR (0.6703). In Approach 2, SVM (0.7062) did not differ from NB (0.6980, P=.13) or LR (F=0.6931, P=.05), but it was over 40% more accurate than VADER (F=0.5030, P<.001). In multiclass task, improvements in sentiment classification (Approach 2 vs Approach 1) did not reach statistical significance (eg, SVM: 0.7062 vs 0.6723, P=.052). In binary sentiment classification (positive vs negative), Approach 2 (focus on personal communication tweets only) improved classification results, compared with Approach 1, for LR (0.8752 vs 0.8516, P=.04) and SVM (0.8800 vs 0.8557, P=.045).ConclusionsThe study provides an example of the use of supervised machine learning methods to categorize cannabis- and synthetic cannabinoid–related tweets with fairly high accuracy. Use of these content analysis tools along with geographic identification capabilities developed by the eDrugTrends platform will provide powerful methods for tracking regional changes in user opinions related to cannabis and synthetic cannabinoids use over time and across different regions.