Incremental proximal support vector classifier for multi-class classification

The focus is made to develop predictor models for wind speed prediction employing the support vector machine neural models. Basically, support vector machines (SVM) is employed as classifiers, but this contribution models variant of SVM to act as predictors. A developed model of linear support vector machine (LSVM) and proximal support vector machine (PSVM) is proposed to carry out the wind speed prediction using the available real time wind farm data. In developed PSVM predictor, it is modeled for both linear PSVM predictor and non-linear PSVM predictor. The difference between the developed linear and non-linear PSVM predictor models lies in their applicability of kernel functions to perform effective wind speed prediction. The prediction application is implemented for the set of wind farm data with a wind mill height of 50 m in a manner to minimize the mean square error. The training process of the neural network algorithmic flow is done with the developed LSVM, L-PSVM (Linear PSVM) and N-PSVM (nonlinear PSVM) for predicting the wind speed in renewable energy systems. Results computed are compared with the other types of predictors to prove the effectiveness of the proposed variants of SVM predictors. The simulated results presents the effectiveness of the proposed predictors for the real time wind farm data and the applicability of the predictors for the considered datasets.

Support Vector Machines and Kernel methods have become very popular as methods for learning from examples. In this paper, in order to solve the non-linear and the time of recognition problem, recent advanced techniques are presented. For example, Kernel Principal Components and Proximal Support Vector Machines, Kernel Principal Components Analysis can describe multiple correlations between pixels in the image recognition, and can handle the high order statistics of the original data, at the same time the nonlinear features of the image can be preferably extracted. Proximal Support Vector Machines is an improvement for these characters of Support Vector Machines, combines the proximal algorithm with Support Vector Machines, has a better ability of nonlinear mapping and a stronger generalization capability, and the pace of Proximal Support Vector Machines is very enhanced. In this paper, the computer simulation was progressed based on ORL face database, the effectiveness of the Proximal Support Vector Machines algorithm and Kernel Principal Components Analysis algorithm were shown by experimental results.

ABSTRACTOne of the major issues in medical field constitutes the correct diagnosis, including the limitation of human expertise in diagnosing the disease in a manual way. Nowadays, the use of machine learning classifiers, such as support vector machines (SVM), in medical diagnosis is increasing gradually. However, traditional classification algorithms can be limited in their performance when they are applied on highly imbalanced data sets, in which negative examples (i.e. negative to a disease) outnumber the positive examples (i.e. positive to a disease). SVM constitutes a significant improvement and its mathematical formulation allows the incorporation of different weights so as to deal with the problem of imbalanced data. In the present work an extensive study of four medical data sets is conducted using a variant of SVM, called proximal support vector machine (PSVM) proposed by Fung and Mangasarian [9]. Additionally, in order to deal with the imbalanced nature of the medical data sets we applied both a variant of SVM, referred as two-cost support vector machine and a modification of PSVM referred as modified PSVM. Both algorithms incorporate different weights one for each class examples.

Proximal support vector machines (PSVMs) are support vector machines (SVMs) version which involve equality instead of inequality constraints assigning data points to one of two classes, and have a significant speedup in comparison with traditional SVM-type classifiers. However, PSVMs are very sensitive to outliers or noises because of overfitting problem. In this paper, a robust proximal support vector machines (RPSVMs) is proposed to deal with the problem of overfitting in PSVMs for two-class classification. In experiments, two methods are used to give membership values to training data points according to their relative importance in the training set. Results show that the proposed RPSVMs actually reduce the effect of outliers and yield higher classification rate than PSVMs do

Soft computing approach to fault diagnosis of centrifugal pump

Research on the non-smooth problems in the nonlinear support vector regression. A nonlinear smooth support vector regression model is proposed. Using a generalized cubic spline function approach the non-smooth part in the support vector regression model. The model of the nonlinear smooth support vector regression is solved by BFGS-Armijo. Then, the approximation accuracy and the astringency of the generalized cubic spline function to the insensitive loss function were analyzed. As a result, we found the four-order and six times spline function's approximation effect is better than other smooth functions, and the nonlinear smooth support vector regression model, which be proposed in this paper is convergent.

Support vector machine (SVM) is a supervised machine learning approach that was recognized as a statistical learning apotheosis for the small-sample database. SVM has shown its excellent learning and generalization ability and has been extensively employed in many areas. This paper presents a performance analysis of six types of SVMs for the diagnosis of the classical Wisconsin breast cancer problem from a statistical point of view. The classification performance of standard SVM (St-SVM) is analyzed and compared with those of the other modified classifiers such as proximal support vector machine (PSVM) classifiers, Lagrangian support vector machines (LSVM), finite Newton method for Lagrangian support vector machine (NSVM), Linear programming support vector machines (LPSVM), and smooth support vector machine (SSVM). The experimental results reveal that these SVM classifiers achieve very fast, simple, and efficient breast cancer diagnosis. The training results indicated that LSVM has the lowest accuracy of 95.6107 %, while St-SVM performed better than other methods for all performance indices (accuracy = 97.71 %) and is closely followed by LPSVM (accuracy = 97.3282). However, in the validation phase, the overall accuracies of LPSVM achieved 97.1429 %, which was superior to LSVM (95.4286 %), SSVM (96.5714 %), PSVM (96 %), NSVM (96.5714 %), and St-SVM (94.86 %). Value of ROC and MCC for LPSVM achieved 0.9938 and 0.9369, respectively, which outperformed other classifiers. The results strongly suggest that LPSVM can aid in the diagnosis of breast cancer.

A boosted SVM classifier trained by incremental learning and decremental unlearning approach

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

P68. Machine learning algorithms for predicting patient-controlled analgesia use after minimally invasive spine surgery

Based on pattern spectrum entropy and proximal support vector machine (PSVM), a motor rolling bearing fault diagnosis method is proposed in this paper. It is very difficult to filter the fault vibration signals from the strong noise background because the roller bearing fault diagnosis is a problem of multi-class classification of inner ring fault, outer ring fault and ball fault. Firstly, vibration signals are processed by the pattern spectrum. Secondly, the morphological pattern spectrum entropy, and pattern spectrum values are utilized to identify the fault features of input parameters of PSVM classifiers. The experiment results demonstrate that the pattern spectrum quantifies various aspects of the shape-size content of a signal, and PSVM costs a little time and has better efficiency than the standard SVM.

Applications of non-linear kernel support vector machines (SVMs) to large data sets is seriously hampered by its excessive training time. We propose a modification, called the approximate extreme points support vector machine (AESVM), that is aimed at overcoming this burden. Our approach relies on conducting the SVM optimization over a carefully selected subset, called the representative set, of the training data set. We present analytical results that indicate the similarity of AESVM and SVM solutions. A linear time algorithm based on convex hulls and extreme points is used to compute the representative set in kernel space. Extensive computational experiments on nine data sets compared AESVM to LIBSVM (Chang and Lin, 2011), CVM (Tsang et al., 2005), BVM (Tsang et al., 2007), LASVM (Bordes et al., 2005), SVMperf (Joachims and Yu, 2009), and the random features method (Rahimi and Recht, 2007). Our AESVM implementation was found to train much faster than the other methods, while its classification accuracy was similar to that of LIBSVM in all cases. In particular, for a seizure detection data set, AESVM training was almost 500 times faster than LIBSVM and LASVM and 20 times faster than CVM and BVM. Additionally, AESVM also gave competitively fast classification times.

Data mining is a new filed in data processing research. Support vector machine (SVM) is a useful method adopted in data mining. However, when the training set of the SVM contains information of uncertainty, the SVM can do nothing about it. In order to solve the problem presented above, this article discusses an algorithm of nonlinear support vector classification machine based on fuzzy theory. With the restriction of the confidence lambda(0<lambdales1), we can using the classification method in fuzzy theory to solve the problem of constraining programming of uncertain chance. By establishing a chain like this: constraining programming of uncertain chance rarr clearly equivalent programming rarr programming of antithesis, the universal algorithm of nonlinearly dividable support vector classification machine based on fuzzy theory can be deduced.

The internet and computer networks are exposed to an increasing number of security threats. With new types of attacks appearing continuously, developing flexible and adaptive security oriented approaches is a severe challenge. In this context, intrusion detection technique is a valuable technology to protect target systems and networks against malicious activities. But this system doesn't provide the required accuracy. Thus to meet this requirement, this paper proposes an intrusion detection system as a model based on Proximal Support Vector Machines (PSVMs) implemented with various combination of basic kernel functions. PSVM is a light and simple modification of support vector machine. We have implemented PSVM for binary classification of intrusion detection data. For experimental training and testing NSL- KDD dataset is preprocessed using Principle Component Analysis technique. Using proposed classification model, we have achieved up to 79% classification accuracy.

In this paper, we have proposed a novel model called proximal support matrix machine (PSMM), which is mainly based on the models of proximal support vector machine (PSVM) and low rank support matrix machine (LRSMM). In design, the PSMM model has comprehensively considered both the relationship between samples of the same class and the structure of rows or columns of matrix data. To a certain extent, our novel model can be regarded as a synthesis of the PSVM model and the LRSMM model. Since the PSMM model is an unconstrained convex problem in essence, we have established an alternating direction method of multipliers algorithm to deal with the proposed model. Finally, since a great deal of experiments on the minst digital database show that the PSMM classifier has a good ability to distinguish two digits with little difference, it encourages us to conduct more complex experiments on MIT face database, INRIA person database, the students face database and Japan female facial expression database. Meanwhile, the final experimental results show that PSMM performs better than PSVM, twin support vector machine, LRSMM and linear twin multiple rank support matrix machine in the demanding image classification tasks.