Valley-loss regular simplex support vector machine for robust multiclass classification

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Smoothing methods, extensively used for solving important mathematical programming problems and applications, are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel. We term such reformulation a smooth support vector machine (SSVM). A fast Newton–Armijo algorithm for solving the SSVM converges globally and quadratically. Numerical results and comparisons are given to demonstrate the effectiveness and speed of the algorithm. On six publicly available datasets, tenfold cross validation correctness of SSVM was the highest compared with four other methods as well as the fastest. On larger problems, SSVM was comparable or faster than SVMlight (T. Joachims, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999), SOR (O.L. Mangasarian and David R. Musicant, IEEE Transactions on Neural Networks, vol. 10, pp. 1032–1037, 1999) and SMO (J. Platt, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999). SSVM can also generate a highly nonlinear separating surface such as a checkerboard.

Automated credit approval helps credit-granting institutions in reducing time and efforts in analyzing credit approval requests and to distinguish good customers from bad ones. Enhancing the automated process of credit approval by integrating it with a good business intelligence (BI) system puts financial institutions and banks in a better position compared to their competitors. In this paper, a novel hybrid approach based on neural network model called Cycle Reservoir with regular Jumps (CRJ) and Support Vector Machines (SVM) is proposed for classifying credit approval requests. In this approach, the readout learning of CRJ will be trained using SVM. Experiments results confirm that in comparison with other data mining techniques, CRJ with SVM readout gives superior classification results.

Structural regularized projection twin support vector machine for data classification

The responsibility weighted Mahalanobis kernel for semi-supervised training of support vector machines for classification

Introduction: The condition of prostate cancer continues to represent a substantial medical issue because it necessitates precise predictive models that support healthcare decisions. A research study investigates how Extreme Gradient Boosting (XGBoost) performs when identifying significant biomarkers to anticipate prostate cancer risks. The research compares XGBoost to Sequential Minimal Optimization (SMO) in Support Vector Machines (SVM) to evaluate its better performance in classification. Prostate-Specific Antigen (PSA) levels together with Gleason scores and molecular markers form the clinical parameters within the dataset. Process data correctly by cleansing it and normalizing it while extracting important features to achieve the optimal model performance. Objectives: This research investigates how well Extreme Gradient Boosting (XGBoost) predicts prostate cancer through an examination of its function to discover complex non-linear data patterns in clinical datasets. The research compares XGBoost to SMO algorithm of Support Vector Machines through a performance assessment of classification metrics including accuracy, precision, recall and F1-score. The research explores two important biomarkers Prostate-Specific Antigen (PSA) levels along with Gleason scores which affect how prostate cancer is expected to progress. The research applies machine learning advanced methods to develop improved early identification systems that lead to better clinical decisions which subsequently enhance patient health results. Methods: The research method relies on a structured machine learning process where multiple steps start with detailed data preprocessing for inconsistent data elimination and variable normalization. Feature extraction technologies increase model interpretability by being applied to the analysis. The available dataset contains two well-separated sections for training and testing purposes which aims to establish reliable validation procedures. Both XGBoost and SVM with SMO operate for classification tasks while their achievement levels are assessed through standard measurement criteria which include accuracy, precision, recall and F1-score. A logistic regression analysis validates whether PSA levels and Gleason scores are significant factors in predictive modeling. Results: XGBoost achieves superior results over SVM with SMO according to every mitigation evaluation indicator. The classification metric for XGBoost displays 0.95 accuracy whereas SVM with SMO achieves 0.87 accuracy. The evaluation of XGBoost produced precision measurements of 0.92 while recall reached 0.93 and F1-score achieved 0.92. These evaluation scores surpassed SVM with SMO which generated precision of 0.85, recall of 0.80 and F1-score of 0.82. XGBoost manages to lower the chances of overfitting while improving biomarker interpretation so it shows clear potential for clinical prostate cancer prediction applications. Conclusions: The conducted research demonstrates XGBoost's effectiveness for prostate cancer classification through better prediction accuracy along with stronger resistance than SVM with SMO. The research demonstrates AI-powered models should become an integral part of clinical practice to improve the detection and treatment preparation for patients.

Sequential minimal optimization (SMO) is widely used for training support vector machines (SVMs) because of fast training. But the training slows down when a large margin parameter value is used. Training by Newton’s method (NM) accelerates training in such a situation but it slows down for a small margin parameter value. To solve this problem, in this paper we fuse SMO with NM and call it SMO-NM. Because slow training is caused by repetitive corrections of the same variables, we modify the working set selection when they are detected. We call the variables that are selected by SMO, SMO variables. At the current step, if a variable selected as an SMO variable was selected in a previous step, we consider that a loop is detected. And in addition to the SMO variables, we add, to the working set, the unbounded variables that were selected as SMO variables and correct the variables by NM. If no loop is detected, the training procedure is the same as that of SMO. As a variant of this working set strategy, we further add violating variables to the working set. We clarify that if the classification problem is not linearly separable in the feature space, the solutions of L1/L2 SVMs (with the linear sum/square sum of slack variables) are unbounded as the margin parameter value approaches infinity but that, if the mapped training data are not linearly independent in the feature space, the solution of the least squares SVM is unbounded as the margin parameter approaches infinity. We also clarify the condition, in which the increment of the objective function value by SMO-NM is larger than that by SMO. We evaluate SMO-NM for several benchmark data sets and confirm the effectiveness over SMO especially for a large margin parameter value.

Cross-validation is a normal method for parameter selection of support vector machine (SVM) which is a novel machine learning method for hyperspectral data classification. Because of the high dimensionality of hyperspectral data, the process of cross-validation will cost more time. For reducing the time of cross-validation and improving classification accuracy, a new combination method of improving sequential minimal optimization (SMO), independent component analysis (ICA) and mixture kernels is proposed. It can be described as follows: first use the improving SMO method to optimize the model of SVM, and then use ICA method to do dimensionality reduction before cross-validation, at last use mixture kernels to do classification of unknown samples. By the experiments, it is proved that this method can guarantee the accuracy of unknown samples classification while reducing the time of cross-validation.

Sequential Minimal Optimization (SMO) is one of the state-of-the-art solutions for Support Vector Machines (SVM) training. SMO sequentially picks a pair of training samples for joint optimization, which minimizes the usage of memory. However, to find such two samples, SMO uses one outer loop through the entire training set looking for a sample which violates the KKT conditions, and one inner loop to look for a sample which heuristically maximizes the increase of the SVM objective function. While there is limited room for improvement in the inner loop, the outer loop can be reduced significantly if we can locate a small subset of training samples which potentially contains the Support Vectors (SVs).We present a Fisher Discriminant Analysis(FDA) based approach to locate potential SVs in a small core subset, which enables the SVM training to be focused effectively on a promising small region instead of the entire SV-sparse training set. Nevertheless, the training points outside the subset can not be blindly ignored. The method iteratively trains the classifier on the core subset and tune it using the remainder subset. Experimental results show that the approach significantly speeds up SMO training by reducing kernel evaluations, which dominates the training time. It converges much faster to optimal solution with much fewer iterations than standard SMO.

Aircraft engine is an important component of the airplane, which lifespan and reliability have directly influence to flight safety. Vibration analysis is an effective fault diagnosis method for airplane's mechanical construction fault. Several typical vibration faults have drawn from the vibration analysis on the basis of fault mechanism in aircraft engine, and the standard vibration fault-symptom (Fuzzy Membership) relationship table have constructed based on Fuzzy Mathematic theory in this paper. Sample data are simulated on the basis of this table. Sequential Minimal Optimization (SMO) is a fast algorithm for training Support Vector Machines (SVM). The simulated result showed a great improvement on computation efficiency by selecting double threshold parameter for SMO algorithm than single threshold. In order to examine the algorithm's effect applying in the aircraft engine fault diagnosis, Radial Basis Function (RBF) Neural Network, C4.5 Decision Tree, SVM (chunking), Platt's SMO and Improved SMO algorithm are applied to the training data and test data. It is shown that the Improved SMO algorithm displayed superior comprehensive performance. Finally, the vibration fault feature is reduced based on Fuzzy-Rough Sets theory. The research shows that the Improved SMO algorithm is quicker than other algorithm in aircraft engine fault diagnosis with the reduced feature attributes, and its accuracy is high.

Proximal support vector machine is a variation of standard support vector machine and can be trained extremely efficiently for binary classification. However in many application fields, multi-class classification and incremental learning must be supported. Incremental linear proximal support vector classifier for multi-class classification has been developed in recent years, but only its performance in one-against-all manner has been investigated, and the application of proximal support vector machine for nonlinear multi-class classification has not been studied. In order to apply proximal support vector machine to more fields, three multi-class classification policies (one-against-all, one-against-one, DAGSVM) applied to incremental linear proximal support vector classifier are compared and incremental nonlinear proximal support vector classifier for multi-class classification based on Gaussian kernel is investigated in the paper. The experiments indicate that one-against-all policy is best for incremental linear proximal support vector classifier according to the tradeoff between computing complexity and correctness, and the introduced incremental nonlinear proximal support vector classifier is effective in one-against-all manner when the reduce rate is below 0.6.

This study compares the performance of local binary patterns (LBP) and scale invariant feature transform (SIFT) with support vector machines (SVM) in automatic classification of discrete facial expressions. Facial expression recognition is a multiclass classification problem and seven classes; happiness, anger, sadness, disgust, surprise, fear and comtempt are classified. Using SIFT feature vectors and linear SVM, 93.1% mean accuracy is acquired on CK+ database. On the other hand, the performance of LBP-based classifier with linear SVM is reported on SFEW using strictly person independent (SPI) protocol. Seven-class mean accuracy on SFEW is 59.76%. Experiments on both databases showed that LBP features can be used in a fairly descriptive way if a good localization of facial points and partitioning strategy are followed.

FAST ONLINE TRAINING OF L1 SUPPORT VECTOR MACHINES By Gabriella Melki A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science at Virginia Commonwealth University. Virginia Commonwealth University, 2016. Directors: Vojislav Kecman, Alberto Cano Department of Computer Science This thesis proposes a novel experimental environment (non-linear stochastic gradient descent, NL-SGD), as well as a novel online learning algorithm (OL SVM), for solving a classic nonlinear Soft Margin L1 Support Vector Machine (SVM) problem using a Stochastic Gradient Descent (SGD) algorithm. The NL-SGD implementation has a unique method of random sampling and alpha calculations. The developed code produces a competitive accuracy and speed in comparison with the solutions of the Direct L2 SVM obtained by software for Minimal Norm SVM (MN-SVM) and Non-Negative Iterative Single Data Algorithm (NN-ISDA). The latter two algorithms have shown excellent performances on large datasets; which is why we chose to compare NL-SGD and OL SVM to them. All experiments have been done under strict double (nested) cross-validation, and the results are reported in terms of accuracy and CPU times. OL SVM has been implemented within MATLAB and is compared to the classic Sequential Minimal Optimization (SMO) algorithm implemented within MATLAB’s solver, fitcsvm. The experiments with OL SVM have been done using k-fold cross-validation and the results reported in % error and % speedup of CPU Time. viii

In this work we address the use of support vector machine (SVM) in the multi-class gas classification system. The objective is to classify single gases and their mixture with a semiconductor-type electronic nose. The SVM has some typical multi-class classification models; One vs. One (OVO) and One vs. All (OVA). However, studies on those models show weaknesses on calculation time, decision time and the reject region. We propose a hierarchical clustering method (HCM) based on the SVM for real-time gas mixture classification. Experimental results show that the proposed method has better performance than the typical multi-class systems based on the SVM, and that the proposed method can classify single gases and their mixture easily and fast in the embedded system compared with BP-MLP and Fuzzy ARTMAP.

Event Abstract Back to Event High Performance EEG Signal Classification using Classifiability and the Twin SVM Sumit Soman1* and Jayadeva .1 1 Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, India Classification of Electroencephalogram (EEG) data for imagined motor movements has been a challenge in the design and development of Brain Computer Interfaces (BCIs). There are two principle challenges. The first is the variability in the recorded EEG data, which manifests across trials as well as across individuals. Consequently, features that are more discriminative need to be identified before any pattern recognition technique can be applied. The second challenge is in the pattern recognition domain. The number of data samples in a class of interest, e.g. a specific action, is a small fraction of the total data, which is composed of samples corresponding to all actions of all users. Building a robust classifier when learning from a highly unbalanced dataset is very difficult; minimizing the classification error typically causes the larger class to overwhelm the smaller one. We show that the combination of ‘classifiability’ for selecting the optimal frequency band and the use of the Twin Support Vector Machine (Twin SVM) for classification, yields significantly improved generalization. On benchmark BCI Competition datasets, the proposed approach often yields up to 20% improvement over the state-of-the-art. Figure 1 References Soman, Sumit and Jayadeva. "High performance EEG signal classification using classifiability and the Twin SVM." Applied Soft Computing Vol. 30 (May 2015): pp. 305-318 [http://www.sciencedirect.com/science/article/pii/S1568494615000204] Keywords: Brain Computer Interface, Classifiability, Support Vector Machines, Motor Imagery (MI), multiclass classification, Unbalanced data Conference: German-Japanese Adaptive BCI Workshop, Kyoto, Japan, 28 Oct - 29 Oct, 2015. Presentation Type: Poster presentation Topic: Adaptive BCI Citation: Soman S and . J (2015). High Performance EEG Signal Classification using Classifiability and the Twin SVM. Front. Comput. Neurosci. Conference Abstract: German-Japanese Adaptive BCI Workshop. doi: 10.3389/conf.fncom.2015.56.00020 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 18 Sep 2015; Published Online: 04 Nov 2015. * Correspondence: Mr. Sumit Soman, Indian Institute of Technology, Delhi, India, Department of Electrical Engineering, Delhi, India, sumit.soman@gmail.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Sumit Soman Jayadeva . Google Sumit Soman Jayadeva . Google Scholar Sumit Soman Jayadeva . PubMed Sumit Soman Jayadeva . Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.