Fast Online Training of L1 Support Vector Machines

Aircraft engine is an important component of the airplane, which lifespan and reliability have directly influence to flight safety. Vibration analysis is an effective fault diagnosis method for airplane's mechanical construction fault. Several typical vibration faults have drawn from the vibration analysis on the basis of fault mechanism in aircraft engine, and the standard vibration fault-symptom (Fuzzy Membership) relationship table have constructed based on Fuzzy Mathematic theory in this paper. Sample data are simulated on the basis of this table. Sequential Minimal Optimization (SMO) is a fast algorithm for training Support Vector Machines (SVM). The simulated result showed a great improvement on computation efficiency by selecting double threshold parameter for SMO algorithm than single threshold. In order to examine the algorithm's effect applying in the aircraft engine fault diagnosis, Radial Basis Function (RBF) Neural Network, C4.5 Decision Tree, SVM (chunking), Platt's SMO and Improved SMO algorithm are applied to the training data and test data. It is shown that the Improved SMO algorithm displayed superior comprehensive performance. Finally, the vibration fault feature is reduced based on Fuzzy-Rough Sets theory. The research shows that the Improved SMO algorithm is quicker than other algorithm in aircraft engine fault diagnosis with the reduced feature attributes, and its accuracy is high.

Introduction: The condition of prostate cancer continues to represent a substantial medical issue because it necessitates precise predictive models that support healthcare decisions. A research study investigates how Extreme Gradient Boosting (XGBoost) performs when identifying significant biomarkers to anticipate prostate cancer risks. The research compares XGBoost to Sequential Minimal Optimization (SMO) in Support Vector Machines (SVM) to evaluate its better performance in classification. Prostate-Specific Antigen (PSA) levels together with Gleason scores and molecular markers form the clinical parameters within the dataset. Process data correctly by cleansing it and normalizing it while extracting important features to achieve the optimal model performance. Objectives: This research investigates how well Extreme Gradient Boosting (XGBoost) predicts prostate cancer through an examination of its function to discover complex non-linear data patterns in clinical datasets. The research compares XGBoost to SMO algorithm of Support Vector Machines through a performance assessment of classification metrics including accuracy, precision, recall and F1-score. The research explores two important biomarkers Prostate-Specific Antigen (PSA) levels along with Gleason scores which affect how prostate cancer is expected to progress. The research applies machine learning advanced methods to develop improved early identification systems that lead to better clinical decisions which subsequently enhance patient health results. Methods: The research method relies on a structured machine learning process where multiple steps start with detailed data preprocessing for inconsistent data elimination and variable normalization. Feature extraction technologies increase model interpretability by being applied to the analysis. The available dataset contains two well-separated sections for training and testing purposes which aims to establish reliable validation procedures. Both XGBoost and SVM with SMO operate for classification tasks while their achievement levels are assessed through standard measurement criteria which include accuracy, precision, recall and F1-score. A logistic regression analysis validates whether PSA levels and Gleason scores are significant factors in predictive modeling. Results: XGBoost achieves superior results over SVM with SMO according to every mitigation evaluation indicator. The classification metric for XGBoost displays 0.95 accuracy whereas SVM with SMO achieves 0.87 accuracy. The evaluation of XGBoost produced precision measurements of 0.92 while recall reached 0.93 and F1-score achieved 0.92. These evaluation scores surpassed SVM with SMO which generated precision of 0.85, recall of 0.80 and F1-score of 0.82. XGBoost manages to lower the chances of overfitting while improving biomarker interpretation so it shows clear potential for clinical prostate cancer prediction applications. Conclusions: The conducted research demonstrates XGBoost's effectiveness for prostate cancer classification through better prediction accuracy along with stronger resistance than SVM with SMO. The research demonstrates AI-powered models should become an integral part of clinical practice to improve the detection and treatment preparation for patients.

Support Vector Machines (SVM) is a practical algorithm that has been widely used in many areas. To guarantee its satisfying performance, it is important to set appropriate parameters of SVM algorithm. Sequential Minimal Optimization (SMO) is an effective training algorithm belonging to SVM, i.e.LS_SVM. Therefore, on the basis of the SMO algorithm and LS_SVM, which integrates SMO algorithm and LS_SVM, we introduced Particle Swarm Optimization (PSO) algorithm, and utilized an example to certify its validity. PSO is proposed to deal with the large amount of data, and the simulation results showed the effectiveness of this method.

Sequential minimal optimization (SMO) algorithm is one of the simplest decomposition methods for learning of support vector machines (SVMs). Keerthi and Gilbert have recently studied the convergence property of SMO algorithm and given a proof that SMO algorithm always stops within a finite number of iterations. In this letter, we point out the incompleteness of their proof and give a more rigorous proof.

Sequential minimal optimization (SMO) is widely used for training support vector machines (SVMs) because of fast training. But the training slows down when a large margin parameter value is used. Training by Newton’s method (NM) accelerates training in such a situation but it slows down for a small margin parameter value. To solve this problem, in this paper we fuse SMO with NM and call it SMO-NM. Because slow training is caused by repetitive corrections of the same variables, we modify the working set selection when they are detected. We call the variables that are selected by SMO, SMO variables. At the current step, if a variable selected as an SMO variable was selected in a previous step, we consider that a loop is detected. And in addition to the SMO variables, we add, to the working set, the unbounded variables that were selected as SMO variables and correct the variables by NM. If no loop is detected, the training procedure is the same as that of SMO. As a variant of this working set strategy, we further add violating variables to the working set. We clarify that if the classification problem is not linearly separable in the feature space, the solutions of L1/L2 SVMs (with the linear sum/square sum of slack variables) are unbounded as the margin parameter value approaches infinity but that, if the mapped training data are not linearly independent in the feature space, the solution of the least squares SVM is unbounded as the margin parameter approaches infinity. We also clarify the condition, in which the increment of the objective function value by SMO-NM is larger than that by SMO. We evaluate SMO-NM for several benchmark data sets and confirm the effectiveness over SMO especially for a large margin parameter value.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Sequential Minimal Optimization (SMO) is one of the state-of-the-art solutions for Support Vector Machines (SVM) training. SMO sequentially picks a pair of training samples for joint optimization, which minimizes the usage of memory. However, to find such two samples, SMO uses one outer loop through the entire training set looking for a sample which violates the KKT conditions, and one inner loop to look for a sample which heuristically maximizes the increase of the SVM objective function. While there is limited room for improvement in the inner loop, the outer loop can be reduced significantly if we can locate a small subset of training samples which potentially contains the Support Vectors (SVs).We present a Fisher Discriminant Analysis(FDA) based approach to locate potential SVs in a small core subset, which enables the SVM training to be focused effectively on a promising small region instead of the entire SV-sparse training set. Nevertheless, the training points outside the subset can not be blindly ignored. The method iteratively trains the classifier on the core subset and tune it using the remainder subset. Experimental results show that the approach significantly speeds up SMO training by reducing kernel evaluations, which dominates the training time. It converges much faster to optimal solution with much fewer iterations than standard SMO.

The nodules and the multiple times larger non-nodules of the regions of interested(ROIs) in lung areas are achieved, that would lead to a serious imbalance on the sample data. Many scholars have proposed some algorithms to solve this problem. In this paper, in order to guarantee that there is no correlation among the extracted characteristics, the PCA method is adopted to optimize and reduce dimensions, and then the modified support vector machine(SVM) classifiers using the sequential minimal optimization(SMO) algorithm and the grid research method are proposed to improve the computing efficiency. Furthermore, the abundant lung CT images from the hospital partnership could confirm the experimental results. We compare the classification performance between the ensemble SVM(ESVM) classifier and the cost-sensitive SVM(CSVM) classifier to deal with this problem. Experimental results show the performance of the CSVM classifier based on grid search is satisfactory than the ESVM.

We propose three techniques for improving accuracy and speed of margin stochastic gradient descent support vector machines (MSGDSVM). The first technique is to use sampling with full replacement. The second technique is to use the new update rule derived from the squared hinge loss function. The third technique is to limit the number of values for tuning of the margin hyperparameter M. We also provide theoretical analysis of a novel optimization problem for the proposed update rule. The first two techniques improve accuracy of MSGDSVM and the last one speed of tuning. Experiments show that the proposed method achieves superior accuracy compared to MSGDSVM for binary and multiclass classification, with similar generalization performance to sequential minimal optimization (SMO) and is faster than MSGDSVM.KeywordsSupport vector machinesStochastic gradient descentEarly stoppingSquared hinge loss function

Insulation defects that occur in gas-insulated switchgear (GIS), which is one of the most important types of equipment in the power grid, can lead to serious accidents. The ultrasonic detection method is commonly used to detect partial discharge (PD) signals in power equipment to discover defects. However, the traditional method to diagnose defects in GIS with ultrasonic PD signals is still based on the experience of testers. In this study, a classification system was proposed to identify insulation defects of GIS, based on voiceprint recognition technology. Twelve coefficients from mel frequency cepstral coefficient (MFCC) and 24 delta MFCC features were extracted as the acoustic features of the system. A support vector machine (SVM) multi-classifier was constructed to perform the classification and the sequential minimal optimization (SMO) algorithm was used to optimize the computational efficiency of the SVM. The experiments were conducted on a 110 kV GIS with different kinds of insulation defects. The results verified that the classification system with SMO-SVM achieved better identification accuracy and efficiency than the system with SVM. Therefore, it reveals the feasibility of the system to realize identification of insulation defects in GIS automatically and accurately.

One of the main causes of death the world over are cardiovascular diseases, of which coronary artery disease (CAD) is a major type. This disease occurs when the diameter narrowing of one of the left anterior descending, left circumflex, or right coronary arteries is equal to or greater than 50 percent. Angiography is the principal diagnostic modality for the stenos is of heart vessels, however, because of its complications and costs, researchers are looking for alternative methods such as data mining. This study conducts data mining algorithms on the Z-Alizadeh Sani dataset which has been collected from 303 random visitors to Tehran's Shaheed Rajaei Cardiovascular, Medical and Research Center. In this paper, the reason of effectiveness of a preprocessing algorithm on the dataset is investigated. This algorithm which has been merely introduced in our previous works, extracts three new features from the dataset. These features are then used to enrich the primary dataset in order to achieve more accurate results. Moreover, despite the fact that misclassification of diseased patients has more side effects than that of healthy ones, to the best of our knowledge cost-sensitive algorithms have yet to be used in this field. Therefore, in this paper 10-fold cross validation on cost-sensitive algorithms along with base classifiers of Naïve Bayes, Sequential Minimal Optimization (SMO), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and C4.5 were employed. As a result, the SMO algorithm has yield to very high sensitivity (97.22%) and accuracy (92.09%) rates, the likes of which have not been reported simultaneously in the existing literature.

To ensure appropriate soundscape management in urban environments, the urban-planning authorities need a range of tools that enable such a task to be performed. An essential step during the management of urban areas from a sound standpoint should be the evaluation of the soundscape in such an area. In this sense, it has been widely acknowledged that a subjective and acoustical categorization of a soundscape is the first step to evaluate it, providing a basis for designing or adapting it to match people's expectations as well. In this sense, this work proposes a model for automatic classification of urban soundscapes. This model is intended for the automatic classification of urban soundscapes based on underlying acoustical and perceptual criteria. Thus, this classification model is proposed to be used as a tool for a comprehensive urban soundscape evaluation. Because of the great complexity associated with the problem, two machine learning techniques, Support Vector Machines (SVM) and Support Vector Machines trained with Sequential Minimal Optimization (SMO), are implemented in developing model classification. The results indicate that the SMO model outperforms the SVM model in the specific task of soundscape classification. With the implementation of the SMO algorithm, the classification model achieves an outstanding performance (91.3% of instances correctly classified).

The paper presents the good classification accuracy of Support Vector Machine (SVM) with optimized parameters by particle swarm optimizer (PSO) in Chinese text classification. Through the simulation we also see that its training speed is slow when we deal with large amounts of texts in dataset, and it affects classification performance. Platt's sequential minimal optimization (SMO) is one of the fastest algorithms for training SVMs, so we introduce the distributed SMO using multiple core processors to process the training data as a fast classification of large amounts of texts in dataset.

Sequential minimal optimization (SMO) and Karush-Kuhn-Tucker condition are often used to solve learning problems in support vector machines. However, during hardware implementation of the SMO algorithm, enhancing chip performance without excessively increasing chip area is often a crucial issue. The solution proposed in this paper is a novel reconfigurable and efficient chip design with SMO-based training accelerator (REC-STA). Two novel methods used in the proposed REC-STA are trimode coarse-grained reconfigurable architecture (TCRA) and triple finite-state-machine with dynamic scheduling The first method modifies the baseline SMO design by developing trimode reconfigurable architectures with parallel and pipeline computing capabilities. The second method provides a schedule for efficient reconfiguration of the TCRA. Use of these methods can remove kernel cache design. For chip manufacturing, the implementation of the REC-STA is synthesized, placed, and routed using the TSMC 0.18-μm technology library. The core size is 2.94 mm × 2.94 mm and the power consumption is 77.3 mW. Compared with the baseline design, the FPGA simulation results show that the proposed architecture requires 50% less memory and 31% fewer gate counts but provides a 16-fold improvement in training performance. The experimental results confirm the efficacy of the proposed architecture and methods.

Sequential Minimal Optimization (SMO) is one of simple but fast iterative algorithm for Support Vector Machine (SVM), while there is a large amount of vector multiplication in SMO, which is still expensive and time-consuming. In this paper, we propose our Semi-sparse algorithm to enhance the vector multiplication in the SMO algorithms for large-scale sparse matrices. In the worst scenario, the traditional sparse algorithm on SMO needs O(n1+n2) times of judgments and addressing on two sparse vectors which own m and n elements respectively, while Semi-sparse algorithm can nearly finish this multiplying process within O(n2). Our experimental results on two benchmarks show that the modified SVMTorch based on our Semi-sparse algorithm can perform significantly faster than SVMTorch based on the original sparse algorithm.