The Upper Preferred Multiple Directed Acyclic Graph Support Vector Machines for Classification

Recursive projection twin support vector machine (PTSVM) and Locality preserving projection twin support vector machine (LPPTSVM) are two extensions of traditional support vector machine (SVM). However, they may lead to a weak classifier in the application where some data points may not be fully assigned to one class. In this paper, we introduce the basic idea of fuzzy membership into LPPTSVM and propose a fuzzy LPPTSVM (FLPPTSVM) algorithm. Through fuzzy weighting technique, input points with different confidence degrees can make different contributions to the learning of decision surface. In this way, FLPPTSVM not only maintains the advantages of LPPTSVM which considers the local geometrical structure of the data and enhances the generalization ability of the algorithm, but also decreases the influence of outliers and noises. We construct experiments on artificial and UCI benchmark datasets and the results show the effectiveness and robustness of the proposed FLPPTSVM method.

Based on the recently proposed projection twin support vector machine (PTSVM) and least squares projection twin support vector machine (LSPTSVM), in this paper, we propose a weighted linear loss projection twin support vector machine, namely WLPTSVM for short. By introducing the weighted linear loss function, the proposed WLPTSVM not only solves systems of linear equations with lower computational cost but also obtains comparable classification accuracy. In addition, it is able to dispose of large scale classification problems efficiently without any extra external optimizers. The experiments conducted on synthetic and several benchmark datasets illustrate the effectiveness of our WLPTSVM.

Classical machine learning algorithms seem to be totally incapable of processing tremendous data, while quantum machine learning algorithms could deal with big data unhurriedly and provide exponential acceleration over classical counterparts. In this paper, we propose two quantum support vector machine algorithms for multi classification. One is the quantum version of the directed acyclic graph support vector machine. The other one is to use the Grover search algorithm before measurement, which amplifies the amplitude of the phase storing of the classification result. For k classification, the former provides quadratic reduction in computational complexity when classifying. The latter accelerates the training speed significantly and more importantly, the classification result can be read out with a probability of at least 50% using only one measurement. We conduct numerical simulations on two algorithms, and their classification success rates are 96% and 88.7%, respectively.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Recursive least squares projection twin support vector machines for nonlinear classification

Weighted least squares projection twin support vector machines with local information

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

Structural regularized projection twin support vector machine for data classification

Automatic classification of electrocardiogram (ECG) is importance in cardiac disease diagnosis. Support vector machine (SVM) has drawn more and more attention on pattern recognition, including ECG feature extraction and cardiac disease detection. The most prominent advantage of SVM can be represent as its excellent performance on simplification of inner product operation from high dimensional space to low dimensional space, avoiding calculations in high dimension space. In this study, a multi-classification method is proposed utilizing wavelet multi-resolution analysis (WMRA) and SVM. WMRA is applied to eliminate interference with frequency beyond the frequency interval of ECG signals (0.05∼100Hz). Meanwhile, WMRA provides detail coefficients and approximation coefficients of different decomposition levels, which are the input features fed into SVM for classification. After that, SVM is employed to recognize 6 types of cardiac beats from MIT-BIH arrhythmia database. Besides, different parameters C and γ are discussed and tested. Experimental results indicate that the classification performance gets better as C increases and γ decreases. When C and γ are set to be 1000 and 0.1 respectively, an overall classification accuracy, sensitivity and positive predictivity of 95.23%, 97.42% and 97.71% respectively are achieved.

In the magnetic-suspension bearing-rotor system, the shape features of the rotor axis trajectory can directly reflect the system operation status and fault information. Accurately classifying the bearing faults can promptly find the cause of the fault and respond quickly and protect it. In this paper, in order to achieve fault identification by identifying and classifying the rotor axis trajectory, we use Hu moment invariants to extract feature vectors from the rotor axis trajectory to identify faults. We use imperial competitive algorithm to reduce redundant data and establish a directed acyclic graph support vector machine to classify and identify feature data. Compared with traditional classification models such as C4.5 and BP neural networks, the experimental results show that the support vector machine model based on Hu moment invariants and simulated annealing has a higher accuracy and robustness in small sample classification. It is suitable for intelligent identification and classification for bearing faults.

Because the solution of Least squares projection twin support vector machine (LSPTSVM) is obtained by solving two smaller-size systems of linear equations, LSPTSVM has closed form solution and it is faster than classical least squares support vector machine (LSSVM). However, LSPTSVM has two disadvantages, it is sensitive to outliers and its solution lacks sparsity. Therefore, the LSPTSVM is difficult to training large-scale datasets. To solve this problem, we propose the robust LSPTSVM (R-LSPTSVM) by applying truncated least squares loss function which is a bounded function. R-LSPTSVM can be represented as difference of convex programming, then the primal solution of R-LSPTSVM can be obtained by using Concave-Convex Procedure. The sparse solution of R-LSPTSVM is iteratively obtained by using the pivoting Cholesky factorization. Furthermore, sparse R-LSPTSVM algorithm (SR-LSPTSVM) is proposed. Experimental results show that the SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fast.

<p style='text-indent:20px;'>Projection twin support vector machine (PTSVM) is an effective tool for classification. However, it is sensitive to outliers or the noise due to the utilization of the squared L2-norm distance. To alleviate the sensitivity to outliers or the noise, we propose a capped L1-norm projection twin support vector machine (CPTSVM), where the L2-norm distance is replaced by the capped L1-norm to confer the robustness to classifiers. CPTSVM is formulated as a pair of non-convex and non-smooth SVM-type problems. To solve these difficult problems, we present an iterative algorithm for CPTSVM as well as its convergence properties. Numerical experiments on artificial and benchmark datasets demonstrate the robustness and feasibility of our proposed method.</p>

This paper is focused on the design of a new approach dedicated to solve classification problems for the detection of broken rotor bar (BRB) fault in induction motors (IM). This new method finds its origins in a novel combination of both recursive undecimated wavelet packet transform (RUWPT) and directed acyclic graph support vector machines (DAG SVMs). Most often, BRB frequency components are hardly detected in the stator current due to its low magnitude and closeness to the supply frequency component. To overcome this drawback, the RUWPT is applied to extract one parameter able to detect the fault with arbitrary working conditions and a great concern of low load cases. Different multiclass support vector machines (MSVMs) methods are evaluated with respect to accuracy, number of support vectors, and testing time. The experimental results confirm that the DAG SVMs and Symlet wavelet kernel function are fast, robust, and give the best classification accuracy of 99%.

The Projection Twin Support Vector Machine (PTSVM) and its variant, the Least Squares PTSVM (LSPTSVM), have demonstrated significant effectiveness in supervised classification tasks due to their strong generalization capabilities. However, their reliance on fully labeled data limits their applicability in real-world scenarios, where obtaining complete labeled datasets is often challenging and expensive. To overcome this limitation, we propose a novel Manifold Energy Projection Twin SVM (MEPTSVM) model for semi-supervised learning, which extends PTSVMs by integrating manifold regularization and energy-based margins. Our proposed MEPTSVM offers the following advancements over traditional PTSVMs: Firstly, MEPTSVM incorporates manifold regularization to leverage unlabeled data, capturing the underlying geometric structure of the dataset. By integrating both labeled and unlabeled samples, the model derives a more generalizable classification boundary that accounts for the global data distribution. Secondly, instead of employing a fixed distance margin between classes, MEPTSVM introduces an energy-based margin for each projection. This adaptive approach better captures the discriminative characteristics of different classes, enhancing classification performance and improving generalization. Finally, the effectiveness and practicality of MEPTSVM are demonstrated through extensive experiments on both synthetic and real-world datasets. The results validate its superiority in leveraging unlabeled data to improve classification accuracy and robustness.

In this paper, a new epileptic seizure detection method using fuzzy-rules-based sub-band specific features and layered directed acyclic graph support vector machine (LDAG-SVM) is proposed for classification of electroencephalogram (EEG) signals. Wavelet transformation is used to decompose the input EEG signals into various sub-bands. The nonlinear features, namely approximate entropy, largest Lyapunov exponent and correlation dimension, are extracted from each sub-band. In this proposed work, sub-band specific feature subset that is reduced in size and capable of discriminating samples is selected by employing fuzzy rules. For classification purpose, a new LDAG-SVM is used for detecting epileptic seizure. Every sub-band has its own characteristics. If appropriate features which characterize the specific sub-band are selected, then the classification accuracy is improved and computational complexity is reduced. The important advantage of the fuzzy logic is its close relation to human thinking. Due to the lengthy record and intra-professional variability, automation of epileptic detection is inevitable. Fuzzy rules are the natural choice of employing human expertise to build machine learning system. Performances of the proposed methods are evaluated using two different benchmark EEG datasets, namely Bonn and CHB-MIT. The performance measures such as classification accuracy, sensitivity, specificity, execution time and receiver operating characteristics are used to measure and analyze the performances of the proposed classifier. The proposed LDAG-SVM with fuzzy-rules-based selected sub-band specific features provides better performance in terms of improved classification accuracy with reduced execution time compared to existing methods.