Fuzzy locality preserving projection twin support vector machine for classification

Based on the recently proposed projection twin support vector machine (PTSVM) and least squares projection twin support vector machine (LSPTSVM), in this paper, we propose a weighted linear loss projection twin support vector machine, namely WLPTSVM for short. By introducing the weighted linear loss function, the proposed WLPTSVM not only solves systems of linear equations with lower computational cost but also obtains comparable classification accuracy. In addition, it is able to dispose of large scale classification problems efficiently without any extra external optimizers. The experiments conducted on synthetic and several benchmark datasets illustrate the effectiveness of our WLPTSVM.

Weighted least squares projection twin support vector machines with local information

The current classification algorithms have weak fault-tolerance. In order to solve problem, a multiple support vector machines method, called Upper preferred Multiple Directed Acyclic Graph Support Vector Machines (UMDAG-SVMs), is proposed. Firstly, we present least squares projection twin support vector machine (LSPTSVM) with confidence-degree for generating binary classifiers. It uses idea that when confidence-degree outputted from node in directed graph, is below threshold, decision-making process will go on along with two branches of node at same time., which strengthens algorithm's fault-tolerance. In order to select parameters of algorithm, we use genetic algorithm to select these parameters. Secondly, according to minimal hypersphere distance, and known principle the upper-level classifiers bring up better performance of classification in DAG-SVMs , we present a new classification algorithm, called UMDAG-SVMs. This algorithm has two advantages of strong fault-tolerance and high classification accuracy. Finally, we make experiments to test performance of algorithm. Experimental results in public datasets show that our UMDAG-SVMs has comparable classification accuracy to that other algorithms but with remarkable less computation.

Structural regularized projection twin support vector machine for data classification

Recursive least squares projection twin support vector machines for nonlinear classification

<p style='text-indent:20px;'>Projection twin support vector machine (PTSVM) is an effective tool for classification. However, it is sensitive to outliers or the noise due to the utilization of the squared L2-norm distance. To alleviate the sensitivity to outliers or the noise, we propose a capped L1-norm projection twin support vector machine (CPTSVM), where the L2-norm distance is replaced by the capped L1-norm to confer the robustness to classifiers. CPTSVM is formulated as a pair of non-convex and non-smooth SVM-type problems. To solve these difficult problems, we present an iterative algorithm for CPTSVM as well as its convergence properties. Numerical experiments on artificial and benchmark datasets demonstrate the robustness and feasibility of our proposed method.</p>

Because the solution of Least squares projection twin support vector machine (LSPTSVM) is obtained by solving two smaller-size systems of linear equations, LSPTSVM has closed form solution and it is faster than classical least squares support vector machine (LSSVM). However, LSPTSVM has two disadvantages, it is sensitive to outliers and its solution lacks sparsity. Therefore, the LSPTSVM is difficult to training large-scale datasets. To solve this problem, we propose the robust LSPTSVM (R-LSPTSVM) by applying truncated least squares loss function which is a bounded function. R-LSPTSVM can be represented as difference of convex programming, then the primal solution of R-LSPTSVM can be obtained by using Concave-Convex Procedure. The sparse solution of R-LSPTSVM is iteratively obtained by using the pivoting Cholesky factorization. Furthermore, sparse R-LSPTSVM algorithm (SR-LSPTSVM) is proposed. Experimental results show that the SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fast.

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

The Projection Twin Support Vector Machine (PTSVM) and its variant, the Least Squares PTSVM (LSPTSVM), have demonstrated significant effectiveness in supervised classification tasks due to their strong generalization capabilities. However, their reliance on fully labeled data limits their applicability in real-world scenarios, where obtaining complete labeled datasets is often challenging and expensive. To overcome this limitation, we propose a novel Manifold Energy Projection Twin SVM (MEPTSVM) model for semi-supervised learning, which extends PTSVMs by integrating manifold regularization and energy-based margins. Our proposed MEPTSVM offers the following advancements over traditional PTSVMs: Firstly, MEPTSVM incorporates manifold regularization to leverage unlabeled data, capturing the underlying geometric structure of the dataset. By integrating both labeled and unlabeled samples, the model derives a more generalizable classification boundary that accounts for the global data distribution. Secondly, instead of employing a fixed distance margin between classes, MEPTSVM introduces an energy-based margin for each projection. This adaptive approach better captures the discriminative characteristics of different classes, enhancing classification performance and improving generalization. Finally, the effectiveness and practicality of MEPTSVM are demonstrated through extensive experiments on both synthetic and real-world datasets. The results validate its superiority in leveraging unlabeled data to improve classification accuracy and robustness.

Smoothing methods, extensively used for solving important mathematical programming problems and applications, are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel. We term such reformulation a smooth support vector machine (SSVM). A fast Newton–Armijo algorithm for solving the SSVM converges globally and quadratically. Numerical results and comparisons are given to demonstrate the effectiveness and speed of the algorithm. On six publicly available datasets, tenfold cross validation correctness of SSVM was the highest compared with four other methods as well as the fastest. On larger problems, SSVM was comparable or faster than SVMlight (T. Joachims, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999), SOR (O.L. Mangasarian and David R. Musicant, IEEE Transactions on Neural Networks, vol. 10, pp. 1032–1037, 1999) and SMO (J. Platt, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999). SSVM can also generate a highly nonlinear separating surface such as a checkerboard.

The inclusion in workplaces of video display terminals has brought multiple benefits for the organization of work. Nevertheless, it also implies a series of risks for the health of the workers, since it can cause ocular and visual disorders, among others. In this research, a group of eye and vision-related problems associated with prolonged computer use (known as computer vision syndrome) are studied. The aim is to select the characteristics of the subject that are most relevant for the occurrence of this syndrome, and then, to develop a classification model for its prediction. The estimate of this problem is made by means of support vector machines for classification. This machine learning technique will be trained with the support of a genetic algorithm. This provides the training of the support vector machine with different patterns of parameters, improving its performance. The model performance is verified in terms of the area under the ROC curve, which leads to a model with high accuracy in the classification of the syndrome.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Integration of feature vector selection and support vector machine for classification of imbalanced data

A Clifford support vector machine (CSVM) learns the decision surface from multi distinct classes of the multiple input points using the Clifford geometric algebra. In many applications, each multiple input point may not be fully assigned to one of these multi-classes. In this paper, we apply a fuzzy membership to each multiple input point and reformulate the CSVM for multiclass classification to make different input points have their own different contributions to the learning of decision surface. We call the proposed method Clifford fuzzy SVM.

The development of DNA microarray makes researchers screen thousands of genes simultaneously and it also helps determine high- and low-expression level genes in normal and disease tissues. Selecting relevant genes for cancer classification is an important issue. Most of the gene selection methods use univariate ranking criteria and arbitrarily choose a threshold to choose genes. However, the parameter setting may not be compatible to the selected classification algorithms. In this paper, we propose a new gene selection method (SVM-t) based on the use of t-statistics embedded in support vector machine. We compared the performance to two similar SVM-based methods: SVM recursive feature elimination (SVMRFE) and recursive support vector machine (RSVM). The three methods were compared based on extensive simulation experiments and analyses of two published microarray datasets. In the simulation experiments, we found that the proposed method is more robust in selecting informative genes than SVMRFE and RSVM and capable to attain good classification performance when the variations of informative and noninformative genes are different. In the analysis of two microarray datasets, the proposed method yields better performance in identifying fewer genes with good prediction accuracy, compared to SVMRFE and RSVM.