Weighted Linear Loss Projection Twin Support Vector Machine for Pattern Classification

Recursive projection twin support vector machine (PTSVM) and Locality preserving projection twin support vector machine (LPPTSVM) are two extensions of traditional support vector machine (SVM). However, they may lead to a weak classifier in the application where some data points may not be fully assigned to one class. In this paper, we introduce the basic idea of fuzzy membership into LPPTSVM and propose a fuzzy LPPTSVM (FLPPTSVM) algorithm. Through fuzzy weighting technique, input points with different confidence degrees can make different contributions to the learning of decision surface. In this way, FLPPTSVM not only maintains the advantages of LPPTSVM which considers the local geometrical structure of the data and enhances the generalization ability of the algorithm, but also decreases the influence of outliers and noises. We construct experiments on artificial and UCI benchmark datasets and the results show the effectiveness and robustness of the proposed FLPPTSVM method.

Recursive least squares projection twin support vector machines for nonlinear classification

Structural regularized projection twin support vector machine for data classification

<p style='text-indent:20px;'>Projection twin support vector machine (PTSVM) is an effective tool for classification. However, it is sensitive to outliers or the noise due to the utilization of the squared L2-norm distance. To alleviate the sensitivity to outliers or the noise, we propose a capped L1-norm projection twin support vector machine (CPTSVM), where the L2-norm distance is replaced by the capped L1-norm to confer the robustness to classifiers. CPTSVM is formulated as a pair of non-convex and non-smooth SVM-type problems. To solve these difficult problems, we present an iterative algorithm for CPTSVM as well as its convergence properties. Numerical experiments on artificial and benchmark datasets demonstrate the robustness and feasibility of our proposed method.</p>

Weighted least squares projection twin support vector machines with local information

Because the solution of Least squares projection twin support vector machine (LSPTSVM) is obtained by solving two smaller-size systems of linear equations, LSPTSVM has closed form solution and it is faster than classical least squares support vector machine (LSSVM). However, LSPTSVM has two disadvantages, it is sensitive to outliers and its solution lacks sparsity. Therefore, the LSPTSVM is difficult to training large-scale datasets. To solve this problem, we propose the robust LSPTSVM (R-LSPTSVM) by applying truncated least squares loss function which is a bounded function. R-LSPTSVM can be represented as difference of convex programming, then the primal solution of R-LSPTSVM can be obtained by using Concave-Convex Procedure. The sparse solution of R-LSPTSVM is iteratively obtained by using the pivoting Cholesky factorization. Furthermore, sparse R-LSPTSVM algorithm (SR-LSPTSVM) is proposed. Experimental results show that the SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fast.

The current classification algorithms have weak fault-tolerance. In order to solve problem, a multiple support vector machines method, called Upper preferred Multiple Directed Acyclic Graph Support Vector Machines (UMDAG-SVMs), is proposed. Firstly, we present least squares projection twin support vector machine (LSPTSVM) with confidence-degree for generating binary classifiers. It uses idea that when confidence-degree outputted from node in directed graph, is below threshold, decision-making process will go on along with two branches of node at same time., which strengthens algorithm's fault-tolerance. In order to select parameters of algorithm, we use genetic algorithm to select these parameters. Secondly, according to minimal hypersphere distance, and known principle the upper-level classifiers bring up better performance of classification in DAG-SVMs , we present a new classification algorithm, called UMDAG-SVMs. This algorithm has two advantages of strong fault-tolerance and high classification accuracy. Finally, we make experiments to test performance of algorithm. Experimental results in public datasets show that our UMDAG-SVMs has comparable classification accuracy to that other algorithms but with remarkable less computation.

The Projection Twin Support Vector Machine (PTSVM) and its variant, the Least Squares PTSVM (LSPTSVM), have demonstrated significant effectiveness in supervised classification tasks due to their strong generalization capabilities. However, their reliance on fully labeled data limits their applicability in real-world scenarios, where obtaining complete labeled datasets is often challenging and expensive. To overcome this limitation, we propose a novel Manifold Energy Projection Twin SVM (MEPTSVM) model for semi-supervised learning, which extends PTSVMs by integrating manifold regularization and energy-based margins. Our proposed MEPTSVM offers the following advancements over traditional PTSVMs: Firstly, MEPTSVM incorporates manifold regularization to leverage unlabeled data, capturing the underlying geometric structure of the dataset. By integrating both labeled and unlabeled samples, the model derives a more generalizable classification boundary that accounts for the global data distribution. Secondly, instead of employing a fixed distance margin between classes, MEPTSVM introduces an energy-based margin for each projection. This adaptive approach better captures the discriminative characteristics of different classes, enhancing classification performance and improving generalization. Finally, the effectiveness and practicality of MEPTSVM are demonstrated through extensive experiments on both synthetic and real-world datasets. The results validate its superiority in leveraging unlabeled data to improve classification accuracy and robustness.

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

Smoothing methods, extensively used for solving important mathematical programming problems and applications, are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel. We term such reformulation a smooth support vector machine (SSVM). A fast Newton–Armijo algorithm for solving the SSVM converges globally and quadratically. Numerical results and comparisons are given to demonstrate the effectiveness and speed of the algorithm. On six publicly available datasets, tenfold cross validation correctness of SSVM was the highest compared with four other methods as well as the fastest. On larger problems, SSVM was comparable or faster than SVMlight (T. Joachims, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999), SOR (O.L. Mangasarian and David R. Musicant, IEEE Transactions on Neural Networks, vol. 10, pp. 1032–1037, 1999) and SMO (J. Platt, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999). SSVM can also generate a highly nonlinear separating surface such as a checkerboard.

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

The use of handwritten recognition tools has increased yearly in various commercialized fields. Due to this, handwritten classification, recognition, and detection have become an exciting research subject for many scholars. Different techniques have been provided to improve character recognition accuracy while reducing time for languages like English, Arabic, Chinese and European languages. The local or regional languages need to consider for research to increase the scope of handwritten recognition tools to the global level. This paper presents a machine learning-based technique that provides an accurate, robust, and fast solution for handwritten Pashto text classification and recognition. Pashto belongs to cursive script division, which has numerous challenges to classify and recognize. The first challenge during this research is developing efficient and full-fledged datasets. The efficient recognition or prediction of Pashto handwritten text is impossible by using ordinary feature extraction due to natural transformations and handwriting variations. We propose some useful invariant features extracting techniques for handwritten Pashto text, i.e., radial, orthographic grid, perspective projection grid, retina, the slope of word trajectories, and cosine angles of tangent lines. During the dataset creation, salt and pepper noise was generated, which was removed using the statistical filter. Another challenge to face was the invalid disconnected handwritten stroke trajectory of words. We also proposed a technique to minimize the problem of disconnection of word trajectory. The proposed approach uses a linear support vector machine (SVM) and RBF-based SVM for classification and recognition.

Introduction: Lobular endocervical glandular hyperplasia (LEGH) is a benign lesion; however, it is considered to be the origin of gastric-type adenocarcinoma in the uterine cervix, and early diagnosis is important. At Shinshu University Hospital, screening of LEGH cells is based on the difference in color tone of cytoplasmic mucin on Papanicolaou staining and detection of gastric mucin using HIK1083-labeled latex agglutination assay. However, it is sometimes difficult to distinguish LEGH cells with subtle nuclear atypia from endocervical (EC) cells. Methods: We calculated the Gabor filter features (mean signal value, standard deviation, skewness, kurtosis) from the nuclei of cytological specimens in EC cells (37 cases) and LEGH cells (33 cases) using microscopic images, and we performed statistical analysis and discriminant analysis by linear support vector machine (LSVM) using these features. A Gabor filter is a linear filter defined as a mathematical representation of the mammalian visual system. Gabor filters with three wavelengths and eight angles were used for analysis. Results: Gabor filter features in EC cells were higher than in LEGH cells, demonstrating that the gradient of LEGH cell nuclei was milder than that of EC cell nuclei. The accuracy calculated using all Gabor filters was 91.0% and the accuracy of four Gabor filters (λ = 2/3π and θ = 0°, 45°, 90°, 135°) was 88.9%. High accuracy with low computation costs was achieved by reducing the number of features used for LSVM. Conclusion: The application of a Gabor filter with convolutional processing resulted in the edges of LEGH cells being slightly rough and thick, whereas those of EC cells were fine and thin. Thus, it is thought that the frequency of abrupt gradients of pixels was higher in EC cells than in LEGH cells, and the gradient of chromatin distribution in LEGH cell nuclei was milder than that in EC cell nuclei. It was possible to evaluate nuclear findings of EC and LEGH cells objectively by quantifying morphological features of nuclei using Gabor filters. It was possible to differentiate EC cells from LEGH cells using LSVM using Gabor filter features.

Laplacian least squares twin support vector machine for semi-supervised classification

This study compares the performance of local binary patterns (LBP) and scale invariant feature transform (SIFT) with support vector machines (SVM) in automatic classification of discrete facial expressions. Facial expression recognition is a multiclass classification problem and seven classes; happiness, anger, sadness, disgust, surprise, fear and comtempt are classified. Using SIFT feature vectors and linear SVM, 93.1% mean accuracy is acquired on CK+ database. On the other hand, the performance of LBP-based classifier with linear SVM is reported on SFEW using strictly person independent (SPI) protocol. Seven-class mean accuracy on SFEW is 59.76%. Experiments on both databases showed that LBP features can be used in a fairly descriptive way if a good localization of facial points and partitioning strategy are followed.