Data-driven mechanism based on fuzzy Lagrangian twin parametric-margin support vector machine for biomedical data analysis

Aiming at the sensitivity of twin support vector machines to noise and outliers, an improved fuzzy twin support vector machine is proposed. Aiming at the shortcomings of fuzzy membership based on the distance between a sample and its class center, this algorithm proposes a new fuzzy membership function that can effectively reflect sample uncertainty— based on intra-class hyperplane and sample affinity. When determining the membership function, the intra-class hyperplane replaces the class center, reducing the dependence on the sample set geometry and improving the model generalization ability. At the same time, considering the relation between each sample and the confusion degree of sample points, the sample affinity is applied to the design of membership function, so as to effectively distinguish the effective sample from noise and outliers and improve the classification accuracy. The experimental results show that compared with the twin support vector machine and the classic fuzzy support vector machine, the improved fuzzy twin support vector machine has better classification performance.

Safe intuitionistic fuzzy twin support vector machine for semi-supervised learning

KNN weighted reduced universum twin SVM for class imbalance learning

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

In this paper, a novel nonparallel hyperplane based classifier termed as “angle-based twin support vector machine” (ATWSVM) has been proposed which is motivated by the concept of twin support vector machine (TWSVM). TWSVM obtains two nonparallel hyperplanes by solving a pair of quadratic programming problems (QPPs). ATWSVM presents a generic classification model, where the first problem can be formulated using a TWSVM-based classifier and the second problem is an unconstrained minimization problem (UMP) which is reduced to solving a system of linear equations. The second hyperplane is determined so that it is proximal to its own class and the angle between the normals to the two hyperplanes is maximized. The notion of angle has been introduced to have maximum separation between the two hyperplanes. In this work, we have presented two versions of ATWSVM: one that solves a QPP and a UMP; second which formulates both the problems as UMPs. The training time of ATWSVM is much less than that of TWSVM because ATWSVM solves the second problem as UMP instead of QPP. To test the efficacy of the proposed classifier, experiments have been conducted on synthetic and benchmark datasets and it is observed that the proposed classifier achieves classification accuracy comparable or better than that of TWSVM. This work also proposes application of ATWSVM for color image segmentation.

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

In the field of machine learning, support vector machine is a supervised learning model which can analyze the data and identify patterns, and this theory is used for classification and regression analysis which is related to the learning algorithm.Support vector machine is a new machine learning method based on statistical learning theory, and has become a hot research topic in the machine learning field because of its excellent learning performance.Support vector machine is also a learning machine based on kernel function, and its generalization ability depends on the chosen kernel function to a great extent.Nevertheless, traditional support vector machine cannot achieve the desired results when training large-scale data set.Therefore, in order to improve training efficiency and generalization performance of support vector machine, we have to improve this algorithm.Although support vector machine has great advantages in theory, the research on its application is comparatively delayed.In this paper, we focus on novel support vector machine generated from multi-subjects amalgamation, such as fuzzy support vector machine, granular support vector machine, twin support vector machine.We analyze a large number of literature review about development trend of support vector machine and its applications in computer field, laying foundation for the systematic research on the correlation algorithm based on support vector machine.

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

Weighted Twin Support Vector Machines with Local Information and its application

EEG signal classification using universum support vector machine

The twin support vector machine (TWSVM) classifier and its fuzzy variant fuzzy twin support vector machine (FTSVM) have received considerable attention due to their low computational complexity. However, their performance often deteriorates when the input data is affected by noise. To overcome this limitation, this study leverages the robustness of granular-ball computing (GBC) against noise to develop more effective classification models by integrating GBC with TWSVM and FTSVM. First, we introduce the granular-ball TWSVM (GBTWSVM) classifier, which incorporates GBC with the TWSVM framework. By replacing traditional point-wise inputs with granular-ball representations, we derive a pair of nonparallel hyperplanes for the GBTWSVM classifier by solving a quadratic programming problem. Afterwards, we develop the granular-ball FTSVM (GBFTSVM) classifier, where the membership and nonmembership functions of granular-balls are defined using Pythagorean fuzzy sets, enabling a more nuanced differentiation of the contributions of granular-balls from distinct regions within the input space. By incorporating these functions into the FTSVM framework, we derive a pair of nonparallel hyperplanes for the GBFTSVM classifier through the solution of a quadratic programming problem. Finally, we present algorithms for the GBTWSVM and GBFTSVM classifiers and evaluate their performance on 21 benchmark datasets. Experimental results demonstrate the superior scalability, computational efficiency, and robustness of the proposed classifiers in pattern recognition, highlighting their potential as advanced tools for noise-tolerant classification.

Improved intuitionistic fuzzy twin support vector machine ensemble training for imbalanced data classification

Truncated loss functions are robust to class noise and outliers. A robust twin bounded support vector machine is proposed in this paper that truncates the growth of its loss functions at a pre-specified point, thus, flattens the function that pre-specified score afterwards. Moreover, to make the proposed method capable of handling datasets with different imbalance ratio, cost-sensitive learning is implemented by scaling the total error of the classes based on the number of samples of each class. However, the adopted loss functions take a non-convex structure which does not always assure global optimum. To handle this issue, we suggest concave-convex procedure (CCCP) to ensure global convergence by decomposing the cost functions into additions of one convex and one concave part. The behaviour of the proposed method for varied imbalance ratio is analysed experimentally and depicted graphically. Classification performance of the proposed method in terms of AUC, F-Measure and G-Mean is compared with other related methods, viz. Hinge loss support vector machine (SVM), Ramp loss SVM (RSVM), twin SVM (TWSVM), twin bounded SVM (TBSVM), Pinball loss SVM (pin-SVM), entropy-based fuzzy SVM (EFSVM), non-parallel hyperplane Universum SVM (U-NHSVM), stochastic gradient twin support vector machine (SGTSVM), k-nearest neighbor (KNN)-based maximum margin and minimum volume hyper-sphere machine (KNN-M3VHM) and affinity and class probability-based fuzzy SVM (ACFSVM) on several real-world datasets with imbalance ratio raging from low to high. Further, to establish the significance of the proposed method in pattern classification, pair-wise statistical comparison of the methods is performed based on their average ranks on AUC. Experimental results are convincing.