Twin support vector machines based on fruit fly optimization algorithm

Twin support vector machines (TWSVM) is based on the idea of proximal SVM based on generalized eigenvalues (GEPSVM), which determines two nonparallel planes by solving two related SVM-type problems, so that its computing cost in the training phase is 1/4 of standard SVM. In addition to keeping the superior characteristics of GEPSVM, the classification performance of TWSVM significantly outperforms that of GEPSVM. However, the stand-alone method requires the solution of two smaller quadratic programming problems. This paper mainly reviews the research progress of TWSVM. Firstly, it analyzes the basic theory and the algorithm thought of TWSVM, then tracking describes the research progress of TWSVM including the learning model and specific applications in recent years, finally points out the research and development prospects.

We propose Twin SVM, a binary SVM classifier that determines two nonparallel planes by solving two related SVM-type problems, each of which is smaller than in a conventional SVM. The Twin SVM formulation is in the spirit of proximal SVMs via generalized eigenvalues. On several benchmark data sets, Twin SVM is not only fast, but shows good generalization. Twin SVM is also useful for automatically discovering two-dimensional projections of the data.

abstractUniversum, a third class not belonging to either class of the classification problem, allows to incorporate the prior knowledge into the learning process. A lot of previous work have demonstrated that the Universum is helpful to the supervised and semi-supervised classification. Moreover, Universum has already been introduced into the support vector machine (SVM) and twin support vector machine (TSVM) to enhance the generalisation performance. To further increase the generalisation performance, we propose a least squares TSVM with Universum data (-TSVM) in this paper. Our -TSVM possesses the following advantages: first, it exploits Universum data to improve generalisation performance. Besides, it implements the structural risk minimisation principle by adding a regularisation to the objective function. Finally, it costs less computing time by solving two small-sized systems of linear equations instead of a single larger-sized quadratic programming problem. To verify the validity of our proposed algorithm, we conduct various experiments around the size of labelled samples and the number of Universum data on data-sets including seven benchmark data-sets, Toy data, MNIST and Face images. Empirical experiments indicate that Universum contributes to making prediction accuracy improved even stable. Especially when fewer labelled samples given, -TSVM is far superior to the improved LS-TSVM (ILS-TSVM), and slightly superior to the -TSVM.

Weighted Twin Support Vector Machines with Local Information and its application

Color image classification and retrieval through ternary decision structure based multi-category TWSVM

Purpose – In order to avoid enterprise crisis and cause the domino effect, which influences the investment return of investors, the national economy, and financial crisis, establishing a complete set of feasible financial early warning model can help to prevent the possibility of enterprise crisis in advance, and thus, reduce the influence on society and the economy. The purpose of this paper is to develop an efficient financial crisis warning model. Design/methodology/approach – First, the fruit fly optimization algorithm (FOA) is used to adjust the coefficients of the parameters in the ZSCORE model (we call it the FOA_ZSCORE model), and the difference between the forecasted value and the real target value is calculated. Afterward, the generalized regressive neural network (GRNN model), with optimized spread by FOA (we call it FOA_GRNN model), is used to forecast the difference to promote the forecasting accuracy. Various models, including ZSCORE, FOA_ZSCORE, FOA_ZSCORE+GRNN, and FOA_ZSCORE+FOA_GRNN, are trained and tested. Finally, different models are compared based on their prediction accuracies and ROC curves. Furthermore, more appropriate parameters, which are different from the parameters in the original ZSCORE model, are selected by using the multivariate adaptive regression splines (MARS) method. Findings – The hybrid model of the FOA_ZSCORE together with the FOA_GRNN offers the highest prediction accuracy, compared to other models; the MARS can be used to select more appropriate parameters to further improve the performance of the prediction models. Originality/value – This paper proposes a hybrid model, FOA_ZSCORE+FOA_GRNN which offers better performance than the original ZSCORE model.

Sparse pinball twin support vector machines

Twin support vector machine (TWSVM) is a classical distance metric learning method for classification problems. The formulation of TWSVM criterion is based on L2-norm distance, which makes TWSVM prone to being influenced by the presence of outliers. In this paper, to develop a robust distance metric learning method, we propose a new objective for TWSVM classifier using L1-norm distance metric, termed as L1-TWSVM. The optimization strategy is to maximize the ratio of the inter-class distance dispersion to the intra-class distance dispersion by using L1-norm distance rather than L2-norm distance. Besides, we design a simple and valid iterative algorithm to solve L1-norm optimal problems, which is easy to actualize and its convergence to an optimum is theoretically ensured. The efficiency and robustness of L1-TWSVM have been validated by experiments on UCI datasets and artificial datasets. The promising experimental results indicate that our proposals outperform relevant state-of-the-art methods in all kinds of experimental settings.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

This paper presents a modified fruit fly optimization algorithm(FOA). The proposed modified FOA establishes a balanced tradeoff between exploration and exploitation, and thus overcomes original FOA's drawbacks of premature convergence and easy trapping in a local optima. In the proposed modified FOA, firstly, the whole population performs a global search; Secondly, the whole population are sequenced in descending order by the individual fitness value; Thirdly, every n consecutive individuals are divided into a meme group, then every meme group iteratively performs a deep search around the local optima; Finally, all the meme groups are mixed, and then the above process is implemented iteratively until meeting the end conditions. The modified FOA efficiently avoids relapsing into local optima and improves convergence precision. Finally, our modified algorithm was validated against the original by testing on six standard benchmark functions, and comparisons show that the performance of the proposed modified FOA is much better than original FOA.

DSLC-FOA : Improved fruit fly optimization algorithm for application to structural engineering design optimization problems

Glaucoma is a group of eye diseases that cause an irreparable decrease in the view field which is the second most common and leading causes of blindness among retinal diseases. Computer aided diagnosis (CAD) system is an emerging field in medical informatics which has high importance for providing prognosis of diseases. Research efforts have reported with increasing confirmation that the twin support vector machines (TWSVM) have greater accurate diagnosis ability. The goal of TWSVM is to construct two non-parallel planes for each class such that each hyper-plane is closer to one of two classes and as far as possible from the other one. In this paper, we propose a new CAD system for glaucoma diagnosis using TWSVM and three heterogeneous families of feature extraction. In this work, we have used 169 images to classify into normal and glaucoma classes. The performance of the method is evaluated using classification accuracy, sensitivity, specificity and receiver operating characteristic (ROC) curves. The results show that the highest classification accuracy (98,53%) is obtained for the TWSVM using Gaussian kernel function, and this is very promising compared to the SVM results.

KNN weighted reduced universum twin SVM for class imbalance learning

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.