Hyperspectral Image Classification Based on Non-Parallel Support Vector Machine

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

Support vector machine (SVM), as a supervised learning method, has different kinds of varieties with significant performance. In recent years, more research focused on nonparallel SVM, where twin SVM (TWSVM) is the typical one. In order to reduce the influence of outliers, more robust distance measurements are considered in these methods, but the discriminability of the models is neglected. In this article, we propose robust manifold twin bounded SVM (RMTBSVM), which considers both robustness and discriminability. Specifically, a novel norm, that is, capped L₁-norm, is used as the distance metric for robustness, and a robust manifold regularization is added to further improve the robustness and classification performance. In addition, we also use the kernel method to extend the proposed RMTBSVM for nonlinear classification. We introduce the optimization problems of the proposed model. Subsequently, effective algorithms for both linear and nonlinear cases are proposed and proved to be convergent. Moreover, the experiments are conducted to verify the effectiveness of our model. Compared with other methods under the SVM framework, the proposed RMTBSVM shows better classification accuracy and robustness.

Discriminative information-based nonparallel support vector machine

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

Twin support vector machine (TWSVM) and twin support vector regression (TSVR) are newly emerging efficient machine learning techniques which offer promising solutions for classification and regression challenges respectively. TWSVM is based upon the idea to identify two nonparallel hyperplanes which classify the data points to their respective classes. It requires to solve two small sized quadratic programming problems (QPPs) in lieu of solving single large size QPP in support vector machine (SVM) while TSVR is formulated on the lines of TWSVM and requires to solve two SVM kind problems. Although there has been good research progress on these techniques; there is limited literature on the comparison of different variants of TSVR. Thus, this review presents a rigorous analysis of recent research in TWSVM and TSVR simultaneously mentioning their limitations and advantages. To begin with, we first introduce the basic theory of support vector machine, TWSVM and then focus on the various improvements and applications of TWSVM, and then we introduce TSVR and its various enhancements. Finally, we suggest future research and development prospects.

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L2-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L2-norm loss that behaves like the quadratic L2-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

Credit consumption has become the choice of more and more people. While the expansion of credit consumption brings convenience, the consequent problem of discredit brings incalculable loss to trust institutions, such as intentional arrears, malicious overdraft consumption, etc. to the trust in the operation of the huge losses caused. On the one hand, our country is unable to establish perfect personal credit record, customer credit information is difficult to share, access to relevant data is limited, personal credit system is not perfect, therefore, it is necessary to establish a perfect and automatic personal credit evaluation system, identify the personal credit risk scientifically, and realize the maximization of bank credit income. In the bank risk management, the main problem is how to measure and avoid the bank's financial risk, and personal credit assessment is the most difficult and important part. In this paper, we use the XGBOOST algorithm to sort the importance of features, and select the number of features and the specific features according to the relationship between the model accuracy and the number of features, then 80% of the data was taken into the traditional Support vector machine model as a training set and tested with the remaining data with low accuracy, so that, in terms of individual credit risk assessment, this paper introduces the quadric surface into the traditional Support vector machine and builds the kernel-free quadric surface Support vector machine model, which improves the accuracy by 9.8 percentage points and has a guiding significance for the bank reference risk control.

In the field of machine learning, support vector machine is a supervised learning model which can analyze the data and identify patterns, and this theory is used for classification and regression analysis which is related to the learning algorithm.Support vector machine is a new machine learning method based on statistical learning theory, and has become a hot research topic in the machine learning field because of its excellent learning performance.Support vector machine is also a learning machine based on kernel function, and its generalization ability depends on the chosen kernel function to a great extent.Nevertheless, traditional support vector machine cannot achieve the desired results when training large-scale data set.Therefore, in order to improve training efficiency and generalization performance of support vector machine, we have to improve this algorithm.Although support vector machine has great advantages in theory, the research on its application is comparatively delayed.In this paper, we focus on novel support vector machine generated from multi-subjects amalgamation, such as fuzzy support vector machine, granular support vector machine, twin support vector machine.We analyze a large number of literature review about development trend of support vector machine and its applications in computer field, laying foundation for the systematic research on the correlation algorithm based on support vector machine.

This paper presents a combined solution for Big Data classification, by using one of the extended versions of the Support Vector Machines (SVM), known as the Parallel Support Vector Machines (PSVM). The main problem assumes that, once a PSVM model is obtained, a feature can be removed overtime, resulting in a decrease of the accuracy with the existing model. While Big Data is one of the interesting contexts, then training a new PSVM with the new data structure is time-consuming. The solution is to use an approach that approximates any SVM model based on the Radial Basis Function (RBF) kernel, and called the Approx SVM. In order to avert a new training step, this paper proposes to apply the Approx SVM in a parallel architecture. Despite that the Approx SVM was not purposely used to deal with large-scaled data set, the experimental results, which will be presented at the end of the article, are proofs that this approach is an appropriate choice for PSVM models.

Union nonparallel support vector machines framework with consistency

The traditional vector-based algorithms, such as Support Vector Machine (SVM) and Twin Support Vector Machine (TSVM), have many limitations especially when tensor is considered as input matrix. In this paper, we proposed a novel algorithm with a tensor-based classification paradigm to utilize the structural information. The new proposed algorithm is called  -Twin Support Tensor Machine ( -TSTM), which is an extension of  -Twin Support Vector Machine ( -TSVM). Similarly,  -TSTM solves a pair of smaller-sized Quadratic Programming Problems (QPPs). It reduces the computational complexity substantially. Besides, we formulate  -TSTM, which separates samples in the tensor space with two non-parallel hyperplanes, and the pair of parameters ( ) have theoretical interpretation which are used to control the bounds of the fractions of support tensors and the error margins. What’s more, the structure information of data is retained by the direct use of tensor representation. The proposed  -TSTM can preferably overcome overfitting problem and deal with big data while most vector-based algorithms could hardly compare. In addition, it has better performances on high dimensional and small-sample-size (S3) problem. The efficiency and superiority of the proposed method are demonstrated by experiments on various datasets.

Twin support vector machine (TWSVM) is a new machine learning method, as opposed to solving a single quadratic programming problem in support vector machine (SVM), which generates two nonparallel hyperplanes by solving two smaller size quadratic programming problems. However, the TWSVM obtains the final classifier by giving the same importance to all training samples which may be important for classification performance. In order to address this problem, in this paper, we propose a novel entropy-based fuzzy twin bounded support vector machine (EFTBSVM) for binary classification problems. By considering the fuzzy membership value for each sample and assigning it based on the entropy value, the samples with higher class certainty are assigned to relatively larger fuzzy membership. In addition, the proposed EFTBSVM not only maintains the superior characteristics of the TWSVM but also exploits the structural risk minimization principle by introducing a regularization term. The experimental results achieved on synthetic datasets and benchmark datasets illustrate the effectiveness of the proposed method.

Hyperspectral images (HSI) offer detailed spectral reflectance information about sensed objects through provision of information on hundreds of narrow spectral bands. HSI have a leading role in a broad range of applications, such as in forestry, agriculture, geology, and environmental sciences. The monitoring and management of agricultural lands is of great importance for meeting the nutritional and other needs of a rapidly and continuously increasing world population. In relation to this, classification of HSI is an effective way for creating land use and land cover maps quickly and accurately. In recent years, classification of HSI using convolutional neural networks (CNN), which is a sub-field of deep learning, has become a very popular research topic and several CNN architectures have been developed by researchers. The aim of this study was to investigate the classification performance of CNN model on agricultural HSI scenes. For this purpose, a 3D-2D CNN framework and a well-known support vector machine (SVM) model were compared using the Indian Pines and Salinas Scene datasets that contain crop and mixed vegetation classes. As a result of this study, it was confirmed that use of 3D-2D CNN offers superior performance for classifying agricultural HSI datasets.

The condition monitoring and fault diagnosis of power transformers plays a significant role in the safe, stable and reliable operation of the whole power system. Dissolved gas analysis (DGA) methods are widely used for fault diagnosis, however, their accuracy is limited by the selection of DGA features and the performance of fault diagnosis models, for example, the classical support vector machine (SVM), is easily affected by unbalanced training samples. This paper presents a transformer fault diagnosis model based on chemical reaction optimization and a twin support vector machine. Twin support vector machines (TWSVMs) are used as classifiers for solving problems involving unbalanced and insufficient samples. Restricted Boltzmann machines (RBMs) are used for data preprocessing to ensure the effective identification of feature parameters and improve the efficiency and accuracy of fault diagnosis. The chemical reaction optimization (CRO) algorithm is used to optimize TWSVM parameters to select the optimal training parameters. The cross-validation (CV) method is used to ensure the reliability and generalization ability of the diagnostic model. Finally, the validity of the model is verified using real fault samples and random testing.

Near infrared spectroscopy was used in discrimination of intact bovine teeth in terms of animal sex, diet, tooth type and place of origin. Discriminant Analysis (DA) models were developed and tested using a stratified random 70:30 data split to calibration and test sets. Of the discriminant techniques of Soft Independent Modelling of Class Analogy (SIMCA), Partial Least Squares – DA, Artificial Neural Networks (ANN) and Support Vector Machine (SVM), SVM and PLS-DA models performed best in most instances, with pretreatment choice impacting technique success. For test set prediction of animal sex, an accuracy of 95% was achieved using a PLS-DA model, and similar performance achieved using a SVM model. SVM, ANN and SIMCA models predicted three categories of tooth type (deciduous, permanent unerupted and erupted) with similar accuracies (89, 79 and 82%, respectively), however, permanent unerupted teeth were conflated with permanent erupted teeth. Prediction accuracy for ANN, SVM and PLS-DA models were also similar for discrimination of teeth from animals on a grain versus grass diet (PLS-DA 82%). There was considerable variation between models in prediction of geographic origin. Of the seven locations, two were discriminated with true positive rate around 70% using SVM and ANN models. Potential applications of this technology are discussed.