New Method Based on Support Vector Machine in Classification for Hyperspectral Data

Band selection (BS), which selects a subset of original bands that contain the most useful information about objects, is an important technique to reduce the dimensionality of hyperspectral data. Dimensionality reduction before hyperspectral data classification can reduce redundancy information and even improve classification accuracy. We propose BS algorithms based on an ant colony optimization (ACO) in conjunction with objective functions such as the supervised Jeffries–Matusita distance and unsupervised simplex volume. Moreover, we propose to use a small number of selected pixels for BS in order to reduce computational cost in the unsupervised BS. In this experiment, the proposed algorithms were applied to three airborne hyperspectral datasets including urban scenes, and the results demonstrated that the ACO-based BS could find a better combination of bands than the widely used sequential forward search-based BS. It was acceptable to use a few pixels to achieve comparable BS performance with our method.

Hyperspectral image classification has been an active topic of research. In recent years, it has been found that light detection and ranging (LiDAR) data provide a source of complementary information that can greatly assist in the classification of hyperspectral data, in particular when it is difficult to separate complex classes. This is because, in addition to the spatial and the spectral information provided by hyperspectral data, LiDAR can provide very valuable information about the height of the surveyed area that can help with the discrimination of classes and their separability. In the past, several efforts have been investigated for fusion of hyperspectral and LiDAR data, with some efforts driven by the morphological information that can be derived from both data sources. However, a main challenge for the learning approaches is how to exploit the information coming from multiple features. Specifically, it has been found that simple concatenation or stacking of features such as morphological attribute profiles (APs) may contain redundant information. In addition, a significant increase in the number of features may lead to very high-dimensional input features. This is in contrast with the limited number of training samples often available in remote-sensing applications, which may lead to the Hughes effect. In this work, we develop a new efficient strategy for fusion and classification of hyperspectral and LiDAR data. Our approach has been designed to integrate multiple types of features extracted from these data. An important characteristic of the presented approach is that it does not require any regularization parameters, so that different types of features can be efficiently exploited and integrated in a collaborative and flexible way. Our experimental results, conducted using a hyperspectral image and a LiDAR-derived digital surface model (DSM) collected over the University of Houston campus and the neighboring urban area, indicate that the proposed framework for multiple feature learning provides state-of-the-art classification results.

The responsibility weighted Mahalanobis kernel for semi-supervised training of support vector machines for classification

Valley-loss regular simplex support vector machine for robust multiclass classification

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The high-dimensional feature vectors of hyper spectral data often impose a high computational cost as well as the risk of “over fitting ” when classification is performed. Therefore it is necessary to reduce the dimensionality through ways like feature selection. Currently, there are two kinds of feature selection methods: filter methods and wrapper methods. The form kind requires no feedback from classifiers and estimates the classification performance indirectly. The latter kind evaluates the “goodness ” of selected feature subset directly based on the classification accuracy. Many experimental results have proved that the wrapper methods can yield better performance, although they have the disadvantage of high computational cost. In this paper, we present a Genetic Algorithm (GA) based wrapper method for classification of hyper spectral data using Support Vector Machine (SVM), a state-of-art classifier that has found success in a variety of areas. The genetic algorithm (GA), which seek to solve optimization problems using the methods of evolution, specifically survival of the fittest, was used to optimize both the feature subset, i.e. band subset, of hyper spectral data and SVM kernel parameters simultaneously. A special strategy was adopted to reduce computation cost caused by the high-dimensional feature vectors of hyper spectral data when the feature subset part of chromosome was designed. The GA-SVM method was realized using the ENVI/IDL language, and was then tested by applying to a HYPERION hyper spectral image. Comparison of the optimized results and the un-optimized results showed that the GA-SVM method could significantly reduce the computation cost while improving the classification accuracy. The number of bands used for classification was reduced from 198 to

This paper addresses the development of a novel zero-shot learning method for remote marine litter hyperspectral imaging data classification. The work consisted of using an airborne acquired marine litter hyperspectral imaging dataset that contains data about different plastic targets and other materials and assessing the viability of detecting and classifying plastic materials without knowing their exact spectral response in an unsupervised manner. The classification of the marine litter samples was divided into known and unknown classes, i.e., classes that were hidden from the dataset during the training phase. The obtained results show a marine litter automated detection for all the classes, including (in the worst case of an unknown class) a precision rate over 56% and an overall accuracy of 98.71%.

Computer vision and pattern recognition systems play an important role in our lives by means of automated face detection, face and gesture recognition, and estimation of gender and age. We have developed a gender classifier with performance superior to existing gender classifiers. This paper addresses the problem of gender classification using frontal facial images. The testbed consists of 500 images (250 females and 250 males) randomly withdrawn from the FERET facial database. Independent component analysis (ICA) is used to represent each image as a feature vector in a low dimensional subspace. Different classifiers are studied in this lower dimensional subspace. Our experimental results show the best accuracy of 96% in gender classification by combining ICA and support vector machines (SVMs).

Introduction: The condition of prostate cancer continues to represent a substantial medical issue because it necessitates precise predictive models that support healthcare decisions. A research study investigates how Extreme Gradient Boosting (XGBoost) performs when identifying significant biomarkers to anticipate prostate cancer risks. The research compares XGBoost to Sequential Minimal Optimization (SMO) in Support Vector Machines (SVM) to evaluate its better performance in classification. Prostate-Specific Antigen (PSA) levels together with Gleason scores and molecular markers form the clinical parameters within the dataset. Process data correctly by cleansing it and normalizing it while extracting important features to achieve the optimal model performance. Objectives: This research investigates how well Extreme Gradient Boosting (XGBoost) predicts prostate cancer through an examination of its function to discover complex non-linear data patterns in clinical datasets. The research compares XGBoost to SMO algorithm of Support Vector Machines through a performance assessment of classification metrics including accuracy, precision, recall and F1-score. The research explores two important biomarkers Prostate-Specific Antigen (PSA) levels along with Gleason scores which affect how prostate cancer is expected to progress. The research applies machine learning advanced methods to develop improved early identification systems that lead to better clinical decisions which subsequently enhance patient health results. Methods: The research method relies on a structured machine learning process where multiple steps start with detailed data preprocessing for inconsistent data elimination and variable normalization. Feature extraction technologies increase model interpretability by being applied to the analysis. The available dataset contains two well-separated sections for training and testing purposes which aims to establish reliable validation procedures. Both XGBoost and SVM with SMO operate for classification tasks while their achievement levels are assessed through standard measurement criteria which include accuracy, precision, recall and F1-score. A logistic regression analysis validates whether PSA levels and Gleason scores are significant factors in predictive modeling. Results: XGBoost achieves superior results over SVM with SMO according to every mitigation evaluation indicator. The classification metric for XGBoost displays 0.95 accuracy whereas SVM with SMO achieves 0.87 accuracy. The evaluation of XGBoost produced precision measurements of 0.92 while recall reached 0.93 and F1-score achieved 0.92. These evaluation scores surpassed SVM with SMO which generated precision of 0.85, recall of 0.80 and F1-score of 0.82. XGBoost manages to lower the chances of overfitting while improving biomarker interpretation so it shows clear potential for clinical prostate cancer prediction applications. Conclusions: The conducted research demonstrates XGBoost's effectiveness for prostate cancer classification through better prediction accuracy along with stronger resistance than SVM with SMO. The research demonstrates AI-powered models should become an integral part of clinical practice to improve the detection and treatment preparation for patients.

Smoothing methods, extensively used for solving important mathematical programming problems and applications, are applied here to generate and solve an unconstrained smooth reformulation of the support vector machine for pattern classification using a completely arbitrary kernel. We term such reformulation a smooth support vector machine (SSVM). A fast Newton–Armijo algorithm for solving the SSVM converges globally and quadratically. Numerical results and comparisons are given to demonstrate the effectiveness and speed of the algorithm. On six publicly available datasets, tenfold cross validation correctness of SSVM was the highest compared with four other methods as well as the fastest. On larger problems, SSVM was comparable or faster than SVMlight (T. Joachims, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999), SOR (O.L. Mangasarian and David R. Musicant, IEEE Transactions on Neural Networks, vol. 10, pp. 1032–1037, 1999) and SMO (J. Platt, in Advances in Kernel Methods—Support Vector Learning, MIT Press: Cambridge, MA, 1999). SSVM can also generate a highly nonlinear separating surface such as a checkerboard.

The inclusion in workplaces of video display terminals has brought multiple benefits for the organization of work. Nevertheless, it also implies a series of risks for the health of the workers, since it can cause ocular and visual disorders, among others. In this research, a group of eye and vision-related problems associated with prolonged computer use (known as computer vision syndrome) are studied. The aim is to select the characteristics of the subject that are most relevant for the occurrence of this syndrome, and then, to develop a classification model for its prediction. The estimate of this problem is made by means of support vector machines for classification. This machine learning technique will be trained with the support of a genetic algorithm. This provides the training of the support vector machine with different patterns of parameters, improving its performance. The model performance is verified in terms of the area under the ROC curve, which leads to a model with high accuracy in the classification of the syndrome.

Hyperspectral data potentially contain more information than multispectral data because of their higher spectral resolution. However, the stochastic data analysis approaches that have been successfully applied to multispectral data are not as effective for hyperspectral data as well. Various investigations indicate that the key problem that causes poor performance in the stochastic approaches to hyperspectral data classification is inaccurate class parameters estimation. It has been found that the conventional approaches can be retained if a preprocessing stage is established before feature extraction procedure in classification of hyperspectral data. For preprocessing stage it has been proposed two steps in this paper including dimensionality reduction and class separability improvement. Sequential Parametric Projection Pursuit was used for dimensionality reduction because of its special characteristics. Projection Pursuit algorithm performs the computation of class parameter estimation at a lower dimensional space, giving better parameter estimation. For class separability improvement a lowpass filter has been used after dimensionality reduction. This paper shows that for different number of features, classification accuracy is improved when the preprocessing stage is applied.

Sequential minimal optimization (SMO) is widely used for training support vector machines (SVMs) because of fast training. But the training slows down when a large margin parameter value is used. Training by Newton’s method (NM) accelerates training in such a situation but it slows down for a small margin parameter value. To solve this problem, in this paper we fuse SMO with NM and call it SMO-NM. Because slow training is caused by repetitive corrections of the same variables, we modify the working set selection when they are detected. We call the variables that are selected by SMO, SMO variables. At the current step, if a variable selected as an SMO variable was selected in a previous step, we consider that a loop is detected. And in addition to the SMO variables, we add, to the working set, the unbounded variables that were selected as SMO variables and correct the variables by NM. If no loop is detected, the training procedure is the same as that of SMO. As a variant of this working set strategy, we further add violating variables to the working set. We clarify that if the classification problem is not linearly separable in the feature space, the solutions of L1/L2 SVMs (with the linear sum/square sum of slack variables) are unbounded as the margin parameter value approaches infinity but that, if the mapped training data are not linearly independent in the feature space, the solution of the least squares SVM is unbounded as the margin parameter approaches infinity. We also clarify the condition, in which the increment of the objective function value by SMO-NM is larger than that by SMO. We evaluate SMO-NM for several benchmark data sets and confirm the effectiveness over SMO especially for a large margin parameter value.

Aircraft engine is an important component of the airplane, which lifespan and reliability have directly influence to flight safety. Vibration analysis is an effective fault diagnosis method for airplane's mechanical construction fault. Several typical vibration faults have drawn from the vibration analysis on the basis of fault mechanism in aircraft engine, and the standard vibration fault-symptom (Fuzzy Membership) relationship table have constructed based on Fuzzy Mathematic theory in this paper. Sample data are simulated on the basis of this table. Sequential Minimal Optimization (SMO) is a fast algorithm for training Support Vector Machines (SVM). The simulated result showed a great improvement on computation efficiency by selecting double threshold parameter for SMO algorithm than single threshold. In order to examine the algorithm's effect applying in the aircraft engine fault diagnosis, Radial Basis Function (RBF) Neural Network, C4.5 Decision Tree, SVM (chunking), Platt's SMO and Improved SMO algorithm are applied to the training data and test data. It is shown that the Improved SMO algorithm displayed superior comprehensive performance. Finally, the vibration fault feature is reduced based on Fuzzy-Rough Sets theory. The research shows that the Improved SMO algorithm is quicker than other algorithm in aircraft engine fault diagnosis with the reduced feature attributes, and its accuracy is high.

By Using data mining techniques, much study has been undertaken to derive information from medical data, which may enhance the curing ability. This study has been compared four automated methods for blood diseases detection using a different method for attributes reduction methods: PCA (Principal Component Analysis), ICA (Independent Component Analysis), original data set, and four algorithms, which are Naive Bayes, K-Nearest Neighbor, Decision Tree, and SVM (Support Vector Machine) for classification. The results show that after applying the PCA technique, the accuracy is: 0.98, 1.0, 0.79, and 0.78 for Naive Bayes, Decision Tree, SVM, and KNN and after applying the ICA technique, the accuracy is 0.76,.76, 0.76, and 0.74 for Naive Bayes, Decision Tree, SVM, and KNN, while the accuracy after applying on the original data set is: 0.98, 1.0, 0.78, and 0.75 for Naive Bayes, Decision Tree, SVM, and KNN. The results appeared that when applying decision tree on the original data set and after applying the PCA method, have been gotten the highest accuracy which is 100 %.