Support vector machines based on subtractive clustering

Lithology prediction by support vector classifiers using inverted seismic attributes data and petrophysical logs as a new approach and investigation of training data set size effect on its performance in a heterogeneous carbonate reservoir

Many studies have focused on the prediction of corporate credit rating using various data mining techniques. One of the most frequently used algorithms is support vector machines (SVM), and recently, novel techniques such as SVM+ and SVM+MTL have emerged. This paper intends to show the applicability of such new techniques to multi-classification and corporate credit rating and compare them with conventional SVM regarding prediction performance. We solve multi-class SVM+ and SVM+MTL problems by constructing several binary classifiers. Furthermore, to demonstrate the robustness and outstanding performance of SVM+MTL algorithm over other techniques, we utilized four typical multi-class processing methods in our experiments. The results show that SVM+MTL outperforms both conventional SVM and novel SVM+ in predicting corporate credit rating. This study contributes to the literature by showing the applicability of new techniques such as SVM+ and SVM+MTL and the outperformance of SVM+MTL over conventional techniques. Thus, this study enriches solving techniques for addressing multi-class problems such as corporate credit rating prediction.

In this paper, a novel semi-supervised subtractive clustering method using pair wise constraints is proposed. Because the traditional subtractive clustering method cannot obtain the better clustering accuracies, some pair wise constraints regarded as supervised information are applied to aiding the process of the traditional subtractive clustering. Experimental results show that the new presented semi-supervised clustering method can improve the clustering performance significantly compared to the unsupervised subtractive clustering.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Prediction of intrinsic solubility of generic drugs using MLR, ANN and SVM analyses

The theoretical analysis of multiclass classification has proved that the existing multiclass classification methods can train a classifier with high classification accuracy on the test set, when the instances are precise in the training and test sets with same distribution and enough instances can be collected in the training set. However, one limitation with multiclass classification has not been solved: how to improve the classification accuracy of multiclass classification problems when only imprecise observations are available. Hence, in this article, we propose a novel framework to address a new realistic problem called multiclass classification with imprecise observations (MCIMO), where we need to train a classifier with fuzzy-feature observations. First, we give the theoretical analysis of the MCIMO problem based on fuzzy Rademacher complexity. Then, two practical algorithms based on support vector machine and neural networks are constructed to solve the proposed new problem. The experiments on both synthetic and real-world datasets verify the rationality of our theoretical analysis and the efficacy of the proposed algorithms.

In the brain Magnetic Resonance (MR) images, the boundary of each encephalic tissue is highly irregular. It is difficult to accurately detect the encephalic tissues. Owing to its powerful capacity in solving non-linearity problems, Support Vector Machine (SVM) has been widely used in object detection. The conventional SVMs, however, assume that each feature of a sample has the same importance degree for the detection result, which is not a true representation of real applications. In addition, the parameters of the SVM and its kernel function also affect detection result. In this study, Immune Algorithm (IA) was introduced in searching for the optimal feature weights and the parameters simultaneously. An Immune Feature Weighted SVM (IFWSVM) method was used to detect encephalic tissues in MR images. Theoretical analysis and experimental results showed that the IFWSVM has better performance than the conventional methods.

Prediction of residual stress in the welding zone of dissimilar metals using data-based models and uncertainty analysis

A kernel-based subtractive clustering method

The computational difficulties occurred when we use a conventional support vector machine with nonlinear kernels to deal with massive datasets. The reduced support vector machine (RSVM) replaces the fully dense square kernel matrix with a small rectangular kernel matrix which is used in the nonlinear SVM formulation to avoid the computational difficulties. In this paper, we propose a new algorithm, Systematic Sampling RSVM (SSRSVM) that selects the informative data points to form the reduced set while the RSVM used random selection scheme. This algorithm is inspired by the key idea of SVM, the SVM classifier can be represented by support vectors and the misclassified points are a part of support vectors. SSRSVM starts with an extremely small initial reduced set and adds a portion of misclassified points into the reduced set iteratively based on the current classifier until the validation set correctness is large enough. In our experiments, we tested SSRSVM on six public available datasets. It turns out that SSRSVM might automatically generate a smaller size of reduced set than the one by random sampling. Moreover, SSRSVM is faster than RSVM and much faster than conventional SVM under the same level of the test set correctness.

Machine learning models play a vital role in the prediction task in several fields of study. In this work, we utilize the ability of machine learning algorithms to predict the occurrence of extreme events in a nonlinear mechanical system. Extreme events are rare events that occur ubiquitously in nature. We consider four machine learning models, namely Logistic Regression, Support Vector Machine, Random Forest and Multi-Layer Perceptron in our prediction task. We train these four machine learning models using training set data and compute the performance of each model using the test set data. We show that the Multi-Layer Perceptron model performs better among the four models in the prediction of extreme events in the considered system. The persistent behaviour of the considered machine learning models is cross-checked with randomly shuffled training set and test set data.

Cancer is the most common disease to threaten human health. The ability to screen individuals with malignant tumours with only a blood sample would be greatly advantageous to early diagnosis and intervention. This study explores the possibility of discriminating between cancer patients and normal subjects with serum surface-enhanced Raman spectroscopy (SERS) and a support vector machine (SVM) through a peripheral blood sample. A total of 130 blood samples were obtained from patients with liver cancer, colonic cancer, esophageal cancer, nasopharyngeal cancer, gastric cancer, as well as 113 blood samples from normal volunteers. Several diagnostic models were built with the serum SERS spectra using SVM and principal component analysis (PCA) techniques. The results show that a diagnostic accuracy of 85.5% is acquired with a PCA algorithm, while a diagnostic accuracy of 95.8% is obtained using radial basis function (RBF), PCA–SVM methods. The results prove that a RBF kernel PCA–SVM technique is superior to PCA and conventional SVM (C-SVM) algorithms in classification serum SERS spectra. The study demonstrates that serum SERS, in combination with SVM techniques, has great potential for screening cancerous patients with any solid malignant tumour through a peripheral blood sample.

Support vector machine (SVM) model is employed and tested for the soil surface roughness classification. SVM is calibrated (trained) and tested with the experimentally obtained data. The experimentally data is obtained by using X-band (9.5 GHz) scatterometer for two soil surface roughness 3.78 cm and 1.83 cm at constant soil surface moisture equal to 22.80%. The measurement of the scattering coefficient was carried out over a range of incidence angle from 20° to 70° at the step of 5° for both the HH and vv polarization. The performance of the SVM model is evaluated from the outcome classification result on trained data set and test data set. Radial Gaussian kernel function results 100% correct Classification and identification of soil surface roughness both in training and validation phase. SVM is a proficient technique for soil surface roughness classification by such experimentation and have numerous of advantages over artificial neural network (ANN) based approaches and other theoretical approaches as its less complexity and less time consumption ability.

Deformability of rock masses influencing their behavior is an important geomechanical property for the rock structures design. Due to the problems in determining the deformability of jointed rock masses at the laboratory-scale, various in situ test methods such as plate loading tests, dilatometer etc. have been developed. Although these methods are currently the best techniques, they are expensive and time consuming, and present operational problems. Furthermore, the influence of the test volume on modulus of deformation depending on the technique used is also important. For these reasons, in this paper, the adaptive network-based fuzzy inference system (ANFIS) was used to build a prediction model for the indirect estimation of deformation modulus of a rock mass. Three ANFIS models were implemented by grid partitioning (GP), subtractive clustering method (SCM) and fuzzy c-means clustering method (FCM). The estimation abilities offered using three ANFIS models were presented by using field data of achieved from road and railway construction sites in Korea. In these models, rock mass rating (RMR), depth, uniaxial compressive strength of intact rock (UCS) and elastic modulus of intact rock (Ei) were utilized as the input parameters, while the deformation modulus of a rock mass was the output parameter. Various statistical performance indexes were utilized to compare the performance of those estimation models. The results achieved indicate that the ANFIS-SCM model has strong potential to indirect estimation of deformation modulus of a rock mass with high degree of accuracy and robustness.

In this paper, a subtractive clustering identification algorithm is introduced to model type-2 Takagi-Sugeno-Kang (TSK) fuzzy logic systems (FLS). The type-2 TSK FLS identification algorithm is an extension of the type-1 TSK FLS modeling algorithm proposed in (S. L. Chiu, 1994), (S. L. Chiu, 1997). In the type-2 algorithm, subtractive clustering method is combined with least squares estimation algorithms to pre-identify a type-1 FLS form input/output data. Then using type-2 TSK FLS theory (J. M. Mendel, 2001), expand the type-1 FLS to a type-2 TSK FLS. Minimum error models are obtained through enumerative search of optimum values for spreading percentage of cluster centers and consequence parameters. By doing so, fuzzy modeling of type-2 TSK FLS is found to be more effective than that of type-1 TSK FLS. Experimental results confirm the effectiveness of this method. A comparison of the Type-1 and -2 TSK FLSs is presented and the limitations of this method are discussed