Fast Support Vector Machine Classification of Very Large Datasets

기업의 부도를 예측하는 것은 회계나 재무 분야에서 중요한 연구주제이다. 지금까지 기업 부도예측을 위해 여러 가지 데이터마이닝 기법들이 적용되었으나 주로 단일 모형을 사용함으로서 복잡한 분류 문제에의 적용에 한계를 갖고 있었다. 본 논문에서는 최근에 각광받고 있는 SVM (support vector machine) 모형들을 결합한 앙상블 SVM 모형 (ensemble SVM model)을 부도예측에 사용하고자 한다. 제안된 앙상블 모형은 v-조각 교차 타당성 (v-fold cross-validation)에 의해 얻어진 여러 가지 모형 중에서 성능이 좋은 상위 k개의 단일 모형으로 구성하고 과반수 투표 방식 (majority voting)을 사용하여 미지의 클래스를 분류한다. 본 논문에서 제안된 앙상블 SVM 모형의 성능을 평가하기 위해 실제 기업의 재무비율 자료와 모의실험자료를 가지고 실험하였고, 실험결과 제안된 앙상블 모형이 여러 가지 평가척도 하에서 단일 SVM 모형들보다 좋은 성능을 보임을 알 수 있었다. Corporate bankruptcy prediction has been an important topic in the accounting and finance field for a long time. Several data mining techniques have been used for bankruptcy prediction. However, there are many limits for application to real classification problem with a single model. This study proposes ensemble SVM (support vector machine) model which assembles different SVM models with each different kernel functions. Our ensemble model is made and evaluated by v-fold cross-validation approach. The k top performing models are recruited into the ensemble. The classification is then carried out using the majority voting opinion of the ensemble. In this paper, we investigate the performance of ensemble SVM classifier in terms of accuracy, error rate, sensitivity, specificity, ROC curve, and AUC to compare with single SVM classifiers based on financial ratios dataset and simulation dataset. The results confirmed the advantages of our method: It is robust while providing good performance.

Kernel selection is a key issue of support vector machine (SVM), however, for the existing criterions of kernel selection in the literature, there are some limitations in generalization ability , computational cost or fair comparison. This paper proposes a novel criterion of kernel selection for SVM named maximum robustness criterion. The properties of maximum robustness criterion guarantee that it has an excellent generalization ability. Moreover, the maximum robustness criterion is fair for different kernel functions because it is calculated in input space and has low computational cost. The experiment results on some benchmark data sets show that it not only achieves almost as good generalization ability as minimum misclassifying rate of k-fold cross validation, but also has less computational cost.

In this paper, we introduce a set of new kernel functions derived from the generalized Legendre polynomials to obtain more robust and higher support vector machine (SVM) classification accuracy. The generalized Legendre kernel functions are suggested to provide a value of how two given vectors are like each other by changing the inner product of these two vectors into a greater dimensional space. The proposed kernel functions satisfy the Mercer’s condition and orthogonality properties for reaching the optimal result with low number support vector (SV). For that, the new set of Legendre kernel functions could be utilized in classification applications as effective substitutes to those generally used like Gaussian, Polynomial and Wavelet kernel functions. The suggested kernel functions are calculated in compared to the current kernels such as Gaussian, Polynomial, Wavelets and Chebyshev kernels by application to various non-separable data sets with some attributes. It is seen that the suggested kernel functions could give competitive classification outcomes in comparison with other kernel functions. Thus, on the basis test outcomes, we show that the suggested kernel functions are more robust about the kernel parameter change and reach the minimal SV number for classification generally.

Data type classification is a significant problem in digital forensics and information security field. Methods based on support vector machine have proven the most successful across varying classification approaches in the previous work. However, the training process of SVM is notably computationally intensive with the number of training vectors increased rapidly. In this study, we proposed parallel distributed SVM (PDSVM) based on Hadoop MapReduce for scalable data type classification. First the map phase determines support vectors (SVs) in the splits of dataset by running the sequential minimal optimization. Then the reduce phase merges SVs and computes the degree of global convergence. Finally, PDSVM utilizes the global convergence SVs to get SVM model. The experimental results demonstrate that PDSVM can not only process large scale training dataset, but also perform well in the term of classification accuracy.

Support Vector Machines (SVMs) is a popular and highly performing classification model with a very good generalization ability for many classification applications. This method uses kernels to classify data that are not linearly separable. The added complexity of the kernels that map data to a higher dimensional space degrade the SVM classifier performance when dealing with large datasets. Moreover, classifying a dataset by choosing an appropriate kernel and finding the best set of its parameters is challenging. Failing in this process can easily cause to an overfit problem.;In this thesis we propose the Piece-wise Linear SVM (PWLSVM) using MagKmeans clustering to address the complexity and computational cost of SVMs. We use a linear SVM to overcome the complexity of dealing with the kernels, and a MagKmeans clustering to cluster the data into balanced groups. MagKmeans which is a supervised technique clusters equal number of each class in one group. It ensures that a linear SVM has balanced training samples for each class and can attain an accurate model.;The detailed mathematical formulation and modeling of the proposed Distributed MagKmeans (D-MagKmeans) is presented. The algorithm uses a Distributed MagKmeans clustering approach to transform the PWLSVM to Distributed Piece-wise Linear SVM (D-PWLSVM). The proposed D-MagKmeans clustering approach makes the MagKmeans clustering work in distributed network by only passing the centroid of each cluster in one node only to its one-hop neighbors. This feature of the D-Magkmeans makes our approach appropriate for a distributed processing and decision making while maximizing privacy and minimizing the communication overload.;The proposed algorithm was validated using four datasets in terms of dimensions on the features and the number of samples. Pima Indian Diabetes, with 768 samples and 8 features, is the smallest dataset of the four. We also examined Abalone, with 4177 samples and 8 features, Waveform, with 5000 samples and 22 features, and EHarmony, with over half million samples and 116 features. The results reveal that a reasonable trade-off is required when dealing with a large dataset. The results also illustrate that PWLSVM and D-PWLSVM outperform SVMs on a relatively large dataset, such as EHarmony, Abalone, and Waveform.

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.

Relative density based support vector machine

Electromyography (EMG) signal is an electrical indicator for neuromuscular activation. It provides direct access to physiological processes enabling the muscle to generate force and produce movement in order to accomplish countless functions. As a successful classification of the EMG signal is basically dependent on the selection of the best parameters carefully, this paper proposes a hybrid optimized classification model for EMG signals classification. The proposed system implements grey wolf optimizer (GWO) combined with support vector machines (SVMs) classification algorithm in order to improve the classification accuracy via selecting the optimal settings of SVMs parameters. The proposed approach consists of three phases; namely pre-processing, feature extraction, and GWO-SVMs classification phases. The obtained experimental results obviously indicate that significant enhancements in terms of classification accuracy have been achieved by the proposed GWO-SVMs classification system. It has outperformed the typical SVMs classification algorithm via achieving an accuracy of over 90 % using the radial basis function (RBF) kernel function.

Search engines are an integral part of web information gathering and retrieval process, but they are significantly dependent upon the words or word phrases input by the end user. The search engine usually contributes no additional semantic data toward the information acquisition process. This paper presents an intelligent search agent "Web host access tool" (WHAT) based on support vector machines (SVM), which introduces the notion of queries conducted within a specific contextual meaning. Given a context and associated keywords that personalize the search history and preferences of the user, WHAT performs more intelligent resource filtering than conventional search engines, providing more relevant results while filtering the irrelevant references. Search results obtained in the form of text from different search engines are processed by the SVM based word classifier that arrange the results obtained according to user preference obtained from previous search processes. The text materials are processed by Latent Semantic Indexing (LSI) for creating a document matrix that gives the probability of a word occurrence in a specified context. For simplicity, this paper considers 5 different contexts: business, education, entertainment, news and information and tourism. The LSI coefficients were used by SVM to yield confidence levels for each search result and according to that the results were sorted and presented to the end user. As an alternative, least square support vector machines (LS-SVM) are also studied in this paper. The system is updated by the user that provides feedback regarding the search relevance, based on which the LSI coefficients are updated to ensure increased relevance in future searches. Three different types of kernel function were considered in this paper -Linear. Radial basis function (RBF) and polynomial kernel. Results claim that Linear SVM perform better than the others, not only in terms of classification accuracy but also in terms of training speed.

Palm tree detection using remote sensing images has received increasing attention in recent years, concerning the issues of sustainability, productivity and profitability. There has been significant progress in the research of using machine learning techniques, especially convolutional neural networks (CNNs) for automatic palm tree detection. However, whether CNNs can actually outperform traditional human-engineered approaches in terms of classification accuracy and detection speed is yet unknown. In the present study, we have compared human-engineered features namely histogram of oriented gradients (HOG), local binary pattern (LBP) and scale-invariant feature transform (SIFT) with features extracted using pre-trained AlexNet model for detecting palm trees in high resolution images obtained via unmanned aerial vehicle (UAV). Support vector machines (SVM) with linear and non-linear kernels were used to classify feature vectors obtained by different feature extractors. Haar-like features used in Viola-Jones framework was also tested in the study. Results showed that features extracted from the fifth convolutional layer of AlexNet achieved the highest accuracy of 96.1% using SVM with RBF kernel as classifier, which surpassed the accuracy obtained by fully-connected CNN, namely 95.6%. The results also suggest that SVM classifier with radial basis function (RBF) kernel using LBP as features is the optimal combination as it achieved comparable accuracy but higher detection speed than CNN approaches.

Support vector machine is a machine learning method. It takes on the good generalization ability and prediction accuracy. But the parameters of SVM model seriously affect the generalization ability and prediction accuracy of SVM model on the great extent. So an improved particle swarm optimization (PSO) algorithm based on chaotic search is introduced into the SVM model in propose a novel data classification (AMPSVM) method for processing the complex data. The first, the ergodicity, stochastic property, and regularity of chaos is used to chaotically search the current best individual, which randomly replaces the selected individual in the population in order to speed up evolution, improve the searching ability, convergence speed and accuracy. Then the improved PSO algorithm is used to select and optimize the parameters of the SVM (AMPSVM) model in order to improve the learning performance and generalization ability of the SVM model. In order to verify the effectiveness of the AMPSVM method, UCI data is selected in here. The experiment results show that the proposed AMPSVM method takes on the strong generalization ability, best sensitivity and higher classification accuracy.

In the field of machine learning, support vector machine is a supervised learning model which can analyze the data and identify patterns, and this theory is used for classification and regression analysis which is related to the learning algorithm.Support vector machine is a new machine learning method based on statistical learning theory, and has become a hot research topic in the machine learning field because of its excellent learning performance.Support vector machine is also a learning machine based on kernel function, and its generalization ability depends on the chosen kernel function to a great extent.Nevertheless, traditional support vector machine cannot achieve the desired results when training large-scale data set.Therefore, in order to improve training efficiency and generalization performance of support vector machine, we have to improve this algorithm.Although support vector machine has great advantages in theory, the research on its application is comparatively delayed.In this paper, we focus on novel support vector machine generated from multi-subjects amalgamation, such as fuzzy support vector machine, granular support vector machine, twin support vector machine.We analyze a large number of literature review about development trend of support vector machine and its applications in computer field, laying foundation for the systematic research on the correlation algorithm based on support vector machine.

State forecast of machine using support vector machines has good generalization ability in situation of rare samples. Appropriate parameter selection is very crucial to the learning results and generalization ability of support vector machines. In addition, embedding dimension influences the phase space reconstitution of nonlinear systems, as well as the precision of machine state forecasting. In this paper, an approach to optimize the parameters of SVM and the embedding dimension based on genetic algorithms was proposed. The proposed model is applied to the tendency forecasting of the vibration of shovel electric drive system. The results show that it can avoid blindness of manually selection of parameters and meanwhile improves the prediction performance greatly.

With the rapid developing of the Internet, more and more business websites are appearing all around the world. As the proportion of C2C business mode increasing, the problem about personal credit evaluation in the business websites also becomes more critical. The deal methods are very different between the C2C mode and the traditional enterprises in evaluating the credit rank due to the feature of customers in the business websites. In this paper, we construct a support vector machines (SVMs) ensemble method based on fuzzy integral to evaluate personal credit under the environment of electronic commerce. This method aggregates the outputs of separate component SVMs with importance of each component SVM. By comparing the experimental result SVMs ensemble with the single SVM, the neural network ensemble, the proposed method outperforms the single SVM, and neural network ensemble in terms of classification accuracy. It proves that the proposed method is stable, highly accurate, strong robust and feasible. It is useful for providing a sound credit assessment system.

Support vector machine (SVM) has been often used in binary classification. In order to seek the guidance principles of the kernel function selection, this paper analyzed a variety of kernel functions used to construct the SVM classifiers and carried out comparative studies on the 4 data sets for binary classification of UCI Machine Learning Repository. The experimental results show that, using the nu-SVC with radial basis kernel function (RBF) has the optimal classification accuracy, but using the C-SVC with RBF kernel function has the best generalization ability.