A Rule Extraction Study from SVM on Sentiment Analysis

In this paper, the algorithm design of the support vector machines (SVMs) ensemble in a practical multi-stage framework is analyzed which can be implemented efficiently by evolutionary multi-objective optimization algorithm. The designing of SVMs ensemble is considered in three stages: first, the bootstrap method and a strategy of dynamical parameter range adjustment are used to generate more diverse base SVMs, and the NSGA-II algorithm which can efficiently tune the parameters of SVMs is applied to ensure the accuracy of base SVMs; Second, the NSGA-II algorithm is used again to select the member of ensemble based on the accuracy and diversity of the ensemble we have measured; Last, the reliability of different class is computed and combined to decide the outputs of ensemble in terms of the decision values of base SVMs. The proposed algorithm is applied to the UCI datasets, some useful results has been concluded for the future work in this field.

In order to develop Human-centric Driver Fatigue Monitoring Systems (HDFMS) with aims to increase driving safety, an efficient combined features extraction from Curvelet transform and Gabor wavelet transform for fatigue expressions descriptions of vehicle drivers is proposed, and Random Subspace Ensemble (RSE) of Support Vector Machines (SVMs) with polynomial kernel as the base classifier is then exploited for classification of three predefined fatigue expressions classes, namely, awake expressions, moderate fatigue expressions, and severe fatigue expressions. The results of holdout and cross-validation experiments show that CF by RSE of SVMs with polynomial kernel outperforms other seven classifiers, i.e., Curvelet features by SVMs classifier, Gabor features by SVMs classifier, CF by five individual SVMs classifiers. With CF and RSE of SVMs with polynomial kernel, the classification accuracies of drivers' fatigue expressions are over 90% in both of the holdout and cross-validation experiments, which show the proposed approach of combined features extraction and RSE of SVMs can be used for developing Human-centric Driver Fatigue Monitoring Systems to increase driving safety.

Support Vector Machines (SVMs) ensembles have been widely used to improve classification accuracy in complicated pattern recognition tasks. In this work we propose to apply an ensemble of SVMs coupled with feature-subset selection methods to aleviate the curse of dimensionality associated with expression-based classification of DNA microarray data. We compare the single SVM classifier to SVM ensembles applying two different feature-subset selection techniques, namely random selection and k-means clustering, the base classifiers are combined using either majority vote or SVM fusion. Two real-world benchmarks datasets are used to evaluate and compare the performance. Experimental results show that the SVM ensemble of SVM base classifiers using k-means clustering for feature-subset selection and employing an SVM combiner achieve the best classification accuracy, and that feature-subset-selection methods can have considerable impact on the classification accuracy.

In this paper we propose a system for visual object detection and tracking based on the extended structural tensor and the ensemble of one-class support vector machines. First, the input color image is transformed with the anisotropic process into the extended structural tensor. Then the tensor space is clustered into the number of partitions which are used to train a corresponding number of one-class support vector machines composing an ensemble of classifiers. In run-time the ensemble classifies the input video stream into an object and background. Thanks to high discriminative properties of the extended structural tensor and to the diversity of the ensemble of classifiers the method shows very good properties which were shown by experiments on real video sequences.

Purpose. At the present stage, network attack detection systems based on the following neural networks are most often offered: multilayer perceptron, Kohonen network or self-organizing map and their combinations. The efficiency problem of two approaches to detect attacks on a computer network using neural network technology based on the normalized data of the open NSL-KDD database is considered. Methodology. As an architectural solution to the network attack detection system, it is proposed to consider the following approaches: based on one neural network determining the attack class (first approach) and an ensemble of five neural networks (second approach), which at the first stage determines the attack category (DoS, Probe, U2R , R2L), and in the second stage, the attack class belonging to a certain category. Findings. Based on the neural networks created in the MatLAB program, a study was conducted of their error on the length of the training sample using various training algorithms: Levenberg-Marquardt; Bayesian Regularization; Scaled Conjugate Gradient with different numbers of hidden neurons (minimum, average and maximum). Certain optimal parameters of neural networks with two approaches were determined. Originality. In the course of conducting experiments with various approaches, the results obtained were: TP (True Positive); FP (False Positive); FN (False Negative); TN (True Negative), based on which the following indicators were calculated for assessing the quality of solutions: correct determination of network attacks; false positives; reliability; accuracy and completeness, which prove the feasibility of using an ensemble of neural networks (second approach). Practical value. On the created neural networks with various approaches, studies were conducted: the operating time of neural networks; errors of the first kind; errors of the second kind. According to the results of the first study, the average operating time of an ensemble of neural networks is 0.92 s, and the operating time of a neural network (according to the first approach) is 2.21 s. According to the results of the second study, the error of the first kind using an ensemble of neural networks is 2.17%, and using the neural network (the first approach) – 7.39%. According to the results of the third study, the error of the second kind using an ensemble of neural networks is 3.91%, and using the neural network (the first approach) – 6.96%, which is confirmed by the efficiency of using an ensemble of neural networks (second approach).

Online dynamic security assessment of wind integrated power system using SDAE with SVM ensemble boosting learner

Flood susceptibility mapping using a novel ensemble weights-of-evidence and support vector machine models in GIS

Fine-grained classification is a recognition task where subtle differences distinguish between different classes. To tackle this classification problem, part-based classification methods are mostly used. Partbased methods learn an algorithm to detect parts of the observed object and extract local part features for the detected part regions. In this paper we show that not all extracted part features are always useful for the classification. Furthermore, given a part selection algorithm that actively selects parts for the classification we estimate the upper bound for the fine-grained recognition performance. This upper bound lies way above the current state-of-the-art recognition performances which shows the need for such an active part selection method. Though we do not present such an active part selection algorithm in this work, we propose a novel method that is required by active part selection and enables sequential part-based classification. This method uses a support vector machine (SVM) ensemble and allows to classify an image based on arbitrary number of part features. Additionally, the training time of our method does not increase with the amount of possible part features. This fact allows to extend the SVM ensemble with an active part selection component that operates on a large amount of part feature proposals without suffering from increasing training time.

Modeling radiation-induced lung injury risk with an ensemble of support vector machines

To obtain the spatial relationship between three or more pixels in the texture image, bispectrum is choosen to extract texture features of the image, and it contains amplitude information and phase information of the image. Due to some problems in neural network, such as unstable classifier design, configuration, training, the research based on the ensemble of neural networks appears. Compared with a single neural network, an ensemble of neural networks has better fault tolerance and generalisation ability. In this paper, bispectrum is used to extract texture features and the neural network ensembles are used to recognize the texture images. The experimental results demonstrate that the ensemble of BP neural networks can effectively improve correct recognition rate of texture images.

The present work presents the experiments performed with two kinds of ensembles to classify infant cry. The ones selected for testing during the presented experiments are: a boosting ensemble of artificial neural networks and a boosting ensemble of support vector machines. The design and implementation of the ensembles as well as the experiments and some of the results are shown. The experiments are aimed to classify the types of pain -no pain and hunger - no hunger cries

A Novel Multi-model Data-Driven Ensemble Technique for the Prediction of Retention Factor in HPLC Method Development

Support Vector Machines (SVM) are believed to be as powerful as Artificial Neural Networks (ANN) in modeling complex problems while avoiding some of the drawbacks of the latter such as local minimæ or reliance on architecture. However, a question that remains to be answered is whether SVM users may expect improvements in the interpretability of their models, namely by using rule extraction methods already available to ANN users. This study successfully applies the Orthogonal Search-based Rule Extraction algorithm (OSRE) to Support Vector Machines. The study evidences the portability of rules extracted using OSRE, showing that, in the case of SVM, extracted rules are as accurate and consistent as those from equivalent ANN models. Importantly, the study also shows that the OSRE method benefits from SVM specific characteristics, being able to extract less rules from SVM than from equivalent ANN models.KeywordsData MiningSupport Vector MachinesArtificial Neural NetworksOrthogonal Search-based AlgorithmOSREPedagogicalDecompositionalRule Extraction

Cancer recognition with bagged ensembles of support vector machines

In this chapter we describe how comprehensible rules can be extracted from artificial neural networks (ANN) and support vector machines (SVM). ANN and SVM are two very popular techniques for pattern classification. In the business intelligence application domain of credit scoring, they have been shown to be effective tools for distinguishing between good credit risks and bad credit risks. The accuracy obtained by these two techniques is often higher than that from decision tree methods. Unlike decision tree methods, however, the classifications made by ANN and SVM are difficult to understand by the end-users as outputs from ANN and SVM are computed as nonlinear mapping of the input data attributes. We describe two rule extraction methods that we have developed to overcome this difficulty. These rule extraction methods enable the users to obtain comprehensible propositional rules from ANN and SVM. Such rules can be easily verified by the domain experts and would lead to a better understanding about the data in hand.