Solving one-class problem with outlier examples by SVM

The SVDD (support vector data description) is one of the most well-known one-class support vector learning methods, in which one tries the strategy of utilizing balls defined on the feature space in order to distinguish a set of normal data from all other possible abnormal objects. The usual strategy of the SVDD depends on the process of finding the region for the normal-class training data with somewhat neglecting the distribution of the abnormal data, thus it may not work well if applied to the binary classification problems in which two classes have similar number of data. In this paper, we consider the problem of performing binary classification based on the SVDD techniques, and in order to overcome the possible drawback of the usual SVDD strategy focusing on the normal-class data only, we propose a new SVDD algorithm which is based on the use of two different SVDD balls for the positive and negative classes along with the SVDD projection and nearest neighbor rule. To investigate how the proposed method works, we compared the performance of the proposed method with SVC (support vector classifier) and conventional SVDD using several real datasets.

Support vector data description (SVDD) is a well-known kernel-based one-class classification method that exhibits intrinsic regularization ability and robustness versus low numbers of high-dimensional samples. However, the efficiency of SVDD is limited by the cubic time complexity. To solve this problem, this paper first investigates the effect of selecting a reduced subset as the training set of SVDD, while guaranteeing the classification quality. To this end, a new heuristic sample condensed rule, termed HSC, is proposed to accurately identify those potential support vectors that characterize the classification boundary. HSC can consider both the spatial distribution and local density features of training samples, and focus on selecting samples very close to the decision boundary. When dealing with the local density computation, we introduce the idea of K nearest neighbors (KNN) to examine the density of samples in the neighbors of the object to be classified. Finally, a condensed but informative subset obtained by HSC will be applied to train SVDD breezily. The experimental results show that HSC-based SVDD sensibly improves over conventional SVDD, in terms of the size of the training set while guaranteeing a comparable classification quality. In addition, it is competitive over other improved SVDD classifiers in terms of training and testing time.

Support vector data description (SVDD) aims at constructing an optimal hypersphere regarded as a data description for a dataset while support vector classification (SVC) aims at separating data of two classes without providing a data description. This paper proposes a unified approach to both SVDD and SVC that aims at separating data of two classes and at the same time provides a data description. A trade off parameter is introduced to control the balance between describing the data and maximising the margin. Experimental results are provided to evaluate the proposed approach.

For generalization to be functional, it must occur with a precision that results in acquired responses occurring under appropriate, nontrained conditions, and acquired responses not occurring under inappropriate, nontrained conditions. This study examines the effect of differing types of negative teaching examples on the precision with which generalized grocery item selection is learned. Within a split-multiple baseline design, six young adults identified as mildly, moderately, or severely mentally retarded were trained to select or to reject grocery items using picture cards as cues. The dependent variables were correct selection of 10 trained “positive” grocery items and the correct rejection of 20 nontrained “negative” grocery items in a nontrained grocery store. Participants were trained in a grocery store to select 10 positive grocery examples matching their picture cards and to reject either (a) a set of negative examples that were maximally different from the positive examples, or (b) a set of negative examples that were minimally different from the positive examples. Both training sets resulted in participants correctly selecting the 10 positive items in a nontrained store. Training with the “minimally different” negative examples was functionally related to improved rejection of nontrained negative items in the nontrained store. The implications of teaching with minimally different, negative examples are discussed.

Monitoring of solid-state fermentation of wheat straw in a pilot scale using FT-NIR spectroscopy and support vector data description

Power electronic circuits (PECs) are prone to various failures, whose classification is of paramount importance. This paper presents a data-driven based fault diagnosis technique, which employs a support vector data description (SVDD) method to perform fault classification of PECs. In the presented method, fault signals (e.g. currents, voltages, etc.) are collected from accessible nodes of circuits, and then signal processing techniques (e.g. Fourier analysis, wavelet transform, etc.) are adopted to extract feature samples, which are subsequently used to perform offline machine learning. Finally, the SVDD classifier is used to implement fault classification task. However, in some cases, the conventional SVDD cannot achieve good classification performance, because this classifier may generate some so-called refusal areas (RAs), and in our design these RAs are resolved with the one-against-one support vector machine (SVM) classifier. The obtained experiment results from simulated and actual circuits demonstrate that the improved SVDD has a classification performance close to the conventional one-against-one SVM, and can be applied to fault classification of PECs in practice.

Amongst all the methods of machine learning, learning from examples is considered as a key to acquire knowledge automatically. Learning from examples is to obtain a general cover through induction from a given set of positive and negative examples of a concept, which may describe all the positive examples, but reject all the negative examples of that concept. Extension matrix theory, a branch of learning from examples, presents the training examples as two matrices which consist of positive and negative examples respectively, and then finds a path satisfied with a positive example against the background of negative examples. Obviously, such a path is a conjunction of conditions that is satisfied by the positive example. However, most of all of pattern recognition problems, such as handwritten Chinese character recognition, have an overlay area. It is necessary to solve the problem of nonlinear boundaries among different classes when machine learning is applied to acquire the recognition rules. An advanced extension matrix algorithm is proposed in this paper, in which a heuristic search based on the average entropy is used to get the approximate solutions of the minimal complex. A potential function is used to estimate the probability density function of the overlay area between positive and negative examples, so that the nonlinear interfaces of the interclass areas may be obtained. This algorithm is applied to supervised learning for Chinese character recognition to acquire recognition rules, and the implementations upon 751,000 loosely-constrained handwritten Chinese characters indicate that these methodologies can be applied to a practical recognition system with promising results.

Anomaly detection using improved deep SVDD model with data structure preservation

Deep learning with support vector data description

Training a classifier for object category recognition using images on the Internet is an attractive approach due to its scalability. However, a big challenge in this approach is that it is difficult to automatically obtain sets of negative samples that are guaranteed to be free of positive samples. In this paper we propose to address this challenge with a Support Vector Data Description (SVDD) classifier. An SVDD classifier does not need negative images in training. It computes a hypersphere around the potentially good images in the feature space and uses this boundary to distinguish images of target visual category from outliers. Evaluation on standard test sets shows that we are able to achieve competitive classification performance using the contaminated training images from the Internet without the need for large datasets of negative examples.

Support vector data description (SVDD) is an effective algorithm for nonlinear process monitoring. However, the conventional SVDD method can't deal with the incipient faults well, which has the small fault amplitude and is easy to be overlapped by industrial noises. Aiming at this problem, an improved SVDD, called probability related SVDD in local variable field (LVPSVDD), is proposed to detect the incipient faults in nonlinear processes. Firstly, the method divides process variables into several local variable fields by hierarchical clustering, and the SVDD model of each variable field is built. Then the sliding window technology is applied to the SVDD distance statistics, and the probability distribution change in the sliding window is measured by Kullback Leibler divergence (KLD). For each local variable field, the corresponding probability related monitoring statistic is developed to replace the original distance statistic. Finally, the global monitoring statistic is obtained by integrating the monitoring results of all local variable fields with Bayesian inference strategy. The method is illustrated with simulation on the continuous stirred tank reactor (CSTR).

Density weighted support vector data description

This paper reports a new document retrieval method using non-relevant documents. Especially, this paper reports a comparison of retrieval efficiency between one class support vector machine (SVM) based and support vector data description (SVDD) based interactive document retrieval method using non-relevant documents only. From a large data set of documents, we need to find documents that relate to human interesting in as few iterations of human testing or checking as possible. In each iteration a comparatively small batch of documents is evaluated for relating to the human interesting. We applied active learning techniques based on support vector machine for evaluating successive batches, which is called relevance feedback. Our proposed approach has been very useful for document retrieval with relevance feedback experimentally. The traditional relevance feedback needs a set of relevant and non-relevant documents to work usefully. However, the initial retrieved documents, which are displayed to a user, sometimes don't include relevant documents. In order to solve this problem, we propose a new feedback method using information of non-relevant documents only. We named this method non-relevance feedback document retrieval. The non-relevance feedback document retrievals are based on one class support vector machine and support vector data description. Our experimental results show that one class support vector machine based method can retrieve relevant documents efficiently using information of non-relevant documents only.

Black rice is an important rice species in Southeast Asia. It is a common phenomenon to pass low-priced black rice off as high-priced ones for economic benefit, especially in some remote towns. There is increasing need for the development of fast, easy-to-use, and low-cost analytical methods for authenticity detection. The feasibility to utilize near-infrared (NIR) spectroscopy and support vector data description (SVDD) for such a goal is explored. Principal component analysis (PCA) is used for exploratory analysis and feature extraction. Another two data description methods, i.e., k-nearest neighbor data description (KNNDD) and GAUSS method, are used as the reference. A total of 142 samples from three brands were collected for spectral analysis. Each time, the samples of a brand serve as the target class whereas other samples serve as the outlier class. Based on both the first two principal components (PCs) and original variables, three types of data descriptions were constructed. On average, the optimized SVDD model achieves acceptable performance, i.e., a specificity of 100% and a sensitivity of 94.2% on the independent test set with tight boundary. It indicates that SVDD combined with NIR is feasible and effective for authenticity detection of black rice.

For the problem of a large dataset, we need to select a subset to represent the original dataset. Many scholars do pattern selection from the problem of the kNN (k-nearest neighbors). The distribution of a pattern's neighbors is usually uneven. In this paper, we define a fixed neighborhood sphere. When the pattern locates near the boundary of the data distribution, there will be fewer neighbors in the neighborhood sphere and when the pattern locates within the data distribution, there will be more neighbors in the neighborhood sphere. According to gather the statistic of the neighbors in a fixed neighborhood sphere, we can find those patterns locating near the boundary of the data distribution. In SVDD (Support Vector Data Description), those patterns are locating near the boundary of the data distribution have more information. They are those patterns which would be support vectors. We can use FNSPS (fixed neighborhood sphere pattern selection) algorithm to select those patterns, which locate near the boundary of the data distribution. The experimental results show that the performance of the SVDD will not go bad. The time complexity of the naive identifying the neighbors in the fixed neighborhood sphere is O(n2). And the time complexity of the SVDD is O(n3). If we set a lower threshold, the FNSPS algorithm can also be used to remove the noise in the targets.