Local Similarity-Based Fuzzy Multiple Kernel One-Class Support Vector Machine

Huberized one-class support vector machine with truncated loss function in the primal space

OCSVM (one-class support vector machine) is a variant of SVM which only use positive class sample set in training. Since only positive samples can be used in OCSVM, Fully exploiting and using the features of the training samples is of great significance to improve its classification performance. Thus, two aspects of study on kernels have been done in this paper: first, we propose a kernel constructing method called WFCD (weighted feature-contribution-degree) kernel constructing method, in which a PCA (principal component analysis) is performed to the training samples to obtain a vector set with the dimension being sorted by corresponding Eigen values and then using this vector set to apply a weighed kernel method to concentrate on the larger Eigen value dimensions, second, we employ the Fisher kernel in OCSVM to decide whether a kernel constructed based on the training sample set has better performance. Experimental results on UCI standard data sets indicate that our method outperforms the general kernel methods and promotes the classification effect considerably.

The classification performances of the traditional one-class support vector machine (OCSVM) and its variants are often not satisfying when outliers are complex. To deal with this case, assigning smaller weights to these outliers may alleviate their influence upon the classification boundary and enhance the robustness of OCSVM. In this paper, a novel adaptive-weighted one-class support vector machine (AWOCSVM) is proposed for dealing with the outlier detection problem. The appropriate weights are assigned to training samples by considering both their local densities and distances between them to their center. Experimental results on two synthetic data sets and eight benchmark data sets demonstrate that the proposed AWOCSVM achieves more compact classification boundary and superior performance in comparison with the traditional OCSVM and one related approach.

This paper studies a specific one-class classification problem where the training data are corrupted by significant outliers. Specifically, we are interested in the one-class support vector machine (OCSVM) approach that normally requires good training data. However, perfect training data are usually hard to obtain in most real-world applications due to the inherent data variability and uncertainty. To address this issue, we propose an OCSVM-based data editing and classification method that can iteratively purify the training data and learn an appropriate classifier from the trimmed training set. The proposed method is compared with a general OCSVM approach trained from two types of bootstrap samples, and applied to the mapping and compliance monitoring tasks for the U.S. Department of Agriculture's Conservation Reserve Program using remotely sensed imagery. Experimental results show that the proposed method outperforms the general OCSVM using bootstrap samples at a lower computational load.

Abstract: Multivariate control charts are essential tools in multivariate statistical process control (MSPC). “Shewhart-type” charts are control charts using rational subgroupings which are effective in the detection of large shifts. Recently, the one-class classification problem has attracted a lot of interest. Three methods are typically used to solve this type of classification problem. These methods include the k−center method, the nearest neighbor method, one-class support vector machine (OCSVM), and the support vector data description (SVDD). In industrial applications, like statistical process control (SPC), practitioners successfully used SVDD to detect anomalies or outliers in the process. In this paper, we reformulate the standard support vector data description and derive a least squares version of the method. This least-squares support vector data description (LS-SVDD) is used to design a control chart for monitoring the mean vector of processes. We compare the performance of the LS-SVDD chart with the SVDD and T2 chart using out-of-control Average Run Length (ARL) as the performance metric. The experimental results indicate that the proposed control chart has very good performance.

Fuzzy quasi-linear SVM classifier: Design and analysis

Usually, the One-Class Support Vector Machine (OC-SVM) requires a large dataset for modeling effectively the target class independently to other classes. For finding the OC-SVM model, the available dataset is subdivided into two subsets namely training and validation, which are used for training and validating the optimal parameters. This approach is effective when a large dataset is available. However, when training samples are reduced, parameters of the OC-SVM are difficult to find in absence of the validation subset. Hence, this paper proposes various techniques for selecting the optimal parameters using only a training subset. The experimental evaluation conducted on several real-world benchmarks proves the effective use of the new selection parameter techniques for validating the model of OC-SVM classifiers versus the standard validation techniques

The anomaly/novelty detection in time series data analysis is one of the most admired area of research, which is specifically a one-class classification problem. Concerning univariate time series data, apart from a huge number of training samples the higher sample span (observation length) also adds computation overhead along with its intrinsic issue of curse of dimensionality (sample length is considered as dimension). In this context, the present research proposes a concurrent way of sample span (treated as dimension) and training sample reduction approach for univariate time series data under the supervision of target class samples. Data representation of time series decides the performance of any machine learning approach, therefore the present research utilizes dissimilarity-based representation (DBR) techniques for time series data representation and later to reduce the sample length, a knowledge grid is computed via eigen space analysis of variance-covariance of target class samples. This knowledge grid is further used to transform the original sample length to reduced one. Afterwards, the training samples are selected using prototype methods. For experiments 16 different DBR measures are used along with 11 prototype techniques. Finally, one-class support vector machine (OCSVM) and 1-nearest neighbour (1-NN) are utilized for classification to validate the performance of proposed approach over 85 UCR/UEA univariate datasets.KeywordsTime seriesTarget classOne-class classificationCurse of dimensionalitySample reductionDissimilarity-based representationPrototype

Intrusion detection system (IDS) has played an important role as a device to defend our networks from cyber attacks. However, since it is unable to detect unknown attacks, i.e., 0-day attacks, the ultimate challenge in intrusion detection field is how we can exactly identify such an attack by an automated manner. Over the past few years, several studies on solving these problems have been made on anomaly detection using unsupervised learning techniques such as clustering, one-class support vector machine (SVM), etc. Although they enable one to construct intrusion detection models at low cost and effort, and have capability to detect unforeseen attacks, they still have mainly two problems in intrusion detection: a low detection rate and a high false positive rate. In this paper, we propose a new anomaly detection method based on clustering and multiple one-class SVM in order to improve the detection rate while maintaining a low false positive rate. We evaluated our method using KDD Cup 1999 data set. Evaluation results show that our approach outperforms the existing algorithms reported in the literature; especially in detection of unknown attacks.

Recently, Anomalous Sound Detection (ASD) has emerged as a promising method for road surveillance. However, since the ratio of anomalous events is generally too small, anomaly detection in general, and ASD in particular, are mainly treated as one-class classification problems. Besides, a common problem in road surveillance is the background noise, which makes it difficult to train effective models based on normal sounds only. Therefore, this work aims to experiment with the use of graph signal processing (GSP) to improve ASD performance. Thus, we propose a Graph-based One-Class SVM technique (GOC-SVM) where the features extracted from audio signals are firstly embedded on graphs, and then filtered through a graph filterbank, before computing their joint Fourier transform magnitude. Subsequently, they are fed into a one-class SVM classifier trained on normal data only. Evaluation results show a threefold advantage of using graph embedding and filtering for ASD: (a) improving the anomaly detection results in comparison to plain features, (b) outperforming the classical OC-SVM baseline, (c) enhancing the performance of the proposed semi-supervised GOC-SVM, so as to reach a comparable level of performance of the fully-supervised binary classification SVM, yielding 0.91 of Area-under-the-curve (AUC), 98% of overall accuracy, 99% and 88% of F1 score for normal and anomalous classes, respectively. Such a performance proves the potential of using GSP to solve the ASD problem in road traffic monitoring.

<p>In one-class classification, the low variance directions in the training data carry crucial information to build a good model of the target class. Boundary-based methods like One-Class Support Vector Machine (OSVM) preferentially separates the data from outliers along the large variance directions. On the other hand, retaining only the low variance directions can result in sacrificing some initial properties of the original data and is not desirable, specially in case of limited training samples. This paper introduces a Covariance-guided One-Class Support Vector Machine (COSVM) classification method which emphasizes the low variance projectional directions of the training data without compromising any important characteristics. COSVM improves upon the OSVM method by controlling the direction of the separating hyperplane through incorporation of the estimated covariance matrix from the training data. Our proposed method is a convex optimization problem resulting in one global optimum solution which can be solved efficiently with the help of existing numerical methods. The method also keeps the principal structure of the OSVM method intact, and can be implemented easily with the existing OSVM libraries. Comparative experimental results with contemporary one-class classifiers on numerous artificial and benchmark datasets demonstrate that our method results in significantly better classification performance.</p>

The goal of this paper is to develop an efficient method for forest change detection using multitemporal stereo panchromatic imagery. Due to the lack of spectral information, it is difficult to extract reliable features for forest change monitoring. Moreover, the forest changes often occur together with other unrelated phenomena, e.g., seasonal changes of land covers such as grass and crops. Therefore, we propose an approach that exploits kernel Minimum Noise Fraction (kMNF) to transform simple change features into high-dimensional feature space. Digital surface models (DSMs) generated from stereo imagery are used to provide information on height difference, which is additionally used to separate forest changes from other land-cover changes. With very few training samples, a change mask is generated with iterated canonical discriminant analysis (ICDA). Two examples are presented to illustrate the approach and demonstrate its efficiency. It is shown that with the same amount of training samples, the proposed method can obtain more accurate change masks compared with algorithms based on k-means, one-class support vector machine, and random forests.

Sugarcane is a significant crop in terms of annual biomass in the world. Timely and accurate mapping of sugarcane planting is important for food security and sustainability. However, accurately remote-sensing-based mapping sugarcane remains challenging due to two reasons: 1) the scarcity of sugarcane training samples, and 2) the diverse sugarcane fields planting dates. This article proposed a novel transfer learning algorithm for accurate sugarcane planting mapping through space and temporary in the U.S. This algorithm only depended on one place's training data to mapping other places’ growing sugarcane. We distilled burning sugarcane fields as training labels from Sentinel-2 in Palm Beach County and neighboring area in 2021. The time-invariant phenology features were calculated from composite Sentinel-2 normalized difference vegetation index (NDVI) series using linear cosine regression (LCR). They integrated as training samples to construct a one-class support vector machine (OCSVM) classifier, generating Jun.–Nov. growing sugarcane maps in Plam Beach County and Lafourche County 2022. Meanwhile, a postprocess method was used to improve mapping quality. These sugarcane maps were validated using surveyed and observed field dataset and compared against the cropland data layer (CDL). As a result, the sugarcane maps achieved the best accuracy in its mature stage (Sep. 2022), which exhibited nearly complete phenological characteristics. The maps show low misclassification rates and higher accuracy compared to the CDL 2022. Moreover, the LCR-OCSVM method was confirmed to have superior transfer learning capability for sugarcane classification across different regions and periods compared to the Harmonic-OCSVM method.

Kernel Methods have been successfully applied in different tasks and used on a variety of data sample sizes. Multiple Kernel Learning (MKL) and Multilayer Multiple Kernel Learning (MLMKL), as new families of kernel methods, consist of learning the optimal kernel from a set of predefined kernels by using an optimization algorithm. However, learning this optimal combination is considered to be an arduous task. Furthermore, existing algorithms often do not converge to the optimal solution (i.e., weight distribution). They achieve worse results than the simplest method, which is based on the average combination of base kernels, for some real-world applications. In this paper, we present a hybrid model that integrates two methods: Support Vector Machine (SVM) and Multiple Classifier (MC) methods. More precisely, we propose a multiple classifier framework of deep SVMs for classification tasks. We adopt the MC approach to train multiple SVMs based on multiple kernel in a multi-layer structure in order to avoid solving the complicated optimization tasks. Since the average combination of kernels gives high performance, we train multiple models with a predefined combination of kernels. Indeed, we apply a specific distribution of weights for each model. To evaluate the performance of the proposed method, we conducted an extensive set of classification experiments on a number of benchmark data sets. Experimental results show the effectiveness and efficiency of the proposed method as compared to various state-of-the-art MKL and MLMKL algorithms.

Nowadays mobile positioning devices, sensor technologies, machine learning methods and cloud computing allow an efficient collection, processing and analysis of dynamic position information (trajectories), which can be used for various applications. Currently, data owners (e.g. vehicle producers or service operators) are reluctant to share data due to data privacy rules or because of the risk of sharing information with competitors, which could jeopardize the data owner's competitive advantage. In this paper, we compare various state-of-the-art anomaly detection methods like one-class support vector machine, isolation forest and bidirectional generative adversarial networks towards the detection of abnormal vehicle trajectories at intersections solving one-class classification problem with unsupervised learning algorithms. The experiments show that the selected models allow to identify anomalies with 98%-99% accuracy for the centralized approach. However, data exists mostly fragmented in form of isolated islands throughout the whole industry. Thus, we focus on a general concept of collaborative learning which provides with an increasing number of partners more accurate anomaly detection models than local models of each partner without exchanging raw data, but only synchronizing model parameters. We propose a federated learning algorithm for an one-class support vector machine model towards the detection of anomalies. The federated collaborative approach allows to construct comparable accuracy with the centralized approach, reduces the local data labeling load and keeps individual data private.