Converting SVDD scores into probability estimates: Application to outlier detection

Anomaly detection using improved deep SVDD model with data structure preservation

As one classical anomaly detection technology, support vector data description (SVDD) has been successfully applied to nonlinear chemical process monitoring. However, the basic SVDD model cannot achieve a satisfactory fault detection performance in the complicated cases because of its intrinsic shallow learning structure. Motivated by the deep learning theory, one improved SVDD method, called ensemble deep SVDD (EDeSVDD), is proposed in order to monitor the process faults more effectively. In the proposed method, a deep support vector data description (DeSVDD) framework is firstly constructed by introducing the deep feature extraction procedure. Different to the traditional SVDD with only one feature extraction layer, DeSVDD is designed with multi-layer feature extraction structure and optimized by minimizing the data-enclosing hypersphere with the regularization of the deep network weights. Further considering the problem that DeSVDD monitoring performance is easily affected by the model structure and the initial weight parameters, an ensemble DeSVDD (EDeSVDD) is presented by applying the ensemble learning strategy based on Bayesian inference. A series of DeSVDD sub-models are generated at the parameter level and the structure level, respectively. These two levels of sub-models are integrated for a holistic monitoring model. To identify the cause variables for the detected faults, a fault isolation scheme is designed by applying the distance correlation coefficients to measure the nonlinear dependency between the original variables and the holistic monitoring index. The applications to the Tennessee Eastman process demonstrate that the proposed EDeSVDD model outperforms the traditional SVDD model and the DeSVDD model in terms of fault detection performance and can identify the fault cause variables effectively.

Key performance indicator (KPI) anomaly detection is the underlying core technology in Artificial Intelligence for IT operations (AIOps). It has an important impact on subsequent anomaly location and root cause analysis. Variational auto-encoder (VAE) is a symmetry network structure composed of encoder and decoder, which has attracted extensive attention because of its ability to capture complex KPI data features and better detection results. However, VAE is not well applied to the modeling of KPI time series data and it is often necessary to set the threshold to obtain more accurate results. In response to these problems, this paper proposes a novel hybrid method for KPI anomaly detection based on VAE and support vector data description (SVDD). This method consists of two modules: a VAE reconstructor and SVDD anomaly detector. In the VAE reconstruction module, firstly, bi-directional long short-term memory (BiLSTM) is used to replace the traditional feedforward neural network in VAE to capture the time correlation of sequences; then, batch normalization is used at the output of the encoder to prevent the disappearance of KL (Kullback–Leibler) divergence, which prevents ignoring latent variables to reconstruct data directly. Finally, exponentially weighted moving average (EWMA) is used to smooth the reconstruction error, which reduces false positives and false negatives during the detection process. In the SVDD anomaly detection module, smoothed reconstruction errors are introduced into the SVDD for training to determine the threshold of adaptively anomaly detection. Experimental results on the public dataset show that this method has a better detection effect than baseline methods.

Monitoring of solid-state fermentation of wheat straw in a pilot scale using FT-NIR spectroscopy and support vector data description

Support Vector Data Description (SVDD) as a kernel-based method constructs a minimum hypersphere so as to enclose all the data of the target class in the kernel mapping space. In this paper, it is found that the kernel matrix G of SVDD can always have the Singular Value Decomposition (SVD) and the corresponding kernel mapping space can be made up of a set of base vectors generated by SVD. In order to make the kernel mapping more flexible, we induce a parameter λ into the set of base vectors and thus propose a novel SVDD with fractional order kernel (named λ-SVDD). In doing so, we can expand the solution space for the optimized dual problem of the SVDD. The experimental results on both synthetic data set and some real data sets show that the proposed method can bring more accurate description for all the tested target cases than the conventional SVDD.

Support vector data description (SVDD) is an effective algorithm for nonlinear process monitoring. However, the conventional SVDD method can't deal with the incipient faults well, which has the small fault amplitude and is easy to be overlapped by industrial noises. Aiming at this problem, an improved SVDD, called probability related SVDD in local variable field (LVPSVDD), is proposed to detect the incipient faults in nonlinear processes. Firstly, the method divides process variables into several local variable fields by hierarchical clustering, and the SVDD model of each variable field is built. Then the sliding window technology is applied to the SVDD distance statistics, and the probability distribution change in the sliding window is measured by Kullback Leibler divergence (KLD). For each local variable field, the corresponding probability related monitoring statistic is developed to replace the original distance statistic. Finally, the global monitoring statistic is obtained by integrating the monitoring results of all local variable fields with Bayesian inference strategy. The method is illustrated with simulation on the continuous stirred tank reactor (CSTR).

The general objective of anomaly detection (AD) in hyperspectral imagery is to detect full-pixel targets. To meet this purpose, the global AD methods can achieve more reliable results than the local methods in terms of time and accuracy. The kernel-based Support Vector Data Description (SVDD) has recently received great attention in the hyperspectral AD applications. This paper presents a global SVDD-based method for autonomous full-pixel AD. The method consists of three steps: clustering, background modeling, and autonomous AD. Experimental results on a hyperspectral dataset show the superiority of the proposed method comparing to the global-based SVDD method.

In this paper, we address the problem of image anomaly detection and segmentation. Anomaly detection involves making a binary decision as to whether an input image contains an anomaly, and anomaly segmentation aims to locate the anomaly on the pixel level. Support vector data description (SVDD) is a long-standing algorithm used for an anomaly detection, and we extend its deep learning variant to the patch-based method using self-supervised learning. This extension enables anomaly segmentation and improves detection performance. As a result, anomaly detection and segmentation performances measured in AUROC on MVTec AD dataset increased by 9.8% and 7.0%, respectively, compared to the previous state-of-the-art methods. Our results indicate the efficacy of the proposed method and its potential for industrial application. Detailed analysis of the proposed method offers insights regarding its behavior, and the code is available online (https://github.com/nuclearboy95/Anomaly-Detection-PatchSVDD-PyTorch).

To improve the efficiency and usability of adaptive anomaly detection system, we propose a new framework based on Support Vector Data Description (SVDD) method. This framework includes two main techniques: online change detection and unsupervised anomaly detection. The first one enables automatically obtain model training data by measuring and distinguishing change caused by intensive attacks from normal behavior change and then filtering most intensive attacks. The second retrains model periodically and detects the forthcoming data. Results of experiments with the KDD’99 network data show that these techniques can handle intensive attacks effectively and adapt to the concept drift while still detecting attacks. As a result, false positive rate is reduced from 13.43% to 4.45%.

Support Vector Data Description (SVDD) is a popular one-class classifier, and well-suited for outlier detection. However, the effectiveness of SVDD depends on selecting good hyperparameter values – a difficult problem that has received significant attention in the literature. Since SVDD is an unsupervised classifier, tuning of hyperparameter values is difficult. This has motivated several methods to estimate hyperparameter values based on data characteristics. But existing methods are purely heuristic, and the conditions under which they work well are largely unclear. This has created a situation where instead of selecting hyperparameter values, one has to choose among several, equally plausible heuristics.In this article, we make some strides towards a principled approach to estimate SVDD hyperparameter values. We propose LAMA (Local Active Min-Max Alignment), the first method to select SVDD hyperparameter values by active learning. The core idea is based on kernel alignment, which we adapt to active learning with small sample sizes. LAMA provides estimates for both of the SVDD hyperparameters. These estimates are evidence-based, i.e., rely on actual class labels, and come with a quality score. This eliminates the need for manual validation, an issue with current heuristics. LAMA outperforms state-of-theart competitors in extensive experiments on real-world data. In several cases, LAMA even yields results close to the empirical upper bound.

Data-driven simulation models can be used for optimal control and decision making in building systems only if they are developed accurately and reliably. An unsolved issue pertains to the data collected from BEMS that generally contain errors owing to sensors, malfunctioning systems, and other unknown reasons. These anomalies generally occur in a multidimensional space and cannot be easily detected. Unfortunately, these anomalies can prevent the simulation model from being accurate, reliable, and scalable. This paper presents an automatic anomaly detection method using a support vector data description (SVDD). The data obtained from a compression chiller in a real office building were classified into raw and filtered datasets based on SVDD. Two artificial neural network (ANN) models were developed (ANNraw and ANNSVDD). Based on the comparison between the two ANNs, the ANNSVDD is found to be better than ANNraw in terms of the model reliability and reproducibility. It is also interesting that the model accuracy differences between the two ANN models are marginal. However, the accuracy of the two ANN models can be improved as long as they are tested against filtered data by SVDD. The improvements in accuracy signify the importance of the elimination of these anomalies.

Complex industrial processes are often non-linear and non-Gaussian, while the traditional principal component analysis (PCA) method assumes that the data are Gaussian and linear. In this paper, a novel process monitoring method based on maximum information coefficient-PCA (MIC-PCA) and support vector data description (SVDD) is proposed. First, the covariance matrix is replaced by the MIC matrix which can measure the non-linear correlation between the variables. Then the SVDD models are built in the principal component subspace (PCS) and the residual subspace (RS) to improve the monitoring of non-linear and non-Gaussian processes. Finally, the feasibility and effectiveness of the proposed method are validated by high-pressure and low-density polyethylene (LDPE) industrial process.

With the rapid development of monitoring systems, extensive amount of water quality high-resolution measurements are accumulated, which make it unrealistic to manually extract the water quality anomaly features from the huge river environment information. In this study, a hybrid anomaly detection framework is developed by the combination of prediction-based and classification-based data-driven methods to provide a scientific indication for river pollution identification. A Variational Mode Decomposition-Back Propagation Neural Network (VMD-BPNN) model is used to analyze the real-time water quality variation tendencies in the first stage. Additionally, a Support Vector Data Description (SVDD) algorithm is adopted to capture the multi-dimensional water quality anomaly characteristics in the second stage. The developed hybrid framework is then applied to the Kansas River in America, to verify its river pollution identification performance in comparison to different anomaly detection methods and in various anomaly-level scenarios. The developed hybrid framework can achieve a maximum Area Under the Curve (AUC) value of 0.932 under a two-dimensional anomaly detection pattern with the True Positive Rate (TPR) and False Positive Rate (FPR) values of 0.861 and 0.142, respectively. The results indicate that the developed hybrid framework can provide an effective river pollution identification performance with dynamically determined warning thresholds. Meanwhile, a vigorous anomaly detection pattern can improve the pollution identification performance by considering the cumulative interactions among the multi-dimensional water quality parameters.

Acoustical damage detection of wind turbine blade using the improved incremental support vector data description

Black rice is an important rice species in Southeast Asia. It is a common phenomenon to pass low-priced black rice off as high-priced ones for economic benefit, especially in some remote towns. There is increasing need for the development of fast, easy-to-use, and low-cost analytical methods for authenticity detection. The feasibility to utilize near-infrared (NIR) spectroscopy and support vector data description (SVDD) for such a goal is explored. Principal component analysis (PCA) is used for exploratory analysis and feature extraction. Another two data description methods, i.e., k-nearest neighbor data description (KNNDD) and GAUSS method, are used as the reference. A total of 142 samples from three brands were collected for spectral analysis. Each time, the samples of a brand serve as the target class whereas other samples serve as the outlier class. Based on both the first two principal components (PCs) and original variables, three types of data descriptions were constructed. On average, the optimized SVDD model achieves acceptable performance, i.e., a specificity of 100% and a sensitivity of 94.2% on the independent test set with tight boundary. It indicates that SVDD combined with NIR is feasible and effective for authenticity detection of black rice.