An Unsupervised Anomaly Detection Method for Nuclear Reactor Coolant Pumps Based on Kernel Self-Organizing Map and Bayesian Posterior Inference

The informed sampler: A discriminative approach to Bayesian inference in generative computer vision models

Parameter estimation and inference for L̩vy models under high-frequency data has been an exciting and important task in the field of financial mathematics and has been found practically useful when analyzing real financial data. One feature of L̩vy models is the allowance of jumps to model the abrupt changes sometimes observed in the market. In this thesis, we discuss some problems related to the statistical inference of L̩vy models based on high-frequency data emphasizing on the presence of the jumps. The first problem we consider focuses on the estimation of the volatility, which is critical to measure and control the risk of financial assets accurately. In order to accommodate the high-frequency data, infinite jump activity and microstructure noise are considered to be present. We propose a "purposely misspecified" posterior of the volatility obtained by ignoring the jump-component of the process. The misspecified posterior is further corrected by a simple estimate of the location shift and re-scaling of the log-likelihood. Our main result establishes a Bernstein-von Mises theorem, which states that the proposed adjusted posterior is asymptotically Gaussian, centered at a consistent estimator, and with variance equal to the inverse of the Fisher information. In the absence of microstructure noise, our approach can be extended to inferences of the integrated variance of general It̫ semimartingales. This method simplifies the complete Bayesian inference of the volatility, which would require the construction of the joint posterior. Thus, it avoids the requirement of a prior distribution on the jump process, the difficulties in specifying the likelihood function, and the implementation of computationally-expensive methods to sample from the joint posterior. Simulations are provided to demonstrate the accuracy of the resulting credible intervals, and the frequentist properties of the approximate Bayesian inference based on the adjusted posterior. As one of the conditions of the Bernstein-von Mises theorem, the local asymptotic normality (LAN) is a key concept in the classical theory of asymptotic analysis. We consider a class of jump processes, which can be represented as a Brownian motion time-changed by a driftless generalized gamma convolution subordinator. Sufficient conditions for LAN property to hold are first summarized. Then, LAN property is proved uniformly for the whole class. We also consider a semiparametric model, including any subordinator-related parameters. Our results show that the LAN derived for the parametric part of the model also regulates the estimation behavior of the semiparametric model. The estimation behavior includes the nonexistence of an estimator, whose convergence rate is faster than the rate imposed by the LAN property, and also an asymptotic risk lower bound of all estimators with the optimal rate.

Anomaly detection is the process of identifying unexpected items or events in datasets, which differ from the norm. In contrast to standard classification tasks, anomaly detection is often applied on unlabeled data, taking only the internal structure of the dataset into account. This challenge is known as unsupervised anomaly detection and is addressed in many practical applications, for example in network intrusion detection, fraud detection as well as in the life science and medical domain. Dozens of algorithms have been proposed in this area, but unfortunately the research community still lacks a comparative universal evaluation as well as common publicly available datasets. These shortcomings are addressed in this study, where 19 different unsupervised anomaly detection algorithms are evaluated on 10 different datasets from multiple application domains. By publishing the source code and the datasets, this paper aims to be a new well-funded basis for unsupervised anomaly detection research. Additionally, this evaluation reveals the strengths and weaknesses of the different approaches for the first time. Besides the anomaly detection performance, computational effort, the impact of parameter settings as well as the global/local anomaly detection behavior is outlined. As a conclusion, we give an advise on algorithm selection for typical real-world tasks.

Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications, where only anomaly-free samples are available for training. Some UAD applications intend to locate the anomalous regions further even without any anomaly information. Although the absence of anomalous samples and annotations deteriorates the UAD performance, an inconspicuous, yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection (AD) and localization in an unsupervised fashion. The flow-based probabilistic models, only trained on anomaly-free data, can efficiently distinguish unpredictable anomalies by assigning them much lower likelihoods than normal data. Nevertheless, the size variation of unpredictable anomalies introduces another inconvenience to the flow-based methods for high-precision AD and localization. To generalize the anomaly size variation, we propose a novel multiscale flow-based framework (MSFlow) composed of asymmetrical parallel flows followed by a fusion flow to exchange multiscale perceptions. Moreover, different multiscale aggregation strategies are adopted for image-wise AD and pixel-wise anomaly localization according to the discrepancy between them. The proposed MSFlow is evaluated on three AD datasets, significantly outperforming existing methods. Notably, on the challenging MVTec AD benchmark, our MSFlow achieves a new state-of-the-art (SOTA) with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8% and PRO score of 97.1%.

Supervised deep learning approaches provide state-of-the-art performance on medical image classification tasks for disease screening. However, these methods require large labeled datasets that involve resource-intensive expert annotation. Further, disease screening applications have low prevalence of abnormal samples; this class imbalance makes the task more akin to anomaly detection. While the machine learning community has proposed unsupervised deep learning methods for anomaly detection, they have yet to be characterized on medical images where normal vs. anomaly distinctions may be more subtle and variable. In this work, we characterize existing unsupervised anomaly detection methods on retinal fundus images, and find that they require significant fine tuning and offer unsatisfactory performance. We thus propose an efficient and effective transfer-learning based approach for unsupervised anomaly detection. Our method employs a deep convolutional neural network trained on ImageNet as a feature extractor, and subsequently feeds the learned feature representations into an existing unsupervised anomaly detection method. We show that our approach significantly outperforms baselines on two natural image datasets and two retinal fundus image datasets, all with minimal fine-tuning. We further show the ability to leverage very small numbers of labelled anomalies to improve performance. Our work establishes a strong unsupervised baseline for image-based anomaly detection, alongside a flexible and scalable approach for screening applications.

Guided wave-based structural health monitoring is extensively utilized in various industrial applications to ensure the integrity of components within industrial systems. Among these monitoring techniques, principal component analysis (PCA) reconstruction methods are widely used for anomaly detection due to their computational efficiency and interoperability. However, existing PCA reconstruction methods are semisupervised anomaly detection approaches that require training on historical normal data and fail to detect anomalous signals within the training set. To address this limitation, this work proposes a correntropy PCA (C-PCA), enabling fully unsupervised anomaly detection on raw training data without requiring label information, when the dataset contains a high proportion of abnormal signals. This method allows anomaly detection on real-time measurements without the need for precleaned historical normal data or can also be used to generate clean data for existing semisupervised anomaly detection methods. In correntropy PCA, principal components are extracted from the correntropy matrix rather than the correlation matrix. The correntropy, representing the statistical dependence between samples of guided waves, is estimated utilizing a Gaussian kernel with a specified kernel width. Through the optimization of the kernel width, the correntropy PCA reconstruction method demonstrates superior anomaly detection performance compared with the standard PCA reconstruction method, especially in scenarios where training data are contaminated by a significant proportion of abnormal signals. Guidelines for the optimization of the kernel width are provided. The effectiveness of the correntropy PCA reconstruction-based anomaly detection method is validated using data collected from ten regions over an 80-day period, encompassing guided waves induced by damage occurring over durations ranging from 2 to 20 days.

To defend a network system from security risks, intrusion detection systems (IDSs) have been playing an important role in recent years. There are two types of detection algorithms of IDSs: misuse detection and anomaly detection. Because misuse detection is based on a signature which is created from the features of attack traffic by security experts, it can achieve accurate and stable detection. However, its weakness is the difficulty of detecting new attacks (i.e., 0-day attack), and the cost of maintaining the latest signature version. Thinking of the increase of the skillful intrusion, e.g., intrusion showing similar access behavior to normal, misuse detection cannot handle these critical attacks, which results in a large number of false alarms. To cope with these problems, we present a clustering algorithm based on an unsupervised anomaly detection. We evaluated our system using Kyoto2006+ data set and KDD Cup 1999 data set. Evaluation results show that our approach achieved a higher detection rate in the region of very low false positive rate and real-time preprocessing capability.

Effective interrelation of Bayesian nonparametric document clustering and embedded-topic modeling

Reservoir development optimization under uncertainty for infill well placement in brownfield redevelopment

Radio environment maps (REMs) have demonstrated their significance in spectrum sensing, control, and sharing, particularly in relation to unmanned systems, the Internet of Things (IoT), and 5G communication. However, they encounter challenges related to meeting the increasing accuracy requirements. This paper introduces the BHM-RK algorithm, a novel approach for constructing REMs in scenarios with multiple transmitters. The algorithm utilizes spatial clustering for transmitter partitioning, Bayesian posterior inference within partitions, and residual Kriging interpolation to address challenges posed by diverse transmitter influences. Experimental validation demonstrates the superior performance of the BHM-RK algorithm in terms of construction accuracy compared to existing methods, highlighting its effectiveness in enhancing precision in REM construction for scenarios with multiple transmitters.

Single-cell RNA-sequencing (scRNA-seq) enables gene expression profiling at single-cell resolution, but it loses the spatial information of cells for solid tissues during the tissue dissociation step before sequencing. In contrast, bulk spatial transcriptomics (ST) methods can measure the expression of spatially organized spots in solid tissues, but as a spot comprises dozens of cells, ST expression levels are averaged signals and lack cellular resolution. Joint analysis of these two complementary data types provides the opportunity to recover the spatial patterns of cell types and obtain the cellular enrichment of spots. However, there is a lack of unified statistical methods to achieve this goal. This study develops a Bayesian statistical method named BEATS to jointly model scRNA-seq data and bulk ST data from a common sample in the presence of cellular and spatial heterogeneity. BEATS can simultaneously (a) discover cell types, where cells in a cell type share mean expression profiles; (b) identify spot regions, where a region is a set of spots with the same cellular compositions; and (c) estimate cell-type proportions for each spot region. The Bayesian posterior inference is performed through a hybrid Markov chain Monte Carlo sampling algorithm. Extensive simulation studies and application to datasets on pancreatic ductal adenocarcinoma tissues demonstrate the practical utility of BEATS.

Anomaly detection (AD) aims to distinguish the data points that are inconsistent with the overall pattern of the data. Recently, unsupervised anomaly detection methods have aroused huge attention. Among these methods, feature representation (FR) plays an important role, which can directly affect the performance of anomaly detection. Sparse representation (SR) can be regarded as one of matrix factorization (MF) methods, which is a powerful tool for FR. However, there are some limitations in the original SR. On the one hand, it just learns the shallow feature representations, which leads to the poor performance for anomaly detection. On the other hand, the local geometry structure information of data is ignored. To address these shortcomings, a graph regularized deep sparse representation (GRDSR) approach is proposed for unsupervised anomaly detection in this work. In GRDSR, a deep representation framework is first designed by extending the single layer MF to a multilayer MF for extracting hierarchical structure from the original data. Next, a graph regularization term is introduced to capture the intrinsic local geometric structure information of the original data during the process of FR, making the deep features preserve the neighborhood relationship well. Then, a L1-norm-based sparsity constraint is added to enhance the discriminant ability of the deep features. Finally, a reconstruction error is applied to distinguish anomalies. In order to demonstrate the effectiveness of the proposed approach, we conduct extensive experiments on ten datasets. Compared with the state-of-the-art methods, the proposed approach can achieve the best performance.

Data anomaly detection plays a pivotal role in structural health monitoring (SHM), enabling reliable and accurate assessment of structural condition. This paper proposes an unsupervised anomaly detection method based on the spatio-temporal sparse integration framework (STSIF), which aims to effectively distinguish various types of anomalous patterns in practical applications. First, the anomaly patterns are classified into three-level categories—normal, mild anomaly, and significant anomaly—and then a three-stage unsupervised anomaly detection framework is designed for these categories. At each stage, the characteristics-based detection indicators are proposed to detect significant, mild, and local anomaly progressively. For significant anomaly, the spatial-sparse fusion indicator is proposed and clustered with the unsupervised Gaussian mixture model (GMM). For mild anomaly, the sparse-based reconstruction detection method is proposed for qualitative analysis of noise levels, in which the Pearson correlation between the original and sparse-based reconstructed signal is computed for the GMM clustering. Overall accuracy of 98.36% and 95.51% is obtained for the data sets on two real-world bridges, respectively. Finally, regarding local anomaly detection, the proposed mask-based ResNet18 effectively emphasizes the local contextual information within the signal and substantially reduces the risk of overgeneralization. In summary, the proposed STSIF could efficiently detect various anomaly patterns without substantial training data or complex parameter tuning, providing a comprehensive detection paradigm for SHM.

An increase in the collection of physiological signals, whether done implicitly in wearable or IoT device or explicitly in experimental and laboratory environments, creates the need for development of smart systems and tools capable of data analysis with limited expert knowledge. Anomaly detection, specifically unsupervised anomaly detection can be used to design a tool or even as a tool to help remedy this issue. This paper will focus on how unsupervised anomaly detection can be utilized for the development of such systems. A systematic, robust, and customizable approach will be presented and preliminary results will be shown that open the door for future research and algorithm development.

With the prevalence of Internet of Things (IoT)-based social media applications, the distance among people has been greatly shortened. As a result, recommender systems in IoT-based social media need to be developed oriented to groups of users rather than individual users. However, existing methods were highly dependent on explicit preference feedbacks, ignoring scenarios of implicit feedbacks. To remedy such gap, this paper proposes an implicit feedback-based group recommender system using probabilistic inference and non-cooperative game (GREPING) for IoT-based social media. Particularly, unknown process variables can be estimated from observable implicit feedbacks via Bayesian posterior probability inference. In addition, the globally optimal recommendation results can be calculated with the aid of non-cooperative game. Two groups of experiments are conducted to assess the GREPING from two aspects: efficiency and robustness. Experimental results show obvious promotion and considerable stability of the GREPING compared to baseline methods.