Memory Augmented Variational Auto-Encoder for Anomaly Detection

Considering the development of deep learning and the emergence of intelligent control demands in nuclear reactors, along with the presence of plant‐level real‐time information monitoring systems in most nuclear power plants, there is a considerable accumulation of sensor measurements from long‐term operation. This makes it feasible to conduct medium to long‐term predictions for various real‐time conditions in nuclear power plants. Therefore, this paper proposes the utilization of a gate‐based recurrent neural network called GRU (Gated Recurrent Unit) and its variants for parameter prediction of LOCA (Loss of Coolant Accident) scenarios. The main content of this paper consists of two parts: (1) Experimental verification is conducted to demonstrate that GRU has excellent capability in capturing long‐term sequential information and generalization ability, making it suitable for predicting accident conditions in nuclear power plants. Two accident trend prediction methods based on the GRU network are proposed for scenarios with limited data. The results show that these methods can effectively provide short‐term development trends for accident conditions. Additionally, by considering the feature extraction capacity of CNN, the fusion of CNN and GRU models is employed for parameter prediction under different sizes of broken area. The results indicate an improvement in the model's generalization ability. (2) In scenarios with limited and incomplete data, a more robust variant of GRU called GRU‐D model is utilized for both univariate and multivariate synchronous prediction of accident conditions with different missing values. Experimental results demonstrate that even with a data missing rate of 90%, the GRU‐D network exhibits excellent predictive accuracy and generalization ability in parameter prediction for the given conditions.

Anomaly Detection in Dynamic Networks plays a critical role in various real-world applications such as cyberse-curity, e-commerce, and social media. Recent approaches based on graph neural networks have achieved fruitful results in static networks, and most of the researches focus on learning node embeddings to represent the large-scale complex networks, thereby facilitating downstream anomaly detection task. However, these methods require the whole network to extract intrinsic properties, so they are sensitive to the frequent changes of nodes and incapable of capturing the dynamism as realworld networks evolve over time. In this paper, we propose a novel framework DynAD for anomalous edge detection on time-evolving networks, which performs adaptive parameter learning in an end-to-end manner. In particular, DynAD first extracts fixed-size node embedding from each snapshot with temporal graph convolution and pooling operations. Then, it captures the temporal information of the graph sequence using Gated recurrent units (GRU) for anomaly detection, where an attention mechanism is employed to highlight the differences between the dynamic patterns. Experimental results on three real-world datasets illustrate that DynAD significantly outperforms the state-of-the-art baseline methods in anomaly detection.

We focus on automatic anomaly detection in SQL databases for security systems. Many logs of database systems, here the Townhall database, contain detailed information about users, like the SQL queries and the response of the database. A database is a list of log instances, where each log instance is a Cartesian product of feature values with an attached anomaly score. All log instances with the anomaly score in the top percentile are identified as anomalous. Our contribution is multi-folded. We define a model for anomaly detection of SQL databases that learns the structure of Bayesian networks from data. Our method for automatic feature extraction generates the maximal spanning tree to detect the strongest similarities between features. Novel anomaly scores based on the joint probability distribution of the database features and the log-likelihood of the maximal spanning tree detect both point and contextual anomalies. Multiple anomaly scores are combined within a robust anomaly analysis algorithm. We validate our method on the Townhall database showing the performance of our anomaly detection algorithm.

Fuzzy anomaly scores for Isolation Forest

In vivo confocal microscopy (IVCM) is a crucial imaging modality for assessing corneal diseases, yet distinguishing pathological features from normal variations remains challenging due to the complex multi-layered corneal structure. Existing anomaly detection methods often struggle to generalize across diverse disease manifestations. To address these limitations, we propose a Transformer-based unsupervised anomaly detection method for IVCM images, capable of identifying corneal abnormalities without prior knowledge of specific disease features. Our method consists of three submodules: an EfficientNet network, a Multi-Scale Feature Fusion Network, and a Transformer Network. A total of 7,063 IVCM images (95 eyes) were included for analysis. The model was trained exclusively on normal IVCM images to capture and differentiate structural variations across four distinct corneal layers: epithelium, sub-basal nerve plexus, stroma, and endothelium. During inference, anomaly scores were computed to distinguish pathological from normal images. The model's performance was evaluated on both internal and external datasets, and comparative analyses were conducted against existing anomaly detection methods, including generative adversarial networks (AnoGAN), generate to detect anomaly model (G2D), and discriminatively trained reconstruction anomaly embedding model (DRAEM). Additionally, explainable anomaly maps were generated to enhance the interpretability of model decisions. The proposed method achieved an the areas under the receiver operating characteristic curve of 0.933 on internal validation and 0.917 on an external test dataset, outperforming AnoGAN, G2D, and DRAEM in both accuracy and generalizability. The model effectively distinguished normal and pathological images, demonstrating statistically significant differences in anomaly scores (p < 0.001). Furthermore, visualization results indicated that the detected anomalous regions corresponded to morphological deviations, highlighting potential imaging biomarkers for corneal diseases. This study presents an efficient and interpretable unsupervised anomaly detection model for IVCM images, effectively identifying corneal abnormalities without requiring labeled pathological samples. The proposed method enhances screening efficiency, reduces annotation costs, and holds great potential for scalable intelligent diagnosis of corneal diseases.

The rapid expansion of blockchain technology has led to increased security challenges, particularly in detecting fraudulent transactions and malicious activities within decentralized networks. Traditional anomaly detection techniques, including rule-based heuristics and supervised learning models, struggle to adapt to the dynamic and complex nature of blockchain transactions. This paper introduces a graph neural network (GNN)-based anomaly detection framework designed to improve blockchain security by leveraging the inherent graph structure of transaction networks. The proposed approach models blockchain transactions as a directed graph, where nodes represent wallet addresses and edges correspond to transaction flows. By applying spatial and temporal graph learning techniques, the framework captures both network topology and transaction evolution over time, allowing for the identification of anomalous activities such as money laundering, phishing scams, and Ponzi schemes. The GNN model incorporates graph convolutional networks (GCN), graph attention networks (GAT), and gated recurrent units (GRU) to learn both spatial dependencies and sequential patterns within blockchain transactions. Experiments conducted on Bitcoin and Ethereum transaction datasets demonstrate that the GNN-based framework outperforms conventional fraud detection methods in terms of precision, recall, and false positive reduction. The model successfully detects fraudulent transactions with an F1-score of 0.92, showing its effectiveness in identifying emerging threats in blockchain networks. These results highlight the potential of deep learning-based anomaly detection in enhancing blockchain security, providing a scalable and adaptive solution for detecting fraud in decentralized financial ecosystems.

Explaining anomalies in coal proximity and coal processing data with Shapley and tree-based models

Predictive anomaly detection for marine diesel engine based on echo state network and autoencoder

Production prediction plays an important role in decision making, development planning, and economic evaluation during the exploration and development period. However, applying traditional methods for production forecasting of newly developed wells in the conglomerate reservoir is restricted by limited historical data, complex fracture propagation, and frequent operational changes. This study proposed a Gated Recurrent Unit (GRU) neural network-based model to achieve batch production forecasting in M conglomerate reservoir of China, which tackles the limitations of traditional decline curve analysis and conventional time-series prediction methods. The model is trained by four features of production rate, tubing pressure (TP), choke size (CS), and shut-in period (SI) from 70 multistage hydraulic fractured horizontal wells. Firstly, a comprehensive data preprocessing is implemented, including excluding unfit wells, data screening, feature selection, partitioning data set, z-score normalization, and format conversion. Then, the four-feature model is compared with the model considering production only, and it is found that with frequent oilfield operations changes, the four-feature model could accurately capture the complex variance pattern of production rate. Further, Random Forest (RF) is employed to optimize the prediction results of GRU. For a fair evaluation, the performance of the proposed model is compared with that of simple Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) neural network. The results show that the proposed approach outperforms the others in prediction accuracy and generalization ability. It is worth mentioning that under the guidance of continuous learning, the GRU model can be updated as soon as more wells become available.

The widespread application of graph data in various high-risk scenarios has increased attention to graph anomaly detection (GAD). Faced with real-world graphs that often carry node descriptions in the form of raw text sequences, termed text-attributed graphs (TAGs), existing graph anomaly detection pipelines typically involve shallow embedding techniques to encode such textual information into features, and then rely on complex self-supervised tasks within the graph domain to detect anomalies. However, this text encoding process is separated from the anomaly detection training objective in the graph domain, making it difficult to ensure that the extracted textual features focus on GAD-relevant information, seriously constraining the detection capability. How to seamlessly integrate raw text and graph topology to unleash the vast potential of cross-modal data in TAGs for anomaly detection poses a challenging issue. This paper presents a novel end-to-end paradigm for text-attributed graph anomaly detection, named CMUCL. We simultaneously model data from both text and graph structures, and jointly train text and graph encoders by leveraging cross-modal and uni-modal multi-scale consistency to uncover potential anomaly-related information. Accordingly, we design an anomaly score estimator based on inconsistency mining to derive node-specific anomaly scores. Considering the lack of benchmark datasets tailored for anomaly detection on TAGs, we release 8 datasets to facilitate future research. Extensive evaluations show that CMUCL significantly advances in text-attributed graph anomaly detection, delivering an 11.13% increase in average accuracy (AP) over the suboptimal.

Nowadays, it is common for applications to require servers to run constantly and aim as close as possible to zero downtime. The slightest failure might cause significant financial losses and sometimes even lives. For this reason, security and management measures against network threats are fundamental and have been researched for years. Software-defined networks (SDN) are an advancement in network management due to their centralization of the control plane, as it facilitates equipment setup and administration over the local network. However, this centralization makes the controller a target to denial of service attacks (DoS). In this study, we aim to develop a network anomaly detection and mitigation system that uses gated recurrent unit (GRU) neural networks combined with fuzzy logic. The neural network is trained to forecast future traffic, and anomalies are detected when the forecasting fails. The system is designed to operate in software-defined networks since they provide network flow information and tools to manage forwarding tables. We also demonstrate how the neural network&#x2019;s hyperparameters affect the detection module. The system was tested using two datasets: one with emulated traffic generated by the data communication and networking research group called Orion, from computer science department at state university of Londrina, and CICDDoS2019, a well-known dataset by the anomaly detection community. The results show that GRU networks combined with fuzzy logic are a viable option to detect anomalies in SDN and possibly in other anomaly detection applications. We compared our model with other deep learning techniques.

The escalating diversity of Internet of Things (IoT) devices has intensified security challenges, as signature-based and heavyweight machine learning (ML) defenses often fall short in resource-constrained edge environments, unable to meet real-time demands. To overcome these limitations, we introduce Patch Gate Time-series Anomaly Detection (PGTAD), a novel, lightweight, unsupervised multivariate time-series (MVTS) anomaly detection algorithm specifically tailored for IoT network traffic. PGTAD is a Gated recurrent units (GRU) autoencoder-based framework that intelligently integrates features from three perspectives—forward sequences, backward sequences, and local patch segments—using a Gate Mechanism (GM). It employs sinusoidal positional encoding for temporal awareness and a patch operation to capture localized patterns. Anomaly detection is achieved via Mahalanobis distance on reconstruction errors. Extensive experiments on publicly available IoT network traffic datasets: Edge-IIoTset, CICIoT2023, CICIoMT2024, and CICIoV2024, utilizing both packet-based and flow-based feature perspectives, demonstrated PGTAD's superior performance. Evaluated using various F-score metrics (including point-wise, composite, and affiliation) and inference throughput, PGTAD achieved an average F-score of 0.92 and sustained over 480K packets per second on an NVIDIA Jetson Orin Nano 8GB development kit. This confirms its effectiveness and generalizability for real-time detection in IoT edge environments.

Detecting anomalous traffic behaviors with seasonal deep Kalman filter graph convolutional neural networks

The anomaly detection method based on system logs can serve as an effective approach to maintain computer systems. Currently, a large number of excellent log anomaly detection methods have emerged. However, these methods usually rely on the template features in each log or combine template features with sequence information for anomaly detection, overlooking the important role of parameter features. Additionally, it is widely recognized that training a model from scratch in a new domain within a real industrial environment is both time-consuming and labor-intensive. In this paper, we introduce a novel log anomaly detection method, named LogPara, which leverages a self-attention enhanced Gated Recurrent Unit (GRU). Our approach begins by transforming each log into a corresponding semantic vector that encapsulates both parameter and template features. We propose a two-stage training objective: first, we train the model on a source domain to enable it to learn the shared features; subsequently, we fine-tune the model on the target domain to optimize its performance and reduce deployment costs. We evaluated the effectiveness of the proposed method using three real-world datasets, and the results demonstrate its superior performance in log anomaly detection.

Wind turbine blade icing data has the characteristics of multi-source and multi variable. It is difficult to characterize and identify the icing failure with multi-scale features. In this paper, a novel Non-negative Matrix Factorization-Dynamic-inner Canonical Correlation Analysis (NMF-DiCCA )based on GRU and SSAE algorithm is proposed to solve this problem. Firstly, using NMF instead of SVD decomposition method in DiCCA algorithm, the NMF-DiCCA is applied to obtain the dynamic latent variable of time serie. Secondly, the latent structure features S of dynamic latent variable is captured by SSAE. Thirdly, the temporal correlation hidden feature H of dynamic latent variable is extracted by Gated Recurrent Unit (GRU). Finally, the attention weight distribution between latent structure S and temporal correlation hidden feature H is integrated using the attention mechanism, and the fusion feature is reconstructed using the improved SSAE(ISSAE) based on GRU and SSAE.