Federated Quantum Kernel Learning for Anomaly Detection in Multivariate IoT Time-Series

Anomaly detection in time series data is a known problem, but recent growth in the number of units that can produce data require models that work on unlabelled and diverse types of data. We propose to adapt the neural network introduced by Simonyan and Zisserman in 2015 called VGG16 and used to detect and classify objects in images. We show that the VGG16 architecture with 2-dimensional convolutions replaced with 1-dimensional version could be a building block of an autoencoder approach to detect anomalies. Additionally we show that the proposed model achieves results that are similar or better than of classical anomaly detection methods.

Nowadays, time series data is more and more likely to appear in various real-world systems, such as power plants, medical care, etc. In these systems, time series anomaly detection is necessary, which involves predictive maintenance, intrusion detection, anti-fraud, cloud platform monitoring and management, etc. Generally, the anomaly detection of time series is regarded as an unsupervised learning problem. However, in a real scenario, in addition to a large set of unlabeled data, there is usually a small set of available labeled data, such as normal or abnormal data sets labeled by experts. Only a few methods use labeled data, and the existing semi-supervised algorithms are not yet suitable for the field of time series anomaly detection. In this work, we propose a semi-supervised time series anomaly detection model based on LSTM autoencoder. We improve the loss function of the LSTM autoencoder so that it can be affected by unlabeled data and labeled data at the same time, and learn the distribution of unlabeled data and labeled data at the same time by minimizing the loss function. In a large number of experiments on the Yahoo! Webscope S5 and NAB data sets, we compared the performance of the unsupervised model and the semi-supervised model of the same network framework to prove that the performance of the semi-supervised model is improved compared to the unsupervised model.KeywordsTime seriesAnomaly detectionSemi-supervised learningAutoencoderLSTM

Time series anomaly detection (TSAD) plays an important role in many domains such as finance, transportation, and healthcare. With the ongoing instrumentation of reality, more time series data will be available, leading also to growing demands for TSAD. While many TSAD methods already exist, new and better methods are still desirable. However, effective progress hinges on the availability of reliable means of evaluating new methods and comparing them with existing methods. We address deficiencies in current evaluation procedures related to datasets and experimental settings and protocols. Specifically, we propose a new time series anomaly detection benchmark, called TAB. First, TAB encompasses 29 public multivariate datasets and 1,635 univariate time series from different domains to facilitate more comprehensive evaluations on diverse datasets. Second, TAB covers a variety of TSAD methods, including Non-learning, Machine learning, Deep learning, LLM-based, and Time-series pre-trained methods. Third, TAB features a unified and automated evaluation pipeline that enables fair and easy evaluation of TSAD methods. Finally, we employ TAB to evaluate existing TSAD methods and report on the outcomes, thereby offering a deeper insight into the performance of these methods.

Time series is a very common class of data sets. Among others, it is very simple to obtain time series data from a variety of various science and finance applications and an anomaly detection technique for time series is becoming a very prominent research topic nowadays. Anomaly identification covers intrusion detection, detection of theft, mistake detection, machine health monitoring, network sensor event detection or habitat disturbance. It is also used for removing suspicious data from the data set before production. This review aims to provide a detailed and organized overview of the Anomaly detection investigation. In this article we will first define what an anomaly in time series is, and then describe quickly some of the methods suggested in the past two or three years for detection of anomaly in time series

Due to connectivity and automation trends, the medical device industry is experiencing increased demand for safety and security mechanisms. Anomaly detection has proven to be a valuable approach for ensuring safety and security in other industries, such as automotive or IT. Medical devices must operate across a wide range of values due to variations in patient anthropometric data, making anomaly detection based on a simple threshold for signal deviations impractical. For example, surgical robots directly contacting the patient's tissue require precise sensor data. However, since the deformation of the patient's body during interaction or movement is highly dependent on body mass, it is impossible to define a single threshold for implausible sensor data that applies to all patients. This also involves statistical methods, such as Z-score, that consider standard deviation. Even pure machine learning algorithms cannot be expected to provide the required accuracy simply due to the lack of available training data. This paper proposes using hybrid filters by combining dynamic system models based on expert knowledge and data-based models for anomaly detection in an operating room scenario. This approach can improve detection performance and explainability while reducing the computing resources needed on embedded devices, enabling a distributed approach to anomaly detection.

Anomaly detection in time series data (e.g., sensor data) is becoming a fundamental research problem that has various applications. Due to the complex inter-sensor relationships, it is challenging to detect anomalous events such as system faults and attacks hidden the high-dimensional time series. Recent advancements in deep learning approaches such as Graph Neural Networks (GNN) have greatly improved anomaly detection performance in time series data. However, existing methods usually use separate components to capture spatial and temporal dependence relationship and ignore the heterogeneities in spatial-temporal data which motivates us to capture them together. In this paper, we propose a novel approach STGDNN (short for Spatial-Temporal Graph Deviation Neural Networks) that learns the spatial-temporal dependence relationship together in structure learning of graph neural networks. In addition, we use the learned spatial-temporal graph structure and attention weights to explain the detected anomalies. The experiments on three real world datasets show our superiority in detection accuracy, anomaly diagnosis, and model interpretation compared with state-of-the-art methods.

Time series anomaly detection is extensively studied in statistics, economics, and computer science. Over the years, numerous methods have been proposed for time series anomaly detection using deep learning-based methods. Many of these methods demonstrate state-of-the-art performance on benchmark datasets, giving the false impression that these systems are robust and deployable in many practical and industrial real-world scenarios. In this paper, we demonstrate that the performance of state-of-the-art anomaly detection methods is degraded substantially by adding only small adversarial perturbations to the sensor data. We use different scoring metrics such as prediction errors, anomaly, and classification scores over several public and private datasets ranging from aerospace applications, server machines, to cyber-physical systems in power plants. Under well-known adversarial attacks from Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) methods, we demonstrate that state-of-the-art deep neural networks (DNNs) and graph neural networks (GNNs) methods, which claim to be robust against anomalies and have been possibly integrated in real-life systems, have their performance drop to as low as 0%. To the best of our understanding, we demonstrate, for the first time, the vulnerabilities of anomaly detection systems against adversarial attacks. The overarching goal of this research is to raise awareness towards the adversarial vulnerabilities of time series anomaly detectors.

We present an end-end system for time series anomaly detection specifically aimed at detecting fraudulent transactions in bank transaction datasets. The goal of anomaly detection is to locate unusual or uncommon occurrences in data. Outlier detection, one of the most crucial data analysis jobs, has many practical uses. For instance, in order to track their company data and send out notifications for outliers, Yahoo and Microsoft have developed their own services for time-series outlier identification. However, majority of outlier identification methods employ classifications of unusual entries without providing any justification. Therefore, the goal is to make the anomalous points as predicted by the model more interpretable and thereby create a flagging system to help human labelers in determining anomalous behavior. This will help reduce human engineering efforts by a huge margin.

Multi-dimensional time series are a series of data recorded by multiple variables in the order of time. Anomaly detection refers to the detection of data that do not conform to normal expectations. Existing multi dimensional time series anomaly detection methods do not well distinguish the anomalies in the data, either in the effectiveness of the algorithm, or in the time cost. Based on the research at home and abroad, an algorithm with both algorithm effectiveness and reducing time cost advantages: Temporal Deep Residual Shrinkage Networks, TDRSN. Combining Time Convolutional Network (TCN) with Deep Residual Shrinkage Network (DRSN), adopt dilated causal convolution to expand the receptive field, introduce the soft threshold of self attention, enabling it to not only remember historical data, reduce the impact of gradient vanishing and gradient exploding, but also automatically set thresholds without expert experience, eliminate redundant features and reduce the influence of unimportant features. The experimental results show that TDRSN has a better anomaly detection effect than TCN, DRSN and other classical time series anomaly detection models.

The escalating diversity of Internet of Things (IoT) devices has intensified security challenges, as signature-based and heavyweight machine learning (ML) defenses often fall short in resource-constrained edge environments, unable to meet real-time demands. To overcome these limitations, we introduce Patch Gate Time-series Anomaly Detection (PGTAD), a novel, lightweight, unsupervised multivariate time-series (MVTS) anomaly detection algorithm specifically tailored for IoT network traffic. PGTAD is a Gated recurrent units (GRU) autoencoder-based framework that intelligently integrates features from three perspectives—forward sequences, backward sequences, and local patch segments—using a Gate Mechanism (GM). It employs sinusoidal positional encoding for temporal awareness and a patch operation to capture localized patterns. Anomaly detection is achieved via Mahalanobis distance on reconstruction errors. Extensive experiments on publicly available IoT network traffic datasets: Edge-IIoTset, CICIoT2023, CICIoMT2024, and CICIoV2024, utilizing both packet-based and flow-based feature perspectives, demonstrated PGTAD's superior performance. Evaluated using various F-score metrics (including point-wise, composite, and affiliation) and inference throughput, PGTAD achieved an average F-score of 0.92 and sustained over 480K packets per second on an NVIDIA Jetson Orin Nano 8GB development kit. This confirms its effectiveness and generalizability for real-time detection in IoT edge environments.

Given high-dimensional time series data (e.g., sensor data), how can we detect anomalous events, such as system faults and attacks? More challengingly, how can we do this in a way that captures complex inter-sensor relationships, and detects and explains anomalies which deviate from these relationships? Recently, deep learning approaches have enabled improvements in anomaly detection in high-dimensional datasets; however, existing methods do not explicitly learn the structure of existing relationships between variables, or use them to predict the expected behavior of time series. Our approach combines a structure learning approach with graph neural networks, additionally using attention weights to provide explainability for the detected anomalies. Experiments on two real-world sensor datasets with ground truth anomalies show that our method detects anomalies more accurately than baseline approaches, accurately captures correlations between sensors, and allows users to deduce the root cause of a detected anomaly.

Multivariate time series anomaly detection has numerous real-world applications and is being extensively studied. Modeling pairwise correlations between variables is crucial. Existing methods employ learnable graph structures and graph neural networks to explicitly model the spatial dependencies between variables. However, these methods are primarily based on prediction or reconstruction tasks, which can only learn similarity relationships between sequence embeddings and lack interpretability in how graph structures affect time series evolution. In this paper, we designed a framework that models spatial dependencies using interpretable causal relationships and detects anomalies through changes in causal patterns. Specifically, we propose a method to dynamically discover Granger causality using gradients in nonlinear deep predictors and employ a simple sparsification strategy to obtain a Granger causality graph, detecting anomalies from a causal perspective. Experiments on real-world datasets demonstrate that the proposed model achieves more accurate anomaly detection compared to baseline methods.

ABSTRACTDeep neural networks have exhibited preeminent performance in anomaly detection, but they struggle to effectively capture changes over time in multivariate time‐series data and suffer from resource consumption issues. Spiking neural networks address these limitations by capturing the change in time‐varying signals and decreasing resource consumption, but they sacrifice performance. This paper develops a novel spiking‐based hybrid model incorporated a temporal prediction network and a reconstruction network. It integrates a unique first‐spike frequency encoding scheme and a firing rate based anomaly score method. The encoding scheme enhances the event representation ability, while the anomaly score enables efficient anomaly identification. Our proposed model not only maintains low resource consumption but also improves the ability of anomaly detection. Experiments on publicly real‐world datasets confirmed that the proposed model acquires state‐of‐the‐art performance superior to existing approaches. Remarkably, it costs 5.04× lower energy consumption compared with the artificial neural network version.

Statistical parametric methodologies are widely employed in the analysis of time series of gas turbine sensor readings. These methodologies identify outliers as a consequence of excessive deviation from a statistically-based model, derived from available observations. Among parametric techniques, the k-σ methodology demonstrates its effectiveness in the analysis of stationary time series. Furthermore, the simplicity and the clarity of this approach justify its direct application to industry. On the other hand, the k-σ methodology usually proves to be unable to adapt to dynamic time series, since it identifies observations in a transient as outliers. As this limitation is caused by the nature of the methodology itself, two improved approaches are considered in this paper in addition to the standard k-σ methodology. The two proposed methodologies maintain the same rejection rule of the standard k-σ methodology, but differ in the portions of the time series from which statistical parameters (mean and standard deviation) are inferred. The first approach performs statistical inference by considering all observations prior to the current one, which are assumed reliable, plus a forward window containing a specified number of future observations. The second approach proposed in this paper is based on a moving window scheme. Simulated data are used to tune the parameters of the proposed improved methodologies and to prove their effectiveness in adapting to dynamic time series. The moving window approach is found to be the best on simulated data in terms of True Positive Rate (TPR), False Negative Rate (FNR) and False Positive Rate (FPR). Therefore, the performance of the moving window approach is further assessed towards both different simulated scenarios and field data taken on a gas turbine.

Anomaly detection in high-dimensional time series is typically tackled using either reconstruction- or forecasting-based algorithms due to their abilities to learn compressed data representations and model temporal dependencies, respectively. However, most existing methods disregard the relationships between features, information that would be extremely useful when incorporated into a model. In this work, we introduce Fused Sparse Autoencoder and Graph Net (FuSAGNet), which jointly optimizes reconstruction and forecasting while explicitly modeling the relationships within multivariate time series. Our approach combines Sparse Autoencoder and Graph Neural Network, the latter of which predicts future time series behavior from sparse latent representations learned by the former as well as graph structures learned through recurrent feature embedding. Experimenting on three real-world cyber-physical system datasets, we empirically demonstrate that the proposed method enhances the overall anomaly detection performance, outperforming baseline approaches. Moreover, we show that mining sparse latent patterns from high-dimensional time series improves the robustness of the graph-based forecasting model. Lastly, we conduct visual analyses to investigate the interpretability of both recurrent feature embeddings and sparse latent representations.