Semi-supervised Time Series Anomaly Detection Model Based on LSTM Autoencoder

Anomaly detection in time series data is a known problem, but recent growth in the number of units that can produce data require models that work on unlabelled and diverse types of data. We propose to adapt the neural network introduced by Simonyan and Zisserman in 2015 called VGG16 and used to detect and classify objects in images. We show that the VGG16 architecture with 2-dimensional convolutions replaced with 1-dimensional version could be a building block of an autoencoder approach to detect anomalies. Additionally we show that the proposed model achieves results that are similar or better than of classical anomaly detection methods.

Anomaly detection in time series data (e.g., sensor data) is becoming a fundamental research problem that has various applications. Due to the complex inter-sensor relationships, it is challenging to detect anomalous events such as system faults and attacks hidden the high-dimensional time series. Recent advancements in deep learning approaches such as Graph Neural Networks (GNN) have greatly improved anomaly detection performance in time series data. However, existing methods usually use separate components to capture spatial and temporal dependence relationship and ignore the heterogeneities in spatial-temporal data which motivates us to capture them together. In this paper, we propose a novel approach STGDNN (short for Spatial-Temporal Graph Deviation Neural Networks) that learns the spatial-temporal dependence relationship together in structure learning of graph neural networks. In addition, we use the learned spatial-temporal graph structure and attention weights to explain the detected anomalies. The experiments on three real world datasets show our superiority in detection accuracy, anomaly diagnosis, and model interpretation compared with state-of-the-art methods.

Time series anomaly detection (TSAD) plays an important role in many domains such as finance, transportation, and healthcare. With the ongoing instrumentation of reality, more time series data will be available, leading also to growing demands for TSAD. While many TSAD methods already exist, new and better methods are still desirable. However, effective progress hinges on the availability of reliable means of evaluating new methods and comparing them with existing methods. We address deficiencies in current evaluation procedures related to datasets and experimental settings and protocols. Specifically, we propose a new time series anomaly detection benchmark, called TAB. First, TAB encompasses 29 public multivariate datasets and 1,635 univariate time series from different domains to facilitate more comprehensive evaluations on diverse datasets. Second, TAB covers a variety of TSAD methods, including Non-learning, Machine learning, Deep learning, LLM-based, and Time-series pre-trained methods. Third, TAB features a unified and automated evaluation pipeline that enables fair and easy evaluation of TSAD methods. Finally, we employ TAB to evaluate existing TSAD methods and report on the outcomes, thereby offering a deeper insight into the performance of these methods.

Time series is a very common class of data sets. Among others, it is very simple to obtain time series data from a variety of various science and finance applications and an anomaly detection technique for time series is becoming a very prominent research topic nowadays. Anomaly identification covers intrusion detection, detection of theft, mistake detection, machine health monitoring, network sensor event detection or habitat disturbance. It is also used for removing suspicious data from the data set before production. This review aims to provide a detailed and organized overview of the Anomaly detection investigation. In this article we will first define what an anomaly in time series is, and then describe quickly some of the methods suggested in the past two or three years for detection of anomaly in time series

Due to connectivity and automation trends, the medical device industry is experiencing increased demand for safety and security mechanisms. Anomaly detection has proven to be a valuable approach for ensuring safety and security in other industries, such as automotive or IT. Medical devices must operate across a wide range of values due to variations in patient anthropometric data, making anomaly detection based on a simple threshold for signal deviations impractical. For example, surgical robots directly contacting the patient's tissue require precise sensor data. However, since the deformation of the patient's body during interaction or movement is highly dependent on body mass, it is impossible to define a single threshold for implausible sensor data that applies to all patients. This also involves statistical methods, such as Z-score, that consider standard deviation. Even pure machine learning algorithms cannot be expected to provide the required accuracy simply due to the lack of available training data. This paper proposes using hybrid filters by combining dynamic system models based on expert knowledge and data-based models for anomaly detection in an operating room scenario. This approach can improve detection performance and explainability while reducing the computing resources needed on embedded devices, enabling a distributed approach to anomaly detection.

Time series anomaly detection is extensively studied in statistics, economics, and computer science. Over the years, numerous methods have been proposed for time series anomaly detection using deep learning-based methods. Many of these methods demonstrate state-of-the-art performance on benchmark datasets, giving the false impression that these systems are robust and deployable in many practical and industrial real-world scenarios. In this paper, we demonstrate that the performance of state-of-the-art anomaly detection methods is degraded substantially by adding only small adversarial perturbations to the sensor data. We use different scoring metrics such as prediction errors, anomaly, and classification scores over several public and private datasets ranging from aerospace applications, server machines, to cyber-physical systems in power plants. Under well-known adversarial attacks from Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) methods, we demonstrate that state-of-the-art deep neural networks (DNNs) and graph neural networks (GNNs) methods, which claim to be robust against anomalies and have been possibly integrated in real-life systems, have their performance drop to as low as 0%. To the best of our understanding, we demonstrate, for the first time, the vulnerabilities of anomaly detection systems against adversarial attacks. The overarching goal of this research is to raise awareness towards the adversarial vulnerabilities of time series anomaly detectors.

The prevalence of streaming data across various sectors poses significant challenges for real-time anomaly detection due to its volume, velocity, and variability. Traditional data processing methods often need to be improved for such dynamic environments, necessitating robust, scalable, and efficient real-time analysis systems. This study compares two advanced machine learning approaches—LSTM autoencoders and Matrix Profile algorithms—to identify the most effective method for anomaly detection in streaming environments using the NYC taxi dataset. Existing literature on anomaly detection in streaming data highlights various methodologies, including statistical tests, window-based techniques, and machine learning models. Traditional methods like the Generalized ESD test have been adapted for streaming data but often require a full historical dataset to function effectively. In contrast, machine learning approaches, particularly those using LSTM networks, are noted for their ability to learn complex patterns and dependencies, offering promising results in real-time applications. In a comparative analysis, LSTM autoencoders significantly outperformed other methods, achieving an F1-score of 0.22 for anomaly detection, notably higher than other techniques. This model demonstrated superior capability in capturing temporal dependencies and complex data patterns, making it highly effective for the dynamic and varied data in the NYC taxi dataset. The LSTM autoencoder's advanced pattern recognition and anomaly detection capabilities confirm its suitability for complex, high-velocity streaming data environments. Future research should explore the integration of LSTM autoencoders with other machine-learning techniques to enhance further the accuracy, scalability, and efficiency of anomaly detection systems. This study advances our understanding of scalable machine-learning approaches and underscores the critical importance of selecting appropriate models based on the specific characteristics and challenges of the data involved.

We present an end-end system for time series anomaly detection specifically aimed at detecting fraudulent transactions in bank transaction datasets. The goal of anomaly detection is to locate unusual or uncommon occurrences in data. Outlier detection, one of the most crucial data analysis jobs, has many practical uses. For instance, in order to track their company data and send out notifications for outliers, Yahoo and Microsoft have developed their own services for time-series outlier identification. However, majority of outlier identification methods employ classifications of unusual entries without providing any justification. Therefore, the goal is to make the anomalous points as predicted by the model more interpretable and thereby create a flagging system to help human labelers in determining anomalous behavior. This will help reduce human engineering efforts by a huge margin.

Multi-dimensional time series are a series of data recorded by multiple variables in the order of time. Anomaly detection refers to the detection of data that do not conform to normal expectations. Existing multi dimensional time series anomaly detection methods do not well distinguish the anomalies in the data, either in the effectiveness of the algorithm, or in the time cost. Based on the research at home and abroad, an algorithm with both algorithm effectiveness and reducing time cost advantages: Temporal Deep Residual Shrinkage Networks, TDRSN. Combining Time Convolutional Network (TCN) with Deep Residual Shrinkage Network (DRSN), adopt dilated causal convolution to expand the receptive field, introduce the soft threshold of self attention, enabling it to not only remember historical data, reduce the impact of gradient vanishing and gradient exploding, but also automatically set thresholds without expert experience, eliminate redundant features and reduce the influence of unimportant features. The experimental results show that TDRSN has a better anomaly detection effect than TCN, DRSN and other classical time series anomaly detection models.

In recent years, anomaly detection techniques in time-series data have been widely used in manufacturing, cybersecurity, and other fields. Meanwhile, various anomaly detection models based on generative adversarial networks (GAN) are gradually used in time-series anomaly detection tasks. However, there are problems of unstable generator training, missed detection of anomalous data, and inconsistency between the discriminator's discriminant and the anomaly detection target in GAN networks. Aiming at the above problems, the paper proposes a DUAL-ADGAN (Dual Anomaly Detection Generative Adversarial Networks) model for the detection of anomalous data in time series. First, the Wasserstein distance satisfying the Lipschitz constraint is used as the loss function of the data reconstruction module, which improves the stability of the traditional GAN network training. Second, by adding a data prediction module to the DUAL-ADGAN model, the distinction between abnormal and normal samples is increased, and the rate of missing abnormal data in the model is reduced. Third, by introducing the Fence-GAN loss function, the discriminator is aligned with the anomaly detection target, which effectively reduces the anomaly data false detection rate of the DUAL-ADGAN model. Finally, anomaly scores derived from the DUAL-ADGAN model are compared with dynamic thresholds to detect anomalies. The experimental results show that the average F1 of the DUAL-ADGAN model is 0.881, which is better than the other nine baseline models. The conclusions demonstrate that the DUAL-ADGAN model proposed in the paper is more stable in training while effectively solving the problems of anomaly miss detection and discriminator inconsistency with the anomaly detection target in the anomaly detection task.

We propose a simple anomaly detection method that is applicable to unlabeled time series data and is sufficiently tractable, even for non-technical entities, by using the density ratio estimation based on the state space model. Our detection rule is based on the ratio of log-likelihoods estimated by the dynamic linear model, i.e. the ratio of log-likelihood in our model to that in an over-dispersed model that we will call the NULL model. Using the Yahoo S5 data set and the Numenta Anomaly Benchmark data set, publicly available and commonly used benchmark data sets, we find that our method achieves better or comparable performance compared to the existing methods. The result implies that it is essential in time series anomaly detection to incorporate the specific information on time series data into the model. In addition, we apply the proposed method to unlabeled Web time series data, specifically, daily page view and average session duration data on an electronic commerce site that deals in insurance goods to show the applicability of our method to unlabeled real-world data. We find that the increase in page view caused by e-mail newsletter deliveries is less likely to contribute to completing an insurance contract. The result also suggests the importance of the simultaneous monitoring of more than one time series.

Recently, Time Series Remote Sensing Images (TSRSIs) have been proven to be a significant resource for land use/land cover (LULC) mapping. Deep learning methods perform well in managing and processing temporal dependencies and have shown remarkable advancements within this domain. Although deep learning methods have exhibited outstanding performance in classifying TSRSIs, they rely on enough labeled time series samples for effective training. Labeling data with a wide geographical range and a long time span is highly time-consuming and labor-intensive. Active learning (AL) is a promising method of selecting the most informative data for labeling to save human labeling efforts. It has been widely applied in the remote sensing community, except for the classification of TSRSIs. The main challenge of AL in TSRSI classification is dealing with the internal temporal dependencies within TSRSIs and evaluating the informativeness of unlabeled time series data. In this paper, we propose a data-driven active deep learning framework for TSRSI classification to address the problem of limited labeled time series samples. First, a temporal classifier for TSRSI classification tasks is designed. Next, we propose an effective active learning method to select informative time series samples for labeling, which considers representativeness and uncertainty. For representativeness, we use the K-shape method to cluster time series data. For uncertainty, we construct an auxiliary deep network to evaluate the uncertainty of unlabeled data. The features with rich temporal information in the classifier's middle-hidden layers will be fed into the auxiliary deep network. Then, we define a new loss function with the aim of improving the deep model's performance. Finally, the proposed method in this paper was verified on two TSRSI datasets. The results demonstrate a significant advantage of our method over other approaches to TSRSI. On the MUDS dataset, when the initial number of samples was 100 after our method selected and labeled 2000 samples, an accuracy improvement of 4.92% was achieved. On the DynamicEarthNet dataset, when the initial number of samples was 1000 after our method selected and labeled 2000 samples, an accuracy improvement of 7.81% was attained. On the PASTIS dataset, when the initial number of samples was 1000 after our method selected and labeled 2000 samples, an accuracy improvement of 4.89% was achieved. Our code is available in Data Availability Statement.

The escalating diversity of Internet of Things (IoT) devices has intensified security challenges, as signature-based and heavyweight machine learning (ML) defenses often fall short in resource-constrained edge environments, unable to meet real-time demands. To overcome these limitations, we introduce Patch Gate Time-series Anomaly Detection (PGTAD), a novel, lightweight, unsupervised multivariate time-series (MVTS) anomaly detection algorithm specifically tailored for IoT network traffic. PGTAD is a Gated recurrent units (GRU) autoencoder-based framework that intelligently integrates features from three perspectives—forward sequences, backward sequences, and local patch segments—using a Gate Mechanism (GM). It employs sinusoidal positional encoding for temporal awareness and a patch operation to capture localized patterns. Anomaly detection is achieved via Mahalanobis distance on reconstruction errors. Extensive experiments on publicly available IoT network traffic datasets: Edge-IIoTset, CICIoT2023, CICIoMT2024, and CICIoV2024, utilizing both packet-based and flow-based feature perspectives, demonstrated PGTAD's superior performance. Evaluated using various F-score metrics (including point-wise, composite, and affiliation) and inference throughput, PGTAD achieved an average F-score of 0.92 and sustained over 480K packets per second on an NVIDIA Jetson Orin Nano 8GB development kit. This confirms its effectiveness and generalizability for real-time detection in IoT edge environments.

Given high-dimensional time series data (e.g., sensor data), how can we detect anomalous events, such as system faults and attacks? More challengingly, how can we do this in a way that captures complex inter-sensor relationships, and detects and explains anomalies which deviate from these relationships? Recently, deep learning approaches have enabled improvements in anomaly detection in high-dimensional datasets; however, existing methods do not explicitly learn the structure of existing relationships between variables, or use them to predict the expected behavior of time series. Our approach combines a structure learning approach with graph neural networks, additionally using attention weights to provide explainability for the detected anomalies. Experiments on two real-world sensor datasets with ground truth anomalies show that our method detects anomalies more accurately than baseline approaches, accurately captures correlations between sensors, and allows users to deduce the root cause of a detected anomaly.

Thyroid nodule is a common clinical disease, although most nodules are benign, the incidence rate of thyroid cancer has risen rapidly in recent years. Even though many methods have achieved automated thyroid nodule segmentation based on deep learning, these methods are based on supervised learning and require a large amount of labeled data for training. However, the labeling work must be carried out by professional doctors, which results in a small number of datasets and difficulty in labeling. To address this problem, this paper proposes a semi-supervised thyroid nodule segmentation model via joint consistency learning and co-training. This model includes two branches: consistency learning and co-training. In the consistency learning branch, based on consistent regularization, the teacher model guides the student model to optimize. In order to make the teacher model more stable, we design a co-training framework to further optimize the teacher model. In co-training branch, the teacher model and TransUNet extract different representations of the same sample and teach each other to prevent consistent but incorrect predictions between the teacher model and the student model. This semi-supervised model can learns useful feature representations from unlabeled data, and effectively trains the model with a small amount of labeled data, reducing the dependence on labeled data during the model training process.