MultiSR: A SR-Based Framework for Multivariate Time Series Anomaly Detection in Complex Unmanned Scenarios

<p>Anomaly detection with multivariate time series data collected by multi-sensors is a challenging problem due to the complexity and high dimension of data and the difficulty of manually labelling data. This paper proposes a novel unsupervised anomaly detection model using spatial-temporal self-attention based on transformer architecture, denoted the biself-attention anomaly detection (BSAAD) model. The BSAAD model not only utilizes a time-step encoder with a self-attention mechanism to capture temporal correlation but also constructs a sensor encoder with a self-attention mechanism to capture spatial correlation among multivariate time series data. To amplify the reconstruction errors of anomalous points during network training, a two-phase training style with an adversarial training strategy is used to improve the anomaly detection performance of the BSAAD model. Experiments on six multivariate time series datasets show that the BSAAD model outperforms state-of-the-art anomaly detection methods.</p> <p> </p>

The prevalence of networked sensors and actuators in many real-world systems such as smart buildings, factories, power plants, and data centers generate substantial amounts of multivariate time series data for these systems. The rich sensor data can be continuously monitored for intrusion events through anomaly detection. However, conventional threshold-based anomaly detection methods are inadequate due to the dynamic complexities of these systems, while supervised machine learning methods are unable to exploit the large amounts of data due to the lack of labeled data. On the other hand, current unsupervised machine learning approaches have not fully exploited the spatial-temporal correlation and other dependencies amongst the multiple variables (sensors/actuators) in the system for detecting anomalies. In this work, we propose an unsupervised multivariate anomaly detection method based on Generative Adversarial Networks (GANs). Instead of treating each data stream independently, our proposed MAD-GAN framework considers the entire variable set concurrently to capture the latent interactions amongst the variables. We also fully exploit both the generator and discriminator produced by the GAN, using a novel anomaly score called DR-score to detect anomalies by discrimination and reconstruction. We have tested our proposed MAD-GAN using two recent datasets collected from real-world CPS: the Secure Water Treatment (SWaT) and the Water Distribution (WADI) datasets. Our experimental results showed that the proposed MAD-GAN is effective in reporting anomalies caused by various cyber-intrusions compared in these complex real-world systems.

In recent years, anomaly detection techniques in time-series data have been widely used in manufacturing, cybersecurity, and other fields. Meanwhile, various anomaly detection models based on generative adversarial networks (GAN) are gradually used in time-series anomaly detection tasks. However, there are problems of unstable generator training, missed detection of anomalous data, and inconsistency between the discriminator's discriminant and the anomaly detection target in GAN networks. Aiming at the above problems, the paper proposes a DUAL-ADGAN (Dual Anomaly Detection Generative Adversarial Networks) model for the detection of anomalous data in time series. First, the Wasserstein distance satisfying the Lipschitz constraint is used as the loss function of the data reconstruction module, which improves the stability of the traditional GAN network training. Second, by adding a data prediction module to the DUAL-ADGAN model, the distinction between abnormal and normal samples is increased, and the rate of missing abnormal data in the model is reduced. Third, by introducing the Fence-GAN loss function, the discriminator is aligned with the anomaly detection target, which effectively reduces the anomaly data false detection rate of the DUAL-ADGAN model. Finally, anomaly scores derived from the DUAL-ADGAN model are compared with dynamic thresholds to detect anomalies. The experimental results show that the average F1 of the DUAL-ADGAN model is 0.881, which is better than the other nine baseline models. The conclusions demonstrate that the DUAL-ADGAN model proposed in the paper is more stable in training while effectively solving the problems of anomaly miss detection and discriminator inconsistency with the anomaly detection target in the anomaly detection task.

Multivariate time seris anomaly detection (MTSAD) is of great significance in various modern industrial applications and IT systems. Recently, some unsupervised deep models have been developed for MTS-AD. However, these methods often struggle to handle the complex temporal patterns and inevitable noise in multivariate time series data, resulting in limited performance. To overcome these challenges, we propose DeAnomaly, a novel anomaly detection framework based on time series decomposition. Specifically, DeAnomaly employs a two-phase training paradigm, consisting of structural pattern elimination and anomaly detection on remainders. The structural pattern elimination phase learns normal trend and seasonal components through spatial relationship modeling and time-frequency analysis, which are subsequently removed from the original time series to overcome the limitation of complex temporal patterns. The anomaly detection phase utilizes the robust characteristics of noise with denoising diffusion models to identify and distinguish between noise and actual anomalies. Since anomalies and small random fluctuations are mainly retained in the remainders, anomalies will be more clearly exposed. In this way, DeAnomaly can detect anomalies more accurately and robustly. We conduct extensive experiments on four real-world datasets and thirteen baselines, experimental results demonstrate that DeAnomaly outperforms these state-of-the-arts.

Anomaly detection in time series data is a critical task with numerous applications across various domains.This study presents a comprehensive empirical review that explores the methodologies, evaluation metrics, benchmark datasets, novel techniques, and real-world case studies in anomaly detection.The study begins by examining the different methodologies employed in anomaly detection, including statistical-based methods, machine learning-based approaches, and deep learning-based techniques.Evaluation metrics such as precision, Recall, F1-score, ROC curve, and AUC are discussed, along with commonly used benchmark datasets that serve as standards for evaluating anomaly detection algorithms.Novel techniques and algorithms for anomaly detection in time series data are critically analyzed, including time series decomposition and reconstruction methods, transfer learning and domain adaptation techniques, online and streaming anomaly detection approaches, and ensemble methods.These techniques' strengths, limitations, and potential applications are discussed in detail.Real-world case studies showcase the practical applications of anomaly detection in different domains.These case studies include anomaly detection in network traffic data, energy consumption patterns, and medical sensor data.The mathematical approaches and algorithms employed in these case studies are examined to provide insights into the specific methodologies used for anomaly detection.Future research directions and challenges in anomaly detection are discussed, highlighting the importance of explainable AI and interpretable models, incorporating domain knowledge and context awareness, privacy-preserving techniques, and integrating anomaly detection with other data analysis techniques.This empirical review contributes to the field by comprehensively analyzing anomaly detection techniques in time series data.The study's findings offer valuable insights into the strengths and limitations of different methodologies and highlight emerging trends in the field.Furthermore, future research directions and challenges provide a roadmap for advancing anomaly detection in time series data analysis.

One of the oldest known predictive analytics techniques is time series prediction. The target in time series prediction is use historical data about a specific quantity to predicts value of the same quantity in the future. Multivariate time series (MTS) data has been widely used in time series prediction research because it is considered better than univariate time series (UTS) data. However, in reality MTS data sets contain various types of information which makes it difficult to extract information to predict the situation. Therefore, UTS data still has a chance to be developed because it is actually simpler than MTS data. UTS prediction treats forecasts as a single variable problem, whereas MTS may employ a large number of time-concurred series to make predictions. Neural Network (NN) model could be built to predict the target variable given the other (predictor) variables. In this study, we used Particle Swarm Optimization (PSO) algorithm to optimize performance of NN on a UTS dataset. Our proposed model is validated using x-validation and and use RMSE to measure its performance. The experimental results show that NN performance after optimization using PSO produces good results compared to classical NN performance. This is evidenced by the value of RMSE = 0.410 which is the smallest RMSE value produced. The smaller the RMSE value, the better the model performance. It can be concluded that the proposed method can improve NN performance on UTS data.

The rise of multivariate time series (MTS) data has made prediction crucial, with deep learning models dominant yet vulnerable to adversarial attacks. These attacks use small perturbations on inputs to cause mispredictions. MTS data's inherent complexity and sensitivity demand stringent perturbation handling. To address this, we propose TFCA, an adversarial attack method focusing on feature dimension impact. TFCA calculates each feature's gradient contribution to predictions independently, establishing a feature importance ranking. It enhances temporal characteristics by integrating multi-dimensional elements (e.g., volatility, cosine direction) to constrain adversarial sample realism. Guided by this ranking, TFCA targets specific attack ratios on subsets of the time series, ensuring attack effectiveness while reducing perturbation size and improving stealthiness. Experiments on real MTS datasets against models (TCN, LSTNet, CNN) demonstrate TFCA's effectiveness, stealthiness, applicability, and transferability. The integration of post hoc explainable AI algorithms also provides interpretability. This research offers insights for enhancing MTS model robustness and security in practice.

A novel anomaly detection method for multivariate time series based on spatial-temporal graph learning

Multivariate time series anomaly detection is a crucial data mining technique with a wide range of applications in areas such as IT applications. Currently, the majority of anomaly detection methods for time series data rely on unsupervised approaches due to the rarity of anomaly labels. However, in real-world scenarios, obtaining a limited number of anomaly labels is feasible and affordable. Effective usage of these labels can offer valuable insights into the temporal characteristics of anomalies and play a pivotal role in guiding anomaly detection efforts. To improve the performance of multivariate time series anomaly detection, we proposed a novel deep learning model named EDD (Encoder-Decoder-Discriminator) that leverages limited anomaly samples. The EDD model innovatively integrates a graph attention network with long short term memory (LSTM) to extract spatial and temporal features from multivariate time series data. This integrated approach enables the model to capture complex patterns and dependencies within the data. Additionally, the model skillfully maps series data into a latent space, utilizing a carefully crafted loss function to cluster normal data tightly in the latent space while dispersing abnormal data randomly. This innovative design results in distinct probability distributions for normal and abnormal data in the latent space, enabling precise identification of anomalous data. To evaluate the performance of our EDD model, we conducted extensive experimental validation across three diverse datasets. The results demonstrate the significant superiority of our model in multivariate time series anomaly detection. Specifically, the average F1-Score of our model outperformed the second-best method by 2.7% and 73.4% in both evaluation approaches, respectively, highlighting its superior detection capabilities. These findings validate the effectiveness of our proposed EDD model in leveraging limited anomaly samples for accurate and robust anomaly detection in multivariate time series data.

Recently, OT (operational technology) networks of industrial control systems have been combined with IT networks. Therefore, OT networks have inherited the vulnerabilities and attack paths existing in IT networks. Consequently, attacks on industrial control systems are increasing, and research on technologies combined with artificial intelligence for detecting attacks is active. Current research focuses on detecting attacks and improving the detection accuracy. Few studies exist on metrics that interpret anomaly detection results. Different analysis metrics are required depending on the characteristics of the industrial control system data used for anomaly detection and the type of attack they contain. We focused on the fact that industrial control system data are time series data. The accuracy and F1-score are used as metrics for interpreting anomaly detection results. However, these metrics are not suitable for evaluating anomaly detection in time series data. Because it is not possible to accurately determine the start and end of an attack, range-based performance metrics must be used. Therefore, in this study, when evaluating anomaly detection performed on time series data, we propose a range-based performance metric with an improved algorithm. The previously studied range-based performance metric time-series aware precision and recall (TaPR) evaluated all attacks equally. In this study, improved performance metrics were studied by deriving ambiguous instances according to the characteristics of each attack and redefining the algorithm of the TaPR metric. This study provides accurate assessments when performing anomaly detection on time series data and allows predictions to be evaluated based on the characteristics of the attack.

The presence of anomalies in time series, i.e., data points that deviate from the expected behavior, can be a symptom of an incoming failure that may lead to costly repair. Thus, the detection of anomalous data by means of diagnostic methodologies can both reduce maintenance actions and asset unscheduled stops. To tackle this challenge, for the first time in the literature, we exploit the capabilities of Convolutional Neural Networks (CNNs), by feeding them with images obtained from multivariate time series data, transformed by means of two different approaches, namely, Gramian Angular Summation Field (GASF) and Markov Transition Field (MTF). Two CNN architectures are investigated, i.e., VGG-19 and SqueezeNet. The performance of both CNNs fed with images is compared to that of i) a Temporal Convolutional Network (TCN) fed with time series data and ii) a Support Vector Machine (SVM) model. In this paper, we present the comprehensive framework, which starts from time series transformation, goes through CNN development and ends with anomaly detection. The framework is applied to field data taken during normal operation of ten SGT-800 gas turbines, operated in combined cycle, running at base load, and located in two different regions. The normal data covers 150 days of operation. Spike faults are implanted in two out of the twenty available measured variables, i.e., compressor discharge temperature and compressor discharge pressure, by considering nine combinations of maximum fault magnitude and number of implanted spikes in each time series. The analyses carried out in this paper demonstrate that both CNNs fed with images achieve significantly higher classification accuracy than both a TCN model fed with time series data and an SVM model. Moreover, the MTF method always proves more robust than GASF method, and also allows higher accuracy values, in the range from 0.85 to 0.99.

Anomalies in time series can be a symptom of an incoming failure. Thus, the detection of anomalous data can both reduce maintenance actions and asset unscheduled stops. To tackle this challenge, we exploit the capabilities of convolutional neural networks (CNNs), fed with images obtained from multivariate time series data, transformed by means of two different approaches. Two CNN architectures are investigated, i.e., VGG-19 and SqueezeNet. The performance of both CNNs fed with images is compared to that of (i) a temporal convolutional network (TCN) fed with time series data and (ii) a support vector machine (SVM) model. In this paper, we present the comprehensive framework, which starts from time series transformation, goes through CNN development and ends with anomaly detection. The framework is applied to field data taken during normal operation of ten SGT-800 gas turbines, located in two different regions. The normal data covers 150 days of operation. Spike faults are implanted in two out of the 20 available measured variables, i.e., compressor discharge temperature and compressor discharge pressure, by considering different combinations of maximum fault magnitude and number of implanted spikes in each time series. The results demonstrate that both CNNs fed with images achieve significantly higher classification accuracy than both a TCN model fed with time series data and an SVM model. Moreover, the Markov transition field (MTF) method always proves more robust than Gramian angular summation field (GASF) method, and also allows higher accuracy values, in the range from 0.85 to 0.99.

Multivariate networked time series data are ubiquitous in many applications, where multiple variables of interest are sequentially collected over time for each vertex as multivariate time series. Data of different vertices may have heterogeneous influence on each other through the network topology. These time series may usually exhibit multimodal marginal distributions, due to complex system variations. In this article, we propose a novel approach for such data modeling. In particular, we assume that each vertex has multiple latent states and exhibits state-switching behaviors according to a Markov process. The multivariate time series of each vertex depend on a defined latent effect variable influenced by both its own latent state and the latent effects of its neighbors through a heterogeneous network autoregressive model according to the network topology. Furthermore, the influence of some exogenous covariates on the time series can also be incorporated in the model. Some model properties are discussed, and a variational EM algorithm is proposed for model parameter estimation and state inference. Extensive synthetic experiments and a real-world case study demonstrate the effectiveness and applicability of the proposed model.

The need for detection, identification and prediction of anomalies emerged just a short time ago. This task brings multiple challenges that are mainly associated with recording and detection of novel attacks and influences. It draws attention of experts in network security and diagnostics of information and industrial systems. Detection of anomalies in dynamic dataflows largely determines the efficiency of management of computer network information security within information and industrial systems. At the same time, the technology of dynamic dataflow prediction is also very important for building systems intended to detect anomalies in data protection within industrial control systems. Any automated systems are based on available computer capabilities and advances in management theory, mathematical modeling and optimizations methods. Processes that occur in industrial automated systems and that are reflected in dataflows being observed constitute complex multi-component processes, thus making it more complicated to predict such processes. In this case, complex multi-component time series data should be predicted at different (short-term, medium-term and long-term) time scales. A neural network architecture matching the structure of the multi-component time series being predicted should be built for generating a multi-component prediction. It is proposed to decompose the complex multi-component time series into several basic components using the digital signal processing technology, i. e. to perform a preliminary structural analysis of multi-component time series within the observed range of all time series that reflect operation of the industrial control system. Separate predictions with different time horizon are formed for each basic component of the multi-component time series using the available neural network architecture and machine learning taking into account dynamic characteristics of the above components. Anomalies in the observed range of multiple time series that reflect operation of the industrial control system are detected (identified) through component-wise comparison of each component (resulted from the above preliminary digital processing) of any time series within the observed range of all time series, with each prediction of the relevant component of the above time series within the observed range of all time series. This approach that implies component-wise comparison will allow to detect anomalies within the range of observed time series of the industrial control system separately by their different dynamic characteristics, and thus will improve the efficiency of management of information security within information and industrial systems.

Anomalies in time series, also called “discord,” are the abnormal subsequences. The occurrence of anomalies in time series may indicate that some faults or disease will occur soon. Therefore, development of novel computational approaches for anomaly detection (discord search) in time series is of great significance for state monitoring and early warning of real-time system. Previous studies show that many algorithms were successfully developed and were used for anomaly classification, e.g., health monitoring, traffic detection, and intrusion detection. However, the anomaly detection of time series was not well studied. In this paper, we proposed a long short-term memory- (LSTM-) based anomaly detection method (LSTMAD) for discord search from univariate time series data. LSTMAD learns the structural features from normal (nonanomalous) training data and then performs anomaly detection via a statistical strategy based on the prediction error for observed data. In our experimental evaluation using public ECG datasets and real-world datasets, LSTMAD detects anomalies more accurately than other existing approaches in comparison.