Inducing Sequential Patterns from Multidimensional Time Series Data

The classification of multidimensional time series holds significant importance across various domains, including action classification, medical diagnosis, and credit assessment. Within multidimensional time series data, features pertinent to classification exhibit variance in their positional distribution along the entirety of the sequence. Moreover, the relative significance of features across distinct dimensions also fluctuates, contributing to suboptimal performance in multidimensional time series classification. Consequently, the proposition of tailored deep learning models for feature extraction specific to multidimensional time series data becomes imperative. This paper introduces attention mechanisms applied to the temporal dimension, graph attention mechanisms for inter-dimensional relationships within multidimensional data, and attention mechanisms applied between channels post-convolutional calculations. These mechanisms are deployed for feature extraction across temporal, variational, and channel dimensions of multidimensional time series data, respectively. Furthermore, attention is directed towards inter-channel interactions within the squeeze-and-excitation network to enhance the model’s representational capacity. Experimental findings substantiate the viability of integrating attention mechanisms into multidimensional time series classification endeavors.

Sequential pattern mining is applicable in a wide range of applications since many types of data sets are in a time related format. Besides mining sequential patterns in a single dimension, mining multidimensional sequential patterns can give us more informative and useful patterns. Due to the huge increase in data volume and also quite large search space, efficient solutions for finding patterns in multidimensional sequence data are nowadays very important. For this reason, developing a parallel algorithm is necessary. In this paper, we present a multidimensional sequence model and a parallel algorithm follows the level-wise approach and all participating processors or workers generate candidate sequence and count their supports independently. Simulation experiments show good load balancing and scalable and acceptable speedup over different processors and problem sizes. dense data sets such as telecommunications, where there are many and long frequent patterns, performance of these algorithms degrades incredibly. On the one side, parallel and distributed computing is expected to relieve current mining methods from the sequential bottleneck, providing the ability to scale to massive datasets, and improving the response time. Achieving good performance on today's multiprocessor systems is a non-trivial task. The main challenges include synchronization and communication minimization, work-load balancing, find good data layout and data decomposition, and disk I/O minimization, which is especially important for data mining (4). On the other side, the basic single dimension can not satisfy the requirement of multi-attribute analysis, which is often the case in actual system practice. To address this problem, multidimensional sequence pattern mining is developed. The most time consuming operation in the discovery process of sequential patterns is the computation of the frequency of the occurrences of interesting subsequences in the sequence database. However, the number of sequential patterns grows exponentially and the task of finding all sequential patterns requires a lot of computational resources, which make it an ideal candidate for parallel processing. However, sequential pattern mining suffers from a scalability problem in both memory use and computational time when a dataset size is large. To perform mining on large datasets, we propose to parallelize this task. We demonstrate that our parallel algorithm can effectively alleviate the scalability problem. In this paper, we present a model for multidimensional sequence pattern and then propose a parallel algorithm for mining sequential patterns from multidimensional sequence data. The algorithm is based on the data parallelism strategy of pattern growth. The experimental results show that our parallel algorithms usually achieve good load balancing and scalability.

Dynamic Time Warping (DTW) is a powerful technique in the time-series similarity search. However, its performance on large-scale data is unsatisfactory because of its high computational cost. Although many methods have been proposed to alleviate this, they are mostly indirect methods, i.e., they do not improve the DTW algorithm itself. In this paper, we propose to incorporate the Ordered Graph Search (OGS) and the lower bound for DTW into an improved DTW algorithm and apply it on time series data. Extensive experiments show that the improved DTW algorithm is faster than the original dynamic programming based algorithm on multi-dimensional time series data. It is also especially useful in the post-processing stage of searching in large time series data based on DTW distance.

In industry, sensor-based monitoring of equipment or environment has become a necessity. Instead of using a single sensor, multi-sensor system is used to fully detect abnormalities in complex scenarios. Recently, physical models, signal processing technology, and various machine learning models have improved the performance. However, these methods either do not consider the potential correlation between features or do not take advantage of the sequential changes of correlation while constructing an anomaly detection model. This paper firstly analyzes the correlation characteristic of a multi-sensor system, which shows a lot of clues to the anomaly/fault propagation. Then, a multi-sensor anomaly detection method, which finds and uses the correlation between features contained in the multidimensional time-series data, is proposed. The method converts the multidimensional time-series data into temporal correlation graphs according to time window. By transforming time-series data into graph structure, the task of anomaly detection is considered as a graph classification problem. Moreover, based on the stability and dynamics of the correlation between features, a structure-sensitive graph neural network is used to establish the anomaly detection model, which is used to discover anomalies from multi-sensor system. Experiments on three real-world industrial multi-sensor systems with anomalies indicate that the method obtained better performance than baseline methods, with the mean value of F1 score reaching more than 0.90 and the mean value of AUC score reaching more than 0.95. That is, the method can effectively detect anomalies of multidimensional time series.

Recently, Body Sensor Networks (BSNs) are being deployed for monitoring and managing medical conditions as well as human performance in sports. These BSNs include various sensors such as accelerometers, gyroscopes, EMG (Electromyogram), EKG (Electro-cardiograms), and other sensors depending on the needs of the medical conditions. Data from these sensors are typically Time Series data and the data from multiple sensors form multiple, multidimensional time series data. This tutorial describes the technologies that go behind BSNs -- both in terms of the hardware infrastructure as well as the basic software. First, we outline the BSN hardware features and the related requirements. We then discuss the energy and communication choices for BSNs. Next, we discuss approaches for classification, data mining, visualization, and securing these data. We also show several demonstrations of body sensor networks as well as the software that aid in analyzing the data.

We present a new methodology for sequential classification, which employs sequential pattern generation and classification, in a two-stage process. In the first phase, a set of sequential patterns are generated from multi-dimensional sequence data. We proposes a novel method for inducing multi-dimensional sequential patterns with the use of Hellinger measure. The importance of each sequential pattern is also calculated. In the second phase, the generated sequential patterns are used for classifying multi-dimensional sequence data. A number of theorems are proposed to reduce the computational complexity of generating sequential patterns. The proposed method is tested on some synthesized sequence databases.

Optimization of Multi dimensional Time Series Data Anomaly Detection Model Based on Graph Deviation Network and Convolutional Neural Network

In VAST Challenge 2017, we developed a visual exploration system for detection of sensor anomaly, pattern of chemical distribution and responsible factory for each release of chemical. In this report, we discuss details of our data preprocessing, design, implementation and how we found our answer.

Time series shapelets are small subsequences that maximally differentiate classes of time series. Since the inception of shapelets, researchers have used shapelets for various data domains including anthropology and health care, and in the process suggested many efficient techniques for shapelet discovery. However, multi-dimensional time series data poses unique challenges to shapelet discovery that are yet to be solved. We show that an ensemble of shapelet-based decision trees on individual dimensions works better than shapelets defined over multiple dimensions. Generating a shapelet ensemble for multidimensional time series is computationally expensive. Most of the existing techniques prune shapelet candidates for speed. In this paper, we propose a novel technique for shapelet discovery that evaluates remaining candidates efficiently. Our algorithm uses a multi-length approximate index for time series data to efficiently find the nearest neighbors of the candidate shapelets. We employ a simple skipping technique for additional candidate pruning and a voting based technique to improve accuracy while retaining interpretability. Not only do we find a significant speed increase, our techniques enable us to efficiently discover shapelets on datasets with multi-dimensional and long time series such as hours of brain activity recordings. We demonstrate our approach on a biomedical dataset and find significant differences between patients with schizophrenia and healthy controls.

Causal relationship mining of multi-dimensional meteorological time series data can reveal the potential connection between visibility and other influencing factors, and it is meaningful in terms of environmental management, air pollution chasing, and haze control. However, because causality analysis based on statistical methods or traditional machine learning techniques cannot represent the complex relationship between visibility and its influencing factors, as an extension of traditional Granger-causality analysis, we propose a causality mining method based on the seq2seq-LSTM deep learning model. The method can profoundly reveal the hidden relationship between different features and visibility and the regular pattern of bad weather, which can provide theoretical support for air pollution control.

Mining frequent sequential patterns from sequence databases has been a central research topic in data mining and various efficient mining sequential patterns algorithms have been proposed and studied. Recently, a novel sequential pattern mining research, called mining repetitive gapped subsequences, has attracted the attention of many researchers. However, the number of repetitive gapped subsequences generated by even these closed mining algorithms may be too large to understand for users, especially when support threshold is low. In this paper, we propose the problem of how to compress repetitive gapped sequential patterns. A novel distance measure of repetitive gapped sequential patterns and an efficient representative pattern checking scheme, δ-dominate sequential pattern checking are proposed. We also develop an efficient algorithm, CRGSgrow ( Compressing Repetitive Gapped Sequential pattern grow), including an efficient pruning strategy, SyncScan. An empirical study with both real and synthetic data sets clearly shows that the CRGSgrow has good performance.Keywordsrepetitive gapped sequential patterncompressing frequent patterns

GRU-Based Interpretable Multivariate Time Series Anomaly Detection in Industrial Control System

An introduction to the Cyrcadia Breast Monitor: A wearable breast health monitoring device

We have proposed an extraction technique of action rules from time series on human actions as situations obtained by long-term observation. In the proposed method, HMM and IF-Then rule style are used in consideration of readability of obtained action rules. In this paper, the usefulness of the proposed method is examined with two dimensional time series data on human actions.

In data mining field, sequential pattern mining can be applied in divers applications such as basket analysis, web access patterns analysis, and quality control in manufactory engineering, etc. Many methods have been proposed for mining sequential patterns. However, conventional methods only consider the occurrences of itemsets in customer sequences. The sequential patterns discovered by these methods are called as positive sequential patterns, i.e., such sequential patterns only represent the occurrences of itemsets. In practice, the absence of a frequent itemset in a sequence may imply significant information. We call a sequential pattern as negative sequential pattern, which also represents the absence of itemsets in a sequence. The two major difficulties in mining sequential patterns, especially negative ones, are that there may be huge number of candidates generated, and most of them are meaningless. In this paper, we proposed a method for mining strong positive and negative sequential patterns, called PNSPM. In our method, the absences of itemsets are also considered. Besides, only sequences with high degree of interestingness will be selected as strong sequential patterns. An example was taken to illustrate the process of PNSPM. The result showed that PNSPM could prune a lot of redundant candidates, and could extract meaningful sequential patterns from a large number of frequent sequences.