Multidimensional time series classification with multiple attention mechanism

In industry, sensor-based monitoring of equipment or environment has become a necessity. Instead of using a single sensor, multi-sensor system is used to fully detect abnormalities in complex scenarios. Recently, physical models, signal processing technology, and various machine learning models have improved the performance. However, these methods either do not consider the potential correlation between features or do not take advantage of the sequential changes of correlation while constructing an anomaly detection model. This paper firstly analyzes the correlation characteristic of a multi-sensor system, which shows a lot of clues to the anomaly/fault propagation. Then, a multi-sensor anomaly detection method, which finds and uses the correlation between features contained in the multidimensional time-series data, is proposed. The method converts the multidimensional time-series data into temporal correlation graphs according to time window. By transforming time-series data into graph structure, the task of anomaly detection is considered as a graph classification problem. Moreover, based on the stability and dynamics of the correlation between features, a structure-sensitive graph neural network is used to establish the anomaly detection model, which is used to discover anomalies from multi-sensor system. Experiments on three real-world industrial multi-sensor systems with anomalies indicate that the method obtained better performance than baseline methods, with the mean value of F1 score reaching more than 0.90 and the mean value of AUC score reaching more than 0.95. That is, the method can effectively detect anomalies of multidimensional time series.

Time series shapelets are small subsequences that maximally differentiate classes of time series. Since the inception of shapelets, researchers have used shapelets for various data domains including anthropology and health care, and in the process suggested many efficient techniques for shapelet discovery. However, multi-dimensional time series data poses unique challenges to shapelet discovery that are yet to be solved. We show that an ensemble of shapelet-based decision trees on individual dimensions works better than shapelets defined over multiple dimensions. Generating a shapelet ensemble for multidimensional time series is computationally expensive. Most of the existing techniques prune shapelet candidates for speed. In this paper, we propose a novel technique for shapelet discovery that evaluates remaining candidates efficiently. Our algorithm uses a multi-length approximate index for time series data to efficiently find the nearest neighbors of the candidate shapelets. We employ a simple skipping technique for additional candidate pruning and a voting based technique to improve accuracy while retaining interpretability. Not only do we find a significant speed increase, our techniques enable us to efficiently discover shapelets on datasets with multi-dimensional and long time series such as hours of brain activity recordings. We demonstrate our approach on a biomedical dataset and find significant differences between patients with schizophrenia and healthy controls.

Optimization of Multi dimensional Time Series Data Anomaly Detection Model Based on Graph Deviation Network and Convolutional Neural Network

Causal relationship mining of multi-dimensional meteorological time series data can reveal the potential connection between visibility and other influencing factors, and it is meaningful in terms of environmental management, air pollution chasing, and haze control. However, because causality analysis based on statistical methods or traditional machine learning techniques cannot represent the complex relationship between visibility and its influencing factors, as an extension of traditional Granger-causality analysis, we propose a causality mining method based on the seq2seq-LSTM deep learning model. The method can profoundly reveal the hidden relationship between different features and visibility and the regular pattern of bad weather, which can provide theoretical support for air pollution control.

GRU-Based Interpretable Multivariate Time Series Anomaly Detection in Industrial Control System

An introduction to the Cyrcadia Breast Monitor: A wearable breast health monitoring device

Inducing sequential patterns from time series data is an important data mining problem. While most of the current methods are generating sequential patterns within a single attribute, this paper proposes a new method, using Hellinger entropy measure, for generating multi-dimensional sequential patterns. A number of theorems are proposed to reduce the computational complexity of the proposed method.

In VAST Challenge 2017, we developed a visual exploration system for detection of sensor anomaly, pattern of chemical distribution and responsible factory for each release of chemical. In this report, we discuss details of our data preprocessing, design, implementation and how we found our answer.

Recently, Body Sensor Networks (BSNs) are being deployed for monitoring and managing medical conditions as well as human performance in sports. These BSNs include various sensors such as accelerometers, gyroscopes, EMG (Electromyogram), EKG (Electro-cardiograms), and other sensors depending on the needs of the medical conditions. Data from these sensors are typically Time Series data and the data from multiple sensors form multiple, multidimensional time series data. This tutorial describes the technologies that go behind BSNs -- both in terms of the hardware infrastructure as well as the basic software. First, we outline the BSN hardware features and the related requirements. We then discuss the energy and communication choices for BSNs. Next, we discuss approaches for classification, data mining, visualization, and securing these data. We also show several demonstrations of body sensor networks as well as the software that aid in analyzing the data.

Dynamic Time Warping (DTW) is a powerful technique in the time-series similarity search. However, its performance on large-scale data is unsatisfactory because of its high computational cost. Although many methods have been proposed to alleviate this, they are mostly indirect methods, i.e., they do not improve the DTW algorithm itself. In this paper, we propose to incorporate the Ordered Graph Search (OGS) and the lower bound for DTW into an improved DTW algorithm and apply it on time series data. Extensive experiments show that the improved DTW algorithm is faster than the original dynamic programming based algorithm on multi-dimensional time series data. It is also especially useful in the post-processing stage of searching in large time series data based on DTW distance.

Identifying abnormalities or anomalies by visual inspection on neurophysiologic signals such as ElectroEncephaloGrams (EEGs), is extremely challenging. We propose a novel Multi-Dimensional Time Series (MDTS) classification technique, called Connectivity Support Vector Machines (C-SVMs) that integrates brain connectivity network with SVMs. To alter noise in EEG data, Independent Component Analysis based on the Unbiased Quasi Newton Method was applied. C-SVM achieved 94.8% accuracy classifying subjects compared to 69.4% accuracy with standard SVMs. It suggests that C-SVM can be a rapid, yet accurate, technique for online differentiation between epileptic and normal subjects. It may solve other classification MDTS problems too.

Accurately predicting the remaining useful life (RUL) of lithium-ion batteries is crucial for ensuring the safe and reliable operation of battery systems. However, due to the highly nonlinear degradation behavior of lithium-ion batteries under complex working conditions, traditional prediction methods often fail to achieve satisfactory accuracy. This study proposes a hybrid prediction model that integrates an attention (ATT) mechanism, a convolutional neural network (CNN), and a bidirectional long short-term memory (BiLSTM) network, termed ATT-CNN-BiLSTM. First, a one-dimensional CNN is employed to extract local degradation patterns and inter-channel correlated features from multi-dimensional sensor time-series data. Then, the BiLSTM network captures long-term dependencies within the degradation process. Furthermore, an attention mechanism is introduced to dynamically adjust the contribution of key time-step features and suppress noise interference. Experimental results based on the NASA battery aging datasets demonstrate that the proposed ATT-CNN-BiLSTM model reduces RMSE and MAE by 49.08% and 54.67%, respectively, compared with the LSTM model, and by 10.89% and 17.85%, respectively, compared with the ACNN-LSTM-MMD model. Ablation experiments also confirm that further confirm that the ATT-CNN-BiLSTM model decreases RMSE and MAE by 26.45% and 33.33%, respectively, compared with CNN-BiLSTM, and by 21.85% and 23.36%, respectively, compared with ATT-BiLSTM.

We have proposed an extraction technique of action rules from time series on human actions as situations obtained by long-term observation. In the proposed method, HMM and IF-Then rule style are used in consideration of readability of obtained action rules. In this paper, the usefulness of the proposed method is examined with two dimensional time series data on human actions.

A novel network with multiple attention mechanisms for aspect-level sentiment analysis

In the current conditions of digitalization, structural transformations of the global labor market, and increased requirements for the adaptability of educational systems, there is a need to revise traditional approaches to regulating the training processes within university education. The article is devoted to the substantiation of the fractal approach as an effective method of strategic management of the educational and economic ecosystem of a digital university. A scientific model is proposed that considers the recursive, self-similar, multilevel, and dynamic nature of the digital educational environment and allows for ensuring the adaptability of educational structures to the challenges of the digital economy. The methodological foundations for the transformation of university education in the digital age are revealed, in particular in terms of building integrated quality indicators based on normalized indicators and principal components. The necessity of using multidimensional data, indices, expert opinions, ratings, and time series to analyze and predict the efficiency of digital university subsystems is substantiated. Mathematical formalizations are proposed that allow for the evaluation of fractal structures of the educational system at different hierarchical levels and the derivation of generalized integral performance indicators. Particular attention is paid to the use of artificial intelligence technologies, precedent modeling, and knowledge-based approaches in management decision-making. The article proves that a digital university is not only an educational organization, but also a complex socio-technical fractal system capable of self-organization, self-development, and flexible functioning under conditions of uncertainty. It is noted that the fractal approach allows for aligning strategic development goals with tactical and operational management actions at different levels of management. The approaches to modeling the educational and economic ecosystem of a digital university proposed by the author open up new prospects for forecasting the development of intellectual capital, optimizing management processes, and ensuring the sustainability of the educational system in the digital environment.