A Robust Approach for Multivariate Time Series Forecasting

A multi-granularity hierarchical network for long- and short-term forecasting on multivariate time series data

Time series forecasting has been regarded as a key research problem in various fields. such as financial forecasting, traffic flow forecasting, medical monitoring, intrusion detection, anomaly detection, and air quality forecasting etc. In this paper, we propose a sequence-to-sequence deep learning framework for multivariate time series forecasting, which addresses the dynamic, spatial-temporal and nonlinear characteristics of multivariate time series data by LSTM based encoder-decoder architecture. Through the air quality multivariate time series forecasting experiments, we show that the proposed model has better forecasting performance than classic shallow learning and baseline deep learning models. And the predicted PM2.5 value can be well matched with the ground truth value under single timestep and multi-timestep forward forecasting conditions. The experiment results show that our model is capable of dealing with multivariate time series forecasting with satisfied accuracy.

The efficiency of business processes relies on business key performance indicators (Biz-KPIs), that can be negatively impacted by IT failures. Business and IT Observability (BizITObs) data fuses both Biz-KPIs and IT event channels together as multivariate time series data. Forecasting Biz-KPIs in advance can enhance efficiency and revenue through proactive corrective measures. However, BizITObs data generally exhibit both useful and noisy inter-channel interactions between Biz-KPIs and IT events that need to be effectively decoupled. This leads to suboptimal forecasting performance when existing multivariate forecasting models are employed. To address this, we introduce AutoMixer, a time-series Foundation Model (FM) approach, grounded on the novel technique of channel-compressed pretrain and finetune workflows. AutoMixer leverages an AutoEncoder for channel-compressed pretraining and integrates it with the advanced TSMixer model for multivariate time series forecasting. This fusion greatly enhances the potency of TSMixer for accurate forecasts and also generalizes well across several downstream tasks. Through detailed experiments and dashboard analytics, we show AutoMixer's capability to consistently improve the Biz-KPI's forecasting accuracy (by 11-15%) which directly translates to actionable business insights.

A dual-topological graph memory network for anti-noise multivariate time series forecasting

The presence of anomalies in time series, i.e., data points that deviate from the expected behavior, can be a symptom of an incoming failure that may lead to costly repair. Thus, the detection of anomalous data by means of diagnostic methodologies can both reduce maintenance actions and asset unscheduled stops. To tackle this challenge, for the first time in the literature, we exploit the capabilities of Convolutional Neural Networks (CNNs), by feeding them with images obtained from multivariate time series data, transformed by means of two different approaches, namely, Gramian Angular Summation Field (GASF) and Markov Transition Field (MTF). Two CNN architectures are investigated, i.e., VGG-19 and SqueezeNet. The performance of both CNNs fed with images is compared to that of i) a Temporal Convolutional Network (TCN) fed with time series data and ii) a Support Vector Machine (SVM) model. In this paper, we present the comprehensive framework, which starts from time series transformation, goes through CNN development and ends with anomaly detection. The framework is applied to field data taken during normal operation of ten SGT-800 gas turbines, operated in combined cycle, running at base load, and located in two different regions. The normal data covers 150 days of operation. Spike faults are implanted in two out of the twenty available measured variables, i.e., compressor discharge temperature and compressor discharge pressure, by considering nine combinations of maximum fault magnitude and number of implanted spikes in each time series. The analyses carried out in this paper demonstrate that both CNNs fed with images achieve significantly higher classification accuracy than both a TCN model fed with time series data and an SVM model. Moreover, the MTF method always proves more robust than GASF method, and also allows higher accuracy values, in the range from 0.85 to 0.99.

Anomalies in time series can be a symptom of an incoming failure. Thus, the detection of anomalous data can both reduce maintenance actions and asset unscheduled stops. To tackle this challenge, we exploit the capabilities of convolutional neural networks (CNNs), fed with images obtained from multivariate time series data, transformed by means of two different approaches. Two CNN architectures are investigated, i.e., VGG-19 and SqueezeNet. The performance of both CNNs fed with images is compared to that of (i) a temporal convolutional network (TCN) fed with time series data and (ii) a support vector machine (SVM) model. In this paper, we present the comprehensive framework, which starts from time series transformation, goes through CNN development and ends with anomaly detection. The framework is applied to field data taken during normal operation of ten SGT-800 gas turbines, located in two different regions. The normal data covers 150 days of operation. Spike faults are implanted in two out of the 20 available measured variables, i.e., compressor discharge temperature and compressor discharge pressure, by considering different combinations of maximum fault magnitude and number of implanted spikes in each time series. The results demonstrate that both CNNs fed with images achieve significantly higher classification accuracy than both a TCN model fed with time series data and an SVM model. Moreover, the Markov transition field (MTF) method always proves more robust than Gramian angular summation field (GASF) method, and also allows higher accuracy values, in the range from 0.85 to 0.99.

Multivariate time series data are becoming increasingly common in numerous real world applications, e.g., power plant monitoring, health care, wearable devices, automobile, etc. As a result, multivariate time series retrieval, i.e., given the current multivariate time series segment, how to obtain its relevant time series segments in the historical data (or in the database), attracts significant amount of interest in many fields. Building such a system, however, is challenging since it requires a compact representation of the raw time series which can explicitly encode the temporal dynamics as well as the correlations (interactions) between different pairs of time series (sensors). Furthermore, it requires query efficiency and expects a returned ranking list with high precision on the top. Despite the fact that various approaches have been developed, few of them can jointly resolve these two challenges. To cope with this issue, in this paper we propose a Deep r-th root of Rank Supervised Joint Binary Embedding (Deep r-RSJBE) to perform multivariate time series retrieval. Given a raw multivariate time series segment, we employ Long Short-Term Memory (LSTM) units to encode the temporal dynamics and utilize Convolutional Neural Networks (CNNs) to encode the correlations (interactions) between different pairs of time series (sensors). Subsequently, a joint binary embedding is pursued to incorporate both the temporal dynamics and the correlations. Finally, we develop a novel r-th root ranking loss to optimize the precision at the top of a Hamming distance ranking list. Thoroughly empirical studies based upon three publicly available time series datasets demonstrate the effectiveness and the efficiency of Deep r-RSJBE.

Multivariate time series forecasting has very great practical significance for a long time, and it has been attracting the attention of researchers from a diverse range of fields. However, it is difficult to analyze the relationship and transformation law among multivariate data. Further, it is hard to obtain a relatively accurate prediction. In recent years, long short-term memory (LSTM) has shown high capability in dealing with nonlinearity and long memory of time series data. Although LSTM can also process multivariate data, it is insufficient to pay various degrees of attention to multivariate data. To address this issue, a multivariate time series prediction model based on multilayer perceptron (MLP), feedforward attention mechanism, and LSTM is proposed in this paper. Firstly, the simulation process utilizes the MLP module to map the multivariate initial sequences into another latent dimensional space, thereby obtaining easily captured mapping features. Then, these features are adaptively assigned attention weights through the feed-forward attention mechanism. Finally, the LSTM module uses these feature sequences with attention weights to make final predictions. The experimental results show that the method that combines the MLP layer with the feed-forward attention layer is effective in extracting multivariate features. Also, the empirical results indicate that our proposed framework (a combined model of MLP-Feedforward attention-LSTM) can achieve better performance than baselines.

Multivariate networked time series data are ubiquitous in many applications, where multiple variables of interest are sequentially collected over time for each vertex as multivariate time series. Data of different vertices may have heterogeneous influence on each other through the network topology. These time series may usually exhibit multimodal marginal distributions, due to complex system variations. In this article, we propose a novel approach for such data modeling. In particular, we assume that each vertex has multiple latent states and exhibits state-switching behaviors according to a Markov process. The multivariate time series of each vertex depend on a defined latent effect variable influenced by both its own latent state and the latent effects of its neighbors through a heterogeneous network autoregressive model according to the network topology. Furthermore, the influence of some exogenous covariates on the time series can also be incorporated in the model. Some model properties are discussed, and a variational EM algorithm is proposed for model parameter estimation and state inference. Extensive synthetic experiments and a real-world case study demonstrate the effectiveness and applicability of the proposed model.

To improve the revenue of dairy farms, cow estrus must be accurately monitored to track mating time. Narrow Band Internet of Things (NB-IoT) is considered as a promising technology to realize cost-effective detection system attributing to its wide coverage and low power consumption. To increase the success rate of real-time detection, machine learning based algorithms have been applied to extract patterns from estrus data. However, due to the lack of multivariate time series data, most previous studies do not consider using the time correlation to guide estrus detection. In this paper, we present a NB-IoT based solution framework where multivariate behavioral time series data collected by neck-mounted sensors and then uploaded to a cloud data center for further analysis through NB-IoT network. Based on the collected data, we propose an estrus prediction algorithm which gives estrus alert by exploiting Long-Short Term Memory (LSTM) and Convolution Neural Network (CNN). Through numerical studies conducted using real data set from the pasture, we show that our proposed solution outperforms exiting detection algorithms in terms of accuracy and efficiency.

In order to reduce unnecessary data transmissions from Internet of Things (IoT) sensors, this article proposes a multivariate-time-series-prediction-based adaptive data transmission period control (PBATPC) algorithm for IoT networks. Based on the spatio-temporal correlation between multivariate time-series data, we developed a novel multivariate time-series data encoding scheme utilizing the proposed time-series distance measure <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textit {ADMWD}$ </tex-math></inline-formula> . Composed of two significant factors for a multivariate time-series prediction, i.e., the absolute deviation from the mean (ADM) and the weighted differential (WD) distance, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textit {ADMWD}$ </tex-math></inline-formula> considers both the time distance from a prediction point and a negative correlation between the time-series data concurrently. Utilizing the convolutional neural network (CNN) model, a subset of IoT sensor readings can be predicted from encoded multivariate time-series measurements, and we compared the predicted sensor values with actual readings to obtain the adaptive data transmission period. Extensive performance evaluations show a substantial performance gain of the proposed algorithm in terms of the average power reduction ratio (approximately 12%) and average data reconstruction error (approximately 8.32% MAPE). Finally, this article also provides a practical implementation of the proposed PBATPC algorithm via the HTTP protocol under the IEEE 802.11-based WLAN network.

Multivariate time series anomaly detection is a crucial data mining technique with a wide range of applications in areas such as IT applications. Currently, the majority of anomaly detection methods for time series data rely on unsupervised approaches due to the rarity of anomaly labels. However, in real-world scenarios, obtaining a limited number of anomaly labels is feasible and affordable. Effective usage of these labels can offer valuable insights into the temporal characteristics of anomalies and play a pivotal role in guiding anomaly detection efforts. To improve the performance of multivariate time series anomaly detection, we proposed a novel deep learning model named EDD (Encoder-Decoder-Discriminator) that leverages limited anomaly samples. The EDD model innovatively integrates a graph attention network with long short term memory (LSTM) to extract spatial and temporal features from multivariate time series data. This integrated approach enables the model to capture complex patterns and dependencies within the data. Additionally, the model skillfully maps series data into a latent space, utilizing a carefully crafted loss function to cluster normal data tightly in the latent space while dispersing abnormal data randomly. This innovative design results in distinct probability distributions for normal and abnormal data in the latent space, enabling precise identification of anomalous data. To evaluate the performance of our EDD model, we conducted extensive experimental validation across three diverse datasets. The results demonstrate the significant superiority of our model in multivariate time series anomaly detection. Specifically, the average F1-Score of our model outperformed the second-best method by 2.7% and 73.4% in both evaluation approaches, respectively, highlighting its superior detection capabilities. These findings validate the effectiveness of our proposed EDD model in leveraging limited anomaly samples for accurate and robust anomaly detection in multivariate time series data.

The extraction of valuable compounds from moringa plants involves complex processes that are highly dependent on various environmental and operational factors. Monitoring these processes using Internet of Things (IoT)-based multivariate time series data presents a unique opportunity for improving efficiency and quality control. Multivariate time series data, characterized by multiple variables recorded over time, provides valuable insights into the behavior, interactions, and dependencies among different components within a system. However, with the increasing complexity and volume of IoT data generated during moringa extraction, the anomaly detection becomes challenging. The objective of this study is to develop a robust and efficient system capable of automatically detecting anomalous patterns in real time, providing early warning signals to operators, and facilitating timely interventions. This paper proposes a novel hybrid unsupervised anomaly detection model combining density-based spatial clustering of applications with noise and k-nearest neighbors for IoT-based multivariate time series data. We conducted extensive experiments on real-world moringa extraction, demonstrating the effectiveness and practicality of our proposed approach. In comparison to other anomaly detection methods, our proposed method has the highest precision value of 0.89, the highest recall value of 0.89, and the highest accuracy value of 0.87. Future research will measure and optimize actuators (relays and motors) from anomaly detection to action. It can also be used with forecasting algorithms to detect anomalies in the coming minutes.

&lt;p&gt;Anomaly detection with multivariate time series data collected by multi-sensors is a challenging problem due to the complexity and high dimension of data and the difficulty of manually labelling data. This paper proposes a novel unsupervised anomaly detection model using spatial-temporal self-attention based on transformer architecture, denoted the biself-attention anomaly detection (BSAAD) model. The BSAAD model not only utilizes a time-step encoder with a self-attention mechanism to capture temporal correlation but also constructs a sensor encoder with a self-attention mechanism to capture spatial correlation among multivariate time series data. To amplify the reconstruction errors of anomalous points during network training, a two-phase training style with an adversarial training strategy is used to improve the anomaly detection performance of the BSAAD model. Experiments on six multivariate time series datasets show that the BSAAD model outperforms state-of-the-art anomaly detection methods.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

Various analytical techniques are available for modeling multivariate time series data. One such approach is the State Space Model, which can be employed to model this type of data. In this study, the data to be analyzed are data on the Indonesia Rupiah (IDR) exchange rate (ExR) against the US Dollar (USD), oil and gas exports (OGE), money supply (MS) and non-oil and gas exports (non-OGE) from January 2008 to December 2019. The aim of this study is to identify the most suitable state space model for the given data. In this research, the state space method will be applied to multivariate time series data, with the state space represented in the Vector Autoregressive (VAR) model to explore the interrelationships among groups of observed variables. The VAR model is a statistical technique used to analyze the relationships between variables in the dataset, employing the Granger Causality Test. The state space model is utilized to model and forecast multiple interconnected time series, where the variables exhibit dynamic interactions and to examine additional unobserved variables in the time series data. Based on the analysis results and the minimum value of the Akaike Information Criterion (AIC), the optimal VAR model identified is the VAR (6) model. The results of forecasting values using the state space model show that the predicted values and the real values for the state space model are very closed to each other.