A Novel Channel and Temporal-Wise Attention in Convolutional Networks for Multivariate Time Series Classification

The multivariate time series (MTS) classification is an important classification problem in which data has the temporal attribute. Because relationships between many variables of the MTS are complex and time-varying, existing methods perform not well in MTS classification with many attribute variables. Thus, in this paper, we propose a novel model-based classification method, called Kullback-Leibler Divergence-based Gaussian Model Classification (KLD-GMC), which converts the original MTS data into two important parameters of the multivariate Gaussian model: the mean vector and the inverse covariance matrix. The inverse covariance is the most important parameter, which can obtain the information between the variables. So that the more variables, the more information could be obtained by the inverse covariance, KLD-GMC can deal with the relationship between variables well in the MTS. Then the sparse inverse covariance of each subsequence is solved by Graphical Lasso. Furthermore, the Kullback-Leibler divergence is used as the similarity measurement to implement the classification of unlabeled subsequences, because it can effectively measure the similarity between different distributions. Experimental results on classical MTS datasets demonstrate that our method can improve the performance of multivariate time series classification and outperform the state-of-the-art methods.

ST-Tree with interpretability for multivariate time series classification

Early classification of time series has been receiving a lot of attention as of late, particularly in the context of gene expression. In the biomédical realm, early classification can be of tremendous help, by identifying the onset of a disease before it has time to fully take hold, or determining that a treatment has done its job and can be discontinued. In this paper we present a state-of-the-art model, which we call the Early Classification Model (ECM), that allows for early, accurate, and patient-specific classification of multivariate time series. The model is comprised of an integration of the widely-used HMM and SVM models, which, while not a new technique per se, has not been used for early classification of multivariate time series classification until now. It attained very promising results on the datasets we tested it on: in our experiments based on a published dataset of response to drug therapy in Multiple Sclerosis patients, ECM used only an average of 40% of a time series and was able to outperform some of the baseline models, which needed the full time series for classification.

Time series classification algorithms have been mainly dominated by non-deep learning models. Deep learning for Multivariate Time Series Classification (MTSC) has gained huge interest in recent years. Most state of the art deep learning methods are convolutional-based where 1-dimensional (1D) convolutions are used to extract features from the 2-dimensional time series. This study shows that factorization of 1D convolution filters into disjoint temporal and spatial components yields significant accuracy improvements with almost no additional computational cost. Based on our study on disjoint temporal-spatial filters, we have designed a novel filter block called "1+1D", which emphasizes the interaction between dimensions to improve the model performance of the convolution based on deep learning MTSC models. We also proposed a new and effective MTSC method called Disjoint-CNN using our proposed 1+1D filter blocks and through our extensive experiments show that our model (called Disjoint-CNN) outperforms the state-of-the-art MTSC models on 26 datasets in the UEA Multivariate time series archive, achieving the highest average rank among 9 MTSC benchmark models.

Multivariate time series classification (MTSC) is a critical task in fields such as human activity recognition, medical diagnosis, and industrial process monitoring. Its core problem lies in effectively capturing the complex nonlinear dynamics within and between multidimensional variables. Reservoir computing (RC), as an efficient feature extraction model, offers advantages including low computational resource requirements and fast training speeds. However, its performance remains highly dependent on hyperparameter tuning, and standard models may fail to fully leverage the global statistical properties of sequences. In order to address these problems, we propose HERA (Hybrid Euler Reservoir Architecture), a novel classification framework specifically designed for MTSC. At the core of HERA lies an innovative hybrid feature design that synergistically integrates two complementary types of information: (1) Dynamic representations of inter-variable interactions captured by the Euler State Network (EuSN); (2) Static statistical features summarizing the global characteristics of each independent variable. Additionally, the architecture embeds a self-optimization module driven by the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to efficiently determine optimal model configurations. Extensive experiments across multiple public MTSC benchmark datasets demonstrate that HERA achieves highly competitive classification accuracy, performing on par with or exceeding various strong baseline models.

PSTFormer: A novel parallel spatial-temporal transformer for remaining useful life prediction of aeroengine

Classification of time series is an essential requirement of various applications that demand continuous monitoring of dynamical systems such as industrial process and health care monitoring. Feature extraction plays a vital role in deciding performance of the time series models. In recent years deep learning techniques have shown an excellent performance to extract highly discriminating features for the classification of the time series. A Convolutional Neural Network (CNN) is a unified framework that performs the feature learning and classification tasks simultaneously. Using the CNN to perform the classification of the multivariate time series is still a challenging task. In this paper, we propose an attention-based multivariate convolutional neural network (AT-MVCNN) that consists of the attention feature-based input tensor scheme to encode informations across the multiple time stamps. The method is capable of learning the temporal characteristics of the multivariate time series. The efficacy of the proposed method is tested on Human Activity Recognition (HAR) and Occupancy Detection datasets. The experiments and results show the proposed method outperforms the other deep learning and traditional machine learning models.

Multivariate Time Series Classification (MTSC) is a challenging task in real-world applications. Current approaches emphasize modeling multiscale relationships over time. However, the Multivariate Time Series (MTS) also exhibits multiscale cross-channel relationships. Furthermore, the long-term temporal relationships in time series are difficult to capture. In this paper, we introduce MD-Former, a Multiscale Dual-Branch Attention network leveraging the Transformer architecture to capture multiscale relationships across time and channels for MTSC. In MD-Former, MTS is embedded into 2D vectors using Channel-Patching (CP) to retain channel information. Following this, we develop two branches: the Interlaced Attention Branch (IAB) and the Channel-Independent Attention Branch (CIAB). The IAB facilitates the fusion of information across channels and time, while the CIAB prevents the loss of information resulting from excessive fusion. Both the IAB and CIAB consist of multiple layers, each representing a distinct time scale. Finally, we utilize features from each layer of both IAB and CIAB as inputs to the Multiscale Classification Head (MCH) for feature fusion and classification. Experimental results show that MD-Former achieves performance levels that are comparable to SOTA methods in MTSC.

Accurate segmentation is a basic and crucial step for medical image processing and analysis. In the last few years, U-Net, and its variants, have become widely adopted models in medical image segmentation tasks. However, the multiple training parameters of these models determines high computation complexity, which is impractical for further applications. In this paper, by introducing depthwise separable convolution and attention mechanism into U-shaped architecture, we propose a novel lightweight neural network (DSCA-Net) for medical image segmentation. Three attention modules are created to improve its segmentation performance. Firstly, Pooling Attention (PA) module is utilized to reduce the loss of consecutive down-sampling operations. Secondly, for capturing critical context information, based on attention mechanism and convolution operation, we propose Context Attention (CA) module instead of concatenation operations. Finally, Multiscale Edge Attention (MEA) module is used to emphasize multi-level representative scale edge features for final prediction. The number of parameters in our network is 2.2 M, which is 71.6% less than U-Net. Experiment results across four public datasets show the potential and the dice coefficients are improved by 5.49% for ISIC 2018, 4.28% for thyroid, 1.61% for lung and 9.31% for nuclei compared with U-Net.

This work presents a novel approach to multivariate time series classification. The method exploits the multivariate structure of the time series and the possibilities of the stacking ensemble method. The basics of the method may be described in three steps: first, decomposing the multivariate time series on its constituent univariate time series; second, inducing a classifier for each univariate time series plus and additional multivariate classifier for the whole time series; third, creating the final multivariate time series classifier stacking the previous classifiers. The ensemble obtained has the potential to improve the accuracy of the single multivariate time series classifier. Several configurations of the stacking method have been tested on seven multivariate time series data sets. In five out of seven data sets, the proposed method obtains the smallest error rate. Moreover, in two out of seven data sets, stacking only the univariate time series classifiers provides the best results. The experimental results show that when a multivariate time series method does not produce an accurate classifier, stacking it with univariate time series classifiers is an alternative worthy of consideration.

Fine-grained image classification is one of the key basic tasks of computer vision. The appearance of traditional deep convolutional neural network (DCNN) combined with attention mechanism can focus on partial and local features of fine-grained images, but it still lacks the consideration of the embedding mode of different attention modules in the network, leading to the unsatisfactory result of classification model. To solve the above problems, three different attention mechanisms are introduced into the DCNN network (like ResNet, VGGNet, etc.), including SE, CBAM and ECA modules, so that DCNN could better focus on the key local features of salient regions in the image. At the same time, we adopt three different embedding modes of attention modules, including serial, residual and parallel modes, to further improve the performance of the classification model. The experimental results show that the three attention modules combined with three different embedding modes can improve the performance of DCNN network effectively. Moreover, compared with SE and ECA, CBAM has stronger feature extraction capability. Among them, the parallelly embedded CBAM can make the local information paid attention to by DCNN richer and more accurate, and bring the optimal effect for DCNN, which is 1.98% and 1.57% higher than that of original VGG16 and Resnet34 in CUB-200-2011 dataset, respectively. The visualization analysis also indicates that the attention modules can be easily embedded into DCNN networks, especially in the parallel mode, with stronger generality and universality.

Traffic flow forecasting is the essential part of intelligent transportation sSystem (ITS), which can fully protect traffic safety and improve traffic system management capability. Nevertheless, it is still a challenging problem, which is influenced by many complex factors, including regional distribution and external factors (e.g., holidays and weather). To combine various factors to forecast traffic flow, we presented a novel neural network structure called ConvLSTM-based Spatiotemporal Attention Network (CLSTAN). Specifically, our proposed model is composed of four modules: a preliminary feature extraction module, a spatial attention module, a temporal attention module, and an information fusion module. The spatiotemporal attention module can efficiently learn the complex spatiotemporal patterns of traffic flow through the attention mechanism. The spatial attention module uses a series of initial traffic flow maps as input and obtains the weights of the various regions through a ConvLSTM. The temporal attention module uses the spatially weighted traffic flow map as input and acquires the complex spatiotemporal patterns of traffic flow by a ConvLSTM that introduces an attention mechanism. Finally, the information fusion module integrates spatiotemporal information from multiple time dimensions to forecast future traffic flow. Moreover, to confirm the validity of our method, our experiments were conducted extensively on the TaxiBJ and BikeNYC datasets, and ultimately, CLSTAN performed better than other baseline experiments.

Classification of multivariate time series data, often including both time points and intervals at variable frequencies, is a challenging task. We introduce the KarmaLegoSification (KLS) framework for classification of multivariate time series analysis, which implements three phases: (1) application of a temporal abstraction process that transforms a series of raw time-stamped data points into a series of symbolic time intervals; (2) mining these symbolic time intervals to discover frequent time-interval-related patterns (TIRPs), using Allen’s temporal relations; and (3) using the TIRPs as features to induce a classifier. To efficiently detect multiple TIRPs (features) in a single entity to be classified, we introduce a new algorithm, SingleKarmaLego, which can be shown to be superior for that purpose over a Sequential TIRPs Detection algorithm. We evaluated the KLS framework on datasets in the domains of diabetes, intensive care, and infectious hepatitis, assessing the effects of the various settings of the KLS framework. Discretization using Symbolic Aggregate approXimation (SAX) led to better performance than using the equal-width discretization (EWD); knowledge-based cut-off definitions when available were superior to both. Using three abstract temporal relations was superior to using the seven core temporal relations. Using an epsilon value larger than zero tended to result in a slightly better accuracy when using the SAX discretization method, but resulted in a reduced accuracy when using EWD, and overall, does not seem beneficial. No feature selection method we tried proved useful. Regarding feature (TIRP) representation, mean duration performed better than horizontal support, which in turn performed better than the default Binary (existence) representation method.

Multivariate Time Series Classification (MTSC) has attracted increasing research attention in the past years due to the wide range applications in e.g., action/activity recognition, EEG/ECG classification, etc. In this paper, we open a novel path to tackle with MTSC: a relational way. The multiple dimensions of MTS are represented in a relational data scheme, then a propositionalisation technique (based on classical aggregation/selection functions from the relational data field) is applied to build interpretable features from secondary tables to “flatten” the data. Finally, the MTS flattened data are classified using a selective Naive Bayes classifier. Experimental validation on various benchmark data sets show the relevance of the suggested approach.

Due to high dimensionality and multiple variables, unsupervised classification of multivariate time series (MTS) involves more challenging problems than those of univariate ones. Unlike the vectorization of a feature matrix in traditional clustering algorithms, an unsupervised pattern recognition scheme based on matrix data is proposed for MTS samples in this paper. To reduce the computational load and time consumption, a novel variable-based principal component analysis (VPCA) is first devised for the dimensionality reduction of MTS samples. Afterward, a spatial weighted matrix distance-based fuzzy clustering (SWMDFC) algorithm is proposed to directly group MTS samples into clusters as well as preserve the structure of the data matrix. The spatial weighted matrix distance (SWMD) integrates the spatial dimensionality difference of elements of data into the distance of MST pairs. In terms of the SWMD, the MTS samples are clustered without vectorization in the dimensionality-reduced feature matrix space. Finally, three open-access datasets are utilized for the validation of the proposed unsupervised classification scheme. The results show that the VPCA can capture more features of MTS data than principal component analysis (PCA) and 2-D PCA. Furthermore, the clustering performance of SWMDFC is superior to that of fuzzy c -means clustering algorithms based on the Euclidean distance or image Euclidean distance.