Spatial-temporal graph neural ODE networks for skeleton-based action recognition

Spatio-temporal Graph Convolutional Networks (ST-GCNs) have shown great performance in the context of skeleton-based action recognition. Nevertheless, ST-GCNs use raw skeleton data as vertex features. Such features have low dimensionality and might not be optimal for action discrimination. Moreover, a single layer of temporal convolution is used to model short-term temporal dependencies but can be insufficient for capturing both long-term. In this paper, we extend the Spatio-Temporal Graph Convolutional Network for skeleton-based action recognition by introducing two novel modules, namely, the Graph Vertex Feature Encoder (GVFE) and the Dilated Hierarchical Temporal Convolutional Network (DH-TCN). On the one hand, the GVFE module learns appropriate vertex features for action recognition by encoding raw skeleton data into a new feature space. On the other hand, the DH-TCN module is capable of capturing both short-term and long-term temporal dependencies using a hierarchical dilated convolutional network. Experiments have been conducted on the challenging NTU RGB-D 60, NTU RGB-D 120 and Kinetics datasets. The obtained results show that our method competes with state-of-the-art approaches while using a smaller number of layers and parameters; thus reducing the required training time and memory.

Spatio-temporal Graph Convolutional Networks (ST-GCNs) have shown great performance in the context of skeleton-based action recognition. Nevertheless, ST-GCNs use raw skeleton data as vertex features. Such features have low dimensionality and might not be optimal for action discrimination. Moreover, a single layer of temporal convolution is used to model short-term temporal dependencies but can be insufficient for capturing both long-term. In this paper, we extend the Spatio-Temporal Graph Convolutional Network for skeleton-based action recognition by introducing two novel modules, namely, the Graph Vertex Feature Encoder (GVFE) and the Dilated Hierarchical Temporal Convolutional Network (DH-TCN). On the one hand, the GVFE module learns appropriate vertex features for action recognition by encoding raw skeleton data into a new feature space. On the other hand, the DH-TCN module is capable of capturing both short-term and long-term temporal dependencies using a hierarchical dilated convolutional network. Experiments have been conducted on the challenging NTU RGB-D 60, NTU RGB-D 120 and Kinetics datasets. The obtained results show that our method competes with state-of-the-art approaches while using a smaller number of layers and parameters; thus reducing the required training time and memory.

Large scale optical character recognition (OCR) refers to or means the computerization of large amounts of documents such as news papers. Despite the diversity of commercial OCR products, this task still remains too far from the mature especially if the input documents are insufficient quality or cursive writing such as the Arabic documents (Vinciarelli, 2002). Indeed, in their project (Holley, 2009), the national library of Australia reports that existing OCR systems are, commonly, weak. Moreover, their conducted experiments on historical newspapers show that the corresponding accuracy raw varied from 71% to 98.02%. This is surely due to the weakness of the approaches and techniques used in these systems. Printed cursive written documents such as the Arabic one presents, in addition, other difficulties which are behind the weaknesses of the existing commercialized systems especially when the quality of the input binary image of the document is not good enough. The first difficulty encountered for such writing is the segmentation of any given input word or sub-word into isolated characters given that the size of each ofwhich is variable. In practice, if the segmentation process is conducted successfully, then it eases the recognition step to a large extent. That is why Latin printedOCR systems are, commonly, more powerful compared to those devoted to the cursive writing documents. Dynamic Time Warp (DTW) algorithm is a well known procedure especially in pattern recognition (Alves et al., 2002; Khemakhem et al., 2005; Philip, 1992; Vuori et al., 2001), (Khemakhem et al., 2009; Kumar et al., 2006; Tapia et al., 2007). The DTW algorithm is the result of the adaptation of dynamic programming to the field of pattern recognition. Printed cursive writing OCR by the DTW algorithm provides very interesting recognition rates without prior character segmentation (such as: the Arabic, Persian, Urdu, latin connected characters,...), (Khemakhem et al., 2005). The purpose of the DTW algorithm is to perform optimal time alignment between a reference pattern and an unknown pattern and evaluate their difference. Intensive experiments show that the recognition rate of the DTW algorithm remains acceptable compared to the existing commercialized systems even when the quality of the input documents is not good enough. Intensive tests on more than 100.000 connected characters (most of them are Arabic cursive and including some important noise) show that the segmentation average rate is greater than 98% and the recognition average rate is 5

Insects play significant role in the human life. And insects pollinate major food crops consumed in the world. Insect pests consume and destroy major crops in the world. Hence to have control over the disease and pests, researches are going on in the area of entomology using chemical, biological and mechanical approaches. The data relevant to the flying insects often changes over time, and classification of such data is a central issue. And such time series mining tasks along with classification is critical nowadays. Most time series data mining algorithms use similarity search and hence time taken for similarity search is the bottleneck and it does not produce accurate results and also produces very poor performance. In this paper, a novel classification method that is based on the dynamic time warping (DTW) algorithm is proposed. The dynamic time warping algorithm is deterministic and lacks in modeling stochastic signals. The dynamic time warping (DTW) algorithm is improved by implementing a nonlinear median filtering (NMF). Recognition accuracy of conventional DTW algorithms is less than that of the hidden Markov model (HMM) by same voice activity detection (VAD) and noise-reduction. With running spectrum filtering (RSF) and dynamic range adjustment (DRA). NMF seek the median distance of every reference of time series data and the recognition accuracy is much improved. In this research work, optical sensors are used to record the sound of insect flight, with invariance to interference from ambient sounds. The implementation of our tool includes two parts, an optical sensor to record the "sound" of insect flight, and a software that leverages on the sensor information, to automatically detect and identify flying insects.

A time series analysis method for spacecraft telemetry data is presented in this paper. For spacecraft testing and on-orbit flight, this method can monitor the changes of telemetry data automatically and identify the failure modes of spacecraft. Using dynamic time warping (DTW) algorithm, combining historical data samples as well as fault cases with this method analyzes the similarity of telemetry data transformed into time series. By comparing the results of analysis with the results of DTW distance calculation, the relative deviation of data is measured and the abnormal data in fault mode is identified. The results show that the telemetry data analysis method based on DTW algorithm can effectively detect data anomalies and realize fault identification, which has a certain application prospect.

Partial matching and search space reduction for QbE-STD

Croplands are commonly mapped using time series of remotely sensed images. The dynamic time warping (DTW) algorithm is an effective method for realizing this. However, DTW algorithm faces the challenge of capturing complete and accurate representative cropland time series on a national scale, especially in Asian countries where climatic and topographic conditions, cropland types, and crop growth patterns vary significantly. This study proposes an automatic cropland extraction method based on the DTW algorithm and density-based spatial clustering of applications with noise (DBSCAN), hereinafter referred to as ACE-DTW, to map croplands in Pakistan in 2015. First, 422 frames of multispectral Landsat-8 satellite images were selected from the Google Earth Engine to construct monthly normalized difference vegetation index (NDVI) time series. Next, a total of 2409 training samples of six land cover types were generated randomly and explained visually using high-resolution remotely sensed images. Then, a multi-layer DBSCAN was used to classify NDVI time series of training samples into different categories automatically based on their pairwise DTW distances, and the mean NDVI time series of each category was used as the standard time series to represent the characteristics of that category. These standard time series attempted to represent cropland information and maximally distinguished croplands from other possible interference land cover types. Finally, image pixels were classified as cropland or non-cropland based on their DTW distances to the standard time series of the six land cover types. The overall cropland extraction accuracy of ACE-DTW was 89.7%, which exceeded those of other supervised classifiers (classification and regression trees: 78.2%; support vector machines: 78.8%) and existing global cropland datasets (Finer Resolution Observation and Monitoring of Global Land Cover: 87.1%; Global Food Security Support Analysis Data: 83.1%). Further, ACE-DTW could produce relatively complete time series of variable cropland types, and thereby provide a significant advantage in mountain regions with small, fragmented croplands and plain regions with large, high-density patches of croplands.

Dynamic Time Warping(DTW) algorithm has strong resistance to synchronous error. Aiming at the similarity and asynchrony of meteorological factor, a method of applying DTW to photovoltaic power prediction is proposed. The meteorological factors which have great influence on photovoltaic power generation are selected by Pearson correlation coefficient method. Based on the DTW algorithm, the similarity of meteorological factors between the predicting days and the historical days can be measured. Through weighted calculation of similarity, the similar days are used to obtain training data of Partial Least Squares(PLS) prediction model. The simulation study on a photovoltaic power station verifies the correctness and effective of the proposed prediction method. The results show that the prediction model has great forecasting accuracy and certain feasibility and practicability.

This paper proposes an improved dynamic time warping (DTW) algorithm with a nonlinear median filtering (NMF). Recognition accuracy of conventional DTW algorithms are less than the hidden Markov model (HMM) by same voice activity detection (VAD) and noise-reduction with running spectrum filtering (RSF) and dynamic range adjustment (DRA). For analyzing some incorrect results, unlike in conventional DTW, we do not use the minimum distance to recognize. we employ NMF and seek the median distance of the every reference word with the unknown speech waveform. All recognition accuracy of conventional DTW algorithms are improved much more by NMF. The recognition accuracy of Itakura's DTW algorithm is the best. Its recognition accuracy similar to that of the HMM approach in 10 dB and 20 dB white noise.

The Dynamic Time Warping (DTW) algorithm is widely used in finding the global alignment of time series. Many time series data mining and analytical problems can be solved by the DTW algorithm. However, using the DTW algorithm to find similar subsequences is computationally expensive or unable to perform accurate analysis. Hence, in the literature, the parallelisation technique is used to speed up the DTW algorithm. However, due to the nature of DTW algorithm, parallelizing this algorithm remains an open challenge. In this paper, we first propose a novel method that finds the similar local subsequence. Our algorithm first searches for the possible start positions of subsequence, and then finds the best-matching alignment from these positions. Moreover, we parallelize the proposed algorithm on GPUs using CUDA and further propose an optimization technique to improve the performance of our parallelization implementation on GPU. We conducted the extensive experiments to evaluate the proposed method. Experimental results demonstrate that the proposed algorithm is able to discover time series subsequences efficiently and that the proposed GPU-based parallelization technique can further speedup the processing.

Graph convolutional networks (GCN), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition. However, in the existing GCN-based methods, graph-structured representation of the human skeleton makes it difficult to be fused with other modalities, especially in the early stages. This may limit their scalability and performance in action recognition tasks. In addition, the pose information, which naturally contains informative and discriminative clues for action recognition, is rarely explored together with skeleton data in existing methods. In this work, we proposed pose-guided GCN (PG-GCN), a multi-modal framework for high-performance human action recognition. In particular, a multi-stream network is constructed to simultaneously explore the robust features from both the pose and skeleton data, while a dynamic attention module is designed for early-stage feature fusion. The core idea of this module is to utilize a trainable graph to aggregate features from the skeleton stream with that of the pose stream, which leads to a network with more robust feature representation ability. Extensive experiments show that the proposed PG-GCN can achieve state-of-the-art performance on the NTU RGB+D 60 and NTU RGB+D 120 datasets.

Adaptive multi-view graph convolutional networks for skeleton-based action recognition

The gesture sequence recognition method based on the traditional Dynamic Time Warping(DTW) algorithm will lead to too long calculation time when the sample data sequence length is too large, and the accuracy of the recognition result will be obviously low due to the influence of noise. The FastDTW algorithm is mainly used for DTW acceleration, but it cannot suppress noise, and its time complexity is O(N). Edit Distance on Real sequence(EDR) algorithm can only suppress noise, but its efficiency is not improved, and its time complexity is O(N2). Based on the above problems, FastEDR algorithm is proposed for the matching verification of gesture sequences. The results show that the computational efficiency of FastEDR algorithm is significantly improved compared with DTW algorithm, the error acceptance rate FAR is reduced by 2.94%, and the error rejection rate FRR is reduced by 4.5%. The gesture sequence recognition based on FastEDR algorithm can achieve a better recognition state. If it is applied to unlock mobile phones, it is expected to become a new generation of unlocking method.

Today, the elderly population is increasing, and there are many drawbacks for them, especially defects in their knee joints which lead to improper gait. To solve this problem, their knee joint can be replaced with knee arthroplasty. In this letter, level of improvement in the human gait before and after total knee arthroplasty (TKA) surgery is investigated using the dynamic time warping (DTW) algorithm. For this purpose, several volunteers who have problems with their knees are incorporated in a test before and after TKA surgery. Then, the data of gait analysis is collected and the data is compared with a reference using the DTW algorithm. The outcome results illustrate an improvement of 89%-97% by the proposed algorithm after TKA surgery. Therefore, patients can see improvement with high accuracy and very fast that result in more use this technique in TKR surgery.

Fault location method combining transient mutation and dynamic time warping algorithm for grounding system