Bandwidth-efficient distributed k-nearest-neighbor search with dynamic time warping

Croplands are commonly mapped using time series of remotely sensed images. The dynamic time warping (DTW) algorithm is an effective method for realizing this. However, DTW algorithm faces the challenge of capturing complete and accurate representative cropland time series on a national scale, especially in Asian countries where climatic and topographic conditions, cropland types, and crop growth patterns vary significantly. This study proposes an automatic cropland extraction method based on the DTW algorithm and density-based spatial clustering of applications with noise (DBSCAN), hereinafter referred to as ACE-DTW, to map croplands in Pakistan in 2015. First, 422 frames of multispectral Landsat-8 satellite images were selected from the Google Earth Engine to construct monthly normalized difference vegetation index (NDVI) time series. Next, a total of 2409 training samples of six land cover types were generated randomly and explained visually using high-resolution remotely sensed images. Then, a multi-layer DBSCAN was used to classify NDVI time series of training samples into different categories automatically based on their pairwise DTW distances, and the mean NDVI time series of each category was used as the standard time series to represent the characteristics of that category. These standard time series attempted to represent cropland information and maximally distinguished croplands from other possible interference land cover types. Finally, image pixels were classified as cropland or non-cropland based on their DTW distances to the standard time series of the six land cover types. The overall cropland extraction accuracy of ACE-DTW was 89.7%, which exceeded those of other supervised classifiers (classification and regression trees: 78.2%; support vector machines: 78.8%) and existing global cropland datasets (Finer Resolution Observation and Monitoring of Global Land Cover: 87.1%; Global Food Security Support Analysis Data: 83.1%). Further, ACE-DTW could produce relatively complete time series of variable cropland types, and thereby provide a significant advantage in mountain regions with small, fragmented croplands and plain regions with large, high-density patches of croplands.

Throughout recent years, dynamic time warping (DTW) has remained as a robust similarity measure in time series classification (TSC). 1-nearest neighbor (1-NN) algorithm with DTW is the most widely used classification method on time series serving as a benchmark. With the increasing demand for TSC on low-resource devices and the widespread of wearable devices, the need for a efficient and accurate time series classifier has never been higher. Although 1-NN DTW attains accurate results, it highly falls back on efficiency due to its quadratic complexity in the length of time series. In this paper, we propose a new approximation method for reducing the length of the time series as the input of DTW. We call it control chart approximation (CCA), after a similar concept used in statistical quality control processing. CCA representation approximates raw time series by transforming them into a set of segments with aggregated values and durations forming a reduced 3-D vector. We also propose an adaptation of DTW in 3-D space as a distance measure for 1-NN classifier, and denote the method as 1-NN 3-D DTW. Our experiments on 85 datasets from UCR archive—including 28 long-length (>500 points) time series datasets—show up to two orders of magnitude performance gain in running time compared to the state-of-the-art 1-NN DTW implementation. Moreover, it shows similar or better accuracy on the long time series in the experiment.

Dynamic Time Warping (DTW) is a well known algorithm for measuring similarity of two time series and widely used in classification, clustering or regression problems related to time series data. Unlike simple Euclid distance measure, DTW can handle time series of unequal lengths and is able to find an optimal alignment between two time sequences. Though very efficient, the computational cost of DTW is very high. There are several suboptimal variants of DTW for lowering computation, none of them is perfect. In this work, an approach to reduce computational burden of DTW has been proposed from the perspective of removing unimportant portion of the time series from computing, selected by a mask generated by edge detection algorithm commonly used in image processing or computer vision. The proposed Edge-Detectional Dynamic Time Warping (EDDTW) has been compared with original DTW by simulation experiments with 43 publicly available benchmark data sets. The simulation results show that EDDTW outperforms DTW regarding classification accuracy in more than half of the data sets, while reducing on the average 60% of the original time series leading to reduction in computational time.

Dynamic time warping (DTW) is a state-of-the-art time series similarity measure method, which warps time axes to match the same shape between two time series with different lengths. However, its quadratic time and space complexity is an obstacle to its applications in the large time series data mining. To address this problem, some lower-bound functions for DTW, fast methods to approximately measure the distance between time series, are used to prune the dissimilar objects from time series database so as to retain the candidates for further measuring their similarity with DTW. So far, the existing lower-bound functions for DTW have been widely accepted for time series similarity search and indexing. In this paper, we propose the extensions of two existing lower-bound functions and discuss the relationships among them. The extensions are improved with high tightness and without much time cost. At the same time, we theoretically prove that these extensions satisfy lower-bound requirement and are better than their old versions respectively. The experimental results demonstrate that in most cases the quality of the proposed extensions of lower-bound functions for DTW outperforms the original versions except for a slightly higher time cost.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2012 SIAM International Conference on Data Mining (SDM)A Novel Approximation to Dynamic Time Warping allows Anytime Clustering of Massive Time Series DatasetsQiang Zhu, Gustavo Batista, Thanawin Rakthanmanon, and Eamonn KeoghQiang Zhu, Gustavo Batista, Thanawin Rakthanmanon, and Eamonn Keoghpp.999 - 1010Chapter DOI:https://doi.org/10.1137/1.9781611972825.86PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Given the ubiquity of time series data, the data mining community has spent significant time investigating the best time series similarity measure to use for various tasks and domains. After more than a decade of extensive efforts, there is increasing evidence that Dynamic Time Warping (DTW) is very difficult to beat. Given that, recent efforts have focused on making the intrinsically slow DTW algorithm faster. For the similarity-search task, an important subroutine in many data mining algorithms, significant progress has been made by replacing the vast majority of expensive DTW calculations with cheap-to-compute lower bound calculations. However, these lower bound based optimizations do not directly apply to clustering, and thus for some realistic problems, clustering with DTW can take days or weeks. In this work, we show that we can mitigate this untenable lethargy by casting DTW clustering as an anytime algorithm. At the heart of our algorithm is a novel data-adaptive approximation to DTW which can be quickly computed, and which produces approximations to DTW that are much better than the best currently known linear-time approximations. We demonstrate our ideas on real world problems showing that we can get virtually all the accuracy of a batch DTW clustering algorithm in a fraction of the time. Previous chapter Next chapter RelatedDetails Published:2012ISBN:978-1-61197-232-0eISBN:978-1-61197-282-5 https://doi.org/10.1137/1.9781611972825Book Series Name:ProceedingsBook Code:PRDT12Book Pages:1-1150Key words:DTW, Clustering, Anytime Algorithm

Dynamic time warping (DTW) is a popular distance measure for time series analysis and has been applied in many research domains. This paper proposes the R package IncDTW for the incremental calculation of DTW, and based on this principle IncDTW also helps to classify or cluster time series, or perform subsequence matching and k-nearest neighbor search. DTW can measure dissimilarity between two temporal sequences which may vary in speed, with a major downside of high computational costs. Especially for analyzing live data streams, subsequence matching or calculating pairwise distance matrices, runtime intensive computations are unfavorable or can even make the analysis intractable. IncDTW tackles this problem by a vector-based implementation of the DTW algorithm to reduce the space complexity from a quadratic to a linear level in number of observations, and an incremental calculation of DTW for updating interim results to reduce the runtime complexity for online applications. We discuss the fundamental functionalities of IncDTW and apply the package to classify multivariate live stream accelerometer time series for activity recognition. Finally, comparative runtime experiments with various R and Python packages for various data analysis tasks emphasize the broad applicability of IncDTW.

Insects play significant role in the human life. And insects pollinate major food crops consumed in the world. Insect pests consume and destroy major crops in the world. Hence to have control over the disease and pests, researches are going on in the area of entomology using chemical, biological and mechanical approaches. The data relevant to the flying insects often changes over time, and classification of such data is a central issue. And such time series mining tasks along with classification is critical nowadays. Most time series data mining algorithms use similarity search and hence time taken for similarity search is the bottleneck and it does not produce accurate results and also produces very poor performance. In this paper, a novel classification method that is based on the dynamic time warping (DTW) algorithm is proposed. The dynamic time warping algorithm is deterministic and lacks in modeling stochastic signals. The dynamic time warping (DTW) algorithm is improved by implementing a nonlinear median filtering (NMF). Recognition accuracy of conventional DTW algorithms is less than that of the hidden Markov model (HMM) by same voice activity detection (VAD) and noise-reduction. With running spectrum filtering (RSF) and dynamic range adjustment (DRA). NMF seek the median distance of every reference of time series data and the recognition accuracy is much improved. In this research work, optical sensors are used to record the sound of insect flight, with invariance to interference from ambient sounds. The implementation of our tool includes two parts, an optical sensor to record the "sound" of insect flight, and a software that leverages on the sensor information, to automatically detect and identify flying insects.

A time series analysis method for spacecraft telemetry data is presented in this paper. For spacecraft testing and on-orbit flight, this method can monitor the changes of telemetry data automatically and identify the failure modes of spacecraft. Using dynamic time warping (DTW) algorithm, combining historical data samples as well as fault cases with this method analyzes the similarity of telemetry data transformed into time series. By comparing the results of analysis with the results of DTW distance calculation, the relative deviation of data is measured and the abnormal data in fault mode is identified. The results show that the telemetry data analysis method based on DTW algorithm can effectively detect data anomalies and realize fault identification, which has a certain application prospect.

Ecological patterns are often fundamentally chronological. However, generalization of data is necessarily accompanied by a loss of detail or resolution. Temporal data in particular contain information not only in data values but in the temporal structure, which is lost when these values are aggregated to provide point estimates. Dynamic time warping (DTW) is a time series comparison method that is capable of efficiently comparing series despite temporal offsets that confound other methods. The DTW method is both efficient and remarkably flexible, capable of efficiently matching not only time series but also any sequentially structured data set, which has made it a popular technique in machine learning, artificial intelligence, and big data analytical tasks. DTW is rarely used in ecology despite the ubiquity of temporally structured data. As technological advances have increased the richness of small‐scale ecological data, DTW may be an attractive analysis technique because it is able to utilize the additional information contained in the temporal structure of many ecological data sets. In this study, we use an example data set of high‐resolution fish movement records obtained from otolith microchemistry to compare traditional analysis techniques with DTW clustering. Our results suggest that DTW is capable of detecting subtle behavioral patterns within otolith data sets which traditional data aggregation techniques cannot. These results provide evidence that the DTW method may be useful across many of the temporal data types commonly collected in ecology, as well other sequentially ordered “pseudo time series” data such as classification of species by shape.

Dynamic Time Warping (DTW) is a widely used distance measurement in time series clustering. DTW distance is invariant to time series phase perturbations but has a quadratic complexity. An effective acceleration method must reduce the DTW utilization ratio during time series clustering; for example, TADPole uses both upper and lower bounds to prune off a large ratio of expensive DTW calculations. To further reduce the DTW utilization ratio, we find that the linear-complexity L1-norm distance (Manhattan distance) is effective enough when the time series only comprise small phase perturbations. Therefore, we propose a novel time series clustering by Minimizing Dynamic Time Warping Utilization (MiniDTW) algorithm to accelerate time series clustering. In MiniDTW, the dataset is first greedily summarized into seed clusters, which comprise time series of small phase perturbations, by L1-norm distance. Then, we develop a new Sparse Symmetric Non-negative Matrix Factorization (SSNMF) algorithm, which factorizes the DTW distance matrix of seed cluster centers, to merge the seed clusters into the final clusters. The experiments on UCR time series datasets demonstrate that MiniDTW, pruning 98.52% of the DTW utilization, is better than the counterpart method, TADPole, which only prunes 75.56% of the DTW utilization; and thus MiniDTW is 10 times faster than TADPole.

Satellite Image Time Series (SITS) have become more accessible in recent years and SITS analysis has attracted increasing research interest. Given that labeled SITS training samples are time and effort consuming to acquire, clustering or unsupervised analysis methods need to be developed. Similarity measure is critical for clustering, however, currently established methods represented by Dynamic Time Warping (DTW) still exhibit several issues when coping with SITS, such as pathological alignment, sensitivity to spike noise, and limitation on capacity. In this paper, we introduce a new time series similarity measure method named time adaptive optimal transport (TAOT) to the application of SITS clustering. TAOT inherits several promising properties of optimal transport for the comparing of time series. Statistical and visual results on two real SITS datasets with two different settings demonstrate that TAOT can effectively alleviate the issues of DTW and further improve the clustering accuracy. Thus, TAOT can serve as a usable tool to explore the potential of precious SITS data.

The accurate identification of the horizontal well marker bed is to guarantee the soft landing of the well trajectory. With the intelligent development of the petroleum industry, it is feasible to apply computers to identify the marker bed automatically. In case-based reasoning technology, the data of well logging while drilling (LWD) as characteristic parameters are compared with those of adjacent well. By taking the depth sequence of LWD data as time series and using Dynamic Time Warping (DTW) similarity measure algorithm, the similarity index of each drilling depth is calculated corresponding to the marker bed in the adjacent well. The total similarity curve is obtained by giving different weights of different feature parameters. Selecting natural gamma, deep resistivity, and shallow resistivity LWD curves as characteristic parameters, two horizontal wells in JL block of Junggar basin are analysed by this method. The result of similarity curve indicates the location of the marker bed and the total similarity value reaches 78%. The research shows that the method based on case-based reasoning can identify the marker bed of the horizontal well accurately and effectively, assist the geologist to carry out formation correlation of multiple wells at the same time, reduce the cost of human labour force, and improve work efficiency.

Dynamic time warping (DTW) has successfully been used in feature extraction and analysis of brain data [1,2]. However, DTW and other dynamic programming (DP) methods are not nearly as widespread in neuroscientific contexts, as they are in bioinformatics, where they are central [3]. DP algorithms are extremely powerful and efficient [4], so we explored further uses of DTW in the analysis of brain data. The first part a) applies DTW to successfully find locally interesting signal changes in EEG and functional magnetic resonance imaging (fMRI) data. In the second part b) we focused on harmonic analysis. Harmonic analysis as conducted via Fourier Transform (FT) or Wavelet Transform (WT) is extremely widespread in analyzing brain data, especially electroencephalography (EEG) data. The fundamental assumption of these methods is that a complex function (the time series waveform) can be represented as a superposition of sinusoidal oscillations. While comparing spectra is useful, it is incorrect to assume that the entirety of the spectral density is due to oscillatory activity proper (especially sinusoidal) [5]. We wanted to get some idea of the amount and effects of non-sinusoidal oscillations in EEG data. In a) we quantify the amount of local signal dissimilarity, which is often equivalent to where there is interesting sudden activity, and in b) we quantify how much non-sinusoidal activity there is in spectra computed in a standard way. For a) we apply the method on a data set from a simultaneous EEG/fMRI session with 18 subjects that has been used in other published works before [6] and show that we can easily detect movement, changes in brain state accompanying large reaction time changes during a choice reaction time task, and other interesting events in the EEG and fMRI data sets. The method is applicable to both modalities, as DTW is very general and can be used with any type of time series. The method we used works by either running DTW on a selected time interval (or volume) against sequential intervals (or volumes), or by running it as a sliding-window for detecting sudden local signal changes in the EEG or fMRI. For b) we show that the spectral power, as it's typically computed, includes non-sinusoidal activity, when compared to the signal compared against a series of equal-length (to the signal) sinusoidal waves of varying frequencies. We also found when the sinusoidal-oscillation deviations occurred during an EEG time series extrapolation which we performed using a deep learning Toolbox. The EEG time series prediction gave another point of view for the oscillation changes. We find that the prediction breakdown points are related to sudden oscillation changes, which are presumably time points when other neural oscillator generators become active and mixed in the signal. We interpret both prediction breakdowns and momentary oscillatory deviations as being possible reflections of sudden phase changes in the local or global brain state related, which may be related to EEG microstate changes. We hope to bring to attention the flexibility and power of DP to the neuroscientific community.

Similarity search in streaming time series is a crucial subroutine of a number of real-time applications dealing with time-series streams. In finding subsequences of time-series streams that match with patterns under Dynamic Time Warping (DTW), data normalization plays a very important role and should not be ignored. SPRING proposed by Sakurai et al. conducts the similarity search by mitigating the time and space complexity of DTW. Unfortunately, SPRING produces inaccurate results since no data normalization is taken into account before the DTW calculation. In this paper, we improve the SPRING method to deal with similarity search for prespecified patterns in streaming time series under DTW by applying incremental min-max normalization before the DTW calculation. For every pattern, our proposed method uses a monitoring window anchored at the entry of one streaming time series to keep track of min-max coefficients, and then the DTW distance between the normalized subsequence and the normalized pattern is incrementally computed. The experimental results reveal that our proposed method obtains best-so-far values better than those of another state-of-the-art method and the wall-clock time of the proposed method is acceptable.

Time series clustering is the process of grouping sequential correspondences in similar clusters. The key feature behind clustering time series data lies on the similarity/distance function used to identify the sequential matches. Dynamic Time Warping (DTW) is one of the common distance measures that have demonstrated competitive results compared to other functions. DTW aims to find the shortest path in the process of identifying sequential matches. DTW relies on dynamic programming to obtain the shortest path where the smaller distance is being computed. However, in the case of equivalent distances, DTW is selecting the path randomly. Hence, the selection could be misguided in such randomization process, which significantly affects the matching quality. This is due to randomization may lead to the longer path which drifts from obtaining the optimum path. This paper proposes a modified DTW that aims to enhance the dynamic selection of the shortest path when handling equivalent distances. Experiments were conducted using twenty UCR benchmark datasets. Also, the proposed modified DTW result has been compared with the state of the art competitive distance measures which is based on precision, recall and f-measure including the original DTW, Minkowski distance measure and Euclidean distance measure. The results showed that the proposed modified DTW reveal superior results in compared to the standard DTW, either using Minkowski or Euclidean. This can demonstrate the effectiveness of the proposed modification in which optimizing the shortest path has enhanced the performance of clustering. The proposed modified DTW can be used for having good clustering method for any time series data.