Featureweighting in dynamic timewarping for gesture recognition in depth data

An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition

There have been many published works on using Dynamic Time Warping (DTW) for word image matching. It has also been shown that constrained DTW works better than classical DTW, due to it's property to limit the warping path and to avoid pathological matching. Moreover, some studies also demonstrate that variants of DTW can perform better, compared to classical DTW. To take advantages from these techniques, we propose in this paper to study how combination of such approaches can even more improve the results. First, we propose to use pseudo Local DTW (LDTW: more efficient than DTW in our experiments) with a constraint band. Next, several other sequence matching techniques are parametrically combined to study their potential advantages. For this, we adapted the approach to the context of word image matching to obtain the value of the parameters. It is shown that, at least on Bentham and George Washington datasets, results can be improved by combining some algorithms. Please note that the goal of this research work is not to propose a competitive word spotting technique but to propose a dynamic programming based learning free approach for word spotting or image matching, which can perform moderately with respect to other existing approaches in this domain.

Hand gesture recognition is becoming increasingly popular for applications in ubiquitous computing environment. Accelerometer based methods have proven themselves to be competitive in terms of both portability and recognition accuracy. But there are only a few algorithms which have achieved moderate accuracies in user independent gesture recognition. A novel accelerometer based user independent hand gesture recognition algorithm is proposed in this paper. For validation of accuracy the test was run over 3200 samples collected from 5 users over 5 days. The overall accuracy was found to be 96.4% for user independent gesture recognition. A high recognition accuracy and possibility of simpler implementation also gives this algorithm an upper hand among competitive methods published in recent literatures. A hardware-accelerated dynamic time warping (DTW) implementation is also proposed to pave the way towards continuous gesture recognition for DTW based methods.

Gesture recognition is a technology often used in human-computer interaction applications. Dynamic time warping (DTW) is one of the techniques used in gesture recognition to find an optimal alignment between two sequences. Oftentimes a pre-processing of sequences is required to remove variations due to different camera or body orientations or due to different skeleton sizes between the reference gesture sequences and the test gesture sequences. We discuss a set of pre-processing methods to make the gesture recognition mechanism robust to these variations. DTW computes a dissimilarity measure by time-warping the sequences on a per sample basis by using the distance between the current reference and test sequences. However, all body joints involved in a gesture are not equally important in computing the distance between two sequence samples. We propose a weighted DTW method that weights joints by optimizing a discriminant ratio. Finally, we demonstrate the performance of our pre-processing and the weighted DTW method and compare our results with the conventional DTW and state-of-the-art.

In the area of advanced human-computer interaction, automatic gesture recognition is an important field. Motion data produced by the accelerometer of a smart watch can be utilized in hand gesture recognition. In this work we examine the use of a commodity smart watch and a smartphone as the capture and the processing units respectively, for recognizing gestures. We claim that if the proper gesture recognition algorithms are applied, the recognition of natural gestures i.e. 3-D gestures easily performed by an individual can be accurate enough to be useful in everyday life activities. Symbolic Aggregate Approximation (SAX) and Dynamic Time Warping (DTW) methodologies are utilized in this context and evaluated using a set of six 3-D natural gestures.

International audience

Template matching methods using Dynamic Time Warping (DTW) have been used recently for online gesture recognition from body-worn motion sensors. However, DTW has been shown sensitive under the strong presence of noise in time series. In sensor readings, labeling temporal boundaries of daily gestures precisely is rarely achievable as they are often intertwined. Moreover, the variation in daily gesture execution always exists. Therefore, here we propose two template matching methods utilizing the Longest Common Subsequence (LCSS) to improve robustness against such noise for online gesture recognition. Segmented LCSS utilizes a sliding window to define the unknown boundaries of gestures in the continuous coming sensor readings and detects efficiently a possibly shorter gesture within it. WarpingLCSS is our novel variant of LCSS to determine occurrences of gestures without segmenting data and performs one order of magnitude faster than the Segmented LCSS. The WarpingLCSS requires low-resource settings to process new arriving samples, thus it is suitable for real-time gesture recognition implemented directly on the small wearable devices. We compare our methods with the existing template matching methods based on Dynamic Time Warping (DTW) on two real-world gesture datasets from arm-worn accelerometer data. The results demonstrate that the LCSS approaches outperform the existing template matching approaches (about 12% in accuracy) in the dataset that suffers from boundary noise and execution variation.

Randomized time warping for motion recognition

This paper presents an algorithm of Hand Gesture Recognition by using Dynamic Time Warping methodology. The system consists of three modules: real time detection of face region and two hand regions, tracking the hands trajectory both in terms of direction among consecutive frames as well as distance from the centre of the frame and gesture recognition based on analyzing variations in the hand locations along with the centre of the face. The proposed technique of ours overcomes not only the limitations of a glove based approach but also most of the vision based approach concerning different illumination conditions, background complexity and distance from camera which is up to 2 meters. Also by using Dynamic Time Warping Algorithm which finds the optimal alignment between the stored database features and query features, improvement in recognition accuracy is observed compared to conventional methods. Experimental results show that the accuracy is 90% in recognizing 24 gestures based on Indian Sign Language.

Cognitive changes in general occur with normal aging. This may lead to the prevalence and effect of age associated diseases. The understanding and identification of these age-related cognitive impairments is an important aspect in elderly population. This leads in the simple case, supporting a functional independence of the elderly and in a complex case, an early identification of dementia in advance. One important change with normal aging is the decline in gait functionality. The decline in gait is more visible in the elderly with more cognitive impairment during dual cognitive tasks, multi-tasking exercises. For the classification of the healthy elderly from the elderly having cognitive impairments, the gait data of the elderly is acquired through Kinect V2. A waking trial of 5m long is used to collect the gait data. 3D based pose estimation using the depth data is performed. Gait parameters and gait cycles of the individual elderly are estimated. In this paper, Dynamic Time Warping (DTW) algorithm is used to compare the patterns of the gait cycles of the individual in different trails such as Regular Gait 1 (RG1), Regular Gait 2 (RG2), Counting Backward 1 (CB1), Counting Backward 3 (CB3), Fast Gait (FG) and Words with Special Letters (WSPL). The identified cross levels along with the estimated gait parameters are used for training the machine learning algorithm. Support Vector Machines (SVM) were used for the classification of the elderly persons with or without cognitive impairments. The experiment results proved that such a classification of cognitive impairment levels using 3D pose estimation and machine learning helps in future for the identification of dementia in advance.

Next chapter Full AccessProceedings Proceedings of the 2001 SIAM International Conference on Data Mining (SDM)Derivative Dynamic Time WarpingEamonn J. Keogh and Michael J. PazzaniEamonn J. Keogh and Michael J. Pazzanipp.1 - 11Chapter DOI:https://doi.org/10.1137/1.9781611972719.1PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract 1 Introduction Time series are a ubiquitous form of data occurring in virtually every scientific discipline. A common task with time series data is comparing one sequence with another. In some domains a very simple distance measure, such as Euclidean distance will suffice. However, it is often the case that two sequences have the approximately the same overall component shapes, but these shapes do not line up in X-axis. Figure 1 shows this with a simple example. In order to find the similarity between such sequences, or as a preprocessing step before averaging them, we must “warp” the time axis of one (or both) sequences to achieve a better alignment. Dynamic time warping (DTW), is a technique for efficiently achieving this warping. In addition to data mining (Keogh & Pazzani 2000, Yi et. al. 1998, Berndt & Clifford 1994), DTW has been used in gesture recognition (Gavrila & Davis 1995), robotics (Schmill et. al 1999), speech processing (Rabiner & Juang 1993), manufacturing (Gollmer & Posten 1995) and medicine (Caiani et. al 1998). Next chapter RelatedDetails Published:2001ISBN:978-0-89871-495-1eISBN:978-1-61197-271-9 https://doi.org/10.1137/1.9781611972719Book Series Name:ProceedingsBook Code:PR105Book Pages:465

Prosthetic hands are of paramount importance in the rehabilitation of upper limbs amputees. Gesture recognition using surface electromyography (sEMG) data has emerged as a great option for controlling prosthetic devices, since these data are acquired by non-intrusive sensors. This work presents a real-time classification system based on artificial neural networks with individualized muscle activity segmentation and using dynamic time warping (DTW) based features. For real-time classification, we modify the size of the sliding windows so that their length is sufficient to fully capture the muscle activity signal. For data segmentation, we propose an enhanced muscle activity detection in which validation is used to fine-tune the thresholds needed to determine the beginning and end of muscle contraction. We used two validation methods: cross-validation and multi-holdout. Moreover, we propose a post-processing technique to choose the most representative class when there are multiple classifications for a given data. By combining all the proposed techniques, the accuracy of the resulting system was (97.2 ± 0.3)% in the classification of 6 hand gestures from 10 healthy people, representing an increase of 7.1% in the mean accuracy compared with the baseline model.

Dynamic time warping(DTW) is widely used for accelero-meter-based gesture recognition. The basic learning strategy applied with DTW in most cases is instance-based learning, where all the feature vectors extracted from labeled training patterns are stored as reference patterns for pattern matching. With the brute-force instance-based learning, the number of reference patterns for a class increases easily to a big number. A smart strategy for generating a small number of good reference patterns is needed. We propose to use DTW-based K-Means clustering algorithm for the purpose. Initial training is performed by brute-force instance-based learning, and then we apply the clustering algorithm over the reference patterns per class so that each class is represented by 5 ~ 10 reference patterns each of which corresponds to the cluster centroid. Experiments were performed on 5200 sample patterns of 26 English uppercase alphabets collected from 40 personals using a handheld device having a 3-d accelerometer inside. Results showed that reducing the number of reference patterns by more than 90% decreased the recognition rate only by 5%, while obtaining more than 10-times faster classification speed.

Gesture recognition has become a very active research area with the advent of the Kinect sensor. The most common approaches for gesture recognition use temporal information and are based on methods such as Hidden Markov Models (HMM) and Dynamic Time Warping (DTW). In this paper, we present a novel non-temporal alternative for gesture recognition using the Microsoft Kinect device. The proposed approach, Recognition by Characteristic Window (RCW), identifies, using clustering techniques and a sliding window, distinctive portions of individual gestures which have low overlapping information with other gestures. Once a distinctive portion has been identified for each gesture, all these sub-sequences are used to recognize a new instance. The proposed method was compared against HMM and DTW on a benchmark gesture’s dataset showing very competitive performance.

At the present time, the facial expressions recognition system from video sequences is a huge challenging problem to face in human and computer interaction. Emotions are important information to retrieve and treat. With the developed system, we were able to decode facial expressions and deduce human emotions, all in keeping a low error rate. The aim of this paper is to bring together two areas of which are K-Nearest Neighbor (KNN) and Dynamic Time Warping (DTW) applying for facial expressions classification. The proposed system constructs the robust features based on Active Shape Models (ASM), computes distance similarity between feature vectors by DTW, then uses KNN as a classifier. Our system achieved 90.1% of recognition accuracy using the Cohn-Kanade (CK+) facial expressions database.