Most discriminating segment – Longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification

Classification algorithms require training data initially labelled by classes to build a model and then to be able to classify the new data. The amount and diversity of training data affect the classification quality and usually the larger the training set, the better the accuracy of classification. In many applications only small amounts of training data are available. This article presents a new time series classification algorithm for problems with small training sets. The algorithm was tested on hand gesture recordings in tasks of person identification and gesture recognition. The algorithm provides significantly better classification accuracy than other machine learning algorithms. For 22 different hand gestures performed by 10 people and the training set size equal to 5 gesture execution records per class, the error rate for the newly proposed algorithm is from 37% to 75% lower than for the other compared algorithms. When the training set consists of only one sample per class the new algorithm reaches from 45% to 95% lower error rate. Conducted experiments indicate that the algorithm outperforms state-of-the-art methods in terms of classification accuracy in the problem of person identification and gesture recognition.

A hidden Markov model (HMM) has been widely used as a cost-effective solution for time series based applications such as hand gesture recognition. A large number of parameters in an HMM often result in a degradation of classification accuracy due to overfitting. Furthermore, the accuracy is often affected by the initialization values of the parameters. Left-right banded (LRB) HMM is one of the representative models which improve the classification accuracy by reducing the number of parameters. However, LRB HMM still suffers from false negative misclassification when it is applied for hand gesture recognition. Misclassification in hand gesture recognition arise from similar trajectories of multiple gestures that are represented by hidden states. This paper proposes a new initialization process that reduces not only state transition parameters but also hidden state observation parameters. To this end, the gesture trajectory is predicted and then the parameters are masked out when their probabilities are negligibly small in the predicted trajectory. Experiment results show that the confusion rate is nearly 14% lower than the LRB HMM while maintaining the classification accuracy.

This chapter presents gesture recognition methods and their application to a dance training system in an instructional, virtual reality (VR) setting. The proposed system is based on the unsupervised parsing of dance movement into a structured posture space using the spherical self-organizing map (SSOM). A unique feature descriptor is obtained from the gesture trajectories through posture space on the SSOM. For recognition, various methods are explored for trajectory analysis, which include sparse coding, posture occurrence, posture transition, and the hidden Markov model. Within the system, the dance sequence of a student can be segmented online and cross-referenced against a library of gestural components performed by the teacher. This facilitates the assessment of the student dance, as well as provides visual feedback for effective training.

A low-power and real-time hidden Markov model (HMM) accelerator is proposed for gesture user interfaces on wearable smart devices. HMM algorithm is widely used for sequence recognitions such as speech recognition and gesture recognition due to its best-in-class recognition accuracy. However, the HMM algorithm incorporates high computational complexity and requires massive memory bandwidth for sequence matches. There have been studies on hardware acceleration of the HMM algorithm to resolve these issues, but they were focused on the speech recognition and did not incorporate the motion orientation capability required for the gesture recognition case. In this paper, we propose an HMM accelerator incorporating the motion orientation block for gesture recognitions on wearable devices. Binary search is exploited in the motion orientation to avoid the division and arctangent associated with the orientation and reduce its arithmetic complexity. In addition, gesture models are clustered in the gesture database to save the memory bandwidth by reducing memory transactions. Moreover, logarithmic arithmetic is used in Viterbi decoding in the HMM for more reduction in its complexity. Thanks to these schemes, this work achieves 25.6% power reduction compared with a plain hardware implementation of the gesture recognizing HMM.

Discrete event recognition based on force measurements in the frequency-domain, is presented. The force signals arise from interaction between the workpiece and the environment in a planar assembly task. The discrete events are modeled as hidden Markov models (HMMs), where the models are trained off-line with the Baum-Welch re-estimation algorithm. After the HMMs have been trained, we use them online in a robotic system to recognise discrete events as they occur. Event recognition with an accuracy as high as 98% was accomplished in 0.5-0.6 s with a relatively small training set.

Template matching methods using Dynamic Time Warping (DTW) have been used recently for online gesture recognition from body-worn motion sensors. However, DTW has been shown sensitive under the strong presence of noise in time series. In sensor readings, labeling temporal boundaries of daily gestures precisely is rarely achievable as they are often intertwined. Moreover, the variation in daily gesture execution always exists. Therefore, here we propose two template matching methods utilizing the Longest Common Subsequence (LCSS) to improve robustness against such noise for online gesture recognition. Segmented LCSS utilizes a sliding window to define the unknown boundaries of gestures in the continuous coming sensor readings and detects efficiently a possibly shorter gesture within it. WarpingLCSS is our novel variant of LCSS to determine occurrences of gestures without segmenting data and performs one order of magnitude faster than the Segmented LCSS. The WarpingLCSS requires low-resource settings to process new arriving samples, thus it is suitable for real-time gesture recognition implemented directly on the small wearable devices. We compare our methods with the existing template matching methods based on Dynamic Time Warping (DTW) on two real-world gesture datasets from arm-worn accelerometer data. The results demonstrate that the LCSS approaches outperform the existing template matching approaches (about 12% in accuracy) in the dataset that suffers from boundary noise and execution variation.

We propose a scheme for supervised image classification that uses privileged information, in the form of keypoint annotations for the training data, to learn strong models from small and/or biased training sets. Our main motivation is the recognition of animal species for ecological applications such as biodiversity modelling, which is challenging because of long-tailed species distributions due to rare species, and strong dataset biases such as repetitive scene background in camera traps. To counteract these challenges, we propose a visual attention mechanism that is supervised via keypoint annotations that highlight important object parts. This privileged information, implemented as a novel privileged pooling operation, is only required during training and helps the model to focus on regions that are discriminative. In experiments with three different animal species datasets, we show that deep networks with privileged pooling can use small training sets more efficiently and generalize better.

An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition

Side channel attacks (SCA) are powerful tools to recover the secret keys of cryptographic devices by employing physical leakages of the devices. In practice, numerous countermeasures (e.g.random process interrupts) and unintentional factors (e.g.imprecise triggering) are able to significantly degrade the performance of side channel attacks (SCA) by producing misaligning leakage measurements. The main solution to over-coming influences of misalignment is to align these traces before mounting an attack. According to this view, this paper proposes a novel leakage alignment method based on the longest common subsequence (LCS) algorithm. Initially, the misaligned traces are preprocessed by non-uniform quantification to remove part of the noise and retain the key-dependent information. Subsequently, the quantified traces are converted into symbols for sequence comparison. Finally, the aligned traces are obtained by inserting and deleting operations. Experiments for different cryptographic implementations with different levels of noise are carried out on several different devices. The experimental results show that the proposed method is effective for countering leakage misalignment and robust under high noise. Especially, it is still effective for multi-channel leakage alignment.

In this paper, we address the problem of recognizing human actions from videos. Human actions recognition is a challenging task in computer vision. We propose a method to solve this problem using Longest Common Sub-Sequence (LCSS) algorithm and Shape Context (SC). Our contributions in this paper are twofold. First, we show the applicability of the SC as a pairwise shape-similarity measurement for generating a sequence that defines a specific motion. Secondly, we demonstrate the usability of LCSS to classify human actions. Experiments were performed on two action datasets to compare the result to the related methods.

This paper describes a method to recognize the alphabets from a single hand motion using Hidden Markov Models (HMM). In our method, gesture recognition for alphabets is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and depth information are used to detect both hands and face in connection with morphological operation. After the detection of the hand, the tracking will take place in further step in order to determine the motion trajectory so-called gesture path. The second stage, feature extraction enhances the gesture path which gives us a pure path and also determines the orientation between the center of gravity and each point in a pure path. Thereby, the orientation is quantized to give a discrete vector that used as input to HMM. In the final stage, the gesture of alphabets is recognized by using Left-Right Banded model (LRB) in conjunction with Baum-Welch algorithm (BW) for training the parameters of HMM. Therefore, the best path is obtained by Viterbi algorithm using a gesture database. In our experiment, 520 trained gestures are used for training and also 260 tested gestures for testing. Our method recognizes the alphabets from A to Z and achieves an average recognition rate of 92.3%.

Cancer centers have an urgent and unmet clinical and research need for AI that can guide patient management. A core component of advancing cancer treatment research is assessing response to therapy. Doing so by hand, for example, as per RECIST or RANO criteria, is tedious and time-consuming, and can miss important tumor response information. Most notably, the prevalent response criteria often exclude lesions, the non-target lesions, altogether. We wish to assess change in a holistic fashion that includes all lesions, obtaining simple, informative, and automated assessments of tumor progression or regression. Because genetic sub-types of cancer can be fairly specific and patient enrollment in therapy trials is often limited in number and accrual rate, we wish to make response assessments with small training sets. Deep neuroevolution (DNE) is a novel radiology artificial intelligence (AI) optimization approach that performs well on small training sets. Here, we use a DNE parameter search to optimize a convolutional neural network (CNN) that predicts progression versus regression of metastatic brain disease. We analyzed 50 pairs of MRI contrast-enhanced images as our training set. Half of these pairs, separated in time, qualified as disease progression, while the other 25 image pairs constituted regression. We trained the parameters of a CNN via "mutations" that consisted of random CNN weight adjustments and evaluated mutation "fitness" as summed training set accuracy. We then incorporated the best mutations into the next generation's CNN, repeating this process for approximately 50,000 generations. We applied the CNNs to our training set, as well as a separate testing set with the same class balance of 25 progression and 25 regression cases. DNE achieved monotonic convergence to 100% training set accuracy. DNE also converged monotonically to 100% testing set accuracy. We have thus shown thatDNE can accurately classify brain metastatic disease progression versus regression. Future work will extend the input from 2D image slices to full 3D volumes, and include the category of "no change." We believe that an approach such as ours can ultimately provide a useful and informative complement to RANO/RECIST assessment and volumetric AI analysis.

A hidden Markov model based classifier is proposed in this paper to perform automatic speech recognition using myoelectric signals from the muscles of vocal articulation. The classifier's resilience to temporal variance is compared to a linear discriminant analysis classifier that was used in a pervious study. Speech recognition was performed, using five channels of myoelectric signals, on isolated words from a 10-word vocabulary. Temporal variance was induced by temporally misaligning data from the test set, with respect to the training set. When compared to the LDA classifier, the hidden Markov model classifier demonstrated a markedly lower variation in classification error due to the temporal misalignment. Characteristics of the hidden Markov model MES classifier suggest that it would effectively complement a conventional acoustic speech recognizer, in a multi-modal speech recognition system.

BackgroundRetroviral LTRs, paired or single, influence the transcription of both retroviral and non-retroviral genomic sequences. Vertebrate genomes contain many thousand endogenous retroviruses (ERVs) and their LTRs. Single LTRs are difficult to detect from genomic sequences without recourse to repetitiveness or presence in a proviral structure. Understanding of LTR structure increases understanding of LTR function, and of functional genomics. Here we develop models of orthoretroviral LTRs useful for detection in genomes and for structural analysis.Principal FindingsAlthough mutated, ERV LTRs are more numerous and diverse than exogenous retroviral (XRV) LTRs. Hidden Markov models (HMMs), and alignments based on them, were created for HML- (human MMTV-like), general-beta-, gamma- and lentiretroviruslike LTRs, plus a general-vertebrate LTR model. Training sets were XRV LTRs and RepBase LTR consensuses. The HML HMM was most sensitive and detected 87% of the HML LTRs in human chromosome 19 at 96% specificity. By combining all HMMs with a low cutoff, for screening, 71% of all LTRs found by RepeatMasker in chromosome 19 were found. HMM consensus sequences had a conserved modular LTR structure. Target site duplications (TG-CA), TATA (occasionally absent), an AATAAA box and a T-rich region were prominent features. Most of the conservation was located in, or adjacent to, R and U5, with evidence for stem loops. Several of the long HML LTRs contained long ORFs inserted after the second A rich module. HMM consensus alignment allowed comparison of functional features like transcriptional start sites (sense and antisense) between XRVs and ERVs.ConclusionThe modular conserved and redundant orthoretroviral LTR structure with three A-rich regions is reminiscent of structurally relaxed Giardia promoters. The five HMMs provided a novel broad range, repeat-independent, ab initio LTR detection, with prospects for greater generalisation, and insight into LTR structure, which may aid development of LTR-targeted pharmaceuticals.

Knowledge transfer is widely held to be a primary mechanism that enables humans to quickly learn new complex concepts when given only small training sets. In this paper, we apply knowledge transfer to deep convolutional neural nets, which we argue are particularly well suited for knowledge transfer. Our initial results demonstrate that components of a trained deep convolutional neural net can constructively transfer information to another such net. Furthermore, this transfer is completed in such a way that one can envision creating a net that could learn new concepts throughout its lifetime. The experiments we performed involved training a Deep Convolutional Neural Net (DCNN) on a large training set containing 20 different classes of handwritten characters from the NIST Special Database 19. This net was then used as a foundation for training a new net on a set of 20 different character classes from the NIST Special Database 19. The new net would keep the bottom layers of the old net (i.e. those nearest to the input) and only allow the top layers to train on the new character classes. We purposely used small training sets for the new net to force it to rely as much as possible upon transferred knowledge as opposed to a large and varied training set to learn the new set of hand written characters. Our results show a clear advantage in relying upon transferred knowledge to learn new tasks when given small training sets, if the new tasks are sufficiently similar to the previously mastered one. However, this advantage decreases as training sets increase in size.