Dynamic Hand Gesture Recognition for Mobile Systems Using Deep LSTM

Human–computer communication using hand gestures has always been difficult. More than half a century ago, people used differentways of interactionwith computers from the early mediums such as perforated game cards. Nowadays, if a richer lexicon of gestures is given, people can communicate more effectively with computers. Machine learning is now used to recognize and classify hand gestures in amore preciseway. In order to increase the communication between computers and humans, we proposed a technique, which uses a wearable low-cost device to generate the electrical impedance tomography (EIT) images to recover the inner impedance structure of a user’s wrist. This is done by measuring the transverse impedance between all the 16 pairs of electrodesofwrist band that lie on the skin of the user hand. The proposedtechnique is enough to integrate the technology into the prototypewrist band tomonitor and classify gestures in real time. We have conducted a study of 16 gestures with a focus on gross hand and pinch finger gestures. The results evaluation shows that the gross hand gestures achieved 90% accuracy inwrist position, while pinch gestures achieved 93% accuracy.

The sign language digits based on hand gestures have been utilized in various applications such as human-computer interaction, robotics, health and medical systems, health assistive technologies, automotive user interfaces, crisis management and disaster relief, entertainment, and contactless communication in smart devices. The color and depth cameras are commonly deployed for hand gesture recognition, but the robust classification of hand gestures under varying illumination is still a challenging task. This work presents the design and deployment of a complete end-to-end edge computing system that can accurately provide the classification of hand gestures captured from thermal images. A thermal dataset of 3200 images was created with each sign language digit having 320 thermal images. The solution presented here utilizes live images taken from a low-resolution thermal camera of 32×32 pixels, feeding into a novel light weight deep learning model based on bottleneck motivated from deep residual learning for classification of hand gestures. The edge computing system presented here utilizes Raspberry pi with a thermal camera making it highly portable. The designed system achieves an accuracy of 99.52% on the test data set with an added advantage of accuracy being invariable to background lighting conditions as it is based on thermal imaging.

Classification of hand gesture is crucial for the development of hand gesture based system for human machine interaction. Gesture recognition system consists of hand gesture acquisition, segmentation, morphological filtering, contour extraction, and classification. This paper aims at classification of hand gesture as a similarity measure using Dynamic Time Warping and Piecewise Dynamic Time Warping. Experiments and evaluation on a subset of American Sign Language (ASL) hand gesture show that, by using Dynamic Time Warping hand gesture can be classified. Additionally, it is also estimated that Piecewise DTW can be efficiently used to speed up the computations of DTW.

ABSTRACTProviding input for the selection of objects in virtual, augmented, and mixed-mode reality can be done with hand-held controllers or hand gestures depending on the complexity and precision required. Free hand gestures have the advantages of eliminating the need for a controller, not needing to see the controls on the controller, and potentially being less fatiguing. However, the designs of hand gestures for object selection have not been thoroughly evaluated for usability and performance. Eighteen participants evaluated four different ray-casting hand gestures (index thrust, index click, palm thrust, and palm click) and three snapback thresholds while selecting 2D targets of different sizes. Dependent variables were mean time to select targets, number of selections not completed, number of incorrect targets selected, and subjective preference. The index thrust and index click gestures were preferred by subjects and had faster mean selection times and lower number of incorrect target selections. There were no significant differences between snapback thresholds on productivity, error, or preference. Overall, the index thrust and index click gestures were associated with greater user confidence, particularly with smaller targets.

Conventional pattern-recognition algorithms for surface electromyography (sEMG)-based hand-gesture classification have difficulties in capturing the complexity and variability of sEMG. The deep structures of deep learning enable the method to learn high-level features of data to improve both accuracy and robustness of a classification. However, the features learned through deep learning are incomprehensible, and this issue has precluded the use of deep learning in clinical applications where model comprehension is required. In this paper, a generative flow model (GFM), which is a recent flourishing branch of deep learning, is used with a SoftMax classifier for hand-gesture classification. The proposed approach achieves accuracy in classifying 53 different hand gestures from the NinaPro database 5. The distribution of all 53 hand gestures is modelled by the GFM, and each dimension of the feature learned by the GFM is comprehensible using the reverse flow of the GFM. Moreover, the feature appears to be related to muscle synergy to some extent.

In this paper, a multimodal hand gesture detection and recognition system using differential Pyroelectric Infrared (PIR) sensors and a regular camera is described. Any movement within the viewing range of the differential PIR sensors are first detected by the sensors and then checked if it is due to a hand gesture or not by video analysis. If the movement is due to a hand, one-dimensional continuous-time signals extracted from the PIR sensors are used to classify/recognize the hand movements in real-time. Classification of different hand gestures by using the differential PIR sensors is carried out by a new winner-take-all (WTA) hash based recognition method. Jaccard distance is used to compare the WTA hash codes extracted from 1-D differential infrared sensor signals. It is experimentally shown that the multimodal system achieves higher recognition rates than the system based on only the on/off decisions of the analog circuitry of the PIR sensors(1).

In this research paper, an effort has been placed to convert 24 American Sign Language (ASL) signer independent, real time hand gesture alphabets into human or machine recognizable English text. In the proposed work, the ASL hand gestures used for cognition and recognition process is completely invariant to scale, luminance, gender, and distance in the complex background of indoor location. The Viola-Jones algorithm, CIE Lab color model and canny approximation to the derivative are used for proper hand segmentation. In both the cognition and recognition process, the various features such as boundary, centroid, entropy, Hu moments, Zernike moments, Gabor filters, Histogram of Oriented Gradients (HOG) and Local Phase Quantization (LPQ) are extracted from the hand gestures. The K-Nearest Neighbor (KNN), Multiclass-Support Vector Machines (M-SVM) and Decision Tree (DT) classifiers are used for classifying the hand gestures. In recognition task, these classifiers are applied independently on the same set of hand gestures to check the optimality of recognition rate and recognition time. With the detailed experimentation, it is found that, the KNN classifier achieved an average recognition rate and average recognition time of 92.71% and 0.48 s per gestures. This recognition rate and time is better and optimal compared to M-SVM and DT classifiers. Also it is an inspiring result compared to state of art techniques in real time environment by considering various invariants.

Human computer interaction is becoming more integrated in daily life with the proliferation of mobile devices and virtual reality technology. Hand gesture recognition is a potentially promising mechanism to facilitate human computer interaction, however wrist-mounted surface electromyography (sEMG) hand gesture classification is particularly challenging given the relatively small sEMG signals as compared to traditional forearm-based sEMG sensing. This paper introduces the development of a wristband for detecting eight air gestures and four surface gestures at two different force levels through sEMG and inertial measurement unit (IMU) sensor fusion. To validate the wrist-worn device, ten healthy subjects performed hand gesture recognition experiments resulting in a total average recognition rate of 92.6% for air gestures and 88.8% for surface gestures. This paper demonstrates the potential of wrist-worn devices for accurate hand gesture recognition applications.

Remote controls are commonly used today as an interface for almost all consumer electronic equipment. Different home appliances use a multitude of remote controllers, offering a variety of problems: confusion in the different button layout, spending time in looking for the right remote, problems in replacement when destroyed and so on. Hand gestures have been one of the oldest and natural methods of human interaction. It is considered as an interaction medium that can possibly deliver more intuitive, creative and natural method for communication between human and a machine. The researchers choose to create an android based hand gesture controlled interface system for home appliances that serves as an alternative to existing remote controllers. The camera of the android device will be used to detect static hand gestures for processing. The gathered data will be compared to a database of recognizable static hand gestures to execute commands on certain home appliances using infrared light. After series of testing and evaluation, the system is able to execute control functions to the home appliances indicated by specific hand gestures. The system required correct execution of the corresponding static gesture to properly execute the command functions of the appliances with proper lighting and illuminance for proper recognition else it would result into loss of accuracy and false detection. For the system to be able to communicate or receive information the transmission of the sender (IR) should be in line of sight to the receivers (IR) with six (6) meters range between the appliances and the mobile phone. A 100% success rate of gesture recognition was observed in Samsung Galaxy S4 with 2.26451 seconds recognition time while there were only three failures of hand gesture detection using Samsung Galaxy Note 3 with 2.6991 seconds recognition time.

This article presents a robotic visual system that allows effective recognition of multiple-angle hand gestures in finger guessing games. We shot images of hand gestures at various angles to train support vector machine (SVM) for the construction of a multiple-angle hand gesture recognition system. Our experimental results show that the hand gesture recognition system presented by this article can effectively recognize hand gestures, at over 95%, of different angles and sizes, with different ornaments, and of different skin colors, while recognizing right or left hand gestures, at up to 90% recognition rate.

Hand gesture for communication has proven effective for humans, and active research is ongoing in replicating the same success in computer vision systems. Human–computer interaction can be significantly improved from advances in systems that are capable of recognizing different hand gestures. In contrast to many earlier works, which consider the recognition of significantly differentiable hand gestures, and therefore often selecting a few gestures from the American Sign Language (ASL) for recognition, we propose applying deep learning to the problem of hand gesture recognition for the whole 24 hand gestures obtained from the Thomas Moeslund’s gesture recognition database. We show that more biologically inspired and deep neural networks such as convolutional neural network and stacked denoising autoencoder are capable of learning the complex hand gesture classification task with lower error rates. The considered networks are trained and tested on data obtained from the above-mentioned public database; results comparison is then made against earlier works in which only small subsets of the ASL hand gestures are considered for recognition.

Recent progress in 3-D, immersive display and virtual reality (VR) technologies has made possible many exciting applications, for example interactive visualization of complex scientific data. To fully exploit this potential there is a need for "natural" interfaces that allow the manipulation of such displays without cumbersome attachments. In this paper we describe the use of visual hand gesture analysis and speech recognition for developing a speech/gesture interface for controlling a 3-D display. The interface enhances an existing application, VMD, which is a VR visual computing environment far molecular biologists. The free hand gestures are used for manipulating the 3-D graphical display together with a set of speech commands. We describe the visual gesture analysis and the speech analysis techniques used in developing this interface. The dual modality of speech/gesture is found to greatly aid the interaction capability.

Hand gesture classification is a key step of gesture based human-computer interaction. The hand gesture dataset with depth information is increasing faster which brings challenges for deep neural network-based hand gesture classification model, such as data loading of big gesture data, optimization of model and visualization of training stage. In this paper, an optimization method of deep neural network-based hand gesture classification model for large-scale dataset is proposed, which improves the efficiency of data loading and makes the model have better time complexity and space complexity. Experimental results show that the proposed optimization method can effectively improve the training efficiency of classification model for large gesture data sets, and make classification models easy to train in a unified framework, which provides a good foundation for the research of hand gesture classification model.

Human hands can perform far more gestures than the number of muscles controlling them, as most gestures result from coordinated combinations of muscle activations and relaxations. This complexity poses a key challenge for human-machine interfaces performing gesture classification based on electromyography (EMG). Rather than identifying all conceivable gestures, it may be simpler to instead identify the activity of the individual muscles which generate a variety of complicated gestures. Here we suggest a three-dimensional model with volume representations of individual digit extensor muscles, averaged across multiple individuals, and evaluate its application and performance in hand gesture classification. Time-domain peaks in high-density surface EMG data from different hand gestures were extracted and localized within the model, from which a gesture classification scheme was generated for both single and multi-label cases. The model was created and tested on a publicly available dataset with 19 participants, leveraging a leave-one-out approach to assess inter-subject generalizability, and multi-label data to assess generalizability to gestures not included in the creation of the model. For single-label classification performance, true positive rates were between 61.9 and 95.1%, with false positive rates between 0 and 24.1%, for different single-digit extensions. The multi-label test demonstrated some degree of generalizability in identifying completely new gesture compositions, while simultaneously maintaining the leave-one-out approach for inter-subject generalizability. A model generated with this approach could be used for gesture classification by anyone, without individual modelling data, with the potential to generalize to any number of gesture compositions.

Recognizing and distinguishing the behavior and gesture of a user has become important owing to an increase in the use of wearable devices, such as a smartwatch. This study aims to propose a method for classifying hand gestures by creating sound in the non-audible frequency range using a smartphone and reflected signal. The proposed method converts the sound data, which has been reflected and recorded, into an image within a short time using short-time Fourier transform, and the obtained data are applied to a convolutional neural network (CNN) model to classify hand gestures. The results showed classification accuracy for 8 hand gestures with an average of 87.75%. Additionally, it is confirmed that the suggested method has a higher classification accuracy than other machine learning classification algorithms.