Sign recognition using key frame selection

Hand gesture recognition for human-computer interaction has become very popular in recent times. The main problem of this technology is how the system can recognize the presence of a gesture in a streaming video. In this paper, we propose a model that can recognize hand motion gesture in avideo stream using Support Vector Machine and Hidden-Markov model. Support Vector Machine has the advantage of generalizing classification. On the other hand, Hidden Markov Model is a statistical model that is capable of modeling Spatial-temporal time series. This system is divided into two main processes. First, this system recognizes hand posture using SMV (static gesture recognition) and generates sequence observation which is used for the second process later. The second process is recognizing dynamic hand gesture with the sequence observation from the static gesture. The implementation shows that static hand gesture recognition achieves average accuracy at 91% using testing dataset. Meanwhile,dynamically isolated hand gesture recognition gets average accuracy at 89% using testing dataset. We also have tested the system with continuous dynamic gesture using video stream. The system can recognize the gesture very well with accuracy of 83%. This achievement shows that the model can be used in human computer-interaction with specific supports such as Vector Machine and Hidden Markov Model parameter.

In this paper, septic safe methods of interaction with smart glasses, due to the health care environment applications consideration, are presented. The main focus is on capabilities of an optical, proximity-based gesture sensor and eye-tracker input systems. The design of both interfaces is being adapted to the open smart glasses platform that is being developed under the eGlasses project. Preliminary results obtained from the proximity sensor show that the recognition of different static and dynamic hand gestures is promising. The experiments performed for the eye-tracker module shown the possibility of interaction with simple Graphical User Interface provided by the near-to-eye display. Research leads to the conclusion of attractiveness of collaborative interfaces for interaction with smart glasses.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

Addressing the challenges of large-scale gesture recognition models, high computational complexity, and inefficient deployment on embedded devices, this study designs and implements a visual dynamic gesture recognition system based on a lightweight CNN-LSTM-DSA model. The system captures user hand images via a camera, extracts 21 keypoint 3D coordinates using MediaPipe, and employs a lightweight hybrid model to perform spatial and temporal feature modeling on keypoint sequences, achieving high-precision recognition of complex dynamic gestures. In static gesture recognition, the system determines the gesture state through joint angle calculation and a sliding window smoothing algorithm, ensuring smooth mapping of the servo motor angles and stability of the robotic hand's movements. In dynamic gesture recognition, the system models the key point time series based on the CNN-LSTM-DSA hybrid model, enabling accurate classification and reproduction of gesture actions. Experimental results show that the proposed system demonstrates good robustness under various lighting and background conditions, with a static gesture recognition accuracy of up to 96%, dynamic gesture recognition accuracy of 90.19%, and an overall response delay of less than 300 ms.

Hand gesture identification is one of problems being widely studied. There are two research trends corresponding to two data types, which are static and dynamic gestures. The static gesture is recognized based on the hand shape, while motion is the main feature in identifying dynamic gestures. In this paper, we propose an approach for modeling the dynamic hand gestures based on a combination of two mentioned information. At first, the hand silhouette is extracted using a skin-color filter. A sequence of geometric manipulations is then performed to remove the possible arm. The characteristics which describe the hand shape and motion orientation are estimated. Finally, the k-means clustering technique is combined with hidden Markov model to model each dynamic gesture. The experiments are performed on human-computer interaction dataset and obtain high efficiency.

With the rapid development of human-computer interaction technology, gesture recognition becomes one of the key technologies of human-computer interaction. In this paper, we propose a new method of dynamic hand gestures recognition. The method adopts the hierarchical identification model for dynamic hand gestures recognition. First, we combine frame fusion with density distribution features for rough gesture recognition, second, we use the hausdorff distance or fingertip detection for accurate gesture recognition. The main innovation of this method lies in that we change the way of dynamic gestures recognition into the recognition of static image, improves the efficiency of gesture recognition effectively. Experimental results showed that our recognition rate is above 90%.

This paper describes a hidden Markov model-based static and dynamic 3D gesture recognition system. The shape and position of the hands, segmented and tracked using a novel 3D statistical model for the upper body in stereo sequences, are used as observation vectors. The upper body model allows for accurate 3D localization of the hands in the presence of partial occlusions, self occlusions and different illumination conditions. The accuracy of our approach is reflected by the performance of our 3D gesture based editing system, that reaches 96% over 12 dynamic gestures and four static gestures.

Dynamic gesture recognition systems face persistent challenges in achieving real-time performance and high recognition efficiency. This paper presents a novel framework integrating computer vision techniques with machine learning algorithms to address these issues. The approach utilizes OpenCV for dynamic gesture detection, employing gesture contour extraction using skin color feature segmentation, fingertip detection for static gesture recognition, and dynamic gesture tracking through Hidden Markov Models (HMM). Experimental results demonstrate high recognition accuracy of 95.8 ± 1.4% across various gesture types, with individual rates ranging from 95.2 ± 1.6% to 98.5 ± 0.8%. Real-time performance is achieved with average processing time of 82.71 ± 3.2 ms per frame (12.1 FPS). Our method demonstrates 1.15× to 1.49× speed improvements over state-of-the-art approaches while maintaining superior accuracy. Validation on the DHG-14/28 public benchmark confirms generalizability with 93.4 ± 1.6% accuracy. The framework shows robust performance under challenging conditions: low-light environments (92.4 ± 2.1%), fast motion (91.8 ± 1.9%), and complex backgrounds (93.2 ± 1.7%). Statistical significance was confirmed through comprehensive evaluation across diverse demographic groups. This research has implications for human-computer interaction applications, including smart home systems, augmented reality, and industrial automation.

Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem. In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper. According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform. There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting. So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition.

Traditional bulky and complex control devices such as remote control and ground station cannot meet the requirement of fast and flexible control of unmanned aerial vehicles (UAVs) in complex environments. Therefore, a data glove based on multi-sensor fusion is designed in this paper. In order to achieve the goal of gesture control of UAVs, the method can accurately recognize various gestures and convert them into corresponding UAV control commands. First, the wireless data glove fuses flexible fiber optic sensors and inertial sensors to construct a gesture dataset. Then, the trained neural network model is deployed to the STM32 microcontroller-based data glove for real-time gesture recognition, in which the convolutional neural network-Attention mechanism (CNN-Attention) network is used for static gesture recognition, and the convolutional neural network-bidirectional long and short-term memory (CNN-Bi-LSTM) network is used for dynamic gesture recognition. Finally, the gestures are converted into control commands and sent to the vehicle terminal to control the UAV. Through the UAV simulation test on the simulation platform, the average recognition accuracy of 32 static gestures reaches 99.7%, and the average recognition accuracy of 13 dynamic gestures reaches 99.9%, which indicates that the system’s gesture recognition effect is perfect. The task test in the scene constructed in the real environment shows that the UAV can respond to the gestures quickly, and the method proposed in this paper can realize the real-time stable control of the UAV on the terminal side.

In this paper, we tackle advantages of cyclical movement patterns of hand gestures. The cyclical patterns are defined as closed-form which hand moves away from a rest position, follows one or more of a series of the movement of hand shapes and returns to its rest position. Due to the cyclical pattern characteristic, phase of gestures are supportive cues for deploying robust recognition schemes. We conduct a spatial-temporal representation of the hand gestures which takes into account both hand shapes and its movements during a gesture. The phase alignment then is deployed in the conducted space. The proposed scheme ensures inter-period phase continuity as well as normalizes length of the hand gestures. Three different datasets of dynamic hand gestures consisting of non-cyclical and cyclical patterns are examined. Evaluation results confirm that the best accuracy rate achieves at 96% for cyclical pattern that is significantly higher than results for typical gestures. The proposed method suggests a feasible and robust solution addressing technical issues in developing human-computer interaction applications such as using hand gestures to control home appliance devices.

Deaf and hard of hearing people communicate with others and within their own communities by using sign language. Computer recognition of sign language begins with the learning of sign gestures and continues through the production of text and speech. There are two types of sign gestures: static and dynamic. Both gesture recognition systems are crucial to the human community, even if static gesture recognition is easier than dynamic gesture recognition. We have conducted research on the steps required to convert static American sign language (ASL) to readable text and selected the best available methods to do so. Examined general steps are the data collection, pre-processing, transformation, feature extraction, and classification. There are also some recommendations for further study in this field.

Airwriting recognition is the task of identifying letters written in free space with finger movement. It is a dynamic gesture recognition with the vocabulary of gestures corresponding to letters in a given language. Electromyography (EMG) is a technique used to record electrical activity during muscle contraction and relaxation as a result of movement and is widely used for gesture recognition. Most of the current research in gesture recognition is focused on identifying static gestures. However, dynamic gestures are natural and user-friendly for being used as alternate input methods in human–computer interaction (HCI) applications. Airwriting recognition using EMG signals recorded from forearm muscles is, therefore, a viable solution. Since the user does not need to learn any new gestures and a large range of words can be formed by concatenating these letters, it is generalizable to a wider population. There has been limited work in recognition of airwriting using EMG signals and forms the core idea of the current work. The SurfMyoAiR dataset comprising of EMG signals recorded during writing English uppercase alphabets is constructed. Several different time-domain features to construct EMG envelope and two different time–frequency image representations: short-time Fourier transform and continuous wavelet transform were explored to form the input to a deep learning model for airwriting recognition. Several different deep learning architectures were exploited for this task. In addition, the effect of various parameters, such as signal length, window length, and interpolation techniques on the recognition performance, is comprehensively explored. The best-achieved accuracy was 78.50% and 62.19% in user-dependent and user-independent scenarios, respectively, by using short-time Fourier transform in conjunction with a 2-D convolutional neural network (CNN)-based classifier. Airwriting has great potential as a user-friendly modality to be used as an alternate input method in HCI applications.

Electrical impedance tomography (EIT) has been applied in the field of human-computer interaction due to its advantages including the fact that it is non-invasive and has both low power consumption and a low cost. Previous work has focused on static gesture recognition based on EIT. Compared with static gestures, dynamic gestures are more informative and can achieve more functions in human-machine collaboration. In order to verify the feasibility of dynamic gesture recognition based on EIT, a traditional excitation drive pattern is optimized in this paper. The drive pattern of the fixed excitation electrode is tested for the first time to simplify the measurement process of the dynamic gesture. To improve the recognition accuracy of the dynamic gestures, a dual-channel feature extraction network combining a convolutional neural network (CNN) and gated recurrent unit (GRU), namely CG-SVM, is proposed. The new center distance loss is designed in order to simultaneously supervise the intra-class distance and inter-class distance. As a result, the discriminability of the confusing data is improved. With the new excitation drive pattern and classification network, the recognition accuracy of different interference data has increased by 2.7~14.2%. The new method has stronger robustness, and realizes the dynamic gesture recognition based on EIT for the first time.

An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition