Highly Accurate Dynamic Gesture Recognition Method Based on Edge Intelligence

In recent years, gesture recognition and speech recognition, as important input methods in Human–Computer Interaction (HCI), have been widely used in the field of virtual reality. In particular, with the rapid development of deep learning, artificial intelligence, and other computer technologies, gesture recognition and speech recognition have achieved breakthrough research progress. The search platform used in this work is mainly the Google Academic and literature database Web of Science. According to the keywords related to HCI and deep learning, such as “intelligent HCI”, “speech recognition”, “gesture recognition”, and “natural language processing”, nearly 1000 studies were selected. Then, nearly 500 studies of research methods were selected and 100 studies were finally selected as the research content of this work after five years (2019–2022) of year screening. First, the current situation of the HCI intelligent system is analyzed, the realization of gesture interaction and voice interaction in HCI is summarized, and the advantages brought by deep learning are selected for research. Then, the core concepts of gesture interaction are introduced and the progress of gesture recognition and speech recognition interaction is analyzed. Furthermore, the representative applications of gesture recognition and speech recognition interaction are described. Finally, the current HCI in the direction of natural language processing is investigated. The results show that the combination of intelligent HCI and deep learning is deeply applied in gesture recognition, speech recognition, emotion recognition, and intelligent robot direction. A wide variety of recognition methods were proposed in related research fields and verified by experiments. Compared with interactive methods without deep learning, high recognition accuracy was achieved. In Human–Machine Interfaces (HMIs) with voice support, context plays an important role in improving user interfaces. Whether it is voice search, mobile communication, or children’s speech recognition, HCI combined with deep learning can maintain better robustness. The combination of convolutional neural networks and long short-term memory networks can greatly improve the accuracy and precision of action recognition. Therefore, in the future, the application field of HCI will involve more industries and greater prospects are expected.

Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem. In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper. According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform. There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting. So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition.

Human-computer interaction for virtual-real fusion

With the rapid development of human-computer interaction technology, gesture recognition becomes one of the key technologies of human-computer interaction. In this paper, we propose a new method of dynamic hand gestures recognition. The method adopts the hierarchical identification model for dynamic hand gestures recognition. First, we combine frame fusion with density distribution features for rough gesture recognition, second, we use the hausdorff distance or fingertip detection for accurate gesture recognition. The main innovation of this method lies in that we change the way of dynamic gestures recognition into the recognition of static image, improves the efficiency of gesture recognition effectively. Experimental results showed that our recognition rate is above 90%.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

An Augmented Reality System for Coronary Artery Diagnosis Planning and Training (ARS-CADPT) is designed and realized in this paper. As the characteristic of ARS-CADPT, the algorithms of static gesture recognition and dynamic gesture spotting and recognition are presented to realize the real-time and friendly Human-Computer Interaction (HCI). The experimental results show that, with the use of ARS-CADPT, the HCI is natural and fluent, which improves the user’s immersion and improves the diagnosis and training effects.

Dynamic gesture recognition systems face persistent challenges in achieving real-time performance and high recognition efficiency. This paper presents a novel framework integrating computer vision techniques with machine learning algorithms to address these issues. The approach utilizes OpenCV for dynamic gesture detection, employing gesture contour extraction using skin color feature segmentation, fingertip detection for static gesture recognition, and dynamic gesture tracking through Hidden Markov Models (HMM). Experimental results demonstrate high recognition accuracy of 95.8 ± 1.4% across various gesture types, with individual rates ranging from 95.2 ± 1.6% to 98.5 ± 0.8%. Real-time performance is achieved with average processing time of 82.71 ± 3.2 ms per frame (12.1 FPS). Our method demonstrates 1.15× to 1.49× speed improvements over state-of-the-art approaches while maintaining superior accuracy. Validation on the DHG-14/28 public benchmark confirms generalizability with 93.4 ± 1.6% accuracy. The framework shows robust performance under challenging conditions: low-light environments (92.4 ± 2.1%), fast motion (91.8 ± 1.9%), and complex backgrounds (93.2 ± 1.7%). Statistical significance was confirmed through comprehensive evaluation across diverse demographic groups. This research has implications for human-computer interaction applications, including smart home systems, augmented reality, and industrial automation.

Stationary hand gesture recognition can be applied in a variety of domains such as human-computer interaction (HCI), remote control, virtual reality etc. Hand gesture recognition is mainly the study of detection and recognition of various hand gestures like Sign Language hand gestures, by a computer. This work is focused on three main issues in developing a gesture recognition system. These are (i) Threshold (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. It is very demanding to segment hand regions from the static hand gesture color images, due to unreliable light conditions and composite background. Since skin pixels can vary with different explanation condition, to find the range of skin pixels, becomes a hard task in case of color space based skin color segmentation. These methods propose a learning algorithm based on customized clustering expanse to remove person skin color regions from the stationary hand gesture color descriptions. To improve the performance of classification, two feature synthesis strategies have been proposed in this work.

With the constant development of smart devices, Gesture recognition is applied on more and more fields. Gesture recognition devices currently on the market are inconvenient and expensive. A gesture recognition method based on Kinect is proposed in this paper. The camera of Kinect is used to get gesture images and then a hidden Markov model is established to recognize dynamic gesture so that the operation is simple. The assessment, decoding algorithms and training issues of hidden Markov model in gesture recognition is focused in this paper. Viterbi algorithm and Baum-Welch algorithm are used to solve the problems above. After simulation experiments, it is discovered that the gesture recognition algorithm based on Kinect proposed in this paper has a high accuracy rate for a variety of gestures, simultaneously the naturalness of human-computer interaction has been greatly improved.

Recently, the new Human Computer Interactive (HCI) technologies such as gesture recognition have drawn extensive attention, which have been extended to such field as the interaction between human and machine. In this paper, the critical technology about the gesture recognition is discussed, a method to locate human hand in high speed is proposed, based on the combination of skincolour based Gauss model and the revised optical flow. Considering the characters of dynamic hand gesture, we adopt the Hidden Markov Model and use it in ‘the remote robot control system based on p2p’.

This article explores the application of YOLOv11 algorithm in gesture recognition field to evaluate its performance in human-computer interaction (HCI). By introducing the YOLOv8 model and corresponding dataset for training, we obtained the confusion matrix predicted by the model, which shows that the model can accurately recognize most gestures, although there are a few cases of misidentification. When the IoU threshold is 0.5, the average accuracy (mAP) of the model steadily improves with the progress of training, indicating that the overall performance of the model in gesture detection tasks has been enhanced. In addition, even under stricter evaluation conditions where the IoU threshold was increased from 0.5 to 0.95, mAP still showed an upward trend, although the growth rate was not as significant as mAP50, which still demonstrated the improvement in model performance. Through the detection of the test set images, we found that the YOLOv11 model can effectively recognize gestures and accurately interpret their meanings, demonstrating high accuracy. This study not only demonstrates the potential of YOLOv11 in gesture recognition tasks, but also provides a new technological path for the future development of HCI field. Overall, the YOLOv11 algorithm has demonstrated strong performance and accuracy in gesture recognition, providing a more natural and intuitive way for interaction between smart devices and humans.

TripCEAiR: A multi-loss minimization approach for surface EMG based airwriting recognition

Gesture is a natural form of human communication, and it is of great significance in human–computer interaction. In the dynamic gesture recognition method based on deep learning, the key is to obtain comprehensive gesture feature information. Aiming at the problem of inadequate extraction of spatiotemporal features or loss of feature information in current dynamic gesture recognition, a new gesture recognition architecture is proposed, which combines feature fusion network with variant convolutional long short‐term memory (ConvLSTM). The architecture extracts spatiotemporal feature information from local, global and deep aspects, and combines feature fusion to alleviate the loss of feature information. Firstly, local spatiotemporal feature information is extracted from video sequence by 3D residual network based on channel feature fusion. Then the authors use the variant ConvLSTM to learn the global spatiotemporal information of dynamic gesture, and introduce the attention mechanism to change the gate structure of ConvLSTM. Finally, a multi‐feature fusion depthwise separable network is used to learn higher‐level features including depth feature information. The proposed approach obtains very competitive performance on the Jester dataset with the classification accuracies of 95.59%, achieving state‐of‐the‐art performance with 99.65% accuracy on the SKIG (Sheffifield Kinect Gesture) dataset.

This paper deals with static and dynamic hand gesture (digits) recognition. The method provides a threefold novel contribution: (1) segmentation algorithm gives better results on any skin colour and any size of hand on complex and non-uniform background; (2) key frame finding algorithm and (3) the recognition technique of signs (static digits, alphabets and dynamic digits). We separate out key frames from a sequence of static gestures, which include correct gestures from a video sequence. The recognition efficiency of key frame detection is 93% using the proposed algorithm. The segmentation efficiency is almost 95%. Features are extracted using the proposed feature extraction algorithm, and gestures are recognised. We propose a novel algorithm for static and dynamic gesture recognition. The proposed algorithm shows recognition efficiency of 94.8% for static gestures and 94% for dynamic gestures.