GF ‐ YOLOv8 : A Lightweight Gesture Recognition Algorithm Based on Adaptive Feature Fusion Pyramid Network

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

This article explores the application of YOLOv11 algorithm in gesture recognition field to evaluate its performance in human-computer interaction (HCI). By introducing the YOLOv8 model and corresponding dataset for training, we obtained the confusion matrix predicted by the model, which shows that the model can accurately recognize most gestures, although there are a few cases of misidentification. When the IoU threshold is 0.5, the average accuracy (mAP) of the model steadily improves with the progress of training, indicating that the overall performance of the model in gesture detection tasks has been enhanced. In addition, even under stricter evaluation conditions where the IoU threshold was increased from 0.5 to 0.95, mAP still showed an upward trend, although the growth rate was not as significant as mAP50, which still demonstrated the improvement in model performance. Through the detection of the test set images, we found that the YOLOv11 model can effectively recognize gestures and accurately interpret their meanings, demonstrating high accuracy. This study not only demonstrates the potential of YOLOv11 in gesture recognition tasks, but also provides a new technological path for the future development of HCI field. Overall, the YOLOv11 algorithm has demonstrated strong performance and accuracy in gesture recognition, providing a more natural and intuitive way for interaction between smart devices and humans.

With the rapid advancement of computer vision and deep learning technologies, the accuracy and efficiency of real-time gesture recognition have significantly improved. This paper introduces a gesture-controlled robot system based on thermal imaging sensors. By replacing traditional physical button controls, this design significantly enhances the interactivity and operational convenience of human–machine interaction. First, a thermal imaging gesture dataset is collected using Python3.9. Compared to traditional RGB images, thermal imaging can better capture gesture details, especially in low-light conditions, thereby improving the robustness of gesture recognition. Subsequently, a neural network model is constructed and trained using Keras, and the model is then deployed to a microcontroller. This lightweight model design enables the gesture recognition system to operate on resource-constrained embedded devices, achieving real-time performance and high efficiency. In addition, using a standalone thermal sensor for gesture recognition avoids the complexity of multi-sensor fusion schemes, simplifies the system structure, reduces costs, and ensures real-time performance and stability. The final results demonstrate that the proposed design achieves a model test accuracy of 99.05%. In summary, through its gesture recognition capabilities—featuring high accuracy, low latency, non-contact interaction, and low-light adaptability—this design precisely meets the core demands for “convenient, safe, and natural interaction” in rehabilitation, smart homes, and elderly assistive devices, showcasing clear potential for practical scenario implementation.

Myoformer: sEMG missing signal recovery for gesture recognition based on multi-channel self-attention mechanism

Hand joints-based gesture recognition using a neural network provides excellent performance in hand gesture recognition. However, during the collection of sequential skeletal datasets, joints identification through hand pose estimations usually includes noise and even errors, which often diminish the accuracy of gesture recognition. To promote the availability of hand gesture recognition for such noisy datasets, this paper presents a nested interval unscented Kalman filter (UKF) with long short-term memory (NIUKF-LSTM) network to improve the accuracy of hand gesture recognition from noisy datasets. This nested interval method with the UKF changes the distribution of the sigma points based on two sampling intervals. By considering the information of previous frames, the nested interval method helps the NIUKF-LSTM network revise the noise in the sequential hand skeletal data and improve the recognition accuracy. The experimental results showing the removal of noisy skeletal data from the dynamic hand gesture dataset demonstrate the effectiveness of our NIUKF-LSTM network, which achieves better performance than do other state-of-the-art methods.

Purpose Gesture recognition based on biological signals played a crucial role in the field of human-computer interaction. Despite achieving decent results in improving gesture recognition accuracy using different algorithms, significant challenges remained in enhancing recognition accuracy for multimodal signals and special patients such as upper limb amputees. Methods This paper proposed a Broad Learning System (BLS) method to recognize hand movements. The input signals fused surface electromyographic (sEMG) and acceleration signals, and the fused data were mapped into contour maps to extract features, which were transformed into two-dimensional grayscale images to achieve efficient encoding. The paper utilized the public databases "Ninapro DB2 and DB7" to evaluate the performance of the proposed method. Results Experimental results showed that the average gesture recognition accuracy of BLS reached 97.73%, with an average testing time of 0.093s, outperforming the other two algorithms, K-Nearest Neighbor (0.497s) and Binary Tree (2.412s). Moreover, the fused signals exhibited higher recognition accuracy for amputees. Conclusion In conclusion, the accuracy and real-time of the method could satisfy the requirements for controlling the prosthetic. Hence, it provided a reliable approach for research in fields such as rehabilitative medicine and prosthetic control.

In complex road scenes, we propose an enhanced road target detection algorithm for YOLOv7-Tiny to address issues such as large model size, misclassification, and low localization accuracy. Our approach involves several key modifi-cations. Firstly, we replace the LeakyReLU activation function in YOLOv7-Tiny with H-swish. This replacement not only reduces the number of parameters in the model but also enhances its feature extraction capabilities. Additionally, we replace the ELAN module in the Neck with the DPCH-ELAN module and introduce the darknetblock module along with the pcconv convolution. These modifications improve the network's ability to comprehend complex patterns and semantics, thereby enabling it to capture features at different levels of input data. Moreover, we introduce the pcconv convolution building block at the output side to handle heterogeneous information in complex road scenes, thereby enhancing the network's performance in detecting road targets and abnormalities. In our experiments using a mul-ti-source dataset, the improved model exhibits a reduction in GFLOPs by 20.90% and a decrease in the number of pa-rameters by 24.49%. Furthermore, the mean average precision scores (map) at thresholds of 0.5 and 0.5~0.9 are im-proved to 77.3% and 51.8%, respectively, compared to the original YOLOv7-Tiny model. These experimental results demonstrate that our enhanced model achieves a reduction in model size while simultaneously enhancing detection accuracy, thereby meeting the requirements for real-time detection. To assess the generalizability of our approach, we conducted comparison experiments on the VOC2012 dataset. The results indicate that the improved algorithm exhibits robust generalization capabilities across different datasets.

In the field of unmanned aerial vehicle-based transmission line inspection, the neural network models currently employed suffer from large model sizes and high computational requirements, resulting in a trade-off between detection speed and accuracy. We propose an improved lightweight YOLOv5 object detection algorithm based on channel pruning to address this issue. The approach incorporates the SimAM attention mechanism into the backbone network to enhance feature extraction capability without increasing computational complexity, thereby improving detection accuracy. Traditional convolutional modules in the neck network are replaced with GSconv modules to reduce convolutional computation and enhance detection speed. The Wise-IoU loss is utilized as the localization loss for the detection model, speeding up model convergence. Finally, a model compression technique is implemented to eliminate unnecessary channels within the network, resulting in a decrease in model size and a boost in detection speed. Experimental results on a constructed grid inspection dataset demonstrate that the improved model achieves a 72.1% reduction in model size, a 62.5% reduction in GFLOPs, a 5.9 FPS improvement, and a 0.3% mAP increase compared to the original YOLOv5. These findings underscore the advantages of the improved lightweight YOLOv5 object detection algorithm, offering a blend of heightened accuracy, streamlined size, and swift detection capabilities.

Aiming at the problem that the accuracy of gesture recognition is not high due to the camera's shooting angle and depth information being affected by illumination, this paper proposes a gesture recognition method based on image processing and SIFT-SVM. First construct a gesture recognition data set, perform image processing such as denoising and segmentation on the collected pictures, then extract the geometric features of gestures through the Scale-invariant feature transform (SIFT) algorithm, and finally use the support vector machine (SVM) to classifies and recognizes different detected gestures. The simulation experiment results show that the accuracy of the method proposed in this paper reaches 99.89%, it has strong robustness, and the recognition speed is faster to meet the needs of general applications.

Aiming to provide feasible solutions for the realization of the robust and natural myoelectric control systems, a novel myoelectric control scheme supporting gesture recognition and muscle force estimation is proposed in this study. Eleven grasping gestures abstracted from daily life are selected as the target gesture set. The high-density surface electromyography (HD-sEMG) of the forearm flexor and the grasping force signal are collected simultaneously. The synchronous prediction of gesture category and instantaneous force is realized by the multi-task learning (MTL) technique. Especially, a post-processing algorithm based on threshold method is conducted to overcome the influence of force variation on the accuracy of gesture recognition. The experimental results show that the proposed post-processing method can decrease the classification error significantly. Specifically, the overall gesture classification error is reduced by 27 ~ 30 percent compared with not using the post-processing method; and 16 ~ 24 percent compared with using classical post-processing methods. The whole scheme can realize the synchronous gesture recognition and force estimation with 9.35 ± 11.48% gesture classification error and 0.1479 ± 0.0436 root-mean-square deviation force estimation accuracy. Meanwhile, it is feasible in different number of electrodes and well meets the real-time requirement of the EMG control system in response time delay (about 28.22 ~ 113.16ms on average). The proposed framework provides the possibility for myoelectric control supporting synchronous gesture recognition and force estimation, which can be extended and applied in the fields of myoelectric prosthesis and exoskeleton devices.

This paper presents the development of a real time American Sign Language (ASL) detection system using Raspberry Pi and a Sequential Convolutional Neural Network (CNN) model. The system aims to bridge the communication gap between the Deaf and Hard of Hearing (DHH) community and the hearing population by translating ASL gestures into text and audio outputs. The proposed system leverages the Raspberry Pi 5, a cost effective and scalable hardware platform, combined with deep learning techniques to achieve high accuracy in gesture recognition. The system utilizes the custom dataset for training and employs hand landmark detection using MediaPipe for real-time gesture analysis. The results demonstrate an 85% accuracy in recognizing ASL gestures, with real-time text and audio outputs. The system is designed for personal, educational, and public applications, offering a practical solution for enhancing communication accessibility for the DHH community. Key Words: American Sign Language(ASL), Raspberry Pi, Machine Learning, Gesture recognition, Sign Language Translation, Text-to-Speech conversion

We present a method for robust and accurate hand pose recognition from egocentric depth cameras. Our method combines CNN based hand pose estimation and joint locations based hand gesture recognition. In pose estimation stage, we use a hand geometry prior network to estimate the hand pose. In gesture recognition stage, we defined a hand language which based on a set of pre-define basic propositions, obtained by applying four predicate types to the fingers and palm states. The hand language is used to convert the estimated joint location to hand gesture. Our experimental results indicate that the method enables robust and accurate gesture recognition in self-occlusion environment.

This work proposes a framework to train and deploy neural network-based gesture recognition algorithms in wearable devices. The approach is demonstrated for a high-density electromyography (HD-EMG) gesture recognition system, where a Siamese convolution neural network (SCNN) learns to associate and dissociate muscle activity patterns from the same or distinct gesture classes. This optimizes learning in low-data environments such as gesture recognition and myoelectric control, where training data must be provided by the end user. Then, using a cosine similarity-based few-shot classifier and inter-session-intra-user transfer of the SCNN’s learning, the proposed model is intended to achieve state-of-the-art results in a framework that is realistically applicable in wearable devices. For an experimented myoelectric interface user, the proposed model achieved 86.49 % accuracy in 6-class gesture recognition, using inter-session-intra-user transfer learning by the SCNN. Few-shot learning demonstrates convergence to this accuracy within 20 shots with the cosine similarity classifier. Across a group of 13 able-bodied participants, 90.66 % and 97.34 % mean and median accuracy were obtained from the 6-way 20-shot learning approach with a majority vote (100 votes) over time-distributed inferences.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem. In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper. According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform. There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting. So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition.