Deep Learning for Gesture Recognition based on Surface EMG Data

Gesture recognition pertains to recognizing meaningful expressions of motion by human, it is utmost important in medical rehabilitation, robot control as well as prosthesis design. Compared with gesture recognition based on machine vision, the gesture recognition based on wearable device, especially wearable surface electromyogram (sEMG) signal acquisition equipment, has more important theoretical and practical application prospects. However, there are still many urgent problems in sEMG signals, involving the signal acquisition and recognition accuracy of multi-channel sEMG signals, to be solved. For these problems, we designed a wearable sEMG armband with convenient acquisition and high precision to record sEMG signals and then done the gesture recognition based on deep learning method. Firstly, sEMG signals are classified, denoised and extracted features, and then extended data by sliding window. Then, Convolutional Neural Networks (CNN) and Multilayer Perceptron (MLP) were constructed to classify the 9 predefined gestures. The result showed that both methods achieve high offline recognition rate. The average gesture recognition accuracy of CNN is 99.47%; The average gesture recognition accuracy of MLP is 98.42%.KeywordsSurface EMGHand gesture recognitionCNNMLP

Recently, deep learning has become a promising technique for constructing gesture recognition classifiers from surface electromyography (sEMG) signals in human–computer interaction. In this paper, we propose a gesture recognition method with sEMG signals based on a deep multi-parallel convolutional neural network (CNN), which solves the problem that traditional machine learning methods may lose too much useful information during feature extraction. CNNs provide an efficient way to constrain the complexity of feedforward neural networks by weight sharing and restriction to local connections. Sophisticated feature extraction is to be avoided and hand gestures are to be classified directly. A multi-parallel and multi-convolution layer convolution structure is proposed to classify hand gestures. Experiment results show that in comparison with five traditional machine learning methods, the proposed method could achieve higher accuracy.

The surface electromyography (sEMG) signal, as a type of bioelectrical signal, has been widely applied in modern human-computer interaction, especially for gesture recognition. The rapid advancement of deep learning has significantly promoted the development of sEMG-based gesture recognition technology. However, existing studies often face challenges such as insufficient feature extraction from sEMG signals and low differentiation between similar gestures. To address these issues, this study proposes a novel dual-branch model architecture specifically designed for sparse-channel sEMG gesture recognition. The model leverages the strengths of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory-Transformer (BiT) networks to process both the time-frequency representations and raw signals of sEMG data, thoroughly extracting spatiotemporal features. Additionally, the proposed Hybrid Attention Block (HAB) further enhances the feature representation capability of the CNN branch. To verify the model's effectiveness, multiple experiments were conducted on the NinaPro-DB1 dataset. The results demonstrate that the proposed model achieved a classification accuracy of 89.23%, outperforming most mainstream models.

Gesture recognition based on surface electromyography (sEMG) has been widely used in the field of human-machine interaction (HMI). However, sEMG has limitations, such as low signal-to-noise ratio and insensitivity to fine finger movements, so we consider adding A-mode ultrasound (AUS) to enhance the recognition impact. To explore the influence of multisource sensing data on gesture recognition and better integrate the features of different modules. We proposed a multimodal multilevel converged attention network (MMCANet) model for multisource signals composed of sEMG and AUS. The proposed model extracts the hidden features of the AUS signal with a convolutional neural network (CNN). Meanwhile, a CNN-LSTM (long-short memory network) hybrid structure extracts some spatial-temporal features from the sEMG signal. Then, two types of CNN features from AUS and sEMG are spliced and transmitted to a transformer encoder to fuse the information and interact with sEMG features to produce hybrid features. Finally, the classification results are output employing fully connected layers. Attention mechanisms are used to adjust the weights of feature channels. We compared MMCANet's feature extraction and classification performance with that of manually extracted sEMG-AUS features using four traditional machine-learning (ML) algorithms. The recognition accuracy increased by at least 5.15%. In addition, we tried deep learning (DL) methods with CNN on single modals. The experimental results showed that the proposed model improved 14.31% and 3.80% over the CNN method with single sEMG and AUS, respectively. Compared with some state-of-the-art fusion techniques, our method also achieved better results.

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

In human-machine interaction systems, continuous movement estimation methods occupy an important position because they are more natural and intuitive than pattern-recognition methods. Essentially, arm position is decided by the shoulder and elbow joint angles. However, the various deformations of muscles around the shoulder and elbow often lead to difficulties in sensor fixation, which results in a loss of synchronization between the surface electromyography (sEMG) signals and joint angles. In order to accurately estimate movement angles using sEMG in situations where the sEMG is not synchronized with joint angles, we utilized a bi-directional long short-term memory (Bi-LSTM) network rather than other deep learning methods to estimate non-dominant arm movements, based on the sEMG signal from the dominant arm. This estimation protocol was designed to avoid a multiplicity of sensors and to simulate more complicated loss of synchronization problems). The performance of the Bi-LSTM was compared with multilayer perceptrons (MLPs), convolutional neural networks (CNNs), and a long short-term memory network (LSTM). The Pearson correlation coefficient (cc) between the estimated and target joint angle sequences was calculated to evaluate the performance of each neural network. The Wilcoxon signed-rank results showed that the Bi-LSTM model significantly outperformed the MLP, CNN, and LSTM models (tested with completely untrained newly recorded free movements).

Surface electromyography (sEMG) signals reflect the local electrical activity of muscle fibers and the synergistic action of the overall muscle group, making them useful for gesture control of myoelectric manipulators. In recent years, deep learning methods have increasingly been applied to sEMG gesture recognition due to their powerful automatic feature extraction capabilities. sEMG signals contain rich local details and global patterns, but single-scale convolutional networks are limited in their ability to capture both comprehensively, which restricts model performance. This paper proposes a deep learning model based on multi-scale feature fusion-MS-CLSTM (MS Block-ResCBAM-Bi-LSTM). The MS Block extracts local details, global patterns, and inter-channel correlations in sEMG signals using convolutional kernels of different scales. The ResCBAM, which integrates CBAM and Simple-ResNet, enhances attention to key gesture information while alleviating overfitting issues common in small-sample datasets. Experimental results demonstrate that the MS-CLSTM model achieves recognition accuracies of 86.66% and 83.27% on the Ninapro DB2 and DB4 datasets, respectively, and the accuracy can reach 89% in real-time myoelectric manipulator gesture prediction experiments. The proposed model exhibits superior performance in sEMG gesture recognition tasks, offering an effective solution for applications in prosthetic hand control, robotic control, and other human-computer interaction fields.

Our purpose is to clear whether the interferential current (IFC) method is useful for information transmission of a myoelectric prosthetic hand to the user. IFC stimulation is composed of two AC output waves. In previously, we demonstrated that IFC method using the two waves with different frequencies could vary the intensity of perceived stimulus. In the present study, we evaluated that the relationship between a movement of position of perceived stimulus and IFC method, using two waves with the same median frequency and a different phase. Moreover, we investigated the effects of a distance of recording electrodes from stimulating electrodes on a degree of interference of IFC with EMG signals. The movement of position of perceived stimulus was caused by the phase difference changing. IFC interfered with EMG signals. Especially, the interference of IFC with EMG of the muscle under stimulating electrodes was large. In contrast, when recording electrodes was distant from the stimulating electrodes, the interference of IFC was small. Moreover, the interference was eliminated by the low-pass filter (< 500Hz). These results support the usefulness of a combination of the IFC method with EMG recording system equipped with the low-pass filter as a sensory feedback system for myoelectric prosthetic hand.

SummaryFor the problem of surface electromyography (sEMG) gesture recognition, considering the fact that the traditional machine learning model is susceptible to the sEMG feature extraction method, it is difficult to distinguish the subtle differences between similar gestures. The NinaPro DB1 dataset is used as the research object, and the sEMG feature image and the Convolutional Neural Network (CNN) are combined to recognize 52 gesture movements. The CNN model effectively solves the limitations of traditional machine learning in sEMG gesture recognition, and combines 1‐dim convolution kernel to extract deep abstract features to improve the recognition effect. Finally, the simulation experiment shows that compared with the accuracy of the raw‐sEMG images based on the CNN and the sEMG‐feature‐images based on the CNN and sEMG based on the traditional machine learning, the multi‐sEMG‐features image based on the CNN is the highest, which coming up to 82.54%.

Prosthetic hands help patients gain greater confidence in daily life, and enabling automatic control of prosthetic hands using real biosignals is an essential task. This paper proposes a prosthetic hand control system based on surface electromyography (sEMG) signals. In particular, the sEMG signals are collected in real time corresponding to each hand gesture and pre-processed to eliminate noise components. A deep learning network is then employed to recognize hand gestures from the pre-processed sEMG signals. Recognized gestures are subsequently used to control the prosthetic hand to perform the corresponding movements. The proposed deep learning network achieves a recognition accuracy of 98.15 % for hand gestures from sEMG signals. Furthermore, the experimental results demonstrate that the proposed system can control the prosthetic hand with an accuracy of up to 96.97 % and a variance of 3.2194 across multiple subjects. These results suggest that the proposed system holds great potential for real-time prosthetic hand control based on sEMG signals, thereby supporting patients in gaining confidence in social interactions.

This research compares classification accuracy obtained with the classical classification techniques and the presented convolutional neural network for the recognition of hand gestures used in robotic prostheses for transradial amputees using surface electromyography (sEMG) signals. The first two classifiers are the most used in the literature: support vector machines (SVM) and artificial neural networks (ANN). A new convolutional neural network (CNN) architecture based on the AtzoriNet network is proposed to assess performance according to amputation-related variables. The results show that convolutional neural networks with a very simple architecture can produce accurate results comparable to the average classical classification methods and The performance it is compared with other CNN proposed by other authors. The performance of the CNN is evaluated with different metrics, providing good results compared to those proposed by other authors in the literature.

PURPOSE: Electromagnetic (EM) kinematic tracking systems have gained popularity in recent years due to their ability to quickly capture 6 degree-of-freedom position data. EM systems work by sequentially creating orthogonally oriented magnetic fields in which the receivers locate themselves. The purpose of this investigation was to determine the impact of the EM signal on surface EMG data and to provide suggested methods for reducing any detrimental effects. EQUIPMENT / METHODS: EM signals are produced by EM kinematic tracking systems. The EM field-strength varies as a function of the distance from the transmitter. The frequency varies as a function of the selected kinematic sampling rate. These combine to produce a non-stationary signal that can contaminate EMG data. Raw EMG data was collected from the quadriceps and hamstring muscles on subjects performing a squatting task and a jumping task in the presence of an EM transmitter. Both tasks were performed with and without the transmitter on. Data were collected at 1000 Hz. ANALYSIS Data from all trials were qualitatively assessed for the appearance of EM contamination. These data were then processed to evaluate the impact of the EM signal on muscle timing analyses. Power spectrums of the data were computed using a Fast Fourier Transform (FFT) algorithm. This spectral information was used to create digital filters to remove the EM signal from the raw data. RESULTS The EM signal was present in the EMG data for all trials performed in the presence of an active EM transmitter. In many cases, this signal was strong enough to completely mask the EMG signal. The muscle timing analysis of contaminated data were unable to accurately quantify onsets. Digital filtering successfully removed the EM noise and provided muscle latencies that were comparable to those found from the analysis of the uncontaminated data. CONCLUSIONS Digital filtering is an effective way of removing EM contamination from EMG data. When using this method, care should be taken to collect raw EMG data at a sampling rate that is consistent with the digital filtering requirements. RELEVANCE Extraneous EM signals have historically been part of EMG data collection. Many commercial systems have built-in notch filters to contend with 60Hz interference. The advent of EM tracking systems produces a need to re-evaluate processing techniques for EMG data collected in these environments.

The abundant movement information carried by the surface electromyography (sEMG) signal can comprehensively reflect the muscle movement patterns on the surface of the human body, and the characteristics of physiological signals involving movement information have promoted the vigorous development in the field of human-computer interaction in machine learning. This paper takes subtle gestures as the research objects, and proposes a subtle gestures recognition system that uses two-channel sEMG signal and utilizes Quadratic Discriminant Analysis (QDA) and Linear Discriminant Analysis (LDA) which are parameter estimation classifiers based on probability density. First, combining the raw sEMG signal and the envelope sEMG signal, the number of channels for sensors and corresponding position of the muscle groups are preferably selected. Then, we propose to use the Pearson correlation coefficient to optimize the four types of features extracted in the time domain, frequency domain, time-frequency domain and AR model parameters, which effectively ensures the uniqueness of each gesture and eliminates individual differences. Finally, this paper compares four classifiers of QDA, LDA, Support Vector Machine (SVM), and K-Nearest Neighbor (KNN), and explores the pattern recognition classifier suitable for subtle actions. The experiment collects 9 kinds of subtle gestures from 8 subjects. The results show that the system can use only two channels of sEMG signal to recognize 9 kinds of subtle gestures which include multi-finger clicking or pinching or holding objects, and the most suitable classifiers QDA and LDA have the average recognition rates of 95.79% and 95.01%, respectively.

In the last years, surface electromyography (sEMG) has become a hot spot for research on signal classification methods due to its many applications with consumer grade sensors. Nonetheless, the correct classification of sEMG signals is not simple due to their stochastic nature and high variability. Our objective was to provide a comprehensive comparison between schemes used on the latest research, namely convolution neural networks (CNN) and hyperdimensional computing (HDC) using a public high-quality dataset. Our results showed that while CNN had substantially higher accuracy (68 vs. 32% for HDC, for 18 gestures), its shortcomings may be more prevalent in this area, as low amounts of training data, and lack of subject specific data can cause an accuracy drop of up to 70%/19% and 56%/7% for CNN/HDC, respectively. Our results point out that while HDC cannot reach CNN classification levels it is more versatile on small datasets and provides more adaptability.KeywordssEMGHand gesture classificationHyperdimensional computingConvolutional neural networksNinapro

Continuous estimation of joints based on surface electromyography (sEMG) signals has been widely used in smooth motion of robots. Currently, deep learning techniques have become the main means of continuous motion estimation. Although the deep learning technique can reduce the workload of manual features by feature learning, its learned features lack the temporal information of continuous joint activity and cannot provide an effective description of the continuous joint trajectory. Therefore, this paper proposes a method for continuous finger joint angle prediction based on time series feature fusion convolutional neural network (CNN), which first extracts feature information of sEMG signals by manual features, then constructs time series, then inputs to convolutional neural network to extract deep features, and finally estimates finger joint angle. The experimental results show that the CNN method based on time series feature fusion is significantly better than the methods of CNN and multilayer perceptron (MLP). It is proved that the method in this study can effectively extract the features of sEMG signals as well as better prediction performance, which provides a new idea for continuous motion estimation methods.