A dual-branch network architecture for sEMG-based gesture recognition

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Gesture recognition pertains to recognizing meaningful expressions of motion by human, it is utmost important in medical rehabilitation, robot control as well as prosthesis design. Compared with gesture recognition based on machine vision, the gesture recognition based on wearable device, especially wearable surface electromyogram (sEMG) signal acquisition equipment, has more important theoretical and practical application prospects. However, there are still many urgent problems in sEMG signals, involving the signal acquisition and recognition accuracy of multi-channel sEMG signals, to be solved. For these problems, we designed a wearable sEMG armband with convenient acquisition and high precision to record sEMG signals and then done the gesture recognition based on deep learning method. Firstly, sEMG signals are classified, denoised and extracted features, and then extended data by sliding window. Then, Convolutional Neural Networks (CNN) and Multilayer Perceptron (MLP) were constructed to classify the 9 predefined gestures. The result showed that both methods achieve high offline recognition rate. The average gesture recognition accuracy of CNN is 99.47%; The average gesture recognition accuracy of MLP is 98.42%.KeywordsSurface EMGHand gesture recognitionCNNMLP

Recently, deep learning has become a promising technique for constructing gesture recognition classifiers from surface electromyography (sEMG) signals in human–computer interaction. In this paper, we propose a gesture recognition method with sEMG signals based on a deep multi-parallel convolutional neural network (CNN), which solves the problem that traditional machine learning methods may lose too much useful information during feature extraction. CNNs provide an efficient way to constrain the complexity of feedforward neural networks by weight sharing and restriction to local connections. Sophisticated feature extraction is to be avoided and hand gestures are to be classified directly. A multi-parallel and multi-convolution layer convolution structure is proposed to classify hand gestures. Experiment results show that in comparison with five traditional machine learning methods, the proposed method could achieve higher accuracy.

Single-channel surface electromyography signal classification with variational mode decomposition and entropy feature for lower limb movements recognition

Surface electromyography (sEMG) signals reflect the local electrical activity of muscle fibers and the synergistic action of the overall muscle group, making them useful for gesture control of myoelectric manipulators. In recent years, deep learning methods have increasingly been applied to sEMG gesture recognition due to their powerful automatic feature extraction capabilities. sEMG signals contain rich local details and global patterns, but single-scale convolutional networks are limited in their ability to capture both comprehensively, which restricts model performance. This paper proposes a deep learning model based on multi-scale feature fusion-MS-CLSTM (MS Block-ResCBAM-Bi-LSTM). The MS Block extracts local details, global patterns, and inter-channel correlations in sEMG signals using convolutional kernels of different scales. The ResCBAM, which integrates CBAM and Simple-ResNet, enhances attention to key gesture information while alleviating overfitting issues common in small-sample datasets. Experimental results demonstrate that the MS-CLSTM model achieves recognition accuracies of 86.66% and 83.27% on the Ninapro DB2 and DB4 datasets, respectively, and the accuracy can reach 89% in real-time myoelectric manipulator gesture prediction experiments. The proposed model exhibits superior performance in sEMG gesture recognition tasks, offering an effective solution for applications in prosthetic hand control, robotic control, and other human-computer interaction fields.

The abundant movement information carried by the surface electromyography (sEMG) signal can comprehensively reflect the muscle movement patterns on the surface of the human body, and the characteristics of physiological signals involving movement information have promoted the vigorous development in the field of human-computer interaction in machine learning. This paper takes subtle gestures as the research objects, and proposes a subtle gestures recognition system that uses two-channel sEMG signal and utilizes Quadratic Discriminant Analysis (QDA) and Linear Discriminant Analysis (LDA) which are parameter estimation classifiers based on probability density. First, combining the raw sEMG signal and the envelope sEMG signal, the number of channels for sensors and corresponding position of the muscle groups are preferably selected. Then, we propose to use the Pearson correlation coefficient to optimize the four types of features extracted in the time domain, frequency domain, time-frequency domain and AR model parameters, which effectively ensures the uniqueness of each gesture and eliminates individual differences. Finally, this paper compares four classifiers of QDA, LDA, Support Vector Machine (SVM), and K-Nearest Neighbor (KNN), and explores the pattern recognition classifier suitable for subtle actions. The experiment collects 9 kinds of subtle gestures from 8 subjects. The results show that the system can use only two channels of sEMG signal to recognize 9 kinds of subtle gestures which include multi-finger clicking or pinching or holding objects, and the most suitable classifiers QDA and LDA have the average recognition rates of 95.79% and 95.01%, respectively.

In human-machine interaction systems, continuous movement estimation methods occupy an important position because they are more natural and intuitive than pattern-recognition methods. Essentially, arm position is decided by the shoulder and elbow joint angles. However, the various deformations of muscles around the shoulder and elbow often lead to difficulties in sensor fixation, which results in a loss of synchronization between the surface electromyography (sEMG) signals and joint angles. In order to accurately estimate movement angles using sEMG in situations where the sEMG is not synchronized with joint angles, we utilized a bi-directional long short-term memory (Bi-LSTM) network rather than other deep learning methods to estimate non-dominant arm movements, based on the sEMG signal from the dominant arm. This estimation protocol was designed to avoid a multiplicity of sensors and to simulate more complicated loss of synchronization problems). The performance of the Bi-LSTM was compared with multilayer perceptrons (MLPs), convolutional neural networks (CNNs), and a long short-term memory network (LSTM). The Pearson correlation coefficient (cc) between the estimated and target joint angle sequences was calculated to evaluate the performance of each neural network. The Wilcoxon signed-rank results showed that the Bi-LSTM model significantly outperformed the MLP, CNN, and LSTM models (tested with completely untrained newly recorded free movements).

Recognizing the breathing resistances of wearing respirators from respiratory and sEMG signals with artificial neural networks

The variation in the distributions of recorded data between individuals leads to low classification accuracy. To address this issue, we introduce a multimodal fusion convolutional neural network (MFCNN). This network extracts common information from surface electromyography (sEMG) and accelerometer signals of different subjects using a two-stream convolutional neural network (CNN). To enhance the classification accuracy of a particular subject, a fine-tuning approach was implemented. The performance of the proposed method was assessed in four different scenarios, which include intersubject classification, intersubject classification when training data from multiple subjects, fine-tuned intersubject classification, and fine-tuned intersubject classification when training data from multiple subjects. The results demonstrate that in the intersubject scenario, when multiple subjects are available for training, the MFCNN achieves higher classification accuracy ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${p} &lt; 0.05$ </tex-math></inline-formula> ) than other neural networks and support vector machines (SVMs) that use sEMG signals [neural network (NN) and SVM], accelerometer signals (accNN and accSVM), sEMG and accelerometer signals [multimodal fusion nerual network (MFNN) and multimodal fusion support vector machine (MFSVM)] as inputs, as well as a CNN that uses sEMG signals as input after fine-tuning. Furthermore, compared with an MFCNN model trained with data from a single subject and an accCNN model trained with data from a single subject or multiple subjects, an MFCNN trained with multiple subjects demonstrated better performance on new subjects after fine-tuning ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${p} &lt; 0.05$ </tex-math></inline-formula> ). This method can learn common features among different subjects and improve the performance of classification among subjects. Our proposed method demonstrates the innovation of using a multimodal fusion approach and two-stream CNN to improve intersubject classification accuracy in upper limb movements.

The EMG signals acquired from acupoint Fu Tu (a motor point in rectus femoris) were reported to be more stable and reproducible than those from the middle point between greater trochanter to tibio-femoral junction of knee in standing position. It was different from the concept of better surface EMG signal from muscle belly than motor point.For studying whether the motor point is a better surface electrode location for stability and reproducibility of EMG signal, 6 normal subjects were examined at right quadriceps in isometric and isokinetic contractions by Cybex 340 dynamometer. The active electrodes of surface electromyography (S-EMG) were placed at midpoint of the rectus femoris and the motor point of rectus femoris for comparison. The integrated EMG (IEMG) signals were recorded and stored by Myosystem 1000. The result in this study showed that surface integrated EMG (S-IEMG) signals from the midpoint of the muscle was bigger in amplitude, more stable in repetitive measurements in within day as well as in between days.The results do not support the motor

In recent years, significant progress has been made in the research of gesture recognition using surface electromyography (sEMG) signals based on machine learning and deep learning techniques. The main motivation for sEMG gesture recognition research is to provide more natural, convenient, and personalized human-computer interaction, which makes research in this field have considerable application prospects in rehabilitation technology. However, the existing gesture recognition algorithms still need to be further improved in terms of global feature capture, model computational complexity, and generalizability. This paper proposes a fusion model of Squeeze-and-Excitation Networks (SE) and DenseNet, inserting attention mechanism between DenseBlock and Transition to focus on the most important information, improving feature representation ability, and effectively solving the problem of gradient vanishing. This proposed method was tested on the electromyographic gesture datasets NinaPro DB2 and DB4, achieving accuracies of 85.93 and 82.39 % respectively. Through ablation experiments, it was found that the method based on DenseNet-101 as the backbone model produced the best results. Compared with existing models, this proposed method has better robustness and generalizability in gesture recognition, providing new ideas for the development of sEMG signal gesture recognition applications in the future.

Freely available dataset of long-lasting surface electromyographic signals during walking

GLEAM: A multimodal deep learning framework for chronic lower back pain detection using EEG and sEMG signals.

Surface electromyography (sEMG) signal is widely used in lower limb exoskeleton system because it can reflect human motion intention. Aiming at the problem of low decoding accuracy of joint motion angle of lower limb exoskeleton robot, a knee motion angle prediction method based on sEMG signal is proposed. Firstly, in order to solve the problem of high time efficiency and insufficient stability of single time domain feature extraction method for sEMG signal, the eigenvalues are extracted from time domain, frequency domain and time-frequency domain respectively, and the prediction error of different channel sEMG signal feature parameter combination model is analyzed. Then, an innovative particle swarm optimization LSTM deep learning algorithm (PSO-LSTM) is proposed to establish the mapping model between sEMG signal and knee joint angle, and determine the PSO-LSTM model corresponding to the optimal feature parameter combination to realize the accurate prediction of knee joint angle. Finally, the experimental results show that the predicted results of the model are highly consistent with the real angle values, which is helpful to improve the human-computer cooperation of the lower limb exoskeleton system.KeywordssEMG signalLower extremity exoskeleton robotIntention perceptionPSO-LSTM algorithm

The development of robust circuit structures remains a pivotal milestone in electronic device research. This article proposes an integrated hardware–software system designed for the acquisition, processing, and analysis of surface electromyographic (sEMG) signals. The system analyzes sEMG signals to understand muscle function and neuromuscular control, employing convolutional neural networks (CNNs) for pattern recognition. The electrical signals analyzed on healthy and unhealthy subjects are acquired using a meticulously developed integrated circuit system featuring biopotential acquisition electrodes. The signals captured in the database are extracted, classified, and interpreted by the application of CNNs with the aim of identifying patterns indicative of neuromuscular problems. By leveraging advanced learning techniques, the proposed method addresses the non-stationary nature of sEMG recordings and mitigates cross-talk effects commonly observed in electrical interference patterns captured by surface sensors. The integration of an AI algorithm with the signal acquisition device enhances the qualitative outcomes by eliminating redundant information. CNNs reveals their effectiveness in accurately deciphering complex data patterns from sEMG signals, identifying subjects with neuromuscular problems with high precision. This paper contributes to the landscape of biomedical research, advocating for the integration of advanced computational techniques to unravel complex physiological phenomena and enhance the utility of sEMG signal analysis.