Robust Continuous Hand Motion Recognition Using Wearable Array Myoelectric Sensor

Consumer electronic devices are becoming more complex and diverse. In order to provide users with an easy and convenient way of controlling various consumer electronics, it is necessary to use a universal way to control various devices. This requires using a user-friendly and intuitive interface. We introduce a wristwatch-type of remote that offers a unified way to control various devices. Total of seven gestures that are based on hand motions are defined. These seven gestures are suitable to the wearable wristwatch-type device. For a variety of systems, these gestures can be used in the same way. Gestures are not designed and analyzed in a discrete manner; instead, they are designed in continuous hand motions and analyzed in gesture commands. Also, a method of modeling a virtual menu from the menu on a consumer electronic device is introduced. With the virtual menu, the user is able to control various devices by using gestures. The virtual menu has to reflect hand motion characteristics and represents functions of electronic appliances. The virtual menu is implemented in XML to represent the basic menu construction and its properties. In order to use virtual menu, we introduce how a user's hand motions can be used in a fast and effective way. Finally we compare our wearable remote control to the conventional remote control through user tests in terms of user convenience and efficiency.

Manipulation performed by humans contains a lot of information that helps robots to learn how to handle objects. Since hand poses and motions are related to manipulated objects, extracting this information is one of the important tasks for robotics community. This paper presents a framework to recognize human manipulations including hand motions, hand poses, and shapes of manipulated objects using egocentric RGB-D videos. Our framework is straightforward but powerful through the efficient use of depth information and egocentric vision. We estimate hand poses with an example-based method through the limited appearances of the hand in egocentric vision. First, from a sensed point cloud, our framework distinguishes hands, manipulated objects and an environment using skin color detection and limitation on the range of the moving hand. Next, we estimate a hand pose by aligning the extracted hand point cloud with a pre-recorded database of hand point clouds of different poses. The position and orientation of the head-mounted sensor are estimated to acquire the hand motion in the world coordinate system. Then, the type of hand motion is classified using Dynamic Programming matching between a series of velocity vectors of estimated and a database of wrist trajectories. Finally, we experiment the effectiveness of our framework for hand motion recognition to validate our work.

Surface electromyography (sEMG) signals are an indirect measurement of muscle activity, and their applications range from biomechanics to control and rehabilitation. Hand movement recognition is a very difficult endeavor due to forearm anatomy. Hence, a multichannel approach for signal acquisition and processing is required. Conventional electrodes can limit the ease-of-use and repeatability of multi-channel sEMG recordings. New techniques have been proposed in this regard, with dry electrodes being one of them. Dry electrode technology has enabled the design of better donning and doffing procedures for multichannel sEMG recording, particularly for rehabilitation and prosthetic applications. However, there is a debate about the quality of the signals recorded with them and their usefulness for the recognition of multiple hand movements. To mitigate these quality issues, this work proposes an array of reusable stainless steel dry electrodes for multichannel sEMG recording with a design that facilitates its positioning on the forearm. The dry electrodes were characterized through electrical impedance measures and a Bland-Altman test. They were found to have similar characteristics to standard, disposable sEMG pre-gelled electrodes. For placement repeatability and application feasibility, an anatomy-based electrode positioning protocol was implemented with 17 healthy subjects and six hand movements. To evaluate the application feasibility of the electrode array, a feed-forward artificial neural network was trained to classify signals from the six movements, with a 97,86±0,58% accuracy. The amplitude of the sEMG signals for two antagonist movements was compared, finding a 24,81% variation. The dry electrode array showed feasibility in acquiring and classifying sEMG signals of hand movements with high accuracy.

Hand motion recognition is pivotal for enhancing skill learning in endovascular surgery. However, catheterization procedures typically involve multimodal sources that introduce data dissimilarities, leading to unstable and poor performance in learning-based methods used for recognizing surgical tasks. This study proposes a resolution-based dynamic time warping (rDTW) method for aligning multimodal signals to improve hand motion recognition in endovascular surgery. The rDTW model is designed to align multimodal signals from wearable sensors, referencing the resolution of a unique source. The reference signal traces the events, creating a consistent progression that can localize a surgeon’s hand motions across all signals. Other signals are stretched or folded to match the reference timeline, facilitating event localization relative to the reference signal. By preprocessing multimodal signals from 33 catheterization trials, we show that the proposed rDTW method enhances hand motion recognition performance in two neural network models. When the data was used directly for the motion recognition task without signal warping, a performance decrease of 2.59% was observed. Also, the rDTW method exhibits higher data similarity index and lower warping time compared to three existing DTW techniques, showing its effectiveness in aligning multimodal signals for surgeons’ hand motion recognition.

Upper limb prosthesis control with pattern recognition (PR)- based on surface Electromyogram (EMG) signals has been heavily investigated in the literature. However, challenges related to signal non-stationarity and its impact by changing force levels, limb positions and many other factors all lead to poor system stability, prevent the widespread adoption of this technology. In this study, we propose an alternative modality based on Acoustic Myography (AMG), a recording of muscle vibrations at low frequencies, as a control signal to decipher the intended hand movements. A custom-built AMG armband, consisting of 4 microphones, has been developed in this study and evaluated on intact-limbed subjects performing six classes of hand and finger movements. Time domain and auto-regression (TD-AR) features, in addition to Bi long-short term memory (BiLSTM) deep learning classifier were utilized to perform the classification. An average classification accuracy of 85% was obtained, which shows the potential of using the developed AMG armband, with PR system for prostheses control.

Reaching with the arm and grasping with the hand and fingers is a complex behavior that appears in utero, is elaborated over the first few years of life, and serves useful everyday functions throughout the course of human life. Several neurological conditions can impair the ability to produce arm and hand movements and so greatly impact on the quality of life and well-being of the affected individuals. Given the fundamental role that arm and hand movements play in everyday life, deficits related to arm and hand function are one of the most debilitating motor conditions. Neurological conditions that can affect arm and hand movements include autism spectrum disorder, Parkinson’s and Huntington’s diseases, amyotrophic lateral sclerosis, cerebral palsy, and stroke-related motor cortex damage as well as spinal cord injury at cervical levels. While arm and hand movement has received considerable attention from both clinicians and researchers from diverse scientific backgrounds, there are a number of broad research questions that still need to be addressed in this research field. The present Research Topic is entirely devoted to arm and hand movement in health as well as in disease. It is a compilation of original research papers and reviews, clinical case studies, hypothesis and theory articles, opinions, commentaries, and methods articles that cover important aspects of the topic from different perspectives. In this volume, de Bruin et al. (1) present data that describe how healthy adults use space while performing a visually guided grasping task. A model for understanding hand functioning in children with cerebral palsy is proposed by Arnould et al. (2) while Johansson et al. (3) explore the effect of timing training on upper limb movement in three children with diplegic cerebral palsy. Parma et al. (4) compare the kinematics of the reach-to grasp movement in patients with vascular and idiopathic Parkinson’s disease whereas Aluru et al. (5) evaluate the effect of auditory constraints on motor performance at different stages after a stroke. Lawrence et al. (6) measure dexterous manipulation in a cross-sectional study comparing gender, age, and absence and presence of disease. Kirsh et al. (7) provide evidence to support the view that neurons outside the primary motor cortex – such as those populating the pontomedullary reticular formation and the spinal cord-drive movement and muscle synergies that primary motor cortex neurons then break up to create individual wrist and finger movements. Sacrey et al. (8) summarize current knowledge related to reaching and grasping in autism spectrum disorder. On the other hand, Whitwell et al. (9) reinstate patient DF’s amazing ability to use information regarding form and orientation of objects to guide skilled reaching actions despite her visual agnosia. In an opinion article, Moore (10) argues that nerve transfer is increasingly popular and is becoming the best treatment strategy for most brachial plexus damage as well as for patients with spinal cord injury at cervical levels. Vicario (11) provides a personal commentary on a paper from Hayashi et al. (12) and, in a review article, Karl and Whishaw (13) summarize the evidence that show that reaching and grasping are from distinct neural and evolutionary origins. Irvine et al. (14) contribute a methods article that assesses the reliability of the Irvine, Beatties, and Bresnahan (IBB) forelimb recovery scale. Fouad et al. (15) demonstrate that continuous viral-mediated brain-derived neurotrophic factor (BDNF) over-expression promotes spasticity in rats with spinal cord hemisections at cervical levels. Alstermark and Pettersson (16) bring evidence to show that lesions to the corticospinal tract that spare the cortico-reticulospinal pathway in the rat have no deleterious effects on skilled reaching and grasping. Finally, Tosolini et al. (17) describe how targeting the full length of the motor endplate region in the mouse forelimb with Fluoro-Gold increases the uptake of this neuroanatomical retrograde tracer in corresponding motor neurons. We are delighted to present “Arm and Hand Movement: Current Knowledge and Future Perspective” as a Research Topic in Frontiers in Neurology. We feel that this wide-ranging compilation of articles by leading experts in upperlimb/forelimb movement and working either in clinical or basic research settings has offered fresh perspectives on the topic. We are thankful for the support of all the scientists who have contributed to this Research Topic and have shared with us their expertise and point of views. Their contributions have deepened our appreciation of the challenge that restoring arm and hand function in different pathologies represents. We invite the readers to experience the diversity in methodological approaches and experimental designs that together have led to broaden our understanding of this particularly wide field of research.

This study aimed to investigate the muscle synergy for hand motions of both the left and right hands based on surface electromyography (sEMG) signals. Six hand motions, including hand grasp (HG), four fingers flexion (FFF), index finger extension (IFE), thumb internal rotation (TIR), thumb opposition (TO), key pinch (KP), were examined in the experiment. The sEMG signals were recorded from brachioradialis (BRA), flexor carpi ulnaris (FCU), flexor carpi radialis (FCR), extensor digitorum communis (EDC), flexor digitorum superficialis (FDS), abductor pollicis brevis (APB), first dorsal interosseous (FDI) and abductor digiti minimi (ADM) of both the left and right hands. A co-contraction index (CI) followed by a correlation coefficients (CC) was applied to quantify the muscle synergy during the hand motions and to compare the synergistic patterns across hands. Results shown that the left and right hands had similar muscle synergy (CC (CC>0.9) for any of the motions. For both hands, there were strong correlations between the synergistic matrix of different motions (CC (CC>0.85), including the correlations between HG and FFF, between FFF and IFE, and between TO and KP. By contrast, weak correlations between the synergistic matrix for IFE and that for TIR (CC (CC<0.1), suggesting a low similarity of the muscle synergy for the two motions. This study shed light on how the muscles of the left and right hands are involved and co-contracted for the dexterous hand motions.

In the context of smart city, hand motion recognition has once again become a research hotspot, especially the surface electromyographic (sEMG) based hand motion recognition. When sEMG signals contain noise, the overfitting problem always exists. To solve this problem, a multi-kernel label relaxation ridge regression with manifold embedding is proposed. Firstly, a non-negative matrix is introduced to relax the label matrix of the training data, so that the proposed model has more freedom in label fitting. At the same time, label relaxation can enlarge the margins between classes as much as possible. Secondly, based on the idea of manifold learning, the compactness graph is introduced to ensure that the training data in the same class should be as close as possible in the conversion space when they are converted from the original feature space to the label space so as to improve the generalization ability. Finally, 15 UCI datasets and the hand motion dataset Ninapro are used to verify the proposed model. The experimental results show that the proposed model has certain advantages over comparative ones in terms of classification accuracy and generalization ability.

Because handicapped people account for a large percentage of our community, we should make an effort to interact with them in order to exchange knowledge, perspectives, and ideas. To that aim, we wish to establish a means of contact. Individuals who are deaf or hard of hearing can communicate with one another using sign language. A handicapped person can communicate without using acoustic noises when they use sign language. The objective of this article is to explain the design and development of a hand gesture-based sign language recognition system. To aid handicapped individuals, particularly those who are unable to communicate verbally, sign language is translated into text and subsequently into speech. The solution is based on a web camera as the major component, which is used to record a live stream video using a proprietary MATLAB algorithm. Recognition of hand movements is possible with the technology. Recognizing hand gestures is a straightforward technique of providing a meaningful, highly flexible interaction between robots and their users. There is no physical communication between the user and the devices. A deep learning system that is efficient at picture recognition is used to locate the dynamically recorded hand movements. Convolutional neural networks are used to optimize performance. A static image of a hand gesture is used to train the model. Without relying on a pre-trained model, the CNN is constructed.KeywordsHuman–computer interactionGesture recognitionWeb cameraCNNMATLAB

The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.

This paper is a comment on “Menkes Disease Presenting with Epilepsia Partialis Continua.” While updating the LOVD (Leiden Open Variation Database) ATP7A mutation database (https://grenada.lumc.nl/LOVD2/MD/home.php) I came across a case described by Rizk and colleagues and published in your journal [1]. Rizk and colleagues report a 17-month-old girl who was referred to their hospital with continuous right hand movements which developed suddenly at age of 14 months, regression of milestones, and repetitive episodes of hypothermia. She had hypotonia and constipation. Brain magnetic resonance imaging showed among others cortical and cerebral atrophy. EEG was abnormal and she was diagnosed with epilepsia partialis continua. Magnetic resonance angiography showed relatively tortuous but patent intracranial vessels, with appearance of “hair pin” sign. She had normal hair also with light microscopy. Blood copper and ceruloplasmin levels were normal. The patient was born full term and had normal birth measures. Genetic testing revealed presence of a heterozygous variation, c.1138G>A [p.(Val380Met)], and the authors concluded the diagnosis of Menkes disease (MD) in this patient. Even though the clinical features of this female patient overlap with some of the features observed in Menkes disease patients, she lacks some of the major pathogenic criteria (such as reduced serum copper and ceruloplasmin levels, abnormal hair, hypopigmentation, and cutis laxa) (review in [2]) questioning the clinical diagnosis of Menkes disease, even though female patients may be less severely affected. Furthermore, the ATP7A variation identified is not convincing enough to establish a genetic diagnosis. Firstly, this variation is described at least once in a healthy individual (rs149523862, dbSNP database of short genetic variations, http://www.ncbi.nlm.nih.gov/SNP/), which questions its pathogenicity especially when taking into consideration that MD is a Mendelian disorder. This variation is a missense substitution in exon 4 of ATP7A proximal to but not within the 4th MBD (metal binding domain) of the ATP7A protein. Until now 70 different missense mutations in ATP7A have been described in 94 unrelated patients and none of these mutations were within the first 7 exons of ATP7A encoding the six MBDs [3]. It is generally believed that variations within these regions are more acceptable and do not necessarily lead to MD [3]. Furthermore, two commonly used and publicly available pathogenicity prediction methods, PhD-SNP (http://snps.biofold.org/phd-snp/phd-snp.html) and nsSNP analyser (http://snpanalyzer.uthsc.edu/), categorize this variation as neutral. Secondly, MD is an X-linked recessive disorder and manifesting heterozygotes are rare [4]. It is therefore extremely important to investigate the inheritance of this mutation and to perform X-inactivation studies before evaluating the clinical effect of this variation. In conclusion, the clinical diagnosis of this patient and the pathogenicity of the identified ATP7A variation should be reevaluated.

This paper proposes a signal processing technique that recognizes hand motions using a 60-GHz frequency-modulated continuous- wave radar system. In the case of hand motion recognition with a radar, the recognition accuracy decreases owing to a clutter signal generated by the upper body and differences between the hand sizes of different people. To solve this problem, this paper proposes a moving target indicator filter with a weighted frequency for each frequency, which improves hand motion recognition accuracy, removes self-interference clutter signals, and uses a range profile and Doppler profile. Neural networks can recognize hand movements; hence, the proposed five hand gesture recognition techniques exhibited an 87.3% accuracy, regardless of the user.

Recent studies have described muscle synergies as overlapping, multimuscle groups defined by synchronous covariation in activation intensity. A different approach regards a synergy as a fixed temporal sequence of bursts of activity across groups of motoneurons. To pursue this latter definition, the present study used a principal component (PC) analysis tailored to reveal the across-muscle temporal synergies of human hand movement. Electromyographic (EMG) activity was recorded as subjects used a manual alphabet to spell a list of words. The analysis was applied to the EMG waveforms from 27 letter-to-letter transitions of equal duration. The first PC (of 27) represented the main temporal synergy; after practice, it began to account for more of the EMG variance (up to 40%). This main synergy began with a burst in the 4-finger extensor and a silent period in the flexors. There were then progressively later and shorter bursts in the thumb abductor, thumb flexor, little finger abductor, and finally the finger flexors. The results suggest that hand movements may be generated by activity waves unfolding in time. Because finger muscles are under relatively direct cortical control, this suggests a specific form of cortical pattern generation.

Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing “Palm Downward” sign gestures from “Palm Inward” ones. Only the “Palm Inward” gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

In neuroscience, the idea that motor behaviors are constructed by a combination of building blocks has been supported by a large amount of experimental evidences. The idea has been very attractive as a powerful strategy for solving the motor redundancy problem. While there are some candidates for motor primitives, such as submovements, oscillations, and mechanical impedances, synergies are one of the candidates for motor modules or composite units for motor control. Synergies are usually extracted by applying statistic techniques to explanatory variables, such as joint angles and electromyography (EMG) signals, and by decomposing these variables into fewer units. The results of factor decomposition are, however, not necessarily interpretable with these explanatory variables, even though the factors successfully reduce the dimensionality of movement; therefore, the physical meaning of synergies is unclear in most cases. To obtain insight into the meaning of synergies, this chapter proposes the agonist-antagonist muscle-pair (A-A) concept and uses other explanatory variables: the A-A ratio, which is related to the equilibrium point (EP), and the A-A sum, which is associated with mechanical stiffness. The A-A concept can be regarded as a form comparable to the EP hypothesis (EPH, λ model), and it can be extended to the novel concept of EP-based synergies. These explanatory variables enable us to identify muscle synergies from human EMG signals and to interpret the physical meaning of the extracted muscle synergies. This chapter introduces the EMG analysis in hand-force generation of a human upper limb and shows that the endpoint (hand) movement is governed by two muscle synergies for (1) radial movement generation and (2) angular movement generation in a polar coordinate system centered on the shoulder joint. On the basis of the analysis, a synergy-based framework of human motor control is hypothesized, and it can explain the mechanism of the movement control in a simple way.