주파수 가중 MTI를 이용한 FMCW 레이다 손동작 인지 기법

Hand motion recognition is pivotal for enhancing skill learning in endovascular surgery. However, catheterization procedures typically involve multimodal sources that introduce data dissimilarities, leading to unstable and poor performance in learning-based methods used for recognizing surgical tasks. This study proposes a resolution-based dynamic time warping (rDTW) method for aligning multimodal signals to improve hand motion recognition in endovascular surgery. The rDTW model is designed to align multimodal signals from wearable sensors, referencing the resolution of a unique source. The reference signal traces the events, creating a consistent progression that can localize a surgeon’s hand motions across all signals. Other signals are stretched or folded to match the reference timeline, facilitating event localization relative to the reference signal. By preprocessing multimodal signals from 33 catheterization trials, we show that the proposed rDTW method enhances hand motion recognition performance in two neural network models. When the data was used directly for the motion recognition task without signal warping, a performance decrease of 2.59% was observed. Also, the rDTW method exhibits higher data similarity index and lower warping time compared to three existing DTW techniques, showing its effectiveness in aligning multimodal signals for surgeons’ hand motion recognition.

Surface electromyography (sEMG) signals are an indirect measurement of muscle activity, and their applications range from biomechanics to control and rehabilitation. Hand movement recognition is a very difficult endeavor due to forearm anatomy. Hence, a multichannel approach for signal acquisition and processing is required. Conventional electrodes can limit the ease-of-use and repeatability of multi-channel sEMG recordings. New techniques have been proposed in this regard, with dry electrodes being one of them. Dry electrode technology has enabled the design of better donning and doffing procedures for multichannel sEMG recording, particularly for rehabilitation and prosthetic applications. However, there is a debate about the quality of the signals recorded with them and their usefulness for the recognition of multiple hand movements. To mitigate these quality issues, this work proposes an array of reusable stainless steel dry electrodes for multichannel sEMG recording with a design that facilitates its positioning on the forearm. The dry electrodes were characterized through electrical impedance measures and a Bland-Altman test. They were found to have similar characteristics to standard, disposable sEMG pre-gelled electrodes. For placement repeatability and application feasibility, an anatomy-based electrode positioning protocol was implemented with 17 healthy subjects and six hand movements. To evaluate the application feasibility of the electrode array, a feed-forward artificial neural network was trained to classify signals from the six movements, with a 97,86±0,58% accuracy. The amplitude of the sEMG signals for two antagonist movements was compared, finding a 24,81% variation. The dry electrode array showed feasibility in acquiring and classifying sEMG signals of hand movements with high accuracy.

Manipulation performed by humans contains a lot of information that helps robots to learn how to handle objects. Since hand poses and motions are related to manipulated objects, extracting this information is one of the important tasks for robotics community. This paper presents a framework to recognize human manipulations including hand motions, hand poses, and shapes of manipulated objects using egocentric RGB-D videos. Our framework is straightforward but powerful through the efficient use of depth information and egocentric vision. We estimate hand poses with an example-based method through the limited appearances of the hand in egocentric vision. First, from a sensed point cloud, our framework distinguishes hands, manipulated objects and an environment using skin color detection and limitation on the range of the moving hand. Next, we estimate a hand pose by aligning the extracted hand point cloud with a pre-recorded database of hand point clouds of different poses. The position and orientation of the head-mounted sensor are estimated to acquire the hand motion in the world coordinate system. Then, the type of hand motion is classified using Dynamic Programming matching between a series of velocity vectors of estimated and a database of wrist trajectories. Finally, we experiment the effectiveness of our framework for hand motion recognition to validate our work.

Upper limb prosthesis control with pattern recognition (PR)- based on surface Electromyogram (EMG) signals has been heavily investigated in the literature. However, challenges related to signal non-stationarity and its impact by changing force levels, limb positions and many other factors all lead to poor system stability, prevent the widespread adoption of this technology. In this study, we propose an alternative modality based on Acoustic Myography (AMG), a recording of muscle vibrations at low frequencies, as a control signal to decipher the intended hand movements. A custom-built AMG armband, consisting of 4 microphones, has been developed in this study and evaluated on intact-limbed subjects performing six classes of hand and finger movements. Time domain and auto-regression (TD-AR) features, in addition to Bi long-short term memory (BiLSTM) deep learning classifier were utilized to perform the classification. An average classification accuracy of 85% was obtained, which shows the potential of using the developed AMG armband, with PR system for prostheses control.

With the advantages of comfortable wearing and outdoor usage, the myoelectric gesture recognition techniques have gained much attention in the field of human-machine interaction (HMI). The purpose of this study is to optimize model structure and transfer generalized features to improve the robustness of myoelectric hand motion decoding. We derived the hand motion recognition framework from the muscle synergy theory, which is formulated as a temporal convolutional (TC) model of array sEMG signals, then a hierarchical myoelectric decoding model was proposed to predict simultaneous and continuous hand motion. The model was trained by the methods of unsupervised low-level feature learning and automated data labeling to minimize training supervision. Extensive experiments on the public sEMG database (17 subjects in Biopatrec) show that the TC model can extract muscle synergy features with higher fidelity ( R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.85±0.23) than the traditional instantaneous mixture model, the results of online test demonstrate robust myoelectric decoding on multiple simultaneous and continuous hand motions. More importantly, the analysis of weights visualization shows that the low-level feature representation layer of TC model can be migrated across the individuals, which provides a transferrable feature extraction layer for generalized hand motion decoding.

In the context of smart city, hand motion recognition has once again become a research hotspot, especially the surface electromyographic (sEMG) based hand motion recognition. When sEMG signals contain noise, the overfitting problem always exists. To solve this problem, a multi-kernel label relaxation ridge regression with manifold embedding is proposed. Firstly, a non-negative matrix is introduced to relax the label matrix of the training data, so that the proposed model has more freedom in label fitting. At the same time, label relaxation can enlarge the margins between classes as much as possible. Secondly, based on the idea of manifold learning, the compactness graph is introduced to ensure that the training data in the same class should be as close as possible in the conversion space when they are converted from the original feature space to the label space so as to improve the generalization ability. Finally, 15 UCI datasets and the hand motion dataset Ninapro are used to verify the proposed model. The experimental results show that the proposed model has certain advantages over comparative ones in terms of classification accuracy and generalization ability.

Because handicapped people account for a large percentage of our community, we should make an effort to interact with them in order to exchange knowledge, perspectives, and ideas. To that aim, we wish to establish a means of contact. Individuals who are deaf or hard of hearing can communicate with one another using sign language. A handicapped person can communicate without using acoustic noises when they use sign language. The objective of this article is to explain the design and development of a hand gesture-based sign language recognition system. To aid handicapped individuals, particularly those who are unable to communicate verbally, sign language is translated into text and subsequently into speech. The solution is based on a web camera as the major component, which is used to record a live stream video using a proprietary MATLAB algorithm. Recognition of hand movements is possible with the technology. Recognizing hand gestures is a straightforward technique of providing a meaningful, highly flexible interaction between robots and their users. There is no physical communication between the user and the devices. A deep learning system that is efficient at picture recognition is used to locate the dynamically recorded hand movements. Convolutional neural networks are used to optimize performance. A static image of a hand gesture is used to train the model. Without relying on a pre-trained model, the CNN is constructed.KeywordsHuman–computer interactionGesture recognitionWeb cameraCNNMATLAB

Crosstalk always exists in monostatic frequency modulated continuous wave (FMCW) radar systems, and obscures echoes of targets located nearby. A crosstalk suppression method based on the polynomial fitting is proposed. It exploits the difference of crosstalk and target echo presence in beat frequency signals and range profile. By minimizing the error function, crosstalk component could be estimated by a low-order fitted model, and then removed from beat frequency signals within each sweep period. Finally, measured data collected from a common millimetre-wave FMCW radar are used to demonstrate rationality and effectiveness of the proposed method, especially for measurement of close-in targets in range profile.

Previous work in recognition of hand gestures has concentrated on classification of hand shapes, with relatively little work done on hand motions. This paper describes a recurrent neural network which has been trained to classify sixteen different hand trajectories, including relatively complex paths such as circles and back-and-forth motions. The network's ability to anticipate the classification of an incomplete gesture is also examined, and its implications for segmentation of gestures is discussed.

The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.

Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing “Palm Downward” sign gestures from “Palm Inward” ones. Only the “Palm Inward” gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

Hand motion tracking and gesture recognition are a fundamental technology in the field of proactive computing for a better human computer interaction system. In this paper, we have developed a 3-D hand motion tracking and gesture recognition system via a data glove (namely the KHU-1 data glove consisting of three tri-axis accelerometer sensors, one controller, and one Bluetooth). The KHU-1 data glove is capable of transmitting hand motion signals to a PC through wireless communication via Bluetooth. Also we have implemented a 3-D digital hand model for hand motion tracking and recognition. The implemented 3-D digital hand model is based on the kinematic chain theory utilizing ellipsoids and joints. Finally, we have utilized a rule-based algorithm to recognize simple hand gestures namely scissor, rock, and paper using the 3-D digital hand model and the KHU-1 data glove. Some preliminary experimental results are presented in this paper.

This work reports on preliminary results about on hand movement recognition with Near InfraRed Spectroscopy (NIRS) and surface ElectroMyoGraphy (sEMG). Either basing on physical contact (touchscreens, data-gloves, etc.), vision techniques (Microsoft Kinect, Sony PlayStation Move, etc.), or other modalities, hand movement recognition is a pervasive function in today environment and it is at the base of many gaming, social, and medical applications. Albeit, in recent years, the use of muscle information extracted by sEMG has spread out from the medical applications to contaminate the consumer world, this technique still falls short when dealing with movements of the hand. We tested NIRS as a technique to get another point of view on the muscle phenomena and proved that, within a specific movements selection, NIRS can be used to recognize movements and return information regarding muscles at different depths. Furthermore, we propose here three different multimodal movement recognition approaches and compare their performances.

Recognition of the Surgeon's Motions During Endoscopic Operation by Statistics Based Algorithm and Neural Networks Based ANARX Models

In this paper we are focusing on the discrimination and recognition of arm movements described in terms of the evolution of the joint angles during the trajectory of the movement. Considering this as a classification problem, we could use a three layered feedforward network trained by the backpropagation learning algorithm to discriminate between arm movements described in the representation proposed by Marr and Vaina (1982) and Vaina and Bennour (1985). We discuss the contribution of the sampling rate for the value of the joint angles in the input layers, the number of angles necessary for a good recognition of the movement. We introduce a new method for determining the size of the network at input stage, which eliminates the unnecessary data in the training set. The effect of changing the number of frames used to train the network for each movement and the properties of the backpropagation algorithm in this application are discussed. We demonstrate that a description of movements at multiple scales of resolution organized from general to particular is conducive to an efficient learning scheme in the network. The main result obtained is that the discrimination between different arm movements is most efficiently obtained at the appropriate and most meaningful scale of resolution, as predicted by the Marr and Vaina's model.