Clustering-Driven DGS-Based Micro-Doppler Feature Extraction for Automatic Dynamic Hand Gesture Recognition

Hand gesture recognition aims to recognize the meaningful expressions of hand motion. It is widely used in information visualization, robotics, sign language understanding, medicine and healthcare. Some methods have been proposed for hand gesture recognition. But no single algorithm can handle all kinds of situations, because of the complex environment. In this study, we propose a hybrid method for hand gesture recognition, which extends our previous work on a gesture recognition method based on concept learning by the addition of an association learning process. We use association learning to reveal the frequent patterns in gesture sequences, and then use such patterns to help recognize incomplete gesture sequences. Experiments show the use of association learning does indeed improve recognition accuracy. Experiments also show the hybrid method is comparable to two state of the art methods (HMMs and DTW) for hand gesture recognition, but outperforms them in the larger datasets.

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture. Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm. The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD). On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures. Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures. Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

The aim of this study is to develop a new method for hand gesture recognition using Leap Motion via deterministic learning. Efficient and accurate extraction and representation of gesture features are achieved. The recognition approach consists of two stages: a training stage and a recognition stage. In the training stage, hand gesture features representing hand motion dynamics, including spatial position and direction of fingers, are derived from Leap Motion. Hand motion dynamics underlying motion patterns of different gestures which represent Arabic numbers (0-9) are locally accurately modeled and approximated by radial basis function (RBF) neural networks. The obtained knowledge of approximated hand motion dynamics is stored in constant RBF networks. In the recognition stage, a bank of dynamical estimators is constructed for all the training patterns. Prior knowledge of hand motion dynamics represented by the constant RBF networks is embedded in the estimators. By comparing the set of estimators with a test gesture pattern to be recognized, a set of recognition errors are generated. The average Li norms of the errors are taken as the recognition measure between the dynamics of the training gesture patterns and the dynamics of the test gesture pattern according to the smallest error principle. Finally, experiments are carried out to demonstrate the high recognition performance of the proposed method. By using the 2-fold and 10-fold cross-validation styles, the correct recognition rates are reported to be 94.2% and 95.1%, respectively.

Recently hand gesture recognition via millimeter-wave radar has attracted a lot of research attention for human-computer interaction. Encouraged by the ability of deep learning models in successfully tackling hand gesture recognition tasks, we propose a deep neural network (DNN) model namely, Res3DTENet that aims to classify dynamic hand gestures using the radio frequency (RF) signals. We propose a scheme that improves the convolutional process of 3DCNNs with residual skip connection (Res3D) to emphasize local-global information and enriches the intra-frame spatio-temporal feature representation. A multi-head attention transformer encoder (TE) network has been trained over the spatio-temporal features to refine the inter-frame temporal dependencies of range-Doppler sequences. The experiments are carried out on the publicly available Soli hand gesture data set. Based on our extensive experiments, we show that the proposed network achieves improved gesture recognition accuracy than the state-of-the-art hand gesture recognition methods.

This paper devoted to the optimization of surface electromyography (sEMG) based hand gesture recognition classifier model. Twenty-two features in time domain, frequency domain and time-frequency domain were extracted from sEMG of 6 muscles to classify 13 hand gestures. Correlation analysis was used to screen the features. Random forest (RF) and principal component analysis (PCA) feature selection were used to further reduce the feature dimensions. Finally, mean absolute value (MAV), root mean square (RMS), slope sign change (SSC), mean frequency (MNF), Willison amplitude (WAMP) and autoregressive coefficients (AR1), totally 6 features were selected. LDA (Linear Discriminant Analysis), SVM (Support Vector Machine), KNN (k-Nearest Neighbor), DT (Decision Tree) and NN (Neural Network), in total of five classification methods were applied. The accuracy of the classifier had no significant reduction, but the dimension of the feature set was reduced from 132 to 36, and the training time was greatly reduced. The result indicated that the combination of correlation analysis and random forest feature selection can optimize the classifier model and greatly reduce the training time of the classifier.

With rapid developments in biometric recognition, a great deal of attention is being paid to robots which interact smartly with humans and communicate certain types of biometrical information. Such human–machine interaction (HMI), also well-known as human–robot interaction (HRI), will, in the future, prove an important development when it comes to automotive manufacturing applications. Currently, hand gesture recognition-based HRI designs are being practically used in various areas of automotive manufacturing, assembly lines, supply chains, and collaborative inspection. However, very few studies are focused on material-handling robot interactions combined with hand gesture communication of the operator. The current work develops a depth sensor-based dynamic hand gesture recognition scheme for continuous-time operations with material-handling robots. The proposed approach properly employs the Kinect depth sensor to extract features of Hu moment invariants from depth data, through which feature-based template match hand gesture recognition is developed. In order to construct continuous-time robot operations using dynamic hand gestures with concatenations of a series of hand gesture actions, the wake-up reminder scheme using fingertip detection calculations is established to accurately denote the starting, ending, and switching timestamps of a series of gesture actions. To be able to perform typical template match on continuous-time dynamic hand gesture recognition with the ability of real-time recognition, representative frame estimates using centroid, middle, and middle-region voting approaches are also presented and combined with template match computations. Experimental results show that, in certain continuous-time periods, the proposed complete hand gesture recognition framework can provide a smooth operation for the material-handling robot when compared with robots controlled using only extractions of full frames; presented representative frames estimated by middle-region voting will maintain fast computations and still reach the competitive recognition accuracy of 90.8%. The method proposed in this study can facilitate the smart assembly line and human–robot collaborations in automotive manufacturing.

In this article, a novel method for continuous hand gesture detection and recognition is proposed based on a frequency modulated continuous wave (FMCW) radar. Firstly, we adopt the 2-Dimensional Fast Fourier Transform (2D-FFT) to estimate the range and Doppler parameters of the hand gesture raw data, and construct the range-time map (RTM) and Doppler-time map (DTM). Meanwhile, we apply the Multiple Signal Classification (MUSIC) algorithm to calculate the angle and construct the angle-time map (ATM). Secondly, a hand gesture detection method is proposed to segment the continuous hand gestures using a decision threshold. Thirdly, the central time-frequency trajectory of each hand gesture spectrogram is clustered using the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means algorithm, and then the Fusion Dynamic Time Warping (FDTW) algorithm is presented to recognize the hand gestures. Finally, experiments show that the accuracy of the proposed hand gesture detection method can reach 96.17%. The hand gesture average recognition accuracy of the proposed FDTW algorithm is 95.83%, while its time complexity is reduced by more than 50%.

Hand gesture is one of the most popular communication methods in everyday life. In human-computer interaction applications, hand gesture recognition provides a natural way of communication between humans and computers. There are mainly two methods of hand gesture recognition: glove-based method and vision-based method. In this paper, we propose a vision-based hand gesture recognition method using Kinect. By using the depth information is efficient and robust to achieve the hand detection process. The finger labeling makes the system achieve pose classification according to the finger name and the relationship between each fingers. It also make the classification more effective and accutate. Two kinds of gesture sets can be recognized by our system. According to the experiment, the average accuracy of American Sign Language(ASL) number gesture set is 94.33%, and that of general gestures set is 95.01%. Since our system runs in real-time and has a high recognition rate, we can embed it into various applications.

For hand gesture recognition (HGR) sensing based on frequency modulated continuous wave (FMCW) radar, it is challenging to cope with the ambiguity and variability of gesture motions performed by different peoples due to the private habits or position, which inevitably reduce the robustness of HGR sensing. To solve this problem, we propose a novel dual-flow fusion deformable residual network (DFDRN). More detailed gesture information, time frequency maps and spectrum map videos, are fed into DFDRN, which improves the generalization and motion modeling ability of HGR through the extensive feature space. To further improve the robustness of HGR sensing, the deformable block is exploited to replace the convolution. To focus on the important features of dual-flow fusion, the convolutional block attention module (CBAM) is embedded in the backbone of DFDRN. Experimental results show that the accuracy of DFDRN, which can reach 97.5%, is obviously better than the state-of-the-art HGR methods.

In recent years, hand gesture recognition (HGR) systems have been extensively developed with the technologies of human-computer interaction (HCI), which enables regular interaction with machines. However, the significance of progress in HGR continues to advance, although the problem of hand segmentation and recognition is challenging due to the unfavorable environment, background illumination, hand size, and shape. To overcome this, we propose an optimal segmentation method for identifying hand gestures from input images, improving recognition performance. For segmenting hand gestures, we compared the segmentation methods of YCbCr, SkinMask, and HSV (hue, saturation, and value). The CR component is extracted from YCbCr, then binarization, erosion, and hole filling are performed. Color segmentation is applied to the SkinMask process that detects pixels that match the color of the hand. In the HSV process, threshold masking determines the dominant features. The Softmax classification is used to classify hand gestures where features are extracted through convolutional neural network (CNN). The proposed segmentation methods are applied to a benchmark dataset and the result shows an improvement in recognition accuracy over state-of-the-art systems.

Hand gesture recognition forms a great difficulty for computer vision especially in dynamics. Sign language has been significant and an interesting application field of dynamic hand gesture recognition system. The recognition of human hands formed an- extremely complicated mission. The solution for such a difficulty requires a robust hand tracking method which depends on an effective feature and classifier. This paper presents a novel, fast and simple method for dynamic hand gesture recognition based on two lines (hundred) of features extracted from two rows of a Real-Time video. Feature selections have been used for hand shape representation to recognize the dynamic word for Kurdish Sign Language. The features extracted in real time from pre-processed hand object were represented through the optimization values of binary captured frame. Finally, an Artificial Neural Network classifier is used to recognize the performed hand gestures by 80% for training and 20% for testing with success 98%.

During the long-term operation of wind turbines, due to environmental factors and equipment aging, the health and reliability of each component will gradually decline, leading to failure. To assess the health status of wind turbines, timely grasp the subsequent changes and development trends, it is necessary to extract degradation characteristics, including time domain, frequency domain, and time-frequency domain characteristics. These degradation characteristics can reflect the operating status of the equipment, help build health indicator curves, and evaluate the health status of high-speed shaft bearings of wind turbines. Selecting reasonable degradation characteristics is an important prerequisite for constructing a health index curve, and using evaluation indicators to construct a comprehensive evaluation function to screen degradation characteristics. The feature fusion method based on a self-organizing feature mapping network is used to fuse multiple selected degradation features and fuse the selected multiple degradation features into a curve that can reflect the bearing degradation process. Finally, a quantitative analysis is performed on the health index curve to scientifically assess the health status of bearings. Bearings are one of the key components of wind turbines. Based on the health index curve constructed in this article, an appropriate prediction model is selected to predict the health index trend of bearings. A timely and effective grasp of the health trends of wind turbine bearings is of great practical significance for formulating scientific and reasonable maintenance measures for wind farms. The work of this article will be divided into the following four parts: (1) Extracting the time domain, frequency domain, and time-frequency domain degradation characteristics of high-speed shaft bearing vibration signals of wind turbines; (2) Comprehensive evaluation using monotonicity, correlation, and robustness constructs function to screen degenerate features; (3) Use self-organizing feature mapping network. The network integrates the selected degradation features and constructs a health index curve; (4) Based on the constructed health index curve, optimize the BiLSTM network hyperparameters through Bayesian and establish a BO-BiLSTM network model to achieve a more accurate and scientific prediction of the health index trend.

In this paper, three different classification methods, including the support vector machine (SVM), the deep learning method based on multi-layer perceptron (MLP) and multichannel convolutional neural network (multi-channel CNN), were used to classify 13 hand gestures. The surface electromyography (sEMG) were extracted from six muscles of hand and forearm. For the SVM and MLP, six features in the time domain, frequency domain and time-frequency domain were extracted. For the multi-channel CNN, a sliding window segment of the original sEMG image was used as the input. Hand gesture recognition based on deep learning had similar performance to traditional machine learning in off-line classification. Considering the high robustness and generalization ability, deep learning is likely a more robust alternative to traditional machine learning in the field of sEMG hand gesture recognition.

The salient feature points of hand gesture play an important role for its representation and recognition. In this paper, a novel hand gesture recognition method based on salient feature point selection is proposed. The raw data of hand gesture is captured by the Kinect sensor and the hand gesture is segmented from the cluttered background. The shape feature of hand gesture is extracted from the contour, and the salient feature points are selected by a new algorithm to represent the hand gesture. Finally, the Dynamic Time Warping algorithm is modified and employed to find the best correspondence between two gestures. Extensive experiments are implemented on three benchmark databases to validate the effectiveness of our method. The experimental results verified the invariance of our method to translation, rotation scaling and articulated deformation. The comparison with state-of-the-art methods demonstrates the accuracy and efficiency of our method.