Dynamic Gesture Contour Feature Extraction Method Using Residual Network Transfer Learning

Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem. In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper. According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform. There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting. So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition.

With the rapid development of human-computer interaction technology, gesture recognition becomes one of the key technologies of human-computer interaction. In this paper, we propose a new method of dynamic hand gestures recognition. The method adopts the hierarchical identification model for dynamic hand gestures recognition. First, we combine frame fusion with density distribution features for rough gesture recognition, second, we use the hausdorff distance or fingertip detection for accurate gesture recognition. The main innovation of this method lies in that we change the way of dynamic gestures recognition into the recognition of static image, improves the efficiency of gesture recognition effectively. Experimental results showed that our recognition rate is above 90%.

Sign languages normally make use of both static and dynamic gestures to achieve communication goal. In automated continuous gesture recognition, one of the problems found is how to separate a transitional gesture from a meaningful dynamic gesture. In addition, a variation in speed, absolute position and orientation of a dynamic gesture makes an automatic recognition task nontrivial. This paper proposes a segmentation method that alleviates these problems. The method is based on the fact that many (meaningful) dynamic gestures as appeared in Thai and other sign languages, are performed in an approximately (quasi) periodic manner. Therefore, such dynamic gestures can be distinguished from a transitional gesture by means of Fourier analysis, performed on samples (discrete signal) of hand shapes, positions and orientation. The analysis, however, requires a good choice of data sample windowing. This paper describes a preprocessing method for estimation of the dynamic gesture data windowing period. The proposed method helps improving the accuracy of segmenting hand shape and location data samples into static, periodic and nonperiodic gestures. In addition, accuracy of the feature extracted from the Fourier analysis of a dynamic gesture signal is improved, thus increasing the recognition rate. Experimental results performed on a real data are included.

Feature extraction is an important factor to improve the recognition rate of coal and gangue. The existing feature extraction methods have some shortcomings in coal and gangue recognition, such as unsatisfactory recognition rate for actual images. Therefore, aiming at the shortcomings of existing coal and gangue feature extraction methods in coal and gangue recognition, the contour feature extraction method is researched based on images analysis to coal and gangue recognition. The center normalization detrended fluctuation analysis (CNDFA) feature extraction algorithm of contour is proposed based on the process of contour feature extraction for coal and gangue. Based on the analysis of the representative features of coal and gangue, the extraction process of target contour features is established based on hardness difference. Combined with the overall features of contour curve after detrending, a center normalized CNDFA feature extraction algorithm is proposed. First, the detrended analysis of contour curve is realized by least square optimal fitting, and then the detrended data are normalized. Finally, the contour features are described quantitatively by multifractal (MF) spectrum to form the geometric features of the target contour curve, which is used to train the support vector machine classifier. The experiment is carried out on the basis of image preprocessing, and the CNDFA method and other feature extraction methods, such as wavelet, gray level co-occurrence matrix, gray level difference statistics, auto-correlation function, and MF, are applied to the contour feature extraction of coal and gangue. Through the comprehensive comparison of the results after different methods recognition in confusion matrix, accuracy, and coal cleanliness, it is concluded that the overall effect of the CNDFA method is better than other methods, and the accuracy is improved by 5% to 25%. The results show that the CNDFA method has better performance. Compared with the other methods, it can better extract the contour features to improve the recognition rate of coal and gangue.

The extraction of dynamic motion features are reported from multiple video frames by three unsupervised learning algorithms, i.e., Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Non-negative Matrix Factorization (NMF). Since the human perception of facial motion goes through two different pathways, i.e., the lateral fusifom gyrus for the invariant aspects and the superior temporal sulcus for the changeable aspects of faces, we extracted the dynamic video features from multiple consecutive frames for the latter. Both the original videos and the frame-difference sequences are used for comparison. The required number of multiframe features for the same representation accuracy is almost independent upon the frame length. Therefore, the multiple-frame features are much more efficient for video representation than the single-frame static features. The extracted features are also used for lipreading, and the features from frame-difference sequences demonstrated better recognition rates than those from original videos.

A dynamic gesture recognition and understanding method in natural human-computer interaction under desktop environment is proposed, including the “reach”, “take up”, “move”, “put down”, “return”, “point” and other natural interactive gestures. In preprocess procedure of each frame of the video, the Gaussian background model and HSV skin-color model is employed to remove background and segment hand gestures. The temporal and spatial information of multi frame images is combined to construct temporal and spatial features of dynamic gestures images. Then a convolution neural network is built for recognize the dynamic characteristics of gesture image. Finally, the classification result is denoised to achieve the robust recognition and understanding of gestures. Experimental results show that the proposed method has a good ability of recognizing and understanding the dynamic gestures in the desktop environment.

Gesture is a natural form of human communication, and it is of great significance in human–computer interaction. In the dynamic gesture recognition method based on deep learning, the key is to obtain comprehensive gesture feature information. Aiming at the problem of inadequate extraction of spatiotemporal features or loss of feature information in current dynamic gesture recognition, a new gesture recognition architecture is proposed, which combines feature fusion network with variant convolutional long short‐term memory (ConvLSTM). The architecture extracts spatiotemporal feature information from local, global and deep aspects, and combines feature fusion to alleviate the loss of feature information. Firstly, local spatiotemporal feature information is extracted from video sequence by 3D residual network based on channel feature fusion. Then the authors use the variant ConvLSTM to learn the global spatiotemporal information of dynamic gesture, and introduce the attention mechanism to change the gate structure of ConvLSTM. Finally, a multi‐feature fusion depthwise separable network is used to learn higher‐level features including depth feature information. The proposed approach obtains very competitive performance on the Jester dataset with the classification accuracies of 95.59%, achieving state‐of‐the‐art performance with 99.65% accuracy on the SKIG (Sheffifield Kinect Gesture) dataset.

In this paper, soft segment model has been evaluated and optimized on a phoneme recognition system in various aspects and the concept of soft segment recognition has been developed as an optimum recognition algorithm for this model. The main idea of this recognition method is consideration of the neighbor segments effects in recognition phase in the same way that has been applied in the training phase. The weighing procedure of feature vectors is modeled as repetitions of the vectors with frequencies less than one. Comparing the results of this model with classic models, it has been shown that there is an impressive improvement in both recognition rate and time at the optimum overlap coefficient. The selection between hard or soft recognition methods is a trade off between recognition rate and time. Comparing with standard CDHMM, a drastic decrease in recognition time and a little increase in recognition rate have been observed in the achieved results, which it shows more compatibility of the proposed model with the speech nature in comparison with other models.

In recent years, gestures have been widely used in many fields. For example, human-computer interaction, virtual reality, gesture translation, etc. The reason for its rapid development is due to the emergence of deep learning and artificial intelligence under today’s society. Due to dynamic gestural interactions, such large intelligent models are often characterized by many parameters, large sample size, frequent parameter updates, and high communication volume. Based on this feature, we propose a cloud-based edge design architecture approach for gesture recognition based on an improved YOLOv5 network model optimization by changing different gestures, background interference, which uses a 21-layer model for the neural network. With the use of edge intelligence, the computational accuracy can be improved by 10.6% over the traditional YOLOv5 with a MAP value of 93.3%. The final recall rate is also improved by 3.6%. The parameter model is only 43.6% of the original one. This shows the practicality and operability of using edge computing as a technique in gesture recognition, as well as the small improvement cost and obvious effect.

Climate change has increased the frequency of various types of meteorological disasters in recent years. Finding the primary factors that limit the emergency response capability of meteorological disasters through the evaluation of that capability and proposing corresponding improvement measures in order to increase that capability is of great practical importance. The evaluation of meteorological disaster emergency response capability still has some issues. The majority of research methods use qualitative analysis, which makes it challenging to deal with fuzzy factors, leading to conclusions that are subjective and insufficiently rigorous. The evaluation models themselves are also complex and challenging to simulate and analyze, making it challenging to promote and use them in practice. Deep learning techniques have made it easier to collect and process large amounts of data, which has opened new avenues for advancement in the emergency management of weather-related disasters. In this paper, we suggest a Recurrent Neural Network (RNN)-based dynamic capability feature extraction method. The process of evaluation content determination and index selection is used to build a meteorological disaster emergency response capability evaluation index system before an encoder, based on the encoder–decoder architecture, is built for dynamic feature extraction. The RNN autoencoder deep learning ability dynamic rating method used in this paper has been shown through a series of experiments to be able to not only efficiently extract ability features from time series data and reduce the dimensionality of ability features, but also to reduce the focus of the ability evaluation model on simple and abnormal samples, concentrate the model learning on difficult samples, and have a higher accuracy. As a result, it is more suitable for the problem situation at evaluation of the disaster capability.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

This work proposes a decision-aid tool for detecting Alzheimer’s disease (AD) at an early stage, based on the Archimedes spiral, executed on a Wacom digitizer. Our work assesses the potential of the task as a dynamic gesture and defines the most pertinent methodology for exploiting transfer learning to compensate for sparse data. We embed directly in spiral trajectory images, kinematic time functions. With transfer learning, we perform automatic feature extraction on such images. Experiments on 30 AD patients and 45 healthy controls (HC) show that the extracted features allow a significant improvement in sensitivity and accuracy, compared to raw images. We study at which level of the deep network features have the highest discriminant capabilities. Results show that intermediate-level features are the best for our specific task. Decision fusion of experts trained on such descriptors outperforms low-level fusion of hybrid images. When fusing decisions of classifiers trained on the best features, from pressure, altitude, and velocity images, we obtain 84% of sensitivity and 81.5% of accuracy, achieving an absolute improvement of 22% in sensitivity and 7% in accuracy. We demonstrate the potential of the spiral task for AD detection and give a complete methodology based on off-the-shelf features.

This paper presents a novel vision-based hand gesture recognition (HGR) and training system for a human-like robot hand. We implemented and trained a multiclass-SVM classifier and N-Dimensional DTW (ND-DTW) classifier for static posture recognition and dynamic gesture recognition. Training features were extracted from the raw gestures depth data captured by Leap Motion Controller. The experimental results show that multiclass SVM method has an average 98.25% recognition rates and the shortest run time when compared to k-NN and ANBC. For dynamic gestures, ND-DTW classifier displays a better performance than DHMM with an average 95.5% recognition rate and significantly shorter run time. In conclusion, the combination of SVMs and DTW proves the efficiency and high accuracy in proposed human-robot interaction system.

There are some problems in the current human motion target gesture recognition algorithms, such as classification accuracy, overlap ratio, low recognition accuracy and recall, and long recognition time. A gesture recognition algorithm of human motion based on deep neural network was proposed. First, Kinect interface equipment was used to collect the coordinate information of human skeleton joints, extract the characteristics of motion gesture nodes, and construct the whole structure of key node network by using deep neural network. Second, the local recognition region was introduced to generate high-dimensional feature map, and the sampling kernel function was defined. The minimum space-time domain of node structure map was located by sampling in the space-time domain. Finally, the deep neural network classifier was constructed to integrate and classify the human motion target gesture data features to realize the recognition of human motion target. The results show that the proposed algorithm has high classification accuracy and overlap ratio of human motion target gesture, the recognition accuracy is as high as 93%, the recall rate is as high as 88%, and the recognition time is 17.8 s, which can effectively improve the human motion target attitude recognition effect.

SUMMARYThis paper presents a method to improve the speed and accuracy rate for space robot visual target recognition based on illumination and affine invariant feature extraction. The method takes illumination changes, strong nonlinear light due to refraction and reflection, target affine transformation and occlusion into consideration, all of which occur on the cabin target surface and affect the target recognition accuracy seriously. In this paper, a method is proposed to capture a same target at multi-viewpoints to establish feature library for high recognition accuracy and speed at any viewpoint. By using an analysis of the light intensity and gray level transformation, we obtain the corrected image which reduce the influence of illumination change. Then the affine moment invariants features of the correction images at multi-viewpoints were extracted and the average feature datum were stored in the library. To verify the validity of the method, a robot vision system provided images, followed by image preprocessing, dynamic local threshold segmentation and feature extraction. These methods were verified on a target recognition system of space robot built for this research. The experimental results showed that the methods were feasible and effective.