Taekwondo Action Recognition Method Based on Partial Perception Structure Graph Convolution Framework

The recognition of sports action is an important research subject, which is conducive to the improvement of athletes’ own level. To improve the accuracy of multi-modal data action recognition, based on the Transformer module, this study introduces a multi-head attention mechanism, fuses multi-modal data, and constructs a multi-stream structured object relationship inference network. Based on PointNet++ network and combining five different data fusion frameworks, a motion recognition model that integrates RGB data and 3D skeleton point cloud is constructed. The results showed that the Top-1 accuracy of multi-stream structured object relationship inference network was 42.5% and 42.7%, respectively, which was better than other algorithms. The accuracy of the multi-modal fusion model was improved by 15.6% and 5.1% compared with the single mode, and by 5.4% and 2.6% compared with the dual mode, which showed its superiority in the action recognition task. This showed that the fusion of multi-modal data can provide more abundant information, so as to improve the accuracy of action recognition. The accuracy of the action recognition model combining RGB data and 3D skeleton point cloud was 84.3%, 87.5%, 90.2%, 90.6% and 91.2% after the combination of different strategies, which effectively compensated for the problem of missing information in 3D skeleton point cloud and significantly improved the accuracy of action recognition. With a small amount of data, the Top-1 accuracy of the multi-stream structured object relationship inference network in this study was superior to other algorithms, showing its advantages in dealing with complex action recognition tasks. In addition, the action recognition model that fuses RGB data and 3D skeleton point cloud also achieved higher accuracy, which is better than other algorithms. This study can meet the needs of motion recognition in different scenarios and has certain reference value.

Action recognition methods based on human skeleton can explicitly represent human actions, and have gradually become one of the important research directions in the field of computer vision. To address the problems that the skeleton graph in graph convolutional networks is fixed to represent only the physical structure of the human body and the lack of adaptive ability to the skeleton topology graph structure, this paper proposes a dual-stream non-local graph convolutional network based on the attention mechanism. First, the temporal convolution layer is extended to a parallel structure with multiple kernels, and different temporal convolution kernel modules are adaptively selected to collect features by channel weights; second, an attention model consisting of an attention pooling layer is proposed to capture the correlation and temporal continuity among joints; finally, the nonlocal graph convolution network is used as the basic framework with joint information, skeletal information and respective motion information A dual-stream fusion model is constructed. The proposed method is compared with the mainstream methods in recent years on the action recognition dataset NTU RGB+D, and the experimental results show that the proposed method achieves high accuracy in action recognition.

Video action recognition technology provides technical support for automated monitoring and early event warning, liberating human resources, preventing anomalies in advance, and handling events more promptly. However, the issue of user privacy leakage is also a growing concern. This paper discusses how to conduct action recognition while protecting privacy, proposing a generative adversarial network architecture that combines a multi-scale feature fusion generator with a spatiotemporal consistency discriminator. The network continuously enhances its capabilities through adversarial training strategies to protect facial privacy in videos. This network extracts features at different levels in the video through a multi-scale feature fusion mechanism so that the video after facial privacy protection still maintains a high degree of realism at the same time, to ensure that the accuracy of action recognition is not compromised, the feature enhancement module is designed to enhance action features and inhibit the privacy features significantly C3D is used as the action recognition model to accurately recognize a variety of actions in the video, such as running, jumping, falling, realizing the action analysis of video content. In this paper, the proposed method is evaluated in terms of privacy protection level and action recognition performance. Experiments are conducted on three datasets, LFW, HMDB51, and Hollywood2, and the results show that this framework effectively protects personal information while maintaining high action recognition accuracy.

SUMMARYRemote surveillance of large‐scale equipment such as power plants and building complexes is important to prevent serious attacks and troubles. Automatic human action recognition can reduce the burdens of the surveillance. Multiview video is useful for human action recognition, because it provides robustness to the changes of people's appearance by orientation and occlusion. One problem of conventional multiview action recognition is that it requires both detection and tracking before action recognition. Human pose and motion vary depending on the person's action, and such variances may cause detection and tracking error. To solve this problem, previous work has proposed simultaneous action recognition and location estimation for single‐view videos using Hough voting. In this paper, we extend the Hough voting approach to simultaneous multiview action recognition and location estimation. Our proposed method independently casts votes for the action labels and spatiotemporal reference positions of actions in each view and integrates them using homographical transformations in the multiview extension. We evaluated our method and confirmed that it achieved high accuracy in action recognition and location estimation. The contribution of this paper is that it enables multiview action recognition without prior human detection and tracking.

With the convenience brought by the rapid development of information, teaching reform has also set off a heat wave. The emergence of Internet+ , multimedia technology, virtual reality technology and artificial intelligence has made education continue to develop towards informatization, and mobile learning has become a way for students to learn. This paper provides a reference for the reform of exercise classes in colleges and universities, enriches the teaching methods, improves the learning quality of students, and makes the action image processing technology better applied to physical education classrooms. In this paper, according to the characteristics of the human motion system, the training actions of sports are collected. After the 3D action data is properly calibrated and normalized, the motion vector is used as the training input method. This method conforms to the laws of the human motion system. In terms of temporal feature extraction, this paper proposes a multi-level long short-term memory neural network structure for action recognition. The action image processing technology improves the accuracy of movements more significantly, and proves that it has a certain promotion effect on the self-editing ability and innovation ability of students'. The final results of the research show that the accuracy of action recognition has been improved to a certain extent after image data processing. The accuracy of action recognition in images has increased from 67.13% to 91.23%, and the accuracy rate has remained above 90% after action image processing which proves the effectiveness of the neural network algorithm processing method.

Graph convolution network can efficiently obtain the spatial features in the task of human action recognition based on skeleton information, and HSGAC network incorporating graph attention convolution mechanism could obtain more abstract spatial features. However, HSGAC does not consider the spatial dynamic importance of joints. In this paper, we propose a two-stream graph attention convolutional network(Two-Stream GAC), which aims to fuse the static and dynamic feature relationships between joints. Two-Stream GAC first preprocesses the original human pose information dynamically, and use two HSGAC models to obtain the static feature relationship of the original pose information and the dynamic feature relationship of the preprocessed pose information respectively. The two network models are fused by weighted sum. The experimental results on the RDT and NTU-RGB+D datasets show that the method achieves better accuracy of action recognition.

The technique of human action recognition can be applied in a number of fields, such as medical rehabilitation, posture in the domain of sports, and emotion perception in counselling. Deep learning action recognition models that consider the continuous changes in the human skeleton can efficiently identify action states, such as rehabilitation processes, posture, and changes in emotion. However, the keypoints of the human skeleton are susceptible to occlusion due to body movement, resulting in a lower confidence level in their positions, which in turn affects the training and execution of human action recognition models. This paper proposes a keypoint correction algorithm based on continuous-time images of temporal variation that can predict and correct the skeleton keypoints with low confidence, based on the previous and subsequent multiple images. The proposed temporal-variation skeleton keypoint correction algorithm can provide accurate feature association training for the locations of the skeleton keypoints. The proposed method is shown to improve the accuracy of human action recognition by at least 30% over three alternative models in the literature: STV-GCN, GCN-NAS, and 2S-AGCN. The proposed algorithm is also improved by at least 30% compared with the above-mentioned methods in the literature, based on an action recognition model with a Gaussian filter.

In the field of human action recognition, a good action classification algorithm has always been the key to measure the accuracy of action recognition. Since the trajectory of human targets is often complex, the recognition of RGB color images is restricted by many influencing factors, such as light intensity, complex background and angle. In this paper, kinect equipment is used to collect depth image data, and the image is generated into bone data. The Alexnet model in convolutional neural network is used to train and classify the data, which effectively improves the accuracy of action recognition.

Physical activity has a positive impact on people's well-being, and it may also decrease the occurrence of chronic diseases. Activity recognition with wearable sensors can provide feedback to the user about his/her lifestyle regarding physical activity and sports, and thus, promote a more active lifestyle. So far, activity recognition has mostly been studied in supervised laboratory settings. The aim of this study was to examine how well the daily activities and sports performed by the subjects in unsupervised settings can be recognized compared to supervised settings. The activities were recognized by using a hybrid classifier combining a tree structure containing a priori knowledge and artificial neural networks, and also by using three reference classifiers. Activity data were collected for 68 h from 12 subjects, out of which the activity was supervised for 21 h and unsupervised for 47 h. Activities were recognized based on signal features from 3-D accelerometers on hip and wrist and GPS information. The activities included lying down, sitting and standing, walking, running, cycling with an exercise bike, rowing with a rowing machine, playing football, Nordic walking, and cycling with a regular bike. The total accuracy of the activity recognition using both supervised and unsupervised data was 89% that was only 1% unit lower than the accuracy of activity recognition using only supervised data. However, the accuracy decreased by 17% unit when only supervised data were used for training and only unsupervised data for validation, which emphasizes the need for out-of-laboratory data in the development of activity-recognition systems. The results support a vision of recognizing a wider spectrum, and more complex activities in real life settings.

Existing studies in skeleton‐based action recognition mainly utilise skeletal data taken from a single camera. Since the quality of skeletal tracking of a single camera is noisy and unreliable, however, combining data from multiple cameras can improve the tracking quality and hence increase the recognition accuracy. In this study, the authors propose a method called weighted averaging fusion which merges skeletal data of two or more camera views. The method first evaluates the reliability of a set of corresponding joints based on their distances to the centroid, then computes the weighted average of selected joints, that is, each joint is weighted by the overall reliability of the camera reporting the joint. Such obtained, fused skeletal data are used as the input to the action recognition step. Experiments using various frame‐level features and testing schemes show that more than 10% improvement can be achieved in the action recognition accuracy using these fused skeletal data as compared with the single‐view case.

The recent development of wireless wearable sensor networks offers a spectrum of new applications in fields of healthcare, medicine, activity monitoring, sport, safety, human-machine interfacing, and beyond. Successful use of this technology depends on lifetime of the battery-powered sensor nodes. This paper presents a new method for extending the lifetime of the wearable sensor networks by avoiding unnecessary data transmissions. The introduced method is based on embedded classifiers that allow sensor nodes to decide if current sensor readings have to be transmitted to cluster head or not. In order to train the classifiers, a procedure was elaborated, which takes into account the impact of data selection on accuracy of a recognition system. This approach was implemented in a prototype of wearable sensor network for human activity monitoring. Real-world experiments were conducted to evaluate the new method in terms of network lifetime, energy consumption, and accuracy of human activity recognition. Results of the experimental evaluation have confirmed that, the proposed method enables significant prolongation of the network lifetime, while preserving high accuracy of the activity recognition. The experiments have also revealed advantages of the method in comparison with state-of-the-art algorithms for data transmission reduction.

Using skeleton data to classify human behavior has been received a lot of attention and various technologies have been developed because it can be utilized in various domains. Among them, Graph Convolutional Networks (GCN) shows high accuracy in action recognition through the effective extraction of features from graph data. GCN has been further developed and two stream adaptive graph convolution networks (2s-AGCN) are made. This model utilizes bone data as well as spatial and temporal information of joints to compensate for the problems of conventional GCN limited to local region. Although 2s-AGCN show the highest accuracy among existing technologies, new approaches using CNN, LSTM and auto encoder continue to emerge and present several methods. In this paper, we verify if 2s-AGCN show higher accuracy than recent CNN-LSTM networks and compare structures and features of their models.

EAAR: Efficient and Accurate Action Recognition model with enhanced spatio-temporal perception.

Understanding semantic meaning of human actions captured in unconstrained environments has broad applications in fields ranging from patient monitoring, human-computer interaction, to surveillance systems. However, while great progresses have been achieved on automatic human action detection and recognition in videos that are captured in controlled/constrained environments, most existing approaches perform unsatisfactorily on videos with uncontrolled/unconstrained conditions (e.g., significant camera motion, background clutter, scaling, and light conditions). To address this issue, the authors propose a robust human action detection and recognition framework that works effectively on videos taken in controlled or uncontrolled environments. Specifically, the authors integrate the optical flow field and Harris3D corner detector to generate a new spatial-temporal information representation for each video sequence, from which the general Gaussian mixture model (GMM) is learned. All the mean vectors of the Gaussian components in the generated GMM model are concatenated to create the GMM supervector for video action recognition. They build a boosting classifier based on a set of sparse representation classifiers and hamming distance classifiers to improve the accuracy of action recognition. The experimental results on two broadly used public data sets, KTH and UCF YouTube Action, show that the proposed framework outperforms the other state-of-the-art approaches on both action detection and recognition.

The accuracy of video-based action recognition depends largely on the extraction and utilization of optical flow, especially in two-stream networks. The original intention of the introduction of optical flow is to use the time information contained in video, however, the subsequent work shows that optical flow is useful for action recognition because it is invariant to appearance. In this article, we study and discuss this point of view, and propose optical flow enhancement algorithms to improve action recognition accuracy. Our enhancement algorithms improve the invariance to appearance of the representation in optical flow without losing time information, and every action recognition network with optical flow can benefit from our algorithms. We conduct a series of experiments to validate the influence of the proposed algorithms with TSN in terms of several datasets and optical flow calculation methods. As a result, we prove that first order differential algorithms are effective, TSN with our enhancement module significantly outperform original network. Based on these experiments, we also verify the importance of invariance to appearance in optical flow, and provide a reference for the follow-up study of improving action recognition accuracy.