Method for Undefined Complex Human Activity Recognition

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.

This paper presents some more recent developments in the research area of human activity recognition. In many cases, human activity recognition can be achieved with the support of body sensors that deliver information about the motion of a person or about acceleration and position. Such data can contribute substantially to the accuracy of the results, however wearing extra equipment is not a very popular practice for most human users and therefore camera-based methods are mostly preferred in today’s systems, which is also a result of the tremendous progress achieved in recent years in pattern recognition and machine learning. Therefore, also this paper concentrates on camera-based methods for human activity recognition. Activity recognition has a large variety of sub-research areas and has many exciting application areas, such as e.g. interaction in smart spaces, surveillance, human-robot interaction or automatic analysis of interest, emotions and human traits.

The recognition of human activities using vision-based techniques has become a crucial research field in video analytics. Over the last decade, there have been numerous advancements in deep learning algorithms aimed at accurately detecting complex human actions in video streams. While these algorithms have demonstrated impressive performance in activity recognition, they often exhibit a bias towards either model performance or computational efficiency. This biased trade-off between robustness and efficiency poses challenges when addressing complex human activity recognition problems. To address this issue, this paper presents a computationally efficient yet robust approach, exploiting saliency-aware spatial and temporal features for human action recognition in videos. To achieve effective representation of human actions, we propose an efficient approach called the dual-attentional Residual 3D Convolutional Neural Network (DA-R3DCNN). Our proposed method utilizes a unified channel-spatial attention mechanism, allowing it to efficiently extract significant human-centric features from video frames. By combining dual channel-spatial attention layers with residual 3D convolution layers, the network becomes more discerning in capturing spatial receptive fields containing objects within the feature maps. To assess the effectiveness and robustness of our proposed method, we have conducted extensive experiments on four well-established benchmark datasets for human action recognition. The quantitative results obtained validate the efficiency of our method, showcasing significant improvements in accuracy of up to 11% as compared to state-of-the-art human action recognition methods. Additionally, our evaluation of inference time reveals that the proposed method achieves up to a 74× improvement in frames per second (FPS) compared to existing approaches, thus showing the suitability and effectiveness of the proposed DA-R3DCNN for real-time human activity recognition.

In order to study the action recognition, tracking, and optimization of the training process based on the support vector regression model, a method of human action recognition based on support vector machine optimization is proposed. This method uses the improved strategy of support vector machine to realize the action recognition through the human action recognition based on the optimization of the vector machine. During the recognition, the DAG SVM strategy is improved according to the recognition accuracy of the classifier, and when outputting the result, output the recognition result and the corresponding confidence level, and use the confidence level to process the recognition result. Finally, through the experimental results, it is realized that the recognition rate based on support vector optimization is 98.7%, indicating that this method is effective and can improve the accuracy and efficiency of human body action recognition.

Human detection and action recognition form the basis for understanding human behaviors. Human detection is used to detect the positions of humans, and action recognition is able to recognize the action of specific humans. However, numerous approaches have been used to handle action recognition and human detection separately. Therefore, three main issues still exist when independent methods of human detection and action recognition are combined, 1) intrinsic errors in object detection impact the performance of action recognition, 2) features common to action recognition and object detection are missed, 3) the combination also has an impact on processing speed. We propose a single framework for human detection and action recognition to solve these issues. It is based on a hierarchical structure called Boosted Randomized Trees. The nodes are trained such that the upper nodes detect humans from the background, while the lower nodes recognize action. We were able to improve both human detection and action recognition rates over earlier hierarchical structure approaches with the proposed method.

Batteryless or so called passive wearables are providing new and innovative methods for human activity recognition (HAR), especially in healthcare applications for older people. Passive sensors are low cost, lightweight, unobtrusive and desirably disposable; attractive attributes for healthcare applications in hospitals and nursing homes. Despite the compelling propositions for sensing applications, the data streams from these sensors are characterised by high sparsity---the time intervals between sensor readings are irregular while the number of readings per unit time are often limited. In this paper, we rigorously explore the problem of learning activity recognition models from temporally sparse data. We describe how to learn directly from sparse data using a deep learning paradigm in an end-to-end manner. We demonstrate significant classification performance improvements on real-world passive sensor datasets from older people over the state-of-the-art deep learning human activity recognition models. Further, we provide insights into the model's behaviour through complementary experiments on a benchmark dataset and visualisation of the learned activity feature spaces.

In order to improve the accuracy of human action recognition and accelerate the recognition speed, we propose a method for human action recognition by modeling the human primary visual cortex neurons. The method firstly extracted motion information by using 3DGabor spatial-temporal filters to model the classical receptive field (CRF) of simple cells in the primary visual cortex. Secondly, conductance-driven integrate and fire neuron model was used to simulate the primary visual cortex neuron, by which motion information was converted into spike train. Finally, the mean firing rate of spike train formed a feature vector that captures the characteristic of human actions in this video sequence. Using Support Vector Machine (SVM), the method is tested on the Weizmann action dataset. The obtained impressive results show that our method was more effective than model of Escobar in human action recognition.

A hierarchical feature graph matching method for recognition of complex human activities

Exercise is an indispensable part of people’s lives and is closely related to their health. Human Activity Recognition (HAR), which involves detects and analyzes human body activity, has become the focus of current research. Photoplethysmography (PPG) has advantages such as convenience for detection and low cost, and is widely used in wearable devices becoming an ideal choice for HAR. In this study, we used wavelet scattering transform (WST) to extract features from PPG and then performed activity recognition on it. We achieved excellent classification accuracy of 92.54% and 97.76% respectively in the experiments of three-class and four-class exercise detection. The results showed this method based on wavelet scattering transform and PPG can accurately detect exercise types and provide effective support for HAR.

Human Activity Recognition (HAR), alongside Ambient Assisted Living (AAL), are integral components of smart homes, sports, surveillance, and investigation activities. To recognize daily activities, researchers are focusing on lightweight, cost-effective, wearable sensor-based technologies as traditional vision-based technologies lack elderly privacy, a fundamental right of every human. However, it is challenging to extract potential features from 1D multi-sensor data. Thus, this research focuses on extracting distinguishable patterns and deep features from spectral images by time-frequency-domain analysis of 1D multi-sensor data. Wearable sensor data, particularly accelerator and gyroscope data, act as input signals of different daily activities, and provide potential information using time-frequency analysis. This potential time series information is mapped into spectral images through a process called use of 'scalograms', derived from the continuous wavelet transform. The deep activity features are extracted from the activity image using deep learning models such as CNN, MobileNetV3, ResNet, and GoogleNet and subsequently classified using a conventional classifier. To validate the proposed model, SisFall and PAMAP2 benchmark datasets are used. Based on the experimental results, this proposed model shows the optimal performance for activity recognition obtaining an accuracy of 98.4% for SisFall and 98.1% for PAMAP2, using Morlet as the mother wavelet with ResNet-101 and a softmax classifier, and outperforms state-of-the-art algorithms.

Human activity recognition (HAR) has been increasingly used in medical care, behavior analysis, and entertainment industry to improve the experience of users. Most of the existing works use fixed models to identify various activities. However, they do not adapt well to the dynamic nature of human activities. We investigated the activity recognition with postural transition awareness. The inertial sensor data was processed by filters and we used both time domain and frequency domain of the signals to extract the feature set. For the corresponding posture classification, three feature selection algorithms were considered to select 585 features to obtain the optimal feature subset for the posture classification. And We adopted three classifiers (support vector machine, decision tree, and random forest) for comparative analysis. After experiments, the support vector machine gave better classification results than other two methods. By using the support vector machine, we could achieve up to 98% accuracy in the Multi-class classification. Finally, the results were verified by probability estimation.

Unsupervised learning for human activity recognition using smartphone sensors

In this paper, we present an original and efficient method of human action recognition in a video sequence. The proposed model is based on the generation and fusion of a set of prototypes generated from different view-points of the data cube of the video sequence. More precisely, each prototype is generated by using a multidimensional scaling (MDS) based nonlinear dimensionality reduction technique both along the temporal axis but also along the spatial axis (row and column) of the binary video sequence of 2D silhouettes. This strategy aims at modeling each human action in a low dimensional space, as a trajectory of points or a specific curve, for each viewpoint of the video cube in a complementary way. A simple K-NN classifier is then used to classify the prototype, for a given viewpoint, associated with each action to be recognized and then the fusion of the classification results for each viewpoint allow us to significantly improve the recognition rate performance. The experiments of our approach have been conducted on the publicly available Weizmann data-set and show the sensitivity of the proposed recognition system to each individual viewpoint and the efficiency of our multi-viewpoint based fusion approach compared to the best existing state-of-the-art human action recognition methods recently proposed in the literature.

Human action recognition from 2D videos is a demanding area due to its broad applications. Many methods have been proposed by the researchers for recognizing human actions. The improved accuracy in identifying human actions is desirable. This paper presents an improved method of human action recognition using support vector machine (SVM) classifier. This paper proposes a novel feature descriptor constructed by fusing the various investigated features. The handcrafted features such as scale invariant feature transform (SIFT) features, speed up robust features (SURF), histogram of oriented gradient (HOG) features and local binary pattern (LBP) features are obtained on online 2D action videos. The proposed method is tested on different action datasets having both static and dynamically varying backgrounds. The proposed method achieves shows best recognition rates on both static and dynamically varying backgrounds. The datasets considered for the experimentation are KTH, Weizmann, UCF101, UCF sports actions, MSR action and HMDB51.The performance of the proposed feature fusion model with SVM classifier is compared with the individual features with SVM. The fusion method showed best results. The efficiency of the classifier is also tested by comparing with the other state of the art classifiers such as k-nearest neighbors (KNN), artificial neural network (ANN) and Adaboost classifier. The method achieved an average of 94.41% recognition rate.

This paper proposes a human activity recognition (HAR) method for frequency-modulated continuous wave (FMCW) radar sensors. The method utilizes a multi-domain feature attention fusion network (MFAFN) model that addresses the limitation of relying on a single range or velocity feature to describe human activity. Specifically, the network fuses time-Doppler (TD) and time-range (TR) maps of human activities, resulting in a more comprehensive representation of the activities being performed. In the feature fusion phase, the multi-feature attention fusion module (MAFM) combines features of different depth levels by introducing a channel attention mechanism. Additionally, a multi-classification focus loss (MFL) function is applied to classify confusable samples. The experimental results demonstrate that the proposed method achieves 97.58% recognition accuracy on the dataset provided by the University of Glasgow, UK. Compared to existing HAR methods for the same dataset, the proposed method showed an improvement of about 0.9–5.5%, especially in the classification of confusable activities, showing an improvement of up to 18.33%.