Human Action Recognition Based on Oriented Motion Salient Regions

Recognising human activity can be advantageous in a number of different scenarios including elder care, healthcare or for training purposes. It can be of direct use to support humans in doing different activities, but is still a challenge for systems to correctly classify the activity in a way that is valuable for the user, as they often times lack the robustness or simplicity for day-to-day use. In this paper an approach for human activity recognition based on object interactions is presented. The proposed system consists of a wireless sensor network, with each sensor node measuring the received signal strength indication (RSSI) to its neighbouring nodes. The accumulated RSSI data is then analyzed by a machine learning algorithm which tries to infer one of several cooked dishes from that data. Experimental studies demonstrate promising results and therefore potential for this technology for recognising human activity in the form of cooking, but its generalised approach makes it suitable for other environments, too.

In this paper, a stereo camera-based novel approach for Human Activity Recognition (HAR) is presented using robust 3-D human body joint features and joint-specific Hidden Markov Models (HMMs). At f...

The existing recognition algorithms always need large number of samples to train models or templates. Nevertheless, they aren’t view-invariant and afiected by performing speed and scale variations of difierent actors. In this paper, we capture the skeleton information by Kinect and present a novel approach based on self-updating template for human action recognition using 4D Quaternions as a compact representation of postures. We adopt the new template generation method and update the template if it satisfles speciflc conditions. This method solves the above problems. On the other hand, we improve the decision function of DTW that the number of alignment paths may have a great in∞uence on recognition. We tested our approach on our dataset containing 960 action samples and the public MSR Action3D dataset. Compared with Li et al.’s [10], our approach produced better results on the most of cases, especially with the increased number of recognition.

This paper proposes a novel approach for human action recognition using multi-resolution feature extraction based on the two-dimensional discrete wavelet transform (2D-DWT). Action representations can be considered as image templates, which can be useful for understanding various actions or gestures as well as for recognition and analysis. An action recognition scheme is developed based on extracting features from the frames of a video sequence. The proposed feature selection algorithm offers the advantage of very low feature dimensionality and therefore lower computational burden. It is shown that the use of wavelet-domain features enhances the distinguish ability of different actions, resulting in a very high within-class compactness and between-class separability of the extracted features, while certain undesirable phenomena, such as camera movement and change in camera distance from the subject, are less severe in the frequency domain. Principal component analysis is performed to further reduce the dimensionality of the feature space. Extensive experimentations on a standard benchmark database confirm that the proposed approach offers not only computational savings but also a very recognition accuracy.

Human action recognition is a research hotspot in the field of ambient intelligence, serving as a foundation for the Internet of Healthcare Things (IoHT) and smart home wellness, with extensive application value. Currently, most research primarily focuses on the performance of single-modal approach for human action recognition. However, single-modal data cannot adequately capture the action characteristics of the human body. For instance, camera data lacks detailed micro-motion information, while radar data lacks visual appearance information. This limitation makes human action recognition systems susceptible to complex covariate influences. To address this issue, this paper proposes a Multi-Modal Fusion Model, M2FM, to accurately recognize various complex human action from millimeter-wave radar signals and video data. In the millimeter-wave radar branch, a LFNet network is constructed to capture richly hierarchical human action representation by extracting micro-Doppler feature and cadence velocity feature simultaneously. Specially, a Linear Feedforward Neural Network (LFNN) module is designed for modeling global-local feature of human action. In the video branch, a lightweight Transformer-based video analysis and action recognition network, STL-Former, is developed. Specially, an Agent downSampling (AS) attention module and a Linear Maxpooling (LM) attention module are designed to achieve context-aware downsampling with global receptive field and spatial vector sequence downsampling, respectively. Additionally, a multi-scale Linear attention (Litner) module is designed for modeling the spatio-temporal information of human action. Finally, the outputs from two branches are fused by a filter-based Dempster-Shafer theory. The experimental results on a self-collected Multi-modal Human Action Dataset, JH-MHAD, show that the M2FM model outperforms other advanced models, with an average recognition accuracy of 99.3% for eight different human actions, and it demonstrates strong adaptability under different conditions.

Human action recognition (HAR)in videos is a challenging task in computer vision. Conventional methods are prone to explore the spatiotemporal or optical representations for video actions. However, optical representation might be inefficient in some real-life situations, such as object occlusion and dim light. To address this issue, this paper presents a novel approach for human action recognition by jointly exploiting video and Wi-Fi clues. We leverage the fact that Wi-Fi signals carry discriminative information of human actions, which is robust to optical limitations. To validate this innovative thought, we conceive a practical framework for HAR and setup a dataset containing both video clips and Wi-Fi Channel State Information of human actions. The 3D convolutional neural network was used to extract the video features and the statistical algorithms were used to extract radio features. A classical linear support vector machine is employed as the classifier after the video and radio feature fusion. Comprehensive experiments on this dataset achieved desirable results with the maximum improvement in accuracy by 10%. This demonstrates our promising findings: with the aid of Wi-Fi Channel State Information, the performance of the video action recognition methods can be improved significantly, even under the optical limitation.

In this paper, we present a “key segment” mining approach for human action recognition. Our model is able to locate discriminative segments for action samples via multiple instance learning. Moreover, we propose a dynamic pooling approach to automatically find the optimal length of segment for each action sample. In addition, an effective feature is proposed for action recognition with 3D skeleton joints. It can effectively capture informative motion and shape cues of skeletons, and leads to a compact and discriminative representation. The experimental results validate the effectiveness of the proposed human action recognition method on two benchmark datasets (i.e., MSR Action3D and UCF-Kinect). Moreover, our method demonstrates superior accuracy than previous methods of using only skeleton data on MSR Action3D, and achieves the state-of-the-art performance on UCF-Kinect.Keywordsaction recognitionmultiple instance learningskeleton featuredynamic pooling

Human Motion Analysis by Fusion of Silhouette Orientation and Shape Features

Action recognition based on 2D information has encountered intrinsic difficulties such as occlusion and view etc. Especially suffering with complicated changes of perspective. In this paper, we present a straightforward and efficient approach for 3D human action recognition based on skeleton sequences. A rough geometric feature, termed planes of 3D joint motions vector (PoJM3D) is extracted from the raw skeleton data to capture the omnidirectional short-term motion cues. A customized 3D convolutional neural network is employed to learn the global long-term representation of spatial appearance and temporal motion information with a scheme called dynamic temporal sparse sampling (DTSS). Extensive experiments on three public benchmark datasets, including UTD-MVAD, UTD-MHAD, and CAS-YNU-MHAD demonstrate the effectiveness of our method compared to the current state-of-the-art in cross-view evaluation, and significant improvement in cross-subjects evaluation. The code of our proposed approach is available at released on GitHub.

This paper presents a new approach for human action recognition around sleeping with the human body parts locations and the positional relationship between human and sleeping environment. Body parts are estimated from the depth image obtained by a time-of-flight (TOF) sensor using oriented 3D normal vector. Issues in action recognition of sleeping situation are the demand of availability in darkness, and hiding of the human body by duvets. Therefore, the extraction of image features is difficult since color and edge features are obscured by covers. Thus, first in our method, positions of four parts of the body (head, torso, thigh, and lower leg) are estimated by using the shape model of bodily surface constructed by oriented 3D normal vector. This shape model can represent the surface shape of rough body, and is effective in robust posture estimation of the body hidden with duvets. Then, action descriptor is extracted from the position of each body part. The descriptor includes temporal variation of each part of the body and spatial vector of position of the parts and the bed. Furthermore, this paper proposes hierarchical action classes and classifiers to improve the indistinct action classification. Classifiers are composed of two layers, and recognize human action by using the action descriptor. First layer focuses on spatial descriptor and classifies action roughly. Second layer focuses on temporal descriptor and classifies action finely. This approach achieves a robust recognition of obscured human by using the posture information and the hierarchical action recognition.

In this work we present an ongoing research which intends to integrate the Block Matching technique into the Human Action Recognition field. We propose a method to create a video descriptor composed of three main steps: Computation of displacement vectors between frames with VSBMA, histogram representation of these vectors, and orientation tensor generation from the histogram. It is a promising approach for human action recognition since it is a simple method with low computational cost which achieves comparable results with state-of-the-art techniques.

ABSTRACTIn the present work, we have proposed an efficient approach for human action recognition (HAR) from silhouette image sequence in videos. The efficiency of the approach lies in feature extraction and action classification. The proposed approach includes scale-shift normalization and distorted silhouette removal for the extraction of newly introduced spatiotemporal features coined as active region energy feature (AREF), and trajectory analysis. On the other hand, classification is done using hierarchical structure. An active region is the changing region in two consecutive silhouettes to accomplish the action. The AREF is estimated using active region energy image (AREI), which embraces the energies of active regions. The higher values in the AREI signify the more activeness (changing) of that region across the silhouette sequences; i.e. the region is used more (active) to complete the action. The silhouette normalization technique makes the feature extraction more robust and scale invariant. Also, the proposed approach uses a low-dimensional feature vector, which makes the whole procedure effective regarding cost in terms of timing requirement. The experimental results on publicly available Weizmann and MuHVAi data-sets clearly validate the efficiency of the proposed technique on that of the related research work regarding accuracy in the human action detection.

The emerging research work in human activity recognition is required for smart environment especially in health and home applications. This paper proposed a bio-inspired human visual system model to extract motion features for human activity recognition. The architecture of classical V1-MT model is modified into hierarchical architecture of feedforward model; (1) three-dimensional (3D) Gabor filter in primary visual cortex (V1) layer, (2) new pooling mechanism of active motion map (3) final motion estimation based on confidence mapping in MT layer. The 3D Gabor filter used to model the spatio-temporal direction and speed tuning of the V1 cells. The active mean motion map indicate the most significant feature vector in representing human actions, while the confidence map used to estimate accurate velocity characteristic of neurons in MT layer. The approach is carried out on the Weizmann and KTH action database. The results demonstrate the effectiveness of proposed method and its advantage in terms of recognition performance and computational efficiency compared to other bio-inspired approaches for human action recognition.

The correct execution of well-defined movements plays a crucial role in physical rehabilitation and sports. While there is an extensive number of well-established approaches for human action recognition, the task of assessing the quality of actions and providing feedback for correcting inaccurate movements has remained an open issue in the literature. We present a learning-based method for efficiently providing feedback on a set of training movements captured by a depth sensor. We propose a novel recursive neural network that uses growing self-organization for the efficient learning of body motion sequences. The quality of actions is then computed in terms of how much a performed movement matches the correct continuation of a learned sequence. The proposed system provides visual assistance to the person performing an exercise by displaying real-time feedback, thus enabling the user to correct inaccurate postures and motion intensity. We evaluate our approach with a data set containing 3 powerlifting exercises performed by 17 athletes. Experimental results show that our novel architecture outperforms our previous approach for the correct prediction of routines and the detection of mistakes both in a single- and multiple-subject scenario.

Human action recognition has been an active research area in recent years. However, building a robust human action recognition system still remains a challenging task due to the large variations in action classes, varying human appearances, illumination changes, camera motion, occlusions and background clutter. Most previous work focus on the goal of improving recognition rates. This paper describes a computationally fast votingbased approach for human action recognition, in which the action in the video sequence is recognized based on the support of the local spatio-temporal features. The proposed technique requires no parameter tuning and can produce recognition rates that are comparable to those in recent published literature. Moreover, the technique can localize the single human action in the video sequence without much additional computation. Recognition results on the KTH and Weizmann action dataset are presented.