A micro-Doppler spectrogram denoising algorithm for radar human activity recognition

In this paper, we present a novel method of 3-D XYT feature extraction for efficient human action recognition. We propose the spatial first 3D feature extraction method to reduce the computation time and data bandwidth compared with Harris 3D corner detection. We also present experimental results through the proposed spatial first feature extraction to reach 83.33% accuracy of human action recognition through KTH dataset. Finally, we demonstrate the 67.8% computation time and 66.7% bandwidth reduction for the efficient human action recognition system.

BackgroundIn digital medicine, human activity recognition (HAR) can be used to track and assess a patient's progress throughout rehabilitation, enhancing the quality of life for the elderly and the disabled.MethodsA patch-type flexible sensor that integrated dynamic electrocardiogram (ECG) and acceleration signal (ACC) was used to record the signals of the various behavioral activities of 20 healthy volunteers and 25 patients with pneumoconiosis. Seven HAR tasks were then carried out on the data using four different deep learning methods (CNN, LSTM, CNN–LSTM and GRU).ResultsWhen ECG and ACC were obtained simultaneously, the overall accuracy rates of HAR for healthy group were 0.9371, 0.8829, 0.9843 and 0.9486 by the CNN, LSTM, CNN–LSTM and GRU models, respectively. In contrast, the overall accuracy rates of HAR for the pneumoconiosis patients’ group were 0.8850, 0.7975, 0.9425 and 0.8525 by the four corresponding models. The accuracy of HAR for both groups using all four models is higher than when only ACC signal is detected.ConclusionThe addition of the ECG signal significantly improves HAR outcomes in the group of healthy individuals, while having relatively less enhancing effects on the group of patients with pneumoconiosis. When ECG and ACC signals were combined, the increase in HAR accuracy was notable compared to cases where no ECG data was provided. These results suggest that the combination of ACC and ECG data can represent a novel method for the clinical application of HAR.

Accurate human activity recognition (HAR) is the key to enable emerging context-aware applications that require an understanding and identification of human behavior, e.g., monitoring disabled or elderly people who live alone. Traditionally, HAR has been implemented either through ambient sensors, e.g., cameras, or through wearable devices, e.g., a smartwatch, with an inertial measurement unit (IMU). The ambient sensing approach is typically more generalizable for different environments as this does not require every user to have a wearable device. However, utilizing a camera in privacy-sensitive areas such as a home may capture superfluous ambient information that a user may not feel comfortable sharing. Radars have been proposed as an alternative modality for coarse-grained activity recognition that captures a minimal subset of the ambient information using micro-Doppler spectrograms. However, training fine-grained, accurate activity classifiers is a challenge as low-cost millimeter-wave (mmWave) radar systems produce sparse and non-uniform point clouds. In this paper, we propose RadHAR, a framework that performs accurate HAR using sparse and non-uniform point clouds. RadHAR utilizes a sliding time window to accumulate point clouds from a mmWave radar and generate a voxelized representation that acts as input to our classifiers. We evaluate RadHAR using a low-cost, commercial, off-the-shelf radar to get sparse point clouds which are less visually compromising. We evaluate and demonstrate our system on a collected human activity dataset with 5 different activities. We compare the accuracy of various classifiers on the dataset and find that the best performing deep learning classifier achieves an accuracy of 90.47%. Our evaluation shows the efficacy of using mmWave radar for accurate HAR detection and we enumerate future research directions in this space.

With the popularity of wearable smart devices, human activity recognition (HAR) based on smart sensors has been widely applied in daily life and medical health fields. In order to balance the accuracy of HAR and the complexity of the algorithm, this paper proposes an HAR algorithm based on the extreme gradient boosting (XGBoost) method. The original data collected by sensors contains noise, thus denoising process is firstly performed. Then multi-dimensional features are extracted from the data because of the limited dimensional features, which cannot be directly utilized in the training process. After that, principal component analysis (PCA) is used to reduce the dimensionality of the data in order to alleviate the input complexity of the training model. Finally, the human activity is recognized using the XGBoost method, and the ultimate goal is to obtain the tradeoff between speed and accuracy of the HAR algorithm.

Targeting the problems of the insufficient utilization of temporal and spatial information in videos and a lower accuracy rate, this paper proposes a human action recognition method for dynamic videos of emergency rescue based on a spatial-temporal fusion network. A time domain segmentation strategy based on random sampling maintains the overall time domain structure of the video. Considering the spatial-temporal asynchronous relationship, multiple asynchronous motion sequences are increased as input of the temporal convolutional network. spatial-temporal features are fused in convolutional layers to reduce feature loss. Because time series information is crucial for human action recognition, the acquired mid-layer spatial-temporal fusion features are sent into Bidirectional Long Short-Term Memory (Bi-LSTM) to obtain the human movement features in the whole video temporal dimension. Experiment results show the proposed method fully fuses spatial and temporal dimension information and improves the accuracy of human action recognition in dynamic scenes. It is also faster than traditional methods.

In order to improve the accuracy of human action recognition and accelerate the recognition speed, we propose a method for human action recognition by modeling the human primary visual cortex neurons. The method firstly extracted motion information by using 3DGabor spatial-temporal filters to model the classical receptive field (CRF) of simple cells in the primary visual cortex. Secondly, conductance-driven integrate and fire neuron model was used to simulate the primary visual cortex neuron, by which motion information was converted into spike train. Finally, the mean firing rate of spike train formed a feature vector that captures the characteristic of human actions in this video sequence. Using Support Vector Machine (SVM), the method is tested on the Weizmann action dataset. The obtained impressive results show that our method was more effective than model of Escobar in human action recognition.

Human action recognition is a fundamental research problem in computer vision. The accuracy of human action recognition has important applications. In this book chapter, the authors use a YOLOv7-based model for human action recognition. To evaluate the performance of the model, the action recognition results of YOLOv7 were compared with those using CNN+LSTM, YOLOv5, and YOLOv4. Furthermore, a small human action dataset suitable for YOLO model training is designed. This data set is composed of images extracted from KTH, Weizmann, MSR data sets. In this book chapter, the authors make use of this data set to verify the experimental results. The final experimental results show that using the YOLOv7 model for human action recognition is very convenient and effective, compared with the previous YOLO model.

Along with the development of the Industrial Internet of Things, human activity recognition (HAR) has received widespread attention in many fields. Support Vector Machine (SVM), is widely used by researchers for human activity recognition. However, the inherent difference of signal properties from different sensors and various orientations is potentially lost when using the vector-based SVM for human activity recognition. What's more, the outlier sensitivity problem of the standard SVM reduces the accuracy of human activity recognition. To tackle this problem, we present a tensor-based feature representation model and a weighted support tensor machine (WSTM) for human activity recognition. Specifically, tensor-based representations are first used to model features from different sensors and various orientations to retain the latent relationship. In addition, the weighted support tensor machine is proposed to classify the human activities in tensor space while avoiding the outlier sensitivity problem. Experimental results demonstrate the proposed WSTM algorithm.

Hyperparameter optimization poses a significant challenge when developing deep neural networks. Building a convolutional neural network (CNN) for implementation can be an arduous and time-intensive task. This work proposed an approach to optimize the hyperparameters of one dimensional (1D-CNN) to improve the accuracy of human activity recognition (HAR). The framework includes a parametric depiction of 1D-CNNs along with an optimization process for hyperparameters aimed at maximizing the model's performance. This work designed the method called OPTConvNet for hyperparameter optimization of 1D-CNN using Hierarchical Particle Swarm Optimization (H-PSO). The H-PSO algorithm is designed to optimize the architectural, layer and training parameters of 1D-CNN. The H-PSO optimizes the architecture of the 1D-CNN at initial level. Layer and training hyperparameters will be optimized at the next level. The proposed approach employs an exponential-like inertia weight to fine-tune the balance between exploration and exploitation of particles to prevent premature convergence to a local optimum solution in the PSO algorithm. The H-PSO- CNN is evaluated on publicly available sensor- human activity recognition (S-HAR) datasets namely, UCI-HAR, Daphnet Gait, Opportunity and PAMPA2 datasets.

In order to improve the accuracy of human action recognition in video and the computational efficiency of large data sets, an action recognition algorithm based on multiple features and modified deep learning model is proposed. First, the deep network pre-training process is used to learn and optimize the RBM parameters, and the deep belief nets (DBN) model is constructed through deep learning. Then, human 13 joint points and critical points of optical flow are automatically extracted by DBN model. Second, these more abstract and more effective human motion features are combined to represent human actions. Ultimately, the entire DBN network structure is fine-tuned by support vector machine (SVM) algorithm to classify human actions. We demonstrate that human 13 joint points and critical points of optical flow are two very effective human action characterizations, our proposed approach greatly reduces the required samples, and shortens the training time of the samples, can efficiently process large data sets and can effectively recognize novel actions. We performed experiments on the KTH data set, Weizmann data set, the ballet data set and UCF101 data set to evaluate the proposed method, the experiment results show that the average recognition accuracy is over 98%, which validates its effectiveness, and show that our results are stable, reliable, and significantly better than the results of two state-of-the-art approaches on four different data sets. So, it lays a good theoretical foundation for practical applications.

Human activity recognition (HAR) can play a vital role in biomedical and health informatics by enabling the monitoring of human daily activities and health behaviors. Accurate HAR can provide valuable insights into patients' physical activity levels, and this helps to manage chronic conditions and promote healthy lifestyles. In this paper, we propose a deep learning model, DKInception, designed for HAR tasks. DKInception integrates deep convolutional residual networks with an attention mechanism and leverages multi-scale convolution kernels to efficiently extract temporal features for activity identification. This model is built on the Inception ResNet architecture and extends its capabilities with effective, fast convergence and robust scaling properties. To evaluate the performance of DKInception, we conduct extensive experiments on four benchmark HAR datasets: UCI-HAR, Opportunity, Daphnet, and PAMAP2. Our comparative analysis with several existing models shows that DKInception outperforms these models using various evaluation metrics. These results demonstrate the model recorded high accuracy of 95.70%, 87.48%, 94.00% and 89.72%, for UCI-HAR, Opportunity, Daphnet, and PAMAP2, respectively.

The technique of human action recognition can be applied in a number of fields, such as medical rehabilitation, posture in the domain of sports, and emotion perception in counselling. Deep learning action recognition models that consider the continuous changes in the human skeleton can efficiently identify action states, such as rehabilitation processes, posture, and changes in emotion. However, the keypoints of the human skeleton are susceptible to occlusion due to body movement, resulting in a lower confidence level in their positions, which in turn affects the training and execution of human action recognition models. This paper proposes a keypoint correction algorithm based on continuous-time images of temporal variation that can predict and correct the skeleton keypoints with low confidence, based on the previous and subsequent multiple images. The proposed temporal-variation skeleton keypoint correction algorithm can provide accurate feature association training for the locations of the skeleton keypoints. The proposed method is shown to improve the accuracy of human action recognition by at least 30% over three alternative models in the literature: STV-GCN, GCN-NAS, and 2S-AGCN. The proposed algorithm is also improved by at least 30% compared with the above-mentioned methods in the literature, based on an action recognition model with a Gaussian filter.

In this paper, we present a home-monitoring oriented human activity recognition benchmark database, based on the combination of a color video camera and a depth sensor. Our contributions are two-fold: 1) We have created a publicly releasable human activity video database (i.e., named as RGBD-HuDaAct), which contains synchronized color-depth video streams, for the task of human daily activity recognition. This database aims at encouraging more research efforts on human activity recognition based on multi-modality sensor combination (e.g., color plus depth). 2) Two multi-modality fusion schemes, which naturally combine color and depth information, have been developed from two state-of-the-art feature representation methods for action recognition, i.e., spatio-temporal interest points (STIPs) and motion history images (MHIs). These depth-extended feature representation methods are evaluated comprehensively and superior recognition performances over their uni-modality (e.g., color only) counterparts are demonstrated.

Human action recognition algorithms are the one of the effective schemes to analyze the action of the people in the surveillance environment.It is one part of the event recognition algorithm.There are many types of action recognition algorithms, and in this paper, some classical methods are surveyed.The detail procedure of human action recognition is described, and the basic theory application of some algorithms is also analyzed.Finally, the conclusion of this paper is given.

Wearables are objective tools for human activity recognition (HAR). Advances in wearables enable synchronized multi-sensing within a single device. This has resulted in studies investigating the use of single or multiple wearable sensor modalities for HAR. Some studies use inertial data, others use surface electromyography (sEMG) from multiple muscles and different post-processing approaches. Yet, questions remain about accuracies relating to e.g., multi-modal approaches, and sEMG post-processing. Here, we explored how inertial and sEMG could be efficiently combined with machine learning and used with post-processing methods for better HAR. This study aims recognition of four basic daily life activities; walking, standing, stair ascent and descent. Firstly, we created a new feature vector based on the domain knowledge gained from previous mobility studies. Then, a feature level data fusion approach was used to combine inertial and sEMG data. Finally, two supervised learning classifiers (Support Vector Machine, SVM, and the k-Nearest Neighbors, kNN) were tested with 5-fold cross-validation. Results show the use of inertial data with sEMG increased overall accuracy by 3.5% (SVM) and 6.3% (kNN). Extracting features from linear envelopes instead of bandpass filtered sEMG improves overall HAR accuracy in both classifiers. Clinical Relevance- Post-processing on sEMG signals can improve the performance of multimodal HAR.