A novel and stable human detection and behavior recognition method based on depth sensor

Big data analysis of human behavior can provide the basis and support for the application of various scenarios. Using sensors for human behavior analysis is an effective means of identification method, which is very valuable for research. To address the problems of low recognition accuracy, low recognition efficiency of traditional human behavior recognition (HBR) algorithms in complex scenes, in this paper, we propose an HBR algorithm for Mobile Big data analytics in wireless sensor network using improved transfer learning. First, different wireless sensors are fused to obtain human behavior mobile big data, and then by analyzing the importance of human behavior features (HBF), the dynamic change parameters of HBF extraction threshold are calculated. Second, combined with the dynamic change parameters of threshold, the HBF of complex scenes are extracted. Finally, the best classification function of human behavior in complex scenes is obtained by using the classification function of HBF in complex scenes. Human behavior in complex scenes is classified according to the HBF in the feature set. The HBR algorithm is designed by using the improved transfer learning network to realize the recognition of human behavior in complex scenes. The results show that the proposed algorithm can accurately recognize up to 22 HBF points, and can control the HBR time within 2 s. The human behavior false recognition rate of miscellaneous scenes is less than 10%. The recognition speed is above 10/s, and the recall rate can reach more than 98%, which improves the HBR ability of complex scenes.

Human behavior recognition is an active research field in computer vision and image processing. A novel method is proposed for human behavior recognition in video image sequences. First of all, a video sequence is represented by extracting space-time interest points. Then Human behavior is represented by activities through Motion Decomposition. The activity comprises labeled bags that are composed of unlabeled instances comprising to action. Final labeled activities are used to train a strong classifier which is used to predict the labels of unseen behavior bags. Experimental results show the effectiveness of the proposed method in comparison with other related works in the literature and can also tolerate noise and interference conditions.

Falling is a common but fatal human behavior in life. With the rapid growth of the aging population, fall-related human behavior recognition has been extensively investigated using radar. Nevertheless, human behavior recognition frequently exhibits suboptimal generalization capabilities due to the scarcity of labeled data. To address this problem, we propose physics-informed data augmentation (PIDA) for the micro-Doppler signature (m-DS) produced from frequency-modulated continuous wave (FMCW) radar to facilitate domain generalization (DG) of human behavior recognition. Specifically, leveraging classical electromagnetic wave propagation theory and radar signal processing principles, we introduce a PIDA strategy: 1) Distance Data Augmentation (DDA), which simulates electromagnetic attenuation effects across varying detection ranges, and 2) Behavior Pattern Data Augmentation (BPDA), designed to preserve intrinsic kinematic features while diversifying motion styles. Subsequently, combined with the proposed PIDA, a multi-source domain adversarial neural (MSDAN) network based on transfer learning is constructed to capture invariant features of human behaviors. To verify the superiority of PIDA, a series of experiments are conducted on various unseen domain data. The results showed that the PIDA method improves average DG accuracy by 1.52%, 2.48% and 4.88%, respectively, on the own dataset and 3.67%, 6.29% and 9.78%, respectively, on the public dataset, compared with baseline data, traditional optical data augmentation (TODA), and generative data augmentation (GDA) methods. Compared with current fall-related human behavior recognition works, our proposed method has a small DG gap.

The accurate analysis of human dynamic behavior is very important for overcoming the limitations of movement diversity and behavioral adaptability. In this paper, a wearable device-based human dynamic behavior recognition method is proposed. The method collects acceleration and angular velocity data through a six-axis sensor to identify information containing specific behavior characteristics in a time series. A human movement data acquisition platform, the DMP attitude solution algorithm, and the threshold algorithm are used for processing. In this experiment, ten volunteers wore wearable sensors on their bilateral forearms, upper arms, thighs, calves, and waist, and movement data for standing, walking, and jumping were collected in school corridors and laboratory environments to verify the effectiveness of this wearable human movement recognition method. The results show that the recognition accuracy for standing, walking, and jumping reaches 98.33%, 96.67%, and 94.60%, respectively, and the average recognition rate is 96.53%. Compared with similar methods, this method not only improves the recognition accuracy but also simplifies the recognition algorithm and effectively saves computing resources. This research is expected to provide a new perspective for the recognition of human dynamic behavior and promote the wider application of wearable technology in the field of daily living assistance and health management.

Human behaviour recognition is one of the most fundamental tasks in Industrial Internet of Behaviour (IIoB) and is crucial for the safe and reliable IIoB. Existing methods lacks adaptability and transferability. In addition, there is a data isolation problem among different users. Therefore, there is an urgent requirement to construct a secure and adaptive human behaviour recognition model in IIoB without violating the privacy of users. Mamba, a structured state space model that integrates a selection mechanism and a scan module, is used for time series modelling tasks. To tackle the aforementioned problems, an Federated Learning-based lightweight human behaviour recognition model with selective state space models is proposed. First, we design a human behaviour recognition model integrating Mamba and residual structure to achieve lightweight and secure human behaviour modelling. In addition, considering data privacy and training efficiency, a decentralised dynamic FL framework is designed to achieve lightweight and secure model collaborative training, including: initial selection of source users, model aggregation strategy based on dynamic weighting, and fine-tuning module based on small-sample data, to improve the training efficiency of the model and the accuracy of human behaviour recognition. Extensive experiments are conducted to prove the superior performance of the proposed method.

In recent years, thermal imaging cameras are widely used in the field of intelligent surveillance because of their special imaging characteristics and better privacy protection properties. However, due to the low resolution and fixed location for current thermal imaging cameras, it is difficult to effectively identify human behavior using a single detection method based on skeletal keypoints. Therefore, a self-update learning method is proposed for fixed thermal imaging camera scenes, called the behavioral parameter field (BPF). This method can express the regularity of human behavior patterns concisely and directly. Firstly, the detection accuracy of small targets under low-resolution video is improved by optimizing the YOLOv4 network to obtain a human detection model under thermal imaging video. Secondly, the BPF model is designed to learn the human normal behavior features at each position. Finally, based on the learned BPF model, we propose to use metric modules, such as cosine similarity and intersection over union matching, to accomplish the classification of human abnormal behaviors. In the experimental stage, the living scene of the indoor elderly living alone is applied as our experimental case, and a variety of detection models are compared to the proposed method for verifying the effectiveness and practicability of the proposed behavioral parameter field in the self-collected thermal imaging dataset for the indoor elderly living alone.

Neuroscience studies have shown that incorporating gaze view with third view perspective has a great influence to correctly infer human behaviors. Given the importance of both first and third person observations for the recognition of human behaviors, we propose a method that incorporates these observations in a technical system to enhance the recognition of human behaviors, thus improving beyond third person observations in a more robust human activity recognition system. First, we present the extension of our proposed semantic reasoning method by including gaze data and external observations as inputs to segment and infer human behaviors in complex real-world scenarios. Then, from the obtained results we demonstrate that the combination of gaze and external input sources greatly enhance the recognition of human behaviors. Our findings have been applied to a humanoid robot to online segment and recognize the observed human activities with better accuracy when using both input sources; for example, the activity recognition increases from 77% to 82% in our proposed pancake-making dataset. To provide completeness of our system, we have evaluated our approach with another dataset with a similar setup as the one proposed in this work, that is, the CMU-MMAC dataset. In this case, we improved the recognition of the activities for the egg scrambling scenario from 54% to 86% by combining the external views with the gaze information, thus showing the benefit of incorporating gaze information to infer human behaviors across different datasets.

This paper presents data set of three commercial physiological sensors i.e. Electrocardiogram (ECG), Galvanic Skin Response (GSR), and Pulse sensor. The paper focuses on experiment to recognize human behavior using these body sensors. An experiment was done with participation of 12 users to observe human behavior i.e. Happy and Neutral mood. Users were asked to watch advertisement video based on comedy and actionscenes. During the implementation some variations were observed in the data-set while users were watching the videos. The results have been discussed at the end of the paper based on the data-set of 12 participants.

Because of its wide application value, behavior recognition has long been one of the research hots pots in the field of computer vision and pattern recognition. At present, the method based on local features and word packet model has been widely used in the field of behavior recognition. However, this method does not consider the temporal and spatial relationship between features, and the local temporal and spatial relationship between features is very important for behavior representation and behavior recognition. In view of the above problems, this paper proposes a modeling method of character behavior recognition based on local spatio-temporal relationship in surveillance video. Firstly, each part of the proposed network model is introduced in detail, and then the proposed model is compared with the advanced skeleton action recognition methods in recent years on several skeleton data sets. Finally, the effectiveness of the proposed method is verified. The experimental results show that, compared with the recognition results of related literatures, the features extracted by choosing the starting point of trajectory have better recognition performance under the fusion framework.

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.

Most researches on human behavior recognition are mainly based on the features of whole body motion. This paper proposed a hierarchical discriminative approach for recognizing human behavior based on limbs motion. The approach consists of feature extraction with mutual motion pattern analysis and discriminative behavior modeling in the hierarchical manifold space. A cascade CRF is introduced to estimate the motion patterns in the corresponding manifold subspace, and the trained SVM classifier is used to predict the behavior label for the current observation. The results on motion capure data prove the significance motion analysis of body parts, and the results on synthetic image sequences are also presented to demonstrate the robustness of the proposed algorithm.

Human behaviour recognition is an important research hotspot in the field of artificial intelligence. Current behaviour recognition methods have low recognition accuracy under different viewing angles, therefore, this paper proposes a novel restricted Boltzmann machine (RBM)-based temporal-spatial correlation method for student behaviour recognition in depth video. The RBM is used to map the human behaviour from different viewing angles to the high-dimensional space. The time level pooling function is applied in the time series activated by each neuron to realise the encoding of the video time sub-series. Finally, behaviour recognition and classification experiments are conducted on different public datasets and real classroom student behaviour datasets with other methods. The results show that the proposed method improves the accuracy of depth video recognition under different viewing angles and has good generalisation performance. The data analysis of abnormal behaviour in class can play an auxiliary role in dynamic classroom management.

In this paper, we propose a novel method based on ontology for human behavior recognition. The state-of-art behavior recognition methods based on low visual features or high visual features have still achieved low recognition accuracy rate in reality. The two most important challenges of this problem are still remaining such as the semantic gap and the variety of appearance of human behaviors in reality. By using prior knowledge, our system could completely detect a behavior without training data of entire process and could be reused in other cases. In experimental results, our method have achieved the encouraging performance on PETS 2006 and PETS 2007 datasets. These results have proved the good prospect of the human behavior recognition system based on behavior ontology.

In order to qualitatively recognize the behaviors and investigate the relationship between fuel consumption and machinery driving modes of the tractor in a low-cost approach, this study proposed a method for behavior recognition and fuel consumption prediction of tractor sowing operations using a smartphone. First, three driving modes were developed for maize sowing scenarios: manual driving assisted driving and unmanned driving. While sowing, smartphone software and CAN (Controller Area Network) storage devices collected both positional data and engine operating conditions. Second, the tractor trajectory points were divided into kinematic sequences, with six driving cycle indicators built in each series based on the time window. Based on the semantic information of the kinematic sequences, the three operations of sowing, seeds filling, and turning round were well recognized. Last, a model for maize sowing fuel consumption forecast was advanced using the principal component analyses and random forest algorithm, regarding three factors: driving cycles, operating behaviors, and driving patterns. When compared to the traditional K-means algorithm, the results demonstrated that the harmonic mean of the precision and recall (F1 score) of sowing behavior recognition, seeds filling behavior recognition, and turning behavior recognition were enhanced by 2.06%, 8.99%, and 21.79%, respectively. In terms of the impacts of driving modes and operating behaviors on fuel consumption, assisted driving mode had the lowest fuel usage for both sowing and turning behavior. Therefore, assisted driving is the most fuel-efficient mode for maize sowing. Combining the three driving modes, the relative error of the fuel consumption prediction model was 0.11 L/h, with the manual driving mode having the lowest relative error at 0.09 L/h. This research method lays the foundation for the optimization of tractor operation behavior, the selection of tractor driving mode, and the fine management of tractor fuel consumption. Keywords: smartphone, kinematic sequence, operating behavior, fuel consumption forecast, tractor DOI: 10.25165/j.ijabe.20221504.7454 Citation: Yang L L, Tian W Z, Zhai W X, Wang X X, Chen Z B, Wen L, et al. Behavior recognition and fuel consumption prediction of tractor sowing operations using a smartphone. Int J Agric & Biol Eng, 2022; 15(4): 154–162.

Human behavior recognition (HBR) has emerged as a core research area in human-computer interaction. In this article, we give a comprehensive introduction to HBR using Wi-Fi channel state information. We first comprehensively review the state-of-art of HBR, based on the two techniques that drove recent progress in Wi-Fi channel-state-information-based HBR -- fingerprint-based and model-based. Specifically, we describe their corresponding characteristics, general architectures, and provide a performance comparison of the two mechanisms. We then provide insights into the future directions of HBR research, and propose two possible new schemes, and the technical challenges coming with them.