Mobile Big Data Analytics for Human Behavior Recognition in Wireless Sensor Network Based on Transfer Learning

To improve traditional video surveillance systems' performance on human behavior recognition, a new system has been built. Not visual camera but depth camera is chosen as sensor. To adapt to the most common forward oblique view of camera, a normalized digital elevation map, whose pixel intensity indicates the elevation of the scene, is built from the depth image. Coordinates and intensity in the digital elevation map represent the position information about the scene. Oriented templates are proposed to match and detect the human head robustly in the elevation map. As for the excellent visibility of human head in the elevation map, we track the human head to get trajectory. By combining the trajectory information of human head with the elevation map, several predefined human behaviors are recognized. Our behavior recognition method is straightforward and robust. This uniqueness has no similar with the traditional machine learning and classification framework about human behavior recognition.

Falling is a common but fatal human behavior in life. With the rapid growth of the aging population, fall-related human behavior recognition has been extensively investigated using radar. Nevertheless, human behavior recognition frequently exhibits suboptimal generalization capabilities due to the scarcity of labeled data. To address this problem, we propose physics-informed data augmentation (PIDA) for the micro-Doppler signature (m-DS) produced from frequency-modulated continuous wave (FMCW) radar to facilitate domain generalization (DG) of human behavior recognition. Specifically, leveraging classical electromagnetic wave propagation theory and radar signal processing principles, we introduce a PIDA strategy: 1) Distance Data Augmentation (DDA), which simulates electromagnetic attenuation effects across varying detection ranges, and 2) Behavior Pattern Data Augmentation (BPDA), designed to preserve intrinsic kinematic features while diversifying motion styles. Subsequently, combined with the proposed PIDA, a multi-source domain adversarial neural (MSDAN) network based on transfer learning is constructed to capture invariant features of human behaviors. To verify the superiority of PIDA, a series of experiments are conducted on various unseen domain data. The results showed that the PIDA method improves average DG accuracy by 1.52%, 2.48% and 4.88%, respectively, on the own dataset and 3.67%, 6.29% and 9.78%, respectively, on the public dataset, compared with baseline data, traditional optical data augmentation (TODA), and generative data augmentation (GDA) methods. Compared with current fall-related human behavior recognition works, our proposed method has a small DG gap.

The proliferation and pervasive use of mobile devices results in the accumulation of massive amounts of wireless data. Mobile big data can be profitable only if suitable analytics and learning methods are utilized to extract meaningful knowledge and hidden patterns. In this article, we propose a novel mobile big data architecture consisting of five layers: the data storage layer, the data fusion layer, the data security layer, the data analysis layer and the data application layer. The functionality of each layer is presented. We consider one illustrative cases under this architecture, namely, user experience prediction by leveraging machine learning techniques. In practice, mobile big data analytics can be used for network planning and parameter dimensioning to facilitate network design, deployment and operation.

Human behavior is often affected by sudden emergency directly or indirectly. In this paper, we focus on two groups of people under different types of medical emergency calls. Based on the quantitative analysis of calling volume and call interval distribution before and after the emergency call, we find that during most types of emergencies, calling volume will increase after the emergency call made and maintain a high level in the next several hours. Medical emergency makes influence on call volumes, but not affects power-law distribution in human communication behavior. From studying entropy of call detail records, we found that during the medical emergency, the richness of outgoing call numbers increases and falls after that. In this paper, we do quantitative analysis of temporal patterns of medical emergency call based on the combination of medical emergency call data and call detail record data, which not only study on the characteristics of human communication behavior in emergency situations, but also provide data basis for scientific rational allocation of aid resources and effective emergency measures.

This paper is concerned with characteristics of human behaviors in Internet-based teleoperation system under communication delay. Especially frequency character and velocity character of human behaviors affected with random delay of the Internet are analyzed. We are researching in human adaptive network mechatronics which can not only evaluate operational behaviors of human, but also help the operator's control according to the evaluation of operations in Internet-based teleoperation system. The characteristics analysis of human behaviors which are affected by delay of the Internet will provide us helpful regulations to design control methods in human adaptive network mechatronics. We develop an experimental system which includes a joystick controller and a virtual inversed pendulum. The joystick can control the virtual pendulum with the Internet delay which has been recorded in a test before. The Laminar Phase analysis and power spectrum is utilized to analyze human behaviors affects by delay of the Internet.

Neuroscience studies have shown that incorporating gaze view with third view perspective has a great influence to correctly infer human behaviors. Given the importance of both first and third person observations for the recognition of human behaviors, we propose a method that incorporates these observations in a technical system to enhance the recognition of human behaviors, thus improving beyond third person observations in a more robust human activity recognition system. First, we present the extension of our proposed semantic reasoning method by including gaze data and external observations as inputs to segment and infer human behaviors in complex real-world scenarios. Then, from the obtained results we demonstrate that the combination of gaze and external input sources greatly enhance the recognition of human behaviors. Our findings have been applied to a humanoid robot to online segment and recognize the observed human activities with better accuracy when using both input sources; for example, the activity recognition increases from 77% to 82% in our proposed pancake-making dataset. To provide completeness of our system, we have evaluated our approach with another dataset with a similar setup as the one proposed in this work, that is, the CMU-MMAC dataset. In this case, we improved the recognition of the activities for the egg scrambling scenario from 54% to 86% by combining the external views with the gaze information, thus showing the benefit of incorporating gaze information to infer human behaviors across different datasets.

Human behaviour recognition is one of the most fundamental tasks in Industrial Internet of Behaviour (IIoB) and is crucial for the safe and reliable IIoB. Existing methods lacks adaptability and transferability. In addition, there is a data isolation problem among different users. Therefore, there is an urgent requirement to construct a secure and adaptive human behaviour recognition model in IIoB without violating the privacy of users. Mamba, a structured state space model that integrates a selection mechanism and a scan module, is used for time series modelling tasks. To tackle the aforementioned problems, an Federated Learning-based lightweight human behaviour recognition model with selective state space models is proposed. First, we design a human behaviour recognition model integrating Mamba and residual structure to achieve lightweight and secure human behaviour modelling. In addition, considering data privacy and training efficiency, a decentralised dynamic FL framework is designed to achieve lightweight and secure model collaborative training, including: initial selection of source users, model aggregation strategy based on dynamic weighting, and fine-tuning module based on small-sample data, to improve the training efficiency of the model and the accuracy of human behaviour recognition. Extensive experiments are conducted to prove the superior performance of the proposed method.

The widespread use of mobile devices, such as smart phones, smart watches, bracelets, equipped with plenty of integrated sensors, provides collection of massive amounts of data and their processing and analysis on edge and cloud computing infrastructures. For the purpose of monitoring of health and activities of people, such Big mobility and IoT data must be properly processed and analyzed. In this paper, the RemoteHealth platform is presented, developed using contemporary mobile, IoT and Big Data technologies. The application implemented on RemoteHealth platform is based on sensor fusion and provides processing and analysis of Big mobility and IoT data originated from smart personal devices with the aim to detect and recognize human activities and behavior. The experimental evaluation shows that the sensor fusion increase the accuracy of activity recognition comparing to single sensor use.

Human behaviour recognition is an important research hotspot in the field of artificial intelligence. Current behaviour recognition methods have low recognition accuracy under different viewing angles, therefore, this paper proposes a novel restricted Boltzmann machine (RBM)-based temporal-spatial correlation method for student behaviour recognition in depth video. The RBM is used to map the human behaviour from different viewing angles to the high-dimensional space. The time level pooling function is applied in the time series activated by each neuron to realise the encoding of the video time sub-series. Finally, behaviour recognition and classification experiments are conducted on different public datasets and real classroom student behaviour datasets with other methods. The results show that the proposed method improves the accuracy of depth video recognition under different viewing angles and has good generalisation performance. The data analysis of abnormal behaviour in class can play an auxiliary role in dynamic classroom management.

In recent years, thermal imaging cameras are widely used in the field of intelligent surveillance because of their special imaging characteristics and better privacy protection properties. However, due to the low resolution and fixed location for current thermal imaging cameras, it is difficult to effectively identify human behavior using a single detection method based on skeletal keypoints. Therefore, a self-update learning method is proposed for fixed thermal imaging camera scenes, called the behavioral parameter field (BPF). This method can express the regularity of human behavior patterns concisely and directly. Firstly, the detection accuracy of small targets under low-resolution video is improved by optimizing the YOLOv4 network to obtain a human detection model under thermal imaging video. Secondly, the BPF model is designed to learn the human normal behavior features at each position. Finally, based on the learned BPF model, we propose to use metric modules, such as cosine similarity and intersection over union matching, to accomplish the classification of human abnormal behaviors. In the experimental stage, the living scene of the indoor elderly living alone is applied as our experimental case, and a variety of detection models are compared to the proposed method for verifying the effectiveness and practicability of the proposed behavioral parameter field in the self-collected thermal imaging dataset for the indoor elderly living alone.

Human behavior recognition is an active research field in computer vision and image processing. A novel method is proposed for human behavior recognition in video image sequences. First of all, a video sequence is represented by extracting space-time interest points. Then Human behavior is represented by activities through Motion Decomposition. The activity comprises labeled bags that are composed of unlabeled instances comprising to action. Final labeled activities are used to train a strong classifier which is used to predict the labels of unseen behavior bags. Experimental results show the effectiveness of the proposed method in comparison with other related works in the literature and can also tolerate noise and interference conditions.

In this paper, we propose a novel method based on ontology for human behavior recognition. The state-of-art behavior recognition methods based on low visual features or high visual features have still achieved low recognition accuracy rate in reality. The two most important challenges of this problem are still remaining such as the semantic gap and the variety of appearance of human behaviors in reality. By using prior knowledge, our system could completely detect a behavior without training data of entire process and could be reused in other cases. In experimental results, our method have achieved the encouraging performance on PETS 2006 and PETS 2007 datasets. These results have proved the good prospect of the human behavior recognition system based on behavior ontology.

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.

Big mobility data (BMD) have shown many advantages in studying human mobility and evaluating the performance of transportation systems. However, the quality of BMD remains poorly understood. This study evaluates biases in BMD and develops mitigation methods. Using Google and Apple mobility data as examples, this study compares them with benchmark data from governmental agencies. Spatio-temporal discrepancies between BMD and benchmark are observed and their impacts on transportation applications are investigated, emphasizing the urgent need to address these biases to prevent misguided policymaking. This study further proposes and tests a bias mitigation method. It is shown that the mitigated BMD could generate valuable insights into large-scale public transit systems across 100+ US counties, revealing regional disparities of the recovery of transit systems from the COVID-19. This study underscores the importance of caution when using BMD in transportation research and presents effective mitigation strategies that would benefit practitioners.

Aiming at the problem of human behavior recognition, this paper proposes a human behavior classification algorithm based on radial basis function (BP) neural network. The singular value decomposition (SVD) and principal component analysis (PCA) are used for feature extraction, and the linear discriminant analysis (LDA) is used for feature transformation to make it easier to distinguish between various categories. Finally, BP neural network classifier is used to recognize human behavior. The experimental results show that compared with the nearest neighbor classification and k-nearest neighbor classification (kNN), the proposed algorithm has a higher recognition rate. The result of BP neural network classifier is the best, it can fit the boundary of each category well.