Direction-Independent Human Activity Recognition Using a Distributed MIMO Radar System and Deep Learning

Deep learning techniques for human activity recognition are gaining popularity due to their effectiveness in identifying intricate tasks and their cost-effectiveness compared to conventional machine learning methods. Human Activity Recognition (HAR) is a research domain concerned with detecting the everyday activities performed by individuals using time-series data captured by sensors. HAR encompasses a wide range of applications, including surveillance, baby monitoring, elderly healthcare, and smart driving. This article provides a brief introduction to the application of deep learning in HAR. It covers the fundamental concepts of CNNs and LSTMs, their strengths in capturing spatial and temporal features, and their integration for enhanced activity recognition. Different approaches are employed in HAR to address problems with efficiency and precision. Conventional human activity recognition (HAR) systems rely on wearable devices like IMUs and stretch sensors to identify different activities. These systems have proven to be effective in recognizing simple user actions like sitting, standing, and walking. However, when it comes to more intricate activities like running, jumping, wrestling, and swinging, sensor-based HAR systems encounter greater misclassification rates due to inaccuracies in sensor readings. These errors significantly impact the accuracy of the HAR system, resulting in suboptimal classification outcomes. In contrast, employing vision-based HAR systems enables improved accuracy in identifying complex activities, leading to enhanced overall performance.

The conventional Human Activity Recognition (HAR) systems use smartwatches, smartphones, and other wearable devices to autonomously derive consumer physical activities. However, implementing real-time HAR in resource-constrained applications is challenging. Computational offloading approaches must address network reliance, latency, and data privacy, which affect system performance. The present study proposes a low-cost, wearable HAR device that collects and transmits the Inertial Measurement Unit (IMU) telemetry signals to smartphones, enabling real-time on-device activity detection. Human actions are subjective and affected by physiological, environmental, and behavioural factors, complicating HAR systems' performance in personalized contexts. Generalized methods for existing systems reduce performance for new consumers or due to alterations in consumer locomotive signatures. Thus, our second goal is to enable the on-device personalization of wearable HAR models with minimum customer calibration. The HAR device, which is in the form of a smart-band, transfers data to and fro smartphones through Bluetooth Low Energy (BLE). A lightweight 1D convolutional neural network (CNN) is built and, using transfer learning, the network model is fine-tuned from real-time sensed data. On-device HAR inferences ensure real-time processing on smartphones without computational offloading. Smartphone-based on-device training for fine-tuning the model allows the HAR system to get customized without compromising privacy or computational costs. The experimental results show that the personalized classification achieves on average 98% accuracy when assessed using four benchmark datasets.

When developing a fully automatic system for evaluating motor activities performed by a person, it is necessary to segment and recognize the different activities in order to focus the analysis. This process must be carried out by a Human Activity Recognition (HAR) system. This paper proposes a user adaptation technique for improving a HAR system based on Hidden Markov Models (HMMs). This system segments and recognizes six different physical activities (walking, walking upstairs, walking downstairs, sitting, standing and lying down) using inertial signals from a smartphone. The system is composed of a feature extractor for obtaining the most relevant characteristics from the inertial signals, a module for training the six HMMs (one per activity), and the last module for segmenting new activity sequences using these models. The user adaptation technique consists of a Maximum A Posteriori (MAP) approach that adapts the activity HMMs to the user, using some activity examples from this specific user. The main results on a public dataset have reported a significant relative error rate reduction of more than 30%. In conclusion, adapting a HAR system to the user who is performing the physical activities provides significant improvement in the system’s performance.

Human activity recognition (HAR) is an important technology for a wide range of applications including elderly people monitoring, ambient assisted living, sport and fitness activities. The aim of this paper is to address the HAR task directly on a wearable device, implementing a recurrent neural network (RNN) on a low cost, low power microcontroller, ensuring the required performance in terms of accuracy and low complexity. To reach this goal we first develop a lightweight RNN on the Human Activity Recognition Using Smartphones dataset in order to accurately detect human activity and then we port the RNN to the embedded device Cloud-JAM L4, based on an STM32 microcontroller. Experimental results show that this HAR RNN-based detector can be effectively implemented on the chosen constrained-resource system, achieving an accuracy of about 90.50% with a very low memory cost (40.883 KB) and inference time (67.131 ms), allowing the design of a wearable embedded system for human activity recognition.

Wireless body area network (WBAN) is a novel technology with the incorporation of numerous types of devices, which is also employed in health monitoring applications. Human activity recognition (HAR) receives more interest in recent times along with wearable sensors. HAR system provides information about a person's identity, personality, and psychological state. In the present scenario, it becomes important to model the active learning paradigms with the help of wearable sensors for analysing human activities. Although various deep learning models and existing algorithms secure better outcomes through the sensor data analysis regarding HAR, the decision‐making evaluation seems to be a complex one. The main intent of this paper is to implement the WBAN‐based HAR system using the improved deep learning model. By connecting the wearable sensors, the signals are gathered regarding human activity from diverse benchmark sources. After collecting the required signals, the pre‐processing of the input signals is done using artefact removal and median filtering. Further, the feature extraction is performed, which intends to extract a set of features by utilizing short‐time Fourier transform (STFT) and statistical features. For reducing the feature‐length, a multi‐objective‐based optimal feature selection is adopted. Human activities such as ‘walking, walking upstairs, walking downstairs, sitting, standing, lying, and jogging’ are recognized with help of selected optimal features. The optimized probabilistic neural network (PNN) and convolutional neural network (CNN) are combined and named as adaptive probabilistic‐based CNN (AP‐CNN). The effective performance of optimal feature selection and recognition is accomplished by incorporating the developed backward updating position‐based sea lion optimization algorithm (BU‐SLnO). Finally, the performance of the BU‐SLnO‐AP‐CNN‐based suggested model is analysed, which shows 1.001%, 1.237%, 0.811%, and 0.859% advanced than SLnO‐AP‐CNN, feed‐forward‐AP‐CNN (FF‐AP‐CNN), grey wolf optimization‐AP‐CNN (GWO‐AP‐CNN), particle swarm optimization‐AP‐CNN (PSO‐AP‐CNN) when observing the dataset 1. The experimental outcomes from comparison with various classification techniques demonstrate the efficiency of the developed technique.

Human Activity Recognition (HAR) is a field that has attracted significant attention due to its wide range of applications, from healthcare monitoring to smart home systems. Because of the aging of the population in first-world countries, it becomes increasingly important to find innovative solutions that reduce risks associated with aging-in-place policies. However, development of robust HAR systems is faced with two main challenges: the lack of generalization capacities of current methods and the lack of sufficient practical labeled data. Traditional data augmentation techniques do improve classification accuracy, but they often don’t fully solve the complexity and variability of human activities. This article introduces an innovative approach to creating synthetic data for HAR using Vector-Quantized Variational Auto Encoders (VQ-VAEs), aiming to address the challenges of data scarcity and improve the ability of HAR systems to generalise. Our approach includes the use of three UWB radars to recognize 14 activities performed by 19 participants in a prototype smart-home apartment as the practical dataset. Data from those three UWB radars are then filtered using a Variational Auto Encoder (VAE) and used as ground for data augmentation. Following best practices in data separation, the Leave-One-Subject-Out (LOSO) evaluation method validates the data augmentation using the MobileNetV2 classifier architecture. Results show that the quality of generated data is similar to the original dataset, potentially enhancing the performance of UWB radar based HAR systems.

The height measurement algorithm of metre wave multiple input multiple output (MIMO) radar based on uniform linear array (ULA) has some problems, such as a sharp decline of angle measurement accuracy of ultra‐low altitude targets and large fluctuation of height measurement with the change of elevation. In order to solve the above difficult problems, starting from the perspective of reducing the influence of multipath effect by array structure, coprime array (CPA) is applied to the height measurement of monostatic metre wave MIMO radar. In this paper, the signal model of monostatic CPA metre wave MIMO radar is established under the condition of flat ground, a height measurement method using physical array is proposed based on the derivation that it is infeasible to use virtual array method for height measurement, it fills the gap of low elevation estimation method based on sparse array metre wave MIMO radar. The proposed method calculates the echo data according to the characteristics of the steering vector of CPA, then obtains the accurate low elevation by using the generalised multiple signal classification algorithm, and finally the target height is obtained by using the geometric relationship. Simulation experiments compare three typical CPAs with ULA using equal array elements, and verify the superiority of height measurement performance of monostatic metre wave MIMO radar based on CPA and the effectiveness of the proposed method.

PurposeTo find a successful human action recognition system (HAR) for the unmanned environments.Design/methodology/approachThis paper describes the key technology of an efficient HAR system. In this paper, the advancements for three key steps of the HAR system are presented to improve the accuracy of the existing HAR systems. The key steps are feature extraction, feature descriptor and action classification, which are implemented and analyzed. The usage of the implemented HAR system in the self-driving car is summarized. Finally, the results of the HAR system and other existing action recognition systems are compared.FindingsThis paper exhibits the proposed modification and improvements in the HAR system, namely the skeleton-based spatiotemporal interest points (STIP) feature and the improved discriminative sparse descriptor for the identified feature and the linear action classification.Research limitations/implicationsThe experiments are carried out on captured benchmark data sets and need to be analyzed in a real-time environment.Practical implicationsThe middleware support between the proposed HAR system and the self-driven car system provides several other challenging opportunities in research.Social implicationsThe authors’ work provides the way to go a step ahead in machine vision especially in self-driving cars.Originality/valueThe method for extracting the new feature and constructing an improved discriminative sparse feature descriptor has been introduced.

In this work, we propose a Human Activity Prediction (HAP) system using activity sequence spanning trees constructed from a life-log created by a video sensor-based daily Human Activity Recognition (HAR) system using time-sequential Independent Component (IC)-based depth silhouette features with Hidden Markov Models (HMMs). In the daily HAR system, the IC features are extracted from the collection of the depth silhouettes containing various daily human activities such as walking, sitting, lying, cooking, eating etc. Using these features, HMMs are used to model the time sequential features and recognize various human activities. The depth silhouette-based human activity recognition system is used to recognize daily human activities automatically in real time, which creates a life-log of daily activity events. In this work, we propose a method for human activity prediction using fixed-length activity sequence spanning trees based on the life-log. Utilizing the consecutive activities recorded in an activity sequence database (i.e. life-log) for a specific period of time of each day over a period such as a month, the fixed-length spanning trees can be constructed for the sequences starting with each activity where the leaf nodes contain the frequency of the fixed-length consecutive activity sequences. Once the trees are constructed, to predict an activity after a sequence of activities, we traverse the spanning trees until a path up to the previous node of the leaf nodes is matched with the testing pattern. Finally, we can predict the next activity based on the highest frequency of the leaf nodes along the matched path. The prediction experiments over the computer simulated data which is based on the daily logs show satisfactory results. Our video sensor-based human activity recognition and prediction systems can be utilized for practical applications such as smart and proactive healthcare.

Human Activity Recognition (HAR) is the automatic detection and understanding of human motion behavior based on data extracted from video camera, ambient sensors or wearable sensors which particularly has recently attracted increased attention from both researchers and industry. However, for running practical HAR systems on wearable devices, there are some requirements such as design and development of small, lightweight, powerful, and low-cost smart sensors. In this context, data must be continuously collected, and edge computing is a viable solution, which is an energy-efficient technique, offering real-time response and privacy requirements for HAR applications. Rather than sending data to the cloud, edge computing is a local process that minimizes the data transmission time and responds with low latency. Recently, HAR system designers have adopted deep learning techniques inspired by their outstanding performance in many application areas and achieved relevant gain in activity recognition performance, however, these techniques were not demonstrated suitable for running on resource constrained devices. Thus, designing energy-efficient deep learning models is critical for realizing efficient HAR for mobile applications. In this work, we present a lightweight framework for the deployment of low-power but accurate HAR systems for these devices. We also implement and run the system in a microcontroller and analyze computational cost and energy consumption, and how different system configurations and deep learning model complexity influence on that.

MIMO radar can provide enhanced spatial resolution and achieve better target detection performance. Hence the parameter identifiability of the system can be significantly improved. In this paper, the issue of direction of arrival (DOA) estimation in multiple-input multiple-output (MIMO) radar is discussed, and several algorithms for DOA tracking based on monostatic MIMO radar in impulsive noise background are proposed and studied. These tracking algorithms are based on the concept of robust covariation (ROC), fractional lower order moment (FLOM), fractional lower order cyclic (FLOC) and similar normalization. The transmit and receive arrays of a monostatic narrowband MIMO radar are assumed to be close to each other and eigenstructure based MUSIC algorithm at the receiver is used to identify the DOA's of the targets of interest. Simulation results verify the usefulness of the proposed algorithms. The comparison of these four tracking algorithms show that the algorithm based on the concept of similar normalization has better performance than the others.

In this study, the authors consider the direction of arrival (DOA) estimation problem for a monostatic multiple-input multiple-output (MIMO) radar. A DOA estimation algorithm for the monostatic MIMO radar using Capon and the reduced-dimension transformation is proposed. The proposed algorithm can achieve lower computational complexity than the Capon algorithm, while not debasing the performance of the angle estimation. The proposed algorithm has better angle estimation performance than the Capon algorithm and estimation of signal parameters via rotational invariance techniques algorithm, and it has close angle estimation performance to the Cramér–Rao bound (CRB). The variance of the estimation error and the CRB of the DOA estimation are derived. Simulation results present the usefulness of the proposed algorithm.

Developments in information and communication technologies have led to the wider usage of Internet of Things (IoT). In the modern health care applications, the usage of IoT technologies brings physicians and patients together for automated and intelligent daily activity monitoring for elderly people. Mobile devices and wearable body sensors are gradually implemented for the monitoring of personal health care and wellbeing. One of the main technologies of IoT improvements in healthcare monitoring system is the wearable sensor technology. Furthermore, integration of IoT in healthcare has led to initiate smart applications such as mobile healthcare (m-Healthcare) and intelligent healthcare monitoring systems. In this study an intelligent m-healthcare system based on IoT technology is presented to provide pervasive human activity recognition by using data mining techniques. In this paper, we present a user-dependent data mining approach for off-line human activity classification and a robust and precise human activity recognition model is developed based on IoT technology. The proposed model utilizes the dataset contains body motion and vital signs recordings for ten volunteers of diverse profile while performing 12 physical activities for human activity recognition purpose. Results show that the proposed system is superior in performance with 99.89 % accuracy and is highly effective, robust and reliable in delivering m-Healthcare services during different activities.

Human Activity Recognition (HAR) systems are designed to read sensor data and analyse it to classify any detected movement and respond accordingly. However, there is a need for more responsive and near real-time systems to distinguish between false and true alarms. To accurately determine alarm triggers, the motion pattern of legitimate users need to be stored over a certain period and used to train the system to recognise features associated with their movements. This training process is followed by a testing cycle that uses actual data of different patterns of activity that are either similar or different to the training data set. This paper evaluates the use of a combined Convolutional Neural Network (CNN) and Naive Bayes for accuracy and robustness to correctly identify true alarm triggers in the form of a buzzer sound for example. It shows that pattern recognition can be achieved using either of the two approaches, even when a partial motion pattern is derived as a subset out of a full-motion path.

Sensor-based Human Activity Recognition (HAR) has been applied across various domains, including healthcare monitoring, fitness tracking, and smart home systems. These applications require the ability to accurately detect and respond to a wide range of human activities, each with varying distributions, which imposes a significant challenge. The task-incremental learning paradigm can address this problem by enabling HAR systems to adapt to changes in distribution and learn new activities over time without forgetting the previously learned ones. Continual adaptation is essential for maintaining high performance, as it allows the system to effectively respond to these changes. Although several strategies in the continual learning literature have been evaluated for task-incremental scenarios in HAR, there is still room for improvement, as the results are not as good as those achieved with conventional approaches. This work proposes two new neuroplasticity-inspired techniques that can be integrated with any learning strategy. Inspired by the brain’s ability to reorganize and strengthen connections over time, these methods focus on enhancing the model’s flexibility and long-term knowledge retention. The proposed techniques were evaluated alongside the WA-ADB and WA-MDF strategies on well-known HAR datasets. Experimental results demonstrated that the new techniques significantly enhanced the models’ ability to retain knowledge, which holds significant potential for improving the robustness and longevity of HAR systems in real-world applications.