Shallow Convolutional Neural Networks for Human Activity Recognition Using Wearable Sensors

Due to the breadth of its application domains, Hu-man Activity Recognition (HAR) is a problematic area of human-computer interaction. HAR can be used in remote monitoring of senior healthcare and concern situations in intelligent man-ufacturing, among other applications. HAR based on wearable inertial sensors has been researched identification efficiency in various kinds of human actions considerably more than vision-based HAR. The sensor-based HAR is generally applicable to indoor and outdoor locations without privacy considerations of implementation. In this research, we explore the recognition performance of multiple deep learning (DL) models to recognize everyday living human activities. We developed a deep residual neural network that employed aggregated multi-branch transformation to boost identification performance. The proposed model is called the ResNeXt model. To evaluate its performance, three standard DL models (CNN, LSTM, and CNN-LSTM) are investigated and compared to our proposed model using a standard HAR dataset called Daily Living Activity dataset. These datasets gathered mobility signal data from multimodal sensors (accelerometer, gyroscope, and magnetometer) in three distinct body areas (wrist, hip, and ankle). The experimental findings reveal that the proposed model surpasses other benchmark DL models with maximum accuracy and F1-scores. Furthermore, the findings show that the ResNeXt model is more resistant than other models with fewer training parameters.

An increase in life expectancy is leading to an aging society, where gradual degradation in physical/cognitive health could result in fall injuries, cardiovascular, neurological/psychological problems, or a sedentary lifestyle. Sensor-based human activity recognition (HAR) is found to be a potential assistive technology in this context to improve the quality of life of an aging society. This chapter presents the state of the art and future trends of Internet of Things (IoT)-enabled sensor data-based Human Activity Recognition (HAR) systems, where machine learning (ML)/deep learning (DL) is used to classify human actions in the context of healthcare applications. In particular, the impact of IoT on the healthcare industry is so immense that a dedicated specialization, termed as Internet of Healthcare Things (IoHT), has emerged. Smart healthcare systems are equipped with different categories of sensors (wearable, wireless, and ambient). The huge amount of sensor data acquired by IoHT sensors allows for performing accurate HAR. The HAR involves detection/classification problems, e.g. fall detection and seizer detection, where ML is used extensively, at its heart. A lot of ML/DL techniques are being used in HAR including support vector machine (SVM), artificial neural network (ANN), recurrent neural network (RNN), convolutional neural network (CNN), deep neural network (DNN), long short-term memory (LSTM), etc. However, there are a few special aspects of HAR sensor data that make it difficult to handle. Involved research is still being pursued across the globe to solve these problems.KeywordsIoTIoHTHealth careHARSensorMachine learning (ML)Deep learning (DL)

Wireless body area network (WBAN) is a novel technology with the incorporation of numerous types of devices, which is also employed in health monitoring applications. Human activity recognition (HAR) receives more interest in recent times along with wearable sensors. HAR system provides information about a person's identity, personality, and psychological state. In the present scenario, it becomes important to model the active learning paradigms with the help of wearable sensors for analysing human activities. Although various deep learning models and existing algorithms secure better outcomes through the sensor data analysis regarding HAR, the decision‐making evaluation seems to be a complex one. The main intent of this paper is to implement the WBAN‐based HAR system using the improved deep learning model. By connecting the wearable sensors, the signals are gathered regarding human activity from diverse benchmark sources. After collecting the required signals, the pre‐processing of the input signals is done using artefact removal and median filtering. Further, the feature extraction is performed, which intends to extract a set of features by utilizing short‐time Fourier transform (STFT) and statistical features. For reducing the feature‐length, a multi‐objective‐based optimal feature selection is adopted. Human activities such as ‘walking, walking upstairs, walking downstairs, sitting, standing, lying, and jogging’ are recognized with help of selected optimal features. The optimized probabilistic neural network (PNN) and convolutional neural network (CNN) are combined and named as adaptive probabilistic‐based CNN (AP‐CNN). The effective performance of optimal feature selection and recognition is accomplished by incorporating the developed backward updating position‐based sea lion optimization algorithm (BU‐SLnO). Finally, the performance of the BU‐SLnO‐AP‐CNN‐based suggested model is analysed, which shows 1.001%, 1.237%, 0.811%, and 0.859% advanced than SLnO‐AP‐CNN, feed‐forward‐AP‐CNN (FF‐AP‐CNN), grey wolf optimization‐AP‐CNN (GWO‐AP‐CNN), particle swarm optimization‐AP‐CNN (PSO‐AP‐CNN) when observing the dataset 1. The experimental outcomes from comparison with various classification techniques demonstrate the efficiency of the developed technique.

Human Activity Recognition (HAR) based on data from wearable sensors has become an attractive research topic thanks to its applications in different fields such as healthcare and smart environments. Recently, the advancement of deep learning with capability to perform automatically high-level feature extraction has achieved promising results. However, the performance of the deep learning models depends deeply on the characteristics of the datasets such as the number of classes, the inter-similarity and intra-variation. Therefore, directly comparing these models has become difficult since a wide variety of experimental protocols, evaluation metrics, and datasets are employed. In this paper, for the first time, a systematic evaluation of several deep learning models for HAR from wearable sensors is provided. In particular, three models named Convolutional Neural Network (CNN) [1], DeepConvLSTM - a combination of CNN and Long Short Term Memory (LSTM) [2], and SensCapsNet - a Capsule Neural Network for wearable sensor-based HAR [3] were implemented and evaluated on three benchmark datasets that are 19NonSens, CMDFall, and UCI-HAR dataset. Moreover, to have an intuitive explanation of deep learning models, a visualization of features learnt from these models is given. The evaluation codebase and results will be made publicly available for community use.

Recently, deep learning, which are able to extract automatically features from data, has achieved state-of-the-art performance across a variety of sensor based human activity recognition (HAR) tasks. However, the existing deep neural networks are usually trained with a global loss, and all hidden layer weights have to be always kept in memory before the forward and backward pass has completed. The backward locking phenomenon prevents the reuse of memory, which is a crucial limitation for wearable activity recognition. In the paper, we proposed a layer-wise convolutional neural networks (CNN) with local loss for the use of HAR task. To our knowledge, this paper is the first that uses local loss based CNN for HAR in ubiquitous and wearable computing arena. We performed experiments on five public HAR datasets including UCI HAR dataset, OPPOTUNITY dataset, UniMib-SHAR dataset, PAMAP dataset, and WISDM dataset. The results show that local loss works better than global loss for tested baseline architectures. At no extra cost, the local loss can approach the state-of-the-arts on a variety of HAR datasets, even though the number of parameters was smaller. We believe that the layer-wise CNN with local loss can be used to update the existing deep HAR methods.

Nowadays, most smartphones feature a number of strong sensors, such as direction, network, location, and motion sensors. Motion and inertial sensors (i.e., accelerometer, gyroscope, etc.) are particularly popular in human activity recognition (HAR) for detecting human physical activity which helps in many research areas of the Internet of healthcare things (IoHT) including several patient’s diseases such as Parkinson, obesity, cardiovascular, and diabetes. Deep learning (DL) techniques have been increasingly popular over the years as a result of their significant outcomes. The use of deep learning techniques to recognize human physical activity in wearable and mobile sensor situations has also gotten a lot of attention from around the world. In this paper, a deep neural network (DNN) combining bidirectional long short-term memory (Bi-LSTM) and convolutional neural network (CNN) is proposed. The performance of the model has been evaluated on two publicly available datasets: WISDM and UCI-HAR. The model has achieved 97.96 and 97.15% accuracy for the WISDM and UCI-HAR, respectively. Moreover, the simulation results show the effectiveness of the proposed work compared to other state-of-the-art methods.KeywordsHuman activity recognitionDeep learningIoHTConvolutional neural networks (CNN)Bidirectional long short-term memory networks (Bi-LSTM)

In recent years, Deep Learning (DL) models, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM), have been widely used for Human Activities Recognition (HAR). They have achieved considerable performance improvements over classical Machine Learning (ML) approaches due to their excellent feature representation capabilities. Simple human activities are performed in sequential order with no overlaps or concurrent actions. The paucity of labeled training activity samples, high computational cost, and system resource requirements of deep learning architectures compared to lightweight models are some of the research challenges confronting the HAR community. We framed the lightweight DL-based CNNs, RNNs, and LSTM model for HAR to tackle these research challenges. The RNNs and LSTM have single layers with softmax activation functions, whereas the one-dimensional CNNs have two convolutional and single max-pooling layers. To evaluate the performance of our models, we used the publicly available benchmark WISDM experimental dataset, which includes the reading of six activities (walking, jogging, upstairs, downstairs, sitting, and standing) performed by 36 participants using a single accelerometer sensor. The experimental result illustrates the efficacy of our models in activity recognition, demonstrating that they attain higher accuracy while being computationally efficient.

In recent years, channel state information (CSI) in WiFi 802.11n has been increasingly used to collect data pertaining to human activity. Such raw data are then used to enhance human activity recognition. Activities such as lying down, falling, walking, running, sitting down, and standing up can now be detected with the use of information collected through CSI. Human activity recognition has a multitude of applications, such as home monitoring of patients. Four deep learning models are presented in this paper, namely: a convolution neural network (CNN) with a Gated Recurrent Unit (GRU); a CNN with a GRU and attention; a CNN with a GRU and a second CNN, and a CNN with Long Short-Term Memory (LSTM) and a second CNN. Those models were trained to perform Human Activity Recognition (HAR) using CSI amplitude data collected by a CSI tool. Experiments conducted to test the efficacy of these models showed superior results compared with other recent approaches. This enhanced performance of our models may be attributable the ability of our models to make full use of available data and to extract all data features, including high dimensionality and time sequence. The highest average recognition accuracy reached by the proposed models was achieved by the CNN-GRU, and the CNN-GRU with attention models, standing at 99.31% and 99.16%, respectively. In addition, the performance of the models was evaluated for unseen CSI data by training our models using a random split-of-dataset method (70% training and 30% testing). Our models achieved impressive results with accuracies reaching 100% for nearly all activities. For the lying down activity, accuracy obtained from the CNN-GRU model stood at 99.46%; slightly higher than the 99.05% achieved by the CNN-GRU with attention model. This confirmed the robustness of our models against environmental changes.

A fingerprint is an impression left by the friction ridges of a human finger. A fingerprint classification system groups fingerprint according to their characteristics and therefore helps to match a fingerprint against an extensive database of fingerprints. The Henry classification system is widely used among fingerprint classification systems. Some researchers have used traditional machine learning or deep learning for fingerprint classification. Nevertheless, traditional algorithms cannot extract the depth features of the fingerprint, and most deep learning algorithms lack fingerprint image enhancement. So, this paper combined the Gabor Filter and Convolutional Neural Network to extract fingerprint features. The model has two channels, one is a Deep Convolutional Neural Network (DCNN), and the other is a Shallow Convolutional Neural Network (SCNN). The DCNN consists of a neural network with eight layers, which can extract deep features of the fingerprint. The SCNN consists of Gabor Filter and a neural network with two layers that can extract features from clear fingerprint images. This paper uses NIST Special Database 4 for experiments. Experimental results show that the model proposed in this paper has achieved 91.4% accuracy. Compared with other algorithms, this model has higher accuracy than others. It shows that combined with the Gabor Filter and Convolutional Neural Network can better extract the ridge features of fingerprint images.

Human Activity Recognition (HAR) is critical in a variety of disciplines, including healthcare and robotics. This paper presents a new Convolutional Neural Network with Bidirectional Long Short-Term Memory and along with Gated Recurrent Unit (CNN-BiLSTM-GRU)hybrid deep learning model designed for Human Activity Recognition (HAR) that makes use of data from wearable sensors and mobile devices. Surprisingly, the model achieves an amazing accuracy rate of 99.7% on the difficult Wireless Sensor Data Mining (WISDM) dataset, demonstrating its ability to properly identify human behaviors. This study emphasizes parameter optimization, with a focus on batch size 0.3 as a significant component in improving the model’s robustness. Furthermore, the findings of this study have far-reaching implications for bipedal robotics, where precise HAR (Human Activity Recognition) is critical to improving human–robot interaction quality and overall work efficiency. These discoveries not only strengthen Human Activity Recognition (HAR) techniques, but also provide practical benefits in real-world applications, particularly in the robotics and healthcare areas. This study thus makes a significant contribution to the continuous development of Human Activity Recognition methods and their actual applications, emphasizing their important role in stimulating innovation and efficiency across a wide range of industries.

Human Activity Recognition (HAR) has become one of the most important research fields to achieve real-time monitoring of human activities for timely decision making in various applications like fall detection, elderly care etc. Now-a-days, most people use smartphones which come with various embedded inertial sensors like accelerometer and gyroscope to monitor acceleration and angular velocity. These smartphone-based sensors have proven to be cost-effective solution in identification of activities belonging to ADL (Activities of Daily Living). Various Machine Learning, Deep learning and hybrid models have been proposed and implemented for HAR. This paper also proposes various hybrid deep learning approaches which combine Deep Neural Networks with other models like LSTM (Long Short Term Memory) Model and GRU (Gated Recurrent Unit) for effective classification of engineered features from CNN (Convolutional Neural Network) Model. A novel architecture that integrates CNN with Random Forest Classifier (DeepCNN-RF) is proposed to add randomness to the model. The proposed models have been tested on publicly available HAR Datasets like UCI HAR and WISDM Activity Recognition Datasets. Experimental results show that the hybrid models outperform the state-of-the-art data mining, machine learning techniques in UCI HAR and WISDM with an overall maximum accuracy of 97.77% and 98.2% respectively.

Human activity recognition (HAR) has a wide range of applications in medical care, elderly care and so on. Due to the development of wearable sensor technology, HAR based on wearable sensors has gradually become a hot research topic. In terms of feature extraction of raw data, traditional methods usually use manually extracted features, which has many limitations. The method based on deep learning has gradually become the mainstream. In this paper, a deep bidirectional GRU network for human activity recognition is proposed, which uses few data preprocessing to classify and recognize daily activities. An accuracy rate of 95.69% is achieved on the UCL-HAR dataset which is an increase of 2.8% compared to the bidirectional LSTM network. In addition, the proposed method in this paper reduces the training time by 11.8 %, which will help the neural network to be deployed in real-time applications.

<p>Human activity recognition (HAR) based on wearable devices is an emerging field of great interest. HAR can provide additional information on a human subject’s physical status. Utilising new technologies for HAR will become very meaningful with the development of deep learning. This study aims to mine deep learning models for HAR prediction with the highest accuracy on the basis of time-series data collected by mobile wearable devices. To this end, convolutional neural networks (CNN) and long short-term memory neural networks (LSTM) are combined in a deep network model to extract behavioural facts. The proposed CNN model contains two convolutional layers and a maximum pooling layer, and batch normalisation is added after each convolutional layer to improve convergence speed and avoid overfitting. This structure yields significant results in terms of performance. The model is evaluated on the MHEALTH dataset with a test set accuracy of 99.61% and can be used for the intelligent recognition of human activity. The results of this study show that the proposed model has better robustness and motion pattern detection capability compared to other models.</p> <p> </p>

Healthcare using body sensor data has been getting huge research attentions by a wide range of researchers because of its good practical applications such as smart health care systems. For instance, smart wearable sensor-based behavior recognition system can observe elderly people in a smart eldercare environment to improve their lifestyle and can also help them by warning about forthcoming unprecedented events such as falls or other health risk, to prolong their independent life. Although there are many ways of using distinguished sensors to observe behavior of people, wearable sensors mostly provide reliable data in this regard to monitor the individual’s functionality and lifestyle. In this paper, we propose a body sensor-based activity modeling and recognition system using time-sequential information-based deep Neural Structured Learning (NSL), a promising deep learning algorithm. First, we obtain data from multiple wearable sensors while the subjects conduct several daily activities. Once the data is collected, the time-sequential information then go through some statistical feature processing. Furthermore, kernel-based discriminant analysis (KDA) is applied to see the better clustering of the features from different activity classes by minimizing inner-class scatterings while maximizing inter-class scatterings of the samples. The robust time-sequential features are then applied with Neural Structured Learning (NSL) based on Long Short-Term Memory (LSTM), for activity modeling. The proposed approach achieved around 99% recall rate on a public dataset. It is also compared to existing different conventional machine learning methods such as typical Deep Belief Network (DBN), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) where they yielded the maximum recall rate of 94%. Furthermore, a fast and efficient explainable Artificial Intelligence (XAI) algorithm, Local Interpretable Model-Agnostic Explanations (LIME) is used to explain and check the machine learning decisions. The robust activity recognition system can be adopted for understanding peoples' behavior in their daily life in different environments such as homes, clinics, and offices.

In recent years, many researchers have focused on using convolutional neural networks to perform human activity recognition as evidenced by the emergence of a number of convolutional neural network architectures such as LeNet-5, AlexNet and VGG16 and modern architectures such as ResNet, Inception V3, Inception-ResNet, MobileNet V2, NASNet and PNASNet. The main characteristic of a convolutional neural network (CNN) is its ability to extract features automatically from input images, which facilitates the processes of activity recognition and classification. Convolutional networks indeed derive more relevant and complex features with every additional layer. In addition, CNNs have achieved perfect classification on highly similar activities that were previously extremely difficult to classify. In this paper, we evaluate modern convolutional neural networks in terms of their human activity recognition accuracy, and we compare the results with the state of the art methods. In our research, we used two public data sets, HMDB (Shooting gun, kicking, falling to the floor, punching) and the Weizman dataset (walking, running, jumping, bending, one hand waving, two-hand waving, jumping in place, jumping jack, skipping). Our experimental results indicated that the CNN with NASNet architecture achieves the best performance of the six CNN architectures on both human activity data sets (HMDB and Weizman).