Unsupervised learning for human activity recognition using smartphone sensors

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.

The article considers the problem of undefined complex classes activity recognition. The human activity recognition relevance in the field of production and services is presented. The classification of physical activity types is described. The possibilities of using methods based on data and knowledge for human activity recognition are considered. An analysis of zero-shot learning is given. A functional model of the method for undefined complex human activity recognition is proposed. The datasets are described. The HH101 and HH105 datasets obtained from CASAS smart homes were used to conduct experimental studies. An experimental study of the developed method for human activity recognition is carried out.

Human Activity Recognition (HAR) using built‐in sensors in wearable devices offers valuable insights for continuous behaviour monitoring and health assessment. However, existing HAR approaches face several key challenges: (1) limited capacity to capture multi‐scale temporal dependencies; (2) difficulty in recognising low‐motion or static activities when relying solely on inertial sensors such as 3D accelerometers (3D‐ACC); and (3) insufficient utilisation of auxiliary physiological signals in conventional single‐task learning frameworks. To address these limitations, we propose a Hierarchical‐Split Multi‐Scale Convolutional Network with Multi‐Task Learning for wearable HAR. The proposed model integrates a hierarchical‐split convolutional block to efficiently extract both local and global temporal features. It further employs a channel‐wise attention mechanism to adaptively fuse photoplethysmography (PPG) and 3D‐ACC signals, enabling the model to capture complementary physiological dynamics and improve the recognition of subtle or low‐motion activities. In addition, we introduce a multi‐task learning framework with an auxiliary heart rate estimation task, which enhances physiological representation learning without requiring additional annotations, thereby improving model robustness and generalisability. Extensive experiments on two public HAR datasets demonstrate that our method consistently outperforms existing approaches under both overall and cross‐subject evaluation protocols, highlighting its effectiveness in realistic, user‐independent settings.

This paper presents some more recent developments in the research area of human activity recognition. In many cases, human activity recognition can be achieved with the support of body sensors that deliver information about the motion of a person or about acceleration and position. Such data can contribute substantially to the accuracy of the results, however wearing extra equipment is not a very popular practice for most human users and therefore camera-based methods are mostly preferred in today’s systems, which is also a result of the tremendous progress achieved in recent years in pattern recognition and machine learning. Therefore, also this paper concentrates on camera-based methods for human activity recognition. Activity recognition has a large variety of sub-research areas and has many exciting application areas, such as e.g. interaction in smart spaces, surveillance, human-robot interaction or automatic analysis of interest, emotions and human traits.

Human detection and action recognition form the basis for understanding human behaviors. Human detection is used to detect the positions of humans, and action recognition is able to recognize the action of specific humans. However, numerous approaches have been used to handle action recognition and human detection separately. Therefore, three main issues still exist when independent methods of human detection and action recognition are combined, 1) intrinsic errors in object detection impact the performance of action recognition, 2) features common to action recognition and object detection are missed, 3) the combination also has an impact on processing speed. We propose a single framework for human detection and action recognition to solve these issues. It is based on a hierarchical structure called Boosted Randomized Trees. The nodes are trained such that the upper nodes detect humans from the background, while the lower nodes recognize action. We were able to improve both human detection and action recognition rates over earlier hierarchical structure approaches with the proposed method.

In order to study the action recognition, tracking, and optimization of the training process based on the support vector regression model, a method of human action recognition based on support vector machine optimization is proposed. This method uses the improved strategy of support vector machine to realize the action recognition through the human action recognition based on the optimization of the vector machine. During the recognition, the DAG SVM strategy is improved according to the recognition accuracy of the classifier, and when outputting the result, output the recognition result and the corresponding confidence level, and use the confidence level to process the recognition result. Finally, through the experimental results, it is realized that the recognition rate based on support vector optimization is 98.7%, indicating that this method is effective and can improve the accuracy and efficiency of human body action recognition.

Batteryless or so called passive wearables are providing new and innovative methods for human activity recognition (HAR), especially in healthcare applications for older people. Passive sensors are low cost, lightweight, unobtrusive and desirably disposable; attractive attributes for healthcare applications in hospitals and nursing homes. Despite the compelling propositions for sensing applications, the data streams from these sensors are characterised by high sparsity---the time intervals between sensor readings are irregular while the number of readings per unit time are often limited. In this paper, we rigorously explore the problem of learning activity recognition models from temporally sparse data. We describe how to learn directly from sparse data using a deep learning paradigm in an end-to-end manner. We demonstrate significant classification performance improvements on real-world passive sensor datasets from older people over the state-of-the-art deep learning human activity recognition models. Further, we provide insights into the model's behaviour through complementary experiments on a benchmark dataset and visualisation of the learned activity feature spaces.

Exercise is an indispensable part of people’s lives and is closely related to their health. Human Activity Recognition (HAR), which involves detects and analyzes human body activity, has become the focus of current research. Photoplethysmography (PPG) has advantages such as convenience for detection and low cost, and is widely used in wearable devices becoming an ideal choice for HAR. In this study, we used wavelet scattering transform (WST) to extract features from PPG and then performed activity recognition on it. We achieved excellent classification accuracy of 92.54% and 97.76% respectively in the experiments of three-class and four-class exercise detection. The results showed this method based on wavelet scattering transform and PPG can accurately detect exercise types and provide effective support for HAR.

Human Activity Recognition (HAR), alongside Ambient Assisted Living (AAL), are integral components of smart homes, sports, surveillance, and investigation activities. To recognize daily activities, researchers are focusing on lightweight, cost-effective, wearable sensor-based technologies as traditional vision-based technologies lack elderly privacy, a fundamental right of every human. However, it is challenging to extract potential features from 1D multi-sensor data. Thus, this research focuses on extracting distinguishable patterns and deep features from spectral images by time-frequency-domain analysis of 1D multi-sensor data. Wearable sensor data, particularly accelerator and gyroscope data, act as input signals of different daily activities, and provide potential information using time-frequency analysis. This potential time series information is mapped into spectral images through a process called use of 'scalograms', derived from the continuous wavelet transform. The deep activity features are extracted from the activity image using deep learning models such as CNN, MobileNetV3, ResNet, and GoogleNet and subsequently classified using a conventional classifier. To validate the proposed model, SisFall and PAMAP2 benchmark datasets are used. Based on the experimental results, this proposed model shows the optimal performance for activity recognition obtaining an accuracy of 98.4% for SisFall and 98.1% for PAMAP2, using Morlet as the mother wavelet with ResNet-101 and a softmax classifier, and outperforms state-of-the-art algorithms.

Human activity recognition (HAR) has been increasingly used in medical care, behavior analysis, and entertainment industry to improve the experience of users. Most of the existing works use fixed models to identify various activities. However, they do not adapt well to the dynamic nature of human activities. We investigated the activity recognition with postural transition awareness. The inertial sensor data was processed by filters and we used both time domain and frequency domain of the signals to extract the feature set. For the corresponding posture classification, three feature selection algorithms were considered to select 585 features to obtain the optimal feature subset for the posture classification. And We adopted three classifiers (support vector machine, decision tree, and random forest) for comparative analysis. After experiments, the support vector machine gave better classification results than other two methods. By using the support vector machine, we could achieve up to 98% accuracy in the Multi-class classification. Finally, the results were verified by probability estimation.

In order to improve the accuracy of human action recognition and accelerate the recognition speed, we propose a method for human action recognition by modeling the human primary visual cortex neurons. The method firstly extracted motion information by using 3DGabor spatial-temporal filters to model the classical receptive field (CRF) of simple cells in the primary visual cortex. Secondly, conductance-driven integrate and fire neuron model was used to simulate the primary visual cortex neuron, by which motion information was converted into spike train. Finally, the mean firing rate of spike train formed a feature vector that captures the characteristic of human actions in this video sequence. Using Support Vector Machine (SVM), the method is tested on the Weizmann action dataset. The obtained impressive results show that our method was more effective than model of Escobar in human action recognition.

In recent years, there has been a significant increase in research focused on Human Activity Analysis (HAA). This field has progressed from basic activity recognition tasks to addressing more challenging ones, such as predicting future human actions based on partially observed videos and even predicting actions before they happen. The evolution of HAA has been driven by recent advancements in attention-based models like Transformers, along with a wide range of applications from security surveillance to advanced monitoring systems, behaviour analysis, and more. A comprehensive review of HAA literature from 2017 to 2025, with a novel taxonomy emphasising activity recognition, prediction, and anticipation, is presented. We critically review and examine recognition methods from trimmed and untrimmed videos, context-aware and trajectory-based prediction, and short-term and long-term anticipation. Through a comprehensive analysis, we review and evaluate key aspects of this domain, including attention-based contextual comprehension, temporal dynamics modelling, and multi-model fusion methods. Furthermore, we critically examine and assess the public datasets utilised in driving this research forward, pinpointing limitations and primary challenges within this domain. Finally, the paper provides a summary of recent developments in HAA and suggests future directions, with the hope that it will serve as a valuable reference for researchers in the field.

Human activity recognition (HAR) is an important research topic in the field of human-computer interaction (HCI), and wearable HAR based on convolutional neural networks (CNNs) has achieved the most advanced performance. In this study, wearable plastic optical fiber (POF) sensors based on GAF-ConvNeXt algorithm are used for HAR for the first time. The GAF-ConvNeXt algorithm, combined with the Gramian Angular Field (GAF) and the latest CNN model ConvNeXt, can convert motion signal of a one-dimensional time series into a two-dimensional image, which can not only retain the complete information of the signal, but also maintain its dependence on time, thereby presenting more depth features and facilitating accurate recognition of human activities. Furthermore, transfer learning (TL) was introduced to further optimize the GAF-ConvNeXt model, which effectively reduces the number of training samples, improves the recognition accuracy and accelerates the convergence speed. The GAF-ConvNeXt algorithm based on TL can recognize seven motion modes with an accuracy of 99.75%, and has great application potential in the fields of human motion recognition and HCI.

Context-aware computing has become a certainty due to the widespread use of smartphone devices equipped with sensors. A wide range of services, such as vehicular traffic monitoring and smart parking, can be accomplished with the help of awareness of user mobility. Transportation mode detection (TMD) using machine learning algorithms and the data captured from smartphone embedded sensors have attracted research community attention. In this research, ensemble learning is utilized to differentiate between transportation modes, including walking, standing, riding a train, driving a car, and riding a bus. The ensemble learning consists of three classifiers; each classifier votes independently on the instances, and the majority vote is applied for robust generalization. The proposed method was validated using three datasets; the samples included in these datasets were gathered by smartphone sensors (belonging to heterogeneous users), such as rotation vector sensors, accelerometers, uncalibrated gyroscopes, linear acceleration, orientation, speed, game rotation vector, sound, and gyroscopes. The proposed ensemble learning method achieves an accuracy of 89%, 93%, and 95% on the first, second, and third datasets, respectively, when 10% and 90% of the data are used for testing and training, respectively. On the other set of experiments, in which 30% and 70% of the data are used for testing and training, respectively, the proposed method yields accuracies of 86.8%, 92.1%, and 94.9% on the first, second, and third datasets, respectively. The proposed method shows promising results compared to existing human activity recognition (HAR) methods.

In this paper, we present an original and efficient method of human action recognition in a video sequence. The proposed model is based on the generation and fusion of a set of prototypes generated from different view-points of the data cube of the video sequence. More precisely, each prototype is generated by using a multidimensional scaling (MDS) based nonlinear dimensionality reduction technique both along the temporal axis but also along the spatial axis (row and column) of the binary video sequence of 2D silhouettes. This strategy aims at modeling each human action in a low dimensional space, as a trajectory of points or a specific curve, for each viewpoint of the video cube in a complementary way. A simple K-NN classifier is then used to classify the prototype, for a given viewpoint, associated with each action to be recognized and then the fusion of the classification results for each viewpoint allow us to significantly improve the recognition rate performance. The experiments of our approach have been conducted on the publicly available Weizmann data-set and show the sensitivity of the proposed recognition system to each individual viewpoint and the efficiency of our multi-viewpoint based fusion approach compared to the best existing state-of-the-art human action recognition methods recently proposed in the literature.