Evolving Markov Chains: Online Mode Discovery and Recognition From Data Streams.

Human Activity Recognition (HAR) is critical in a variety of disciplines, including healthcare and robotics. This paper presents a new Convolutional Neural Network with Bidirectional Long Short-Term Memory and along with Gated Recurrent Unit (CNN-BiLSTM-GRU)hybrid deep learning model designed for Human Activity Recognition (HAR) that makes use of data from wearable sensors and mobile devices. Surprisingly, the model achieves an amazing accuracy rate of 99.7% on the difficult Wireless Sensor Data Mining (WISDM) dataset, demonstrating its ability to properly identify human behaviors. This study emphasizes parameter optimization, with a focus on batch size 0.3 as a significant component in improving the model’s robustness. Furthermore, the findings of this study have far-reaching implications for bipedal robotics, where precise HAR (Human Activity Recognition) is critical to improving human–robot interaction quality and overall work efficiency. These discoveries not only strengthen Human Activity Recognition (HAR) techniques, but also provide practical benefits in real-world applications, particularly in the robotics and healthcare areas. This study thus makes a significant contribution to the continuous development of Human Activity Recognition methods and their actual applications, emphasizing their important role in stimulating innovation and efficiency across a wide range of industries.

The concept of Human Activity Recognition (HAR) is integral to applications based on Internet of Things (IoT)-enabled devices, particularly in healthcare, fitness tracking, and smart environments. The streams of data from wearable sensors are rich in information, yet their high dimensionality and variability pose a significant challenge to proper classification. To address this problem, this paper proposes hybrid architectures that integrate traditional machine learning models with a deep neural network (DNN) to deliver improved performance and enhanced capabilities for HAR tasks. Multi-sensor HAR data were used to systematically test several hybrid models, including: RF + DNN (Random Forest + Deep Neural Network), XGB + DNN (XGBoost + DNN), GB + DNN (Gradient Boosting + DNN), KNN + DNN (K-Nearest Neighbors + DNN), and DT + DNN (Decision Tree + DNN). The RF + DNN model was the most accurate, achieving a 97.03% score with excellent precision, recall, and F1-score. These findings demonstrate that hybrid machine learning and deep learning systems have a promising future in IoT-based HAR applications. The model provides a novel solution for developing smart and trustworthy monitoring systems that support real-time analytics, patient surveillance, and other IoT applications.

Smartphone sensor-based Human Activity Recognition (HAR) is a significant facilitator for many real-world applications such as smart homes, personal healthcare, and illness detection because it has the potential to discover activity patterns in daily life. While solutions for detecting sequential activities like walking or running are already matured, however recognizing additional forms of complex, concurrent, interleaved, and heterogeneous human activities remains a research challenge. Furthermore, several solutions have been developed for specific types of activity complexity. According to the best of our knowledge, there is no common solution for detecting all types of action-based activities. In this paper, we address the problem of recognizing all types of action-based human activities i.e. sequential, complex, concurrent, interleaved, and heterogeneous activities. The intention behind this paper is to present a generic methodology for HAR by leveraging transfer learning and evaluating the performance on a plethora of HAR datasets. In this regard, we have proposed a model called RecurrentHAR, which stands for Recurrent layers for HAR, a novel adversarial knowledge transfer approach that uses the Gated Recurrent Units (GRUs) architecture for smartphone sensor-based HAR. The performance of the proposed model is evaluated through extensive experiments using three public datasets namely WISDM (sequential activities), PAMAP2 (complex, concurrent, and interleaved activities), and KU-HAR (Heterogeneous activities), and the RecurrentHAR model has outperformed compared with other state-of-the-art approaches. The proposed model achieved 96.26%, 94.77%, and 98.98% of F1-score for WISDM, PAMAP2, and KU-HAR datasets respectively. The experimental results provide insight into the applicability of the proposed model and future research possibilities.

Sensor-based Human Activity Recognition (HAR) has been applied across various domains, including healthcare monitoring, fitness tracking, and smart home systems. These applications require the ability to accurately detect and respond to a wide range of human activities, each with varying distributions, which imposes a significant challenge. The task-incremental learning paradigm can address this problem by enabling HAR systems to adapt to changes in distribution and learn new activities over time without forgetting the previously learned ones. Continual adaptation is essential for maintaining high performance, as it allows the system to effectively respond to these changes. Although several strategies in the continual learning literature have been evaluated for task-incremental scenarios in HAR, there is still room for improvement, as the results are not as good as those achieved with conventional approaches. This work proposes two new neuroplasticity-inspired techniques that can be integrated with any learning strategy. Inspired by the brain’s ability to reorganize and strengthen connections over time, these methods focus on enhancing the model’s flexibility and long-term knowledge retention. The proposed techniques were evaluated alongside the WA-ADB and WA-MDF strategies on well-known HAR datasets. Experimental results demonstrated that the new techniques significantly enhanced the models’ ability to retain knowledge, which holds significant potential for improving the robustness and longevity of HAR systems in real-world applications.

Human Activity Recognition (HAR) plays an important role in many real world applications. Currently, various techniques have been proposed for sensor-based HAR in daily health monitoring, rehabilitative training and disease prevention. However, non-visual sensors in general and wearable sensors in specific have several limitations: acceptability and willingness to use wearable sensors; battery life; ease of use; size and effectiveness of the sensors. Therefore, adopting vision-based human activity recognition approach is more viable option since its diversity would enable the application to be deployed in wide range of domains. The most popular technique of vision based activity recognition, Deep Learning, however, requires huge domain-specific datasets for training which, is time consuming and expensive. To address this problem this paper proposes a Transfer Learning technique by adopting vision-based approach to HAR by using already trained Deep Learning models. A new stochastic model is developed by borrowing the concept of Alloaction from Latent Dirichlet Allocation (LDA) for an inference of the posterior distribution of the variables relating the deep learning classifiers predicted labels with the corresponding activities. Results show that an average accuracy of 95.43% is achieved during training the model as compared to 74.88 and 61.4% of Decision Tree and SVM respectively.

Batteryless or so called passive wearables are providing new and innovative methods for human activity recognition (HAR), especially in healthcare applications for older people. Passive sensors are low cost, lightweight, unobtrusive and desirably disposable; attractive attributes for healthcare applications in hospitals and nursing homes. Despite the compelling propositions for sensing applications, the data streams from these sensors are characterised by high sparsity---the time intervals between sensor readings are irregular while the number of readings per unit time are often limited. In this paper, we rigorously explore the problem of learning activity recognition models from temporally sparse data. We describe how to learn directly from sparse data using a deep learning paradigm in an end-to-end manner. We demonstrate significant classification performance improvements on real-world passive sensor datasets from older people over the state-of-the-art deep learning human activity recognition models. Further, we provide insights into the model's behaviour through complementary experiments on a benchmark dataset and visualisation of the learned activity feature spaces.

Causality testing methods are being widely used in various disciplines of science. Model-free methods for causality estimation are very useful as the underlying model generating the data is often unknown. However, existing model-free measures assume separability of cause and effect at the level of individual samples of measurements and unlike model-based methods do not perform any intervention to learn causal relationships. These measures can thus only capture causality which is by the associational occurrence of ‘cause’ and ‘effect’ between well separated samples. In real-world processes, often ‘cause’ and ‘effect’ are inherently inseparable or become inseparable in the acquired measurements. We propose a novel measure that uses an adaptive interventional scheme to capture causality which is not merely associational. The scheme is based on characterizing complexities associated with the dynamical evolution of processes on short windows of measurements. The formulated measure, Compression- Complexity Causality is rigorously tested on simulated and real datasets and its performance is compared with that of existing measures such as Granger Causality and Transfer Entropy. The proposed measure is robust to presence of noise, long-term memory, filtering and decimation, low temporal resolution (including aliasing), non-uniform sampling, finite length signals and presence of common driving variables. Our measure outperforms existing state-of-the-art measures, establishing itself as an effective tool for causality testing in real world applications.

Human activity recognition is one of the most intensively studied areas of computer vision and pattern recognition in recent years. A wide variety of approaches have shown to work well against challenging image variations such as appearance, pose and illumination. However, the problem of low video quality remains an unexplored and challenging issue in real-world applications. In this paper, we investigate the effects of low video quality in human action recognition from two perspectives: videos that are poorly sampled spatially (low resolution) and temporally (low frame rate), and compressed videos affected by motion blurring and artifacts. In order to increase the robustness of feature representation under these conditions, we propose the usage of textural features to complement the popular shape and motion features. Extensive experiments were carried out on two well-known benchmark datasets of contrasting nature: the classic KTH dataset and the large-scale HMDB51 dataset. Results obtained with two popular representation schemes (Bag-of-Words, Fisher Vectors) further validate the effectiveness of the proposed approach.

This paper introduces a novel Human Action Recognition (HAR) dataset designed to improve Human-Robot Collaboration (HRC) in green electrolyzer production. Recorded in a lab using RGB, depth, and skeletal data from Azure Kinect, the dataset focuses on assembly tasks, labeled with Methods-Time Measurement (MTM) primitives. The use of a green screen enables the study of background effects on HAR algorithms. The dataset addresses the challenges of data imbalance and limited training data in industrial HAR applications, offering standardized, mergeable, and extendable data for the research community. It aims to enhance the development of HAR algorithms in manufacturing contexts, with future plans for collaborative expansion and real-world application. The dataset and further information are available via a GitHub repository.

Human Activity Recognition (HAR) has emerged as a critical research domain with applications spanning healthcare monitoring to smart environments. Despite significant advances in deep learning approaches for HAR, fundamental challenges persist in modeling the inherent variability of human movements and the sequential nature of physical activities. In this paper, we present a Multi-Head Convolutional Neural Network with an attention mechanism (MH-CNN) that addresses these limitations by dynamically focusing on relevant sensor inputs. Our approach integrates four parallel convolutional pathways with attention mechanisms, enabling the model to process different feature subspaces while maintaining computational efficiency simultaneously. Experiments conducted on the challenging Opportunity dataset (Roggen in UCI Mach Learn Repos, 2010. https://doi.org/10.24432/C5M027 ) [6] reveal that our architecture achieves superior performance (99.78% test accuracy and F1-score), outperforming conventional CNN approaches (99.75%) and single-head attention models (99.73%). While the accuracy improvement of 0.03% may appear modest, it represents approximately 30 additional correctly classified instances in our test set of approximately 10,000 samples, which is meaningful in safety–critical HAR applications where misclassifications can have significant consequences. Interestingly, we identify and analyze an unexpected phenomenon where single- head attention slightly underperforms compared to standard CNNs, while multi- head configurations provide substantial improvements. These findings demonstrate that our architecture effectively captures the complex temporal dependencies in human activities, offering a promising direction for next- generation HAR systems in real-world applications.

One of the major open problems in sensor-based Human Activity Recognition (HAR) is the scarcity of labeled data. Among the many solutions to address this challenge, semi-supervised learning approaches represent a promising direction. However, their centralized architecture incurs in the scalability and privacy problems that arise when the process involves a large number of users. Federated learning (FL) is a promising paradigm to address these problems. However, the FL methods that have been proposed for HAR assume that the participating users can always obtain labels to train their local models (i.e., they assume a fully supervised setting). In this work, we propose FedAR: a novel hybrid method for HAR that combines semi-supervised and federated learning to take advantage of the strengths of both approaches. FedAR combines active learning and label propagation to semi-automatically annotate the local streams of unlabeled sensor data, and it relies on FL to build a global activity model in a scalable and privacy-aware fashion. FedAR also includes a transfer learning strategy to fine-tune the global model on each user. We evaluated our method on two public datasets, showing that FedAR reaches recognition rates and personalization capabilities similar to state-of-the-art FL supervised approaches. As a major advantage, FedAR only requires a very limited number of annotated data to populate a pre-trained model and a small number of active learning questions that quickly decrease while using the system, leading to an effective and scalable solution for the data scarcity problem of HAR.

Abstract: HAR is a field of study that aims to develop algorithms and techniques for automatically identifying and classifying human activities. Different people can execute an activity in various ways, and the same action might have numerous versions. Modern human activity recognition systems are largely taught and utilised on video stream and picture data in order to identify the features and activities, variations in the data having similar or related movements. Human-to-human and human-computer contact both heavily rely on human activity recognition. Systems that are manually operated take much longer and cost more money. The goal of this project is to develop a quicker, more affordable system for recognizing human activity and detect threats proactively through live camera. This system will help the end user in a variety of applications, such as surveillance and assistance in examination centres, hospitals, restricted areas, banks etc. by identifying the activity being carried out in the video or picture and give out alerts if detected any suspicious activity. Not only will this system be affordable, but it also functions as a utility-based system that can be integrated into a variety of applications to speed up and assist with various tasks that require recognition, which will result in significant time savings and high accuracy.

Human activity recognition (HAR) plays a pivotal role in digital healthcare, enabling applications such as exercise monitoring and elderly care. However, traditional HAR methods relying on accelerometer data often require complex preprocessing steps, including noise reduction and manual feature extraction. Deep learning-based human activity recognition (HAR) using one-dimensional accelerometer data often suffers from noise and limited feature extraction. Transforming time-series signals into two-dimensional representations has shown potential for enhancing feature extraction and reducing noise. However, existing methods relying on single-feature inputs or extensive preprocessing face limitations in robustness and accuracy. This study proposes a multi-input, two-dimensional CNN architecture using three distinct data reconstruction methods. By fusing features from reconstructed images, the model enhances feature extraction capabilities. This method was validated on a custom HAR dataset without requiring complex preprocessing steps. The proposed method outperformed models using single-reconstruction methods or raw one-dimensional data. Compared to a one-dimensional baseline, it achieved 16.64%, 13.53%, and 16.3% improvements in accuracy, precision, and recall, respectively. We tested across various levels of noise, and the proposed model consistently demonstrated greater robustness than the time-series-based approach. Fusing features from three inputs effectively captured latent patterns and variations in accelerometer data. This study demonstrates that HAR can be effectively improved using a multi-input CNN approach with reconstructed data. This method offers a practical and efficient solution, streamlining feature extraction and enhancing performance, making it suitable for real-world applications.

Real-time human activity recognition plays an essential role in real-world human-centered robotics applications, such as assisted living and human-robot collaboration. Although previous methods based on skeletal data to encode human poses showed promising results on real-time activity recognition, they lacked the capability to consider the context provided by objects within the scene and in use by the humans, which can provide a further discriminant between human activity categories. In this paper, we propose a novel approach to real-time human activity recognition, through simultaneously learning from observations of both human poses and objects involved in the human activity. We formulate human activity recognition as a joint optimization problem under a unified mathematical framework, which uses a regression-like loss function to integrate human pose and object cues and defines structured sparsity-inducing norms to identify discriminative body joints and object attributes. To evaluate our method, we perform extensive experiments on two benchmark datasets and a physical robot in a home assistance setting. Experimental results have shown that our method outperforms previous methods and obtains real-time performance for human activity recognition with a processing speed of 104 Hz.

Stream data is the data that is generated continuously from the different data sources and ideally defined as the data that has no discrete beginning or end. Processing the stream data is a part of big data analytics that aims at querying the continuously arriving data and extracting meaningful information from the stream. Although earlier processing of such stream was using batch analytics, nowadays there are applications like the stock market, patient monitoring, and traffic analysis which can cause a drastic difference in processing, if the output is generated in levels of hours and minutes. The primary goal of any real-time stream processing system is to process the stream data as soon as it arrives. Correspondingly, analytics of the stream data also needs consideration of surrounding dependent data. For example, stock market analytics results are often useless if we do not consider their associated or dependent parameters which affect the result. In a real-world application, these dependent stream data usually arrive from the distributed environment. Hence, the stream processing system has to be designed, which can deal with the delay in the arrival of such data from distributed sources. We have designed the stream processing model which can deal with all the possible latency and provide an end-to-end low latency system. We have performed the stock market prediction by considering affecting parameters, such as USD, OIL Price, and Gold Price with an equal arrival rate. We have calculated the Normalized Root Mean Square Error (NRMSE) which simplifies the comparison among models with different scales. A comparative analysis of the experiment presented in the report shows a significant improvement in the result when considering the affecting parameters. In this work, we have used the statistical approach to forecast the probability of possible data latency arrives from distributed sources. Moreover, we have performed preprocessing of stream data to ensure at-least-once delivery semantics. In the direction towards providing low latency in processing, we have also implemented exactly-once processing semantics. Extensive experiments have been performed with varying sizes of the window and data arrival rate. We have concluded that system latency can be reduced when the window size is equal to the data arrival rate.