Neuroevolution based hierarchical activation function for long short-term model network

Recent advancement in smart phones and computing technologies has played a vital role in people’s life. Develop a model to detect the human basic dynamic activities such as Amble, Climb stairs, coming down the stairs into the floor and human basic static activities like Sitting, Standing or Laying using the person’s smart phone and computers are the major work of this paper. Conventional Machine learning models like Logistic Regression, SVC, Decision tree, etc. results are compared with a recurrent deep neural network model named as Long Short Term Memory (LSTM). LSTM is proposed to detect the human behavior based on Human Activity Recognition (HAR) dataset. The data is monitored and recorded with the aid of sensors like accelerometer and Gyroscope in the user smart phone. HAR dataset is collected from 30 persons, performing different activities with a smart phone to their waists. The testing of the model is evaluated with respect to accuracy and efficiency. The designed activity recognition system can be manipulated in other activities like predicting abnormal human actions, disease by human actions, etc. The overall accuracy has improved to 95.40%.

Human Activity Recognition (HAR) has become one of the most important research fields to achieve real-time monitoring of human activities for timely decision making in various applications like fall detection, elderly care etc. Now-a-days, most people use smartphones which come with various embedded inertial sensors like accelerometer and gyroscope to monitor acceleration and angular velocity. These smartphone-based sensors have proven to be cost-effective solution in identification of activities belonging to ADL (Activities of Daily Living). Various Machine Learning, Deep learning and hybrid models have been proposed and implemented for HAR. This paper also proposes various hybrid deep learning approaches which combine Deep Neural Networks with other models like LSTM (Long Short Term Memory) Model and GRU (Gated Recurrent Unit) for effective classification of engineered features from CNN (Convolutional Neural Network) Model. A novel architecture that integrates CNN with Random Forest Classifier (DeepCNN-RF) is proposed to add randomness to the model. The proposed models have been tested on publicly available HAR Datasets like UCI HAR and WISDM Activity Recognition Datasets. Experimental results show that the hybrid models outperform the state-of-the-art data mining, machine learning techniques in UCI HAR and WISDM with an overall maximum accuracy of 97.77% and 98.2% respectively.

This paper introduces a novel Human Action Recognition (HAR) dataset designed to improve Human-Robot Collaboration (HRC) in green electrolyzer production. Recorded in a lab using RGB, depth, and skeletal data from Azure Kinect, the dataset focuses on assembly tasks, labeled with Methods-Time Measurement (MTM) primitives. The use of a green screen enables the study of background effects on HAR algorithms. The dataset addresses the challenges of data imbalance and limited training data in industrial HAR applications, offering standardized, mergeable, and extendable data for the research community. It aims to enhance the development of HAR algorithms in manufacturing contexts, with future plans for collaborative expansion and real-world application. The dataset and further information are available via a GitHub repository.

Human action recognition has been a popular issue recently because it can be applied in many applications such as intelligent surveillance systems, human-robot interaction, and autonomous vehicle control. Human action recognition using RGB video is a challenging task because the learning of actions is easily affected by the cluttered background. To cope with this problem, the proposed method estimates 3D human poses first which can help remove the cluttered background and focus on the human body. In addition to the human poses, the proposed method also utilizes appearance features nearby the predicted joints to make our action prediction context-aware. Instead of using 3D convolutional neural networks as many action recognition approaches did, the proposed method uses a two-stream architecture that aggregates the results from skeleton-based and appearance-based approaches to do action recognition. Experimental results show that the proposed method achieved state-of-the-art performance on NTU RGB+D which is a large-scale dataset for human action recognition.

The recognition of human activities using vision-based techniques has become a crucial research field in video analytics. Over the last decade, there have been numerous advancements in deep learning algorithms aimed at accurately detecting complex human actions in video streams. While these algorithms have demonstrated impressive performance in activity recognition, they often exhibit a bias towards either model performance or computational efficiency. This biased trade-off between robustness and efficiency poses challenges when addressing complex human activity recognition problems. To address this issue, this paper presents a computationally efficient yet robust approach, exploiting saliency-aware spatial and temporal features for human action recognition in videos. To achieve effective representation of human actions, we propose an efficient approach called the dual-attentional Residual 3D Convolutional Neural Network (DA-R3DCNN). Our proposed method utilizes a unified channel-spatial attention mechanism, allowing it to efficiently extract significant human-centric features from video frames. By combining dual channel-spatial attention layers with residual 3D convolution layers, the network becomes more discerning in capturing spatial receptive fields containing objects within the feature maps. To assess the effectiveness and robustness of our proposed method, we have conducted extensive experiments on four well-established benchmark datasets for human action recognition. The quantitative results obtained validate the efficiency of our method, showcasing significant improvements in accuracy of up to 11% as compared to state-of-the-art human action recognition methods. Additionally, our evaluation of inference time reveals that the proposed method achieves up to a 74× improvement in frames per second (FPS) compared to existing approaches, thus showing the suitability and effectiveness of the proposed DA-R3DCNN for real-time human activity recognition.

Action class relation detection and classification across multiple video datasets

In internet-of-things and big data applications, multiple sensors are required to acquire information about the same event. The dependencies among these modalities have to be leveraged to gain new perspectives and to improve the accuracy of the underlying performance measures. Retrieving this dependency information from the multimodal data helps in dimensionality reduction, and performance improvements. However, capturing these dependencies can be a major challenge due to a complex heterogeneity profile in the data. In this paper, we develop a dependency-based model for classification with multimodal data representation and information fusion using regular vine (R-Vine) copula theory. The efficacy of the developed models is tested on STISEN and TMD datasets for human activity recognition and transport mode detection applications. Further, we show that the F1 scores obtained for our model-98.50 percent and 98.649 percent for STISEN and TMD respectively - are better compared to existing dependency-blind architectures. These dependency-blind architectures use feature extraction using a deep neural network followed by (a) no information fusion, and (b) R-Vine-based fusion.

Human activity recognition has been a prominent research focus for several decades. While computer vision-based passive sensing is desirable for many applications, traditional RGB-based methods face significant limitations, including high computational cost, sensitivity to ambient lighting conditions, and privacy concerns. Single-photon LiDAR (light detection and ranging) is emerging as a robust alternative, offering efficient and high-resolution 3D imaging over long ranges and in difficult conditions while preserving privacy. In this study, we combine an eye-safe single-photon LiDAR system with a deep learning pipeline to achieve fast, long-range human activity recognition. To address the issue of limited available data for training, we generate synthetic datasets by combining real motion capture data with virtual models in a 3D modeling environment. A state-of-the-art recurrent neural network is trained on short-duration depth-image sequences of six activities. We also contribute two long-range, video frame rate single-photon LiDAR datasets for human activity recognition, recorded at distances of 325 m and 1.4 km, which exhibit markedly different noise levels. When evaluated on these data, the network maintains more than 80% accuracy even in the most challenging scenario, while supporting continuous and fast inference.

The bag-of-words approach with local spatio-temporal features have become a popular video representation for action recognition in videos. Together these techniques have demonstrated high recognition results for a number of action classes. Recent approaches have typically focused on capturing global statistics of features. However, existing methods ignore relations between features and thus may not be discriminative enough. Therefore, we propose a novel feature representation which captures statistics of pairwise co-occurring local spatio-temporal features. Our representation captures not only global distribution of features but also focuses on geometric and appearance (both visual and motion) relations among the features. Calculating a set of bag-of-words representations with different geometrical arrangement among the features, we keep an important association between appearance and geometric information. Using two benchmark datasets for human action recognition, we demonstrate that our representation enhances the discriminative power of features and improves action recognition performance.

Numerous practical datasets have been developed to recognize human actions from videos. However, many of them were constructed by collecting videos within a limited domain; thus, a model trained using one of the existing datasets often fails to classify videos in a different domain accurately. A possible solution for this drawback is to enhance the domain of each action label, i.e., to import videos associated with a given action label from the other datasets, and then, to train a model using the enhanced dataset. To realize this solution, we constructed a meta video dataset from the existing datasets for human action recognition, referred to as MetaVD. MetaVD comprises six popular human action recognition datasets, which we integrated by annotating 568,015 relation labels in total. These relation labels reflect equality, similarity, and hierarchy between action labels of the original datasets. We further present simple yet effective dataset enhancement methods using MetaVD, which are useful for training models with higher generalization performance, as established by experiments on human action classification. As a further contribution of MetaVD, we show that its analysis can provide useful insight into the datasets.

Existing activity tracker datasets for human activity recognition are typically obtained by having participants perform predefined activities in an enclosed environment under supervision. This results in small datasets with a limited number of activities and heterogeneity, lacking the mixed and nuanced movements normally found in free-living scenarios. As such, models trained on laboratory-style datasets may not generalise out of sample. To address this problem, we introduce a new dataset involving wrist-worn accelerometers, wearable cameras, and sleep diaries, enabling data collection for over 24 hours in a free-living setting. The result is CAPTURE-24, a large activity tracker dataset collected in the wild from 151 participants, amounting to 3883 hours of accelerometer data, of which 2562 hours are annotated. CAPTURE-24 is two to three orders of magnitude larger than existing publicly available datasets, which is critical to developing accurate human activity recognition models.

In general, dynamic systems are systems with time-dependent behavior. Dynamic systems are characterized by the non-stationary data sequences they emit. One particular way to model these non-stationary sequences is to consider them as a sequence of stationary segments, regimes, where each regime is separated by regime switching points from both the preceding and subsequent regimes. In system identification and monitoring applications, it is crucial to correctly and timely detect these regime switching points. One promising estimation method that may be used for detecting regime switching points is the stochastic learning based weak estimation (SLWE) method by Oommen and Rueda. We use SLWE for estimating First Order Markov (FOM) probabilities between symbols emitted by a system and for predicting regime switching points. A switching point is detected when the SLWE estimator unlearns, i.e., adapts estimates of FOM probabilities to new observations, such that the estimate re-converges to a new value that reflects, for the new regime, the FOM dependency of system output tokens. In experiments with a real Dataset for Human Activity Recognition, we see that our method has attractive efficiency (time and space) and similar accuracy compared with the state-of the-art. Experiments with synthetic data, where we controlled noise and Hamming distance between regimes, show promising accuracy for noise rates up to 25%, a rate at which accuracy of state-of-the-art methods deteriorates. Our method is flexible and can be configured to use not only FOM but also second-order and prior symbol probabilities, and combinations thereof.

The research area of ambient assisted living has led to the development of activity recognition systems (ARS) based on human activity recognition (HAR). These systems improve the quality of life and the health care of the elderly and dependent people. However, before making them available to end users, it is necessary to evaluate their performance in recognizing activities of daily living, using data set benchmarks in experimental scenarios. For that reason, the scientific community has developed and provided a huge amount of data sets for HAR. Therefore, identifying which ones to use in the evaluation process and which techniques are the most appropriate for prediction of HAR in a specific context is not a trivial task and is key to further progress in this area of research. This work presents a systematic review of the literature of the sensor-based data sets used to evaluate ARS. On the one hand, an analysis of different variables taken from indexed publications related to this field was performed. The sources of information are journals, proceedings, and books located in specialized databases. The analyzed variables characterize publications by year, database, type, quartile, country of origin, and destination, using scientometrics, which allowed identification of the data set most used by researchers. On the other hand, the descriptive and functional variables were analyzed for each of the identified data sets: occupation, annotation, approach, segmentation, representation, feature selection, balancing and addition of instances, and classifier used for recognition. This paper provides an analysis of the sensor-based data sets used in HAR to date, identifying the most appropriate dataset to evaluate ARS and the classification techniques that generate better results.

<strong>Abstract:</strong> This work introduces the harAGEdataset: a novel multimodal smartwatch-based dataset for Human Activity Recognition (HAR) with more than 17 hours of data collected from 19 participants using a Garmin Vivoactive 3 device. The dataset contains samples from resting, lying, sitting, standing, washing hands, walking, running, stairs climbing, strength workout, flexibility workout, and cycling activities. The resting activity, excluded from the set of activities to recognise, was explicitly conducted while avoiding stressors and external stimuli, so the data collected can be used to compute the personal, baseline heart rate at rest. We also present the HAR-based models trained using the accelerometer data to recognise different sets of activities. Specifically, we focus on different strategies to combine, fuse, and enrich the accelerometer measurements, so they can be used end-to-end. Model performances are assessed following a Leave-One-Subject-Out Cross-Validation (LOSO-CV) approach, and we use the Unweighted Average Recall (UAR) as the evaluation metric to compare the ground truth and the inferred information. The best UAR score of 98.1 % is obtained when recognising the static and the dynamic activities, excluding the samples corresponding to the washing hands, strength workout, and flexibility workout activities. When recognising the specific activities included in these two sets, the model with the best performance scores a UAR of 70.1 %. Finally, when recognising all the activities considered in the harAGEdataset, the highest UAR achieved is 64.3 %.

This paper presents a novel approach for counting hand-performed activities using deep learning and inertial measurement units (IMUs). The particular challenge in this task is finding the correct window size for capturing activities with different durations. Traditionally, fixed window sizes have been used, which occasionally result in incorrectly represented activities. To address this limitation, we propose segmenting the time series data into variable-length sequences using ragged tensors to store and process the data. Additionally, our approach utilizes weakly labeled data to simplify the annotation process and reduce the time to prepare annotated data for machine learning algorithms. Thus, the model receives only partial information about the performed activity. Therefore, we propose an LSTM-based architecture, which takes into account both the ragged tensors and the weak labels. To the best of our knowledge, no prior studies attempted counting utilizing variable-size IMU acceleration data with relatively low computational requirements using the number of completed repetitions of hand-performed activities as a label. Hence, we present the data segmentation method we employed and the model architecture that we implemented to show the effectiveness of our approach. Our results are evaluated using the Skoda public dataset for Human activity recognition (HAR) and demonstrate a repetition error of ±1 even in the most challenging cases. The findings of this study have applications and can be beneficial for various fields, including healthcare, sports and fitness, human-computer interaction, robotics, and the manufacturing industry.