Optimized Skeleton graph based CNN for Human Abnormal Detection in Video Streams

Human behavior has always been a significant parameter in social interaction and human activity recognition is a major clue that eases the analysis of human character. In recent decades, researchers have gained more attention towards human activity detection that plays a huge part in different domains, like intelligent video surveillance, and human computer communication in order to improve security level. In order to recognize the human action from videos, effective features are required. However, extraction of effective features is still resembles as a challenging process due to changes occurred in human action. In order to counterpart such limitations, this article proposes an effective strategy for human action recognition and abnormality detection using Chronological Poor and Rich Tunicate Swarm Algorithm (CPRTSA)-based Deep Maxout Network. However, the human action recognition and abnormality detection is performed using same Deep Maxout Network, where the network classifier is trained using proposed CPRTSA. The proposed approach provides better accuracy in terms of human action recognition with higher efficiency and the proposed CPRTSA achieved a maximum accuracy of 0.959, maximum sensitivity of 0.963, and a maximum specificity of 0.965.

Human action detection and recognition has become a significant topic in computer vision research over the last two decades. Intelligent techniques (such as machine learning and deep learning) have grown in popularity as a result of technological advancements in visual data. Due to the exponential growth of video data generated by surveillance cameras, intelligent systems are in great demand for detecting specific human activities. In this work, we critically examine the state-of-the-art (SOA) methodologies for human activity detection (HAD) approaches. The study illustrates a generic classification for HAD methods anda paradigm shift is observed from traditional to modern deep learning-based techniques. We also provide an analysis of publicly available datasets for the classification of human activities. The paper outlines various open research issues as well as future directions for HAD methods. Due to problems such as a dynamic and complex background, camera motion, occlusion, and bad weather, it is evident that detecting human behaviors in surveillance videos is a challenging task. Convolution neural networks (CNNs) are used in the bulk of HAD techniques, prompting researchers to look at sequence learning models like Recurrent Neural Networks (RNNs) and Long-Short-Term Neural Networks (LSTM). Furthermore, our analysis indicates that only a few research articles on the detection of anomalous behavior have been published, with the majority of the work focusing on human action detection. Furthermore, existing HAD deep learning models could be improved by including the notion of transfer learning, which saves training time and enhances accuracy.

Human Activity Detection and Action Recognition in videos using Convolutional Neural Networks

Human action abnormality detection has been attempted by various sensors for application domains like rehabilitation, healthcare, and assisted living. Since the release of motion sensors that ease the human body skeleton retrieval, skeleton-based human action recognition has recently been an active topic in the area of artificial intelligence. Unlike human action recognition, human action abnormality detection is an emerging field that aims to detect the incorrect action from the same action class. Graph convolutional network has been widely adopted for human action recognition. However, to the best of our knowledge, whether it could be effective for the task of human action abnormality detection has not been attempted. To advance prior work in the emerging field of human action abnormality detection, we propose a novel method that uses graph convolutional network to detect abnormal actions in skeleton data. To validate the effectiveness of our proposed method, we conduct extensive experiments on a public dataset called UI-PRMD. Based on the experimental results, our proposed method achieved superior action abnormality detection performance comparing with existing deep learning methods.

Understanding the action of human plays an important role in public gatherings and recognition of human action is a major problem which leads to analysis the human activities. In many years, people are interested for detecting the human activity. Human behavior analysis are used in many areas like video surveillance, banks to increase public security. To detect the human behavior from the videos, essential features are to be detected. The major challenge in human action recognition is to generate the required features significance changes occurred in human action. Nowadays skeleton data-based action detection becoming more popular. In order to counterpart such limitations, this paper brings a method using Skeleton Graph based deep learning convolutional neural network. The proposed method gives accuracy of 0.93.

Bidirectional Long Short-Term Memory with Temporal Dense Sampling for human action recognition

In recent years, many researchers have focused on using convolutional neural networks to perform human activity recognition as evidenced by the emergence of a number of convolutional neural network architectures such as LeNet-5, AlexNet and VGG16 and modern architectures such as ResNet, Inception V3, Inception-ResNet, MobileNet V2, NASNet and PNASNet. The main characteristic of a convolutional neural network (CNN) is its ability to extract features automatically from input images, which facilitates the processes of activity recognition and classification. Convolutional networks indeed derive more relevant and complex features with every additional layer. In addition, CNNs have achieved perfect classification on highly similar activities that were previously extremely difficult to classify. In this paper, we evaluate modern convolutional neural networks in terms of their human activity recognition accuracy, and we compare the results with the state of the art methods. In our research, we used two public data sets, HMDB (Shooting gun, kicking, falling to the floor, punching) and the Weizman dataset (walking, running, jumping, bending, one hand waving, two-hand waving, jumping in place, jumping jack, skipping). Our experimental results indicated that the CNN with NASNet architecture achieves the best performance of the six CNN architectures on both human activity data sets (HMDB and Weizman).

In recent years, channel state information (CSI) in WiFi 802.11n has been increasingly used to collect data pertaining to human activity. Such raw data are then used to enhance human activity recognition. Activities such as lying down, falling, walking, running, sitting down, and standing up can now be detected with the use of information collected through CSI. Human activity recognition has a multitude of applications, such as home monitoring of patients. Four deep learning models are presented in this paper, namely: a convolution neural network (CNN) with a Gated Recurrent Unit (GRU); a CNN with a GRU and attention; a CNN with a GRU and a second CNN, and a CNN with Long Short-Term Memory (LSTM) and a second CNN. Those models were trained to perform Human Activity Recognition (HAR) using CSI amplitude data collected by a CSI tool. Experiments conducted to test the efficacy of these models showed superior results compared with other recent approaches. This enhanced performance of our models may be attributable the ability of our models to make full use of available data and to extract all data features, including high dimensionality and time sequence. The highest average recognition accuracy reached by the proposed models was achieved by the CNN-GRU, and the CNN-GRU with attention models, standing at 99.31% and 99.16%, respectively. In addition, the performance of the models was evaluated for unseen CSI data by training our models using a random split-of-dataset method (70% training and 30% testing). Our models achieved impressive results with accuracies reaching 100% for nearly all activities. For the lying down activity, accuracy obtained from the CNN-GRU model stood at 99.46%; slightly higher than the 99.05% achieved by the CNN-GRU with attention model. This confirmed the robustness of our models against environmental changes.

The conventional Human Activity Recognition (HAR) systems use smartwatches, smartphones, and other wearable devices to autonomously derive consumer physical activities. However, implementing real-time HAR in resource-constrained applications is challenging. Computational offloading approaches must address network reliance, latency, and data privacy, which affect system performance. The present study proposes a low-cost, wearable HAR device that collects and transmits the Inertial Measurement Unit (IMU) telemetry signals to smartphones, enabling real-time on-device activity detection. Human actions are subjective and affected by physiological, environmental, and behavioural factors, complicating HAR systems' performance in personalized contexts. Generalized methods for existing systems reduce performance for new consumers or due to alterations in consumer locomotive signatures. Thus, our second goal is to enable the on-device personalization of wearable HAR models with minimum customer calibration. The HAR device, which is in the form of a smart-band, transfers data to and fro smartphones through Bluetooth Low Energy (BLE). A lightweight 1D convolutional neural network (CNN) is built and, using transfer learning, the network model is fine-tuned from real-time sensed data. On-device HAR inferences ensure real-time processing on smartphones without computational offloading. Smartphone-based on-device training for fine-tuning the model allows the HAR system to get customized without compromising privacy or computational costs. The experimental results show that the personalized classification achieves on average 98% accuracy when assessed using four benchmark datasets.

Human Action Recognition (HAR)is a challenging time series classification problem that has received significant attention from computer vision researchers. In this paper, different techniques used for human activities are investigated, and a human action recognition approach using a Convolutional Neural Network (CNN) with Long Short Term Memory (LSTM) and generative adversarial network is proposed. The proposed research evaluates the performance of cross-entropy and adversarial loss function for HAR analysis. Two different datasets UFC101 and the classic KTH dataset, are used for experimental purposes. The UFC101 dataset contains 13k videos in which 101 human actions are included i.e., playing instrument, makeup, etc. In contrast, KTH dataset contains 600 videos containing six human activities, including walking, running, jogging, hand clapping and hand waving, performed by 25 different persons. Also, demonstrates the process of HAR by mixing both datasets and evaluate the performance. The GAN enhances the model robustness by applying adversarial training which fully discovers the underlying connections in both intra-view and cross-view aspects.

Understanding semantic meaning of human actions captured in unconstrained environments has broad applications in fields ranging from patient monitoring, human-computer interaction, to surveillance systems. However, while great progresses have been achieved on automatic human action detection and recognition in videos that are captured in controlled/constrained environments, most existing approaches perform unsatisfactorily on videos with uncontrolled/unconstrained conditions (e.g., significant camera motion, background clutter, scaling, and light conditions). To address this issue, the authors propose a robust human action detection and recognition framework that works effectively on videos taken in controlled or uncontrolled environments. Specifically, the authors integrate the optical flow field and Harris3D corner detector to generate a new spatial-temporal information representation for each video sequence, from which the general Gaussian mixture model (GMM) is learned. All the mean vectors of the Gaussian components in the generated GMM model are concatenated to create the GMM supervector for video action recognition. They build a boosting classifier based on a set of sparse representation classifiers and hamming distance classifiers to improve the accuracy of action recognition. The experimental results on two broadly used public data sets, KTH and UCF YouTube Action, show that the proposed framework outperforms the other state-of-the-art approaches on both action detection and recognition.

The identification of human activities holds paramount importance in diverse domains, including CCTV monitoring and human-robot interaction. While existing Human Activity Recognition (HAR) techniques have made substantial progress, challenges persist due to their reliance on manual feature extraction. This research introduces an innovative approach to analyze and identify human actions within small video sequences, employing a convolutional neural network (CNN). The suggested CNN approach combines a 3D convolution technique with a convolutional Long Short-Term Memory process, enabling concurrent examination of spatiotemporal data spanning multiple video frames. In contrast to conventional frame-byframe analysis, this CNN detects and measures distinctions and resemblances among objects in consecutive frames, enhancing the capability for more intuitive identifications. This approach maximizes processing resources, ensuring efficient training and classification, thereby yielding precise results rapidly. Our novel approach addresses the challenge of model generalization by overcoming the limitations associated with manual feature extraction. By leveraging the combined power of 3D convolution and LSTM, our CNN model demonstrates enhanced capabilities in recognizing and understanding human actions within dynamic video sequences. The presented methodology not only contributes to the advancement of HAR techniques but also opens avenues for more effective utilization of processing resources in real-world applications.

Human Activity Recognition (HAR) has recently attracted the attention of researchers. Human behavior and human intention are driving the intensification of HAR research rapidly. This paper proposes a novel Motion History Mapping (MHI) and Orientation-based Convolutional Neural Network (CNN) framework for action recognition and classification using Machine Learning. The proposed method extracts oriented rectangular patches over the entire human body to represent the human pose in an action sequence. This distribution is represented by a spatially oriented histogram. The frames were trained with a 3D Convolution Neural Network model, thus saving time and increasing the Classification Correction Rate (CCR). The K-Nearest Neighbor (KNN) algorithm is used for the classification of human actions. The uniqueness of our model lies in the combination of Motion History Mapping approach with an Orientation-based 3D CNN, thereby enhancing precision. The proposed method is demonstrated to be effective using four widely used and challenging datasets. A comparison of the proposed method’s performance with current state-of-the-art methods finds that its Classification Correction Rate is higher than that of the existing methods. Our model’s CCRs are 92.91%, 98.88%, 87.97.% and 87.77% which are remarkably higher than the existing techniques for KTH, Weizmann, UT-Tower and YouTube datasets, respectively. Thus, our model significantly outperforms the existing models in the literature.

Traditional methods of human action recognition focus on internal links between human joints, that is, local neighbor connections. However, some external links that cannot exist in the skeleton graph are also important for identifying human behaviors, such as hands and feet in harmonious movement. To capture richer spatial features in a skeleton graph, it is necessary to add external links and distinguish internal links from external links in a skeleton graph. Therefore, this paper designs two different adjacency matrices to characterize the internal links and external links of the human body respectively, and set different edge weights and feature weights to them for autonomous learning during the convolution process. Furthermore, a spatio-temporal graph convolution network called ieSTGCN is proposed. It consists of two modules: graph convolution network supporting internal and external links (ieGCN) and temporal convolution network in human joints (joTCN). Experiments on the Kinetics and the NTU-RGB+D datasets demonstrate that our model can obtain better recognition accuracy than some benchmark models.

Human detection and action recognition form the basis for understanding human behaviors. Human detection is used to detect the positions of humans, and action recognition is able to recognize the action of specific humans. However, numerous approaches have been used to handle action recognition and human detection separately. Therefore, three main issues still exist when independent methods of human detection and action recognition are combined, 1) intrinsic errors in object detection impact the performance of action recognition, 2) features common to action recognition and object detection are missed, 3) the combination also has an impact on processing speed. We propose a single framework for human detection and action recognition to solve these issues. It is based on a hierarchical structure called Boosted Randomized Trees. The nodes are trained such that the upper nodes detect humans from the background, while the lower nodes recognize action. We were able to improve both human detection and action recognition rates over earlier hierarchical structure approaches with the proposed method.