Trajectory-Pooled Deep Convolutional Networks for Violence Detection in Videos

The violence detection is mostly achieved through handcrafted feature descriptors, while some researchers have also employed deep learning-based representation models for violent activity recognition. Deep learning-based models have achieved encouraging results for fight activity recognition on benchmark data sets such as hockey and movies. However, these models have limitations in learning discriminating features for violence activity classification with abrupt camera motion. This research work investigated deep representation models using transfer learning for handling the issue of abrupt camera motion. Consequently, a novel deep multi-net (DMN) architecture based on AlexNet and GoogleNet is proposed for violence detection in videos. AlexNet and GoogleNet are top-ranked pre-trained models for image classification with distinct pre-learnt potential features. The fusion of these models can yield superior performance. The proposed DMN unleashed the integrated potential by concurrently coalescing both networks. The results confirmed that DMN outperformed state-of-the-art methods by learning finest discriminating features and achieved 99.82% and 100% accuracy on hockey and movies data sets, respectively. Moreover, DMN has faster learning capability i.e. 1.33 and 2.28 times faster than AlexNet and GoogleNet, which makes it an effective learning architecture on images and videos.

WITHDRAWN: Inception ResNet deep transfer learning model for human action recognition using LSTM

Violence detection in videos is very promising in practical applications due to the emergence of massive videos in recent years. Most previous works define violence detection as a simple video classification task and use the single modality of small-scale datasets, e.g., visual signal. However, such solutions are undersupplied. To mitigate this problem, we study weakly supervised violence detection on the large-scale audio-visual violence data, and first introduce two complementary tasks, i.e., coarse-grained violent frame detection and fine-grained violent event detection, to advance the simple violence video classification to frame-level violent event localization, which aims to accurately locate the violent events on untrimmed videos. We then propose a novel network that takes as input audio-visual data and contains three parallel branches to capture different relationships among video snippets and further integrate features, where similarity branch and proximity branch capture long-range dependencies using similarity prior and proximity prior, respectively, and score branch dynamically captures the closeness of predicted score. In both coarse-grained and fine-grained tasks, our approach outperforms other state-of-the-art approaches on two public datasets. Moreover, experiment results also show the positive effect of audio-visual input and relationship modeling.

Violence detection in videos using interest frame extraction and 3D convolutional neural network

Convolutional neural networks (CNN) have shown remarkable performance in enormous computer vision applications over the years, and many works have been done for human activity recognition (HAR) using CNN. However, most of deep learning-based architectures require large training data and plenty of computational resources. Therefore, we proposed a simple and efficient deep learning model for human activity recognition, which works on complex visual data and require less computational resources. In the proposed work, we designed a novel CNN architecture by stacking the repeated components called micro-networks to incorporate multiscale processing in the network layers. We have used a feature fusion strategy to pass the previous layer’s abstract complementary information to the next adjacent layer, believing that each layer encapsulates specific feature maps. Therefore, the hidden complementary information can potentially enhance the feature discrimination capacity of the network and help in learning the network. The proposed architecture is fully trained from scratch with stochastic gradient descent (SGD) optimizer at 0.05 initial learning rate and a softmax classifier is used for activity recognition. The merit of the proposed method over standard deep learning models is its computational efficiency in terms of learnable parameters and computational resources. The proposed model gives good performance on small as well as large size datasets. For authentication of the proposed method several extensive experiments are conducted on publically available datasets, namely UCF-101, HMDB-51, YouTube, and IXMAS datasets. The results have shown the outperformance of the proposed method over the existing state-of-the-art methods.

Human activity recognition is an important and challenging topic for computer vision research community. Action representation using deep learning models are currently the dominant technique compared with other methods. However, supervised convolutional neural networks require large computational and memory resources to optimize their parameters. Recently, a simple unsupervised deep learning architecture Principal Component Analysis Network (PCANet) has emerged as an alternative of Convolutional Neural Networks (CNNs) and has significant accomplishments in various vision applications. Meanwhile, encoding and representation techniques using Bag of Words (BoW) and Vector of Locally Aggregated Descriptors (VLAD) have demonstrated great success for several visual tasks specifically in activity recognition. This work presents a novel human activity recognition technique by combining global and local features of PCANet with BoW and VLAD encoding schemes. Both global and local features are learned by PCANet utilizing selected frames from each action video. After that the dimensionality of these features is decreased via Whitening PCA (WPCA). Then encoding schemes are applied on both features to represent the final descriptors for each action. Ultimately, Support Vector Machines classifier (SVM) is trained for recognition process. Several experiments are conducted on UCF sports dataset to evaluate our method. All experimental results utilizing leave-one-out-cross validation (LOOCV) strategy are satisfactorv and comparable.

Human activity recognition (HAR) is one of the important research areas in pervasive computing. Among HAR, sensor-based activity recognition refers to acquiring a high-level knowledge about human activities from readings of many low-level sensor. In recent years, although traditional methods of deep learning (DL) have been widely used for sensor-based HAR with some good performance, they still face such challenges as feature extraction and characterization, continuous action segmentation in dealing with time series problems. In this study, a multichannel fusion model is proposed by the idea of dividing. In this proposed architecture, a multichannel convolutional neural network (CNN) is used to enhance the ability to extract features at different scales, and then the fused features are fed into gated recurrent unit (GRU) for feature labeling and enhanced feature representation, through the learning of temporal relationships. Finally, the multichannel CNN-GRU model is designed using global average pooling (GAP) to connect the feature maps with the final classification. The model performance was conducted on three benchmark datasets of WISDM, UCI-HAR, and PAMAP2 with the accuracy of 96.41%, 96.67%, and 96.25% respectively. The results show that the proposed model demonstrates better activity detection capability than some of the reported results.

Human action recognition is a fundamental research problem in computer vision. The accuracy of human action recognition has important applications. In this book chapter, the authors use a YOLOv7-based model for human action recognition. To evaluate the performance of the model, the action recognition results of YOLOv7 were compared with those using CNN+LSTM, YOLOv5, and YOLOv4. Furthermore, a small human action dataset suitable for YOLO model training is designed. This data set is composed of images extracted from KTH, Weizmann, MSR data sets. In this book chapter, the authors make use of this data set to verify the experimental results. The final experimental results show that using the YOLOv7 model for human action recognition is very convenient and effective, compared with the previous YOLO model.

Clinicians lack objective means for monitoring if their knee osteoarthritis patients are improving outside of the clinic (e.g., at home). Previous human activity recognition (HAR) models using wearable sensor data have only used data from healthy people and such models are typically imprecise for people who have medical conditions affecting movement. HAR models designed for people with knee osteoarthritis have classified rehabilitation exercises but not the clinically relevant activities of transitioning from a chair, negotiating stairs and walking, which are commonly monitored for improvement during therapy for this condition. Therefore, it is unknown if a HAR model trained on data from people who have knee osteoarthritis can be accurate in classifying these three clinically relevant activities. Therefore, we collected inertial measurement unit (IMU) data from 18 participants with knee osteoarthritis and trained convolutional neural network models to identify chair, stairs and walking activities, and phases. The model accuracy was 85% at the first level of classification (activity), 89–97% at the second (direction of movement) and 60–67% at the third level (phase). This study is the first proof-of-concept that an accurate HAR system can be developed using IMU data from people with knee osteoarthritis to classify activities and phases of activities.

Due to the increasing number of mobile robots including domestic robots for cleaning and maintenance in developed countries, human activity recognition is inevitable for congruent human-robot interaction. Needless to say that this is indeed a challenging task for robots, it is expedient to learn human activities for autonomous mobile robots (AMR) for navigating in an uncontrolled environment without any guidance. Building a correct classifier for complex human action is non-trivial since simple actions can be combined to recognize a complex human activity. In this paper, we trained a model for human activity recognition using convolutional neural network. We trained and validated our model using the Vicon physical action dataset and also tested the model on our generated dataset (VMCUHK). Our experiment shows that our method performs with high accuracy, human activity recognition task both on the Vicon physical action dataset and VMCUHK dataset.

According to the National Highway Traffic Safety Administration, one in ten fatal crashes and two in ten injury crashes were reported as distracted driver accidents in the United State during 2014. In an attempt to mitigate these alarming statistics, this paper explores using a dashboard camera along with computer vision and machine learning to automatically detect distracted drivers. We consider a dataset that incorporates drivers engaging in seven different distracting behaviors using left and/or right hands. Traditional handcrafted features paired with a Support Vector Machine classifier are contrasted with deep Convolutional Neural Networks. The traditional features include a blend of Histogram of Oriented Gradients and Scale-Invariant Feature Transform descriptors used to create Bags of Words. The deep convolutional methods use transfer learning on AlexNet, VGG-16, and ResNet-152. The results yield 85% accuracy with ResNet and 82.5% accuracy with VGG-16, which outperformed AlexNet by almost 10%. Replacing the fully connected layers by a Support Vector Machine classifier did not improve the classification accuracy. The traditional features yielded much lower accuracy than the deep convolutional networks.

Anomaly detection in crowd surveillance videos is a critical task for ensuring public safety and security. In this research paper, we propose a comprehensive framework for anomaly detection using deep learning techniques, specifically Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Generative Adversarial Networks (GANs). Our objective is to narrow the gap between computational complexity and detection effectiveness while satisfying the demand for explainability in anomaly detection systems. The proposed framework leverages the UCSD Ped2 dataset and focuses on incorporating spatial constraints to enhance anomaly detection performance. We introduce novel approaches for feature extraction and anomaly detection using GANs, aiming to capture essential spatial and temporal information while reducing computational complexity. Specifically, we develop encoders within the GAN framework to map input data to a lower-dimensional latent space and perform anomaly detection based on the similarity between latent codes. Through extensive experimentation and evaluation, we compare the performance of RNNs, CNNs, and GANs in detecting anomalies in crowd surveillance videos. Various evaluation metrics, including precision, recall, F1 score, ROC-AUC, and PR-AUC, is considered to assess the effectiveness of each approach. Additionally, they studied hyperparameters such as learning rate, batch size, network architecture, and GAN-specific parameters to optimize anomaly detection performance. Our results demonstrate the usefulness of the proposed framework in achieving robust and explainable anomaly detection in crowd surveillance videos. By combining advanced deep learning techniques with spatial constraints and explainability considerations, our research contributes to the development of more reliable and interpretable anomaly detection systems for real-world applications.

FTCF: Full temporal cross fusion network for violence detection in videos

For this work to be done, I relied on the help and support of many people.My family, for the unconditional love and support through such long distances.My advisors, Anderson and Zanoni, for the patience and the help throughout this journey.My friend Lavi, who believed so much in

Human Violence Detection in Videos Using Key Frame Identification and 3D CNN with Convolutional Block Attention Module