Video Abnormal Behavior Detection Based on Optical Flow Method and Convolutional Neural Network

As society develops, the demand for security is rapidly increasing. Accordingly, there is growing interest in research on methods to detect and prevent abnormal behavior using surveillance cameras in public places and private spaces such as shopping malls and airports for human safety. Many detection techniques based on deep learning models have been researched in the field of abnormal behavior detection. However, due to the lack of labeled abnormal behavior data, there are significant difficulties in developing an effective detection system. This paper surveys methods for deep learning methods to detect abnormal human behavior in surveillance video and presents recent techniques. First, I will introduce popular datasets that have often been used in previous research. After that, we categorized the existing methods for detecting abnormal behavior using deep learning into three types: supervised learning, unsupervised learning, and partially-supervised learning. We then explained the basic concepts and advantages of each method and summarized their shortcomings. We also briefly describe future research directions based on the advantages and disadvantages of each method. Based on this, it is expected that the technology of video surveillance systems that apply abnormal behavior detection will further develop.

Abnormal behavior detection in surveillance videos based on deep learning has becomea hot topic in current research. However, due to the complexity and variability of crowd movement and external environment, detecting abnormal behavior faces great challenges. Current abnormal behavior recognition models have limitations in feature extraction and pay insufficient attention to dynamic temporal features. To address these problems, a spatiotemporal enhancement anomaly detection method based on multi-channel coupling is proposed. Based on the SlowFast network, a multi-channel coupled temporal enhancement module and a spatial enhancement module are introduced respectively to extract more discriminative static and dynamic feature information. A large number of experiments are carried out on three benchmark datasets (Violent Flow, Hockey Fight, and Real-life Violence Situations). The results show that the prediction accuracy of the proposed abnormal behavior recognition method reaches 95.3%、97.3%and respectively 94%, thus verifying the effectiveness of the method.

Convolution Neural Networks (CNNs) have become an attractive method for the detection of anomalous behaviors. However, designing an efficient CNN model in terms of classification accuracy remains a challenging problem. Furthermore, the existing datasets for abnormal behavior detection are limited, with each focusing on a certain context. Therefore, a CNN model trained on a certain dataset will be adaptive for a particular context and not suitable for other contexts. This study proposes a CNN framework with an efficient attention mechanism to capture key information from multiple inputs, namely RGB, optical flow, and heatmap. Experiments were carried out on several benchmark datasets and a self-collected dataset, and the evaluation involved both single- and cross-dataset strategies. The results show the superior performance of the proposed frameworks compared to other SOTA methods in detection accuracy.

With the development of track and field, people pay more and more attention to the quality of classroom teaching of track and field technology, and the evaluation of teaching quality plays a key role in it. In today's educational reform, teaching evaluation plays an important role as an important method to test teachers' teaching and students' learning. With the rapid development of machine learning, especially deep learning, image-based individual abnormal behavior detection technology is becoming more and more mature, but there are still many difficulties to be solved in video-based group abnormal behavior detection technology. Therefore, it is necessary to study the detection algorithm. Based on machine vision theory, image processing theory and video analysis technology, this paper studies three key technologies involved in human abnormal behavior detection in video: moving target detection, moving target tracking and abnormal behavior detection.

Abnormal crowd behavior detection has recently attracted increasing attention due to its wide applications in computer vision research areas. However, it is still an extremely challenging task due to the great variability of abnormal behavior coupled with huge ambiguity and uncertainty of video contents. To tackle these challenges, we propose a new probabilistic framework named variational abnormal behavior detection (VABD), which can detect abnormal crowd behavior in video sequences. We make three major contributions: (1) We develop a new probabilistic latent variable model that combines the strengths of the U-Net and conditional variational auto-encoder, which also are the backbone of our model; (2) We propose a motion loss based on an optical flow network to impose the motion consistency of generated video frames and input video frames; (3) We embed a Wasserstein generative adversarial network at the end of the backbone network to enhance the framework performance. VABD can accurately discriminate abnormal video frames from video sequences. Experimental results on UCSD, CUHK Avenue, IITB-Corridor, and ShanghaiTech datasets show that VABD outperforms the state-of-the-art algorithms on abnormal crowd behavior detection. Without data augmentation, our VABD achieves 72.24% in terms of AUC on IITB-Corridor, which surpasses the state-of-the-art methods by nearly 5%.

In this paper; we propose a novel crowd behavior representation method to detect abnormal behaviors in videos. An adaptive optical flow filtering method is proposed to utilize low-level optical flow informations. Furthermore, a simple framework is developed to detect and to localize abnormal crowd behavior using adaptive optical flow filtering result. The proposed method is more robust than other modeling methods in representing different behaviors. In this model, a normal behavior is presented by the general value. Some outliers in the temporal domain or spatial domain are presented by a higher value. Spatio-temporal cuboids are extracted from the filtering result to present the likelihood of anomaly in the frame. Experimental evaluations are performed on two public datasets with comparison to the provisos abnormal behavior detection methods in the literature. Experimental results show that the proposed methods outperform previous abnormal behavior detection techniques in the literature.

Detecting human beings accurately in a visual surveillance system is crucial for diverse application areas including abnormal event detection, human gait characterization, congestion analysis, person identification, gender classification and fall detection for elderly people. The first step of the detection process is to detect an object which is in motion. Object detection could be performed using background subtraction, optical flow and spatio-temporal filtering techniques. Once detected, a moving object could be classified as a human being using shape-based, texture-based or motion-based features. A comprehensive review with comparisons on available techniques for detecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmark datasets as well as the future research directions on human detection have also been discussed.

In this paper, global abnormal events detection in the crowded scene is proposed. The proposed algorithm is based on a hierarchical approach. The method uses the histogram of optical flow orientation as a feature descriptor along with the magnitude of the optical flow to capture the motion. The nonlinear one-class support vector machine (SVM) classification algorithm is used to learn the normal events from the training data. After learning, one-class SVM detects abnormal events in the frame. The algorithm is fast with improved accuracy since it omits the background subtraction step and adopts hierarchy. The proposed method is tested on benchmark UMN unusual crowd activity dataset which contain global abnormal events. Promising results are reported which evaluates the performance of the proposed method.

For abnormal gait detection in surveillance videos, the existing methods suffer from that they are unable to recognize novel types of anomalies if the corresponding prototypes have not been included in the training data for supervised machine learning but it is impractical to foresee all types of anomalies. This research aims to solve the problem in an unsupervised manner, which does not rely on any prior knowledge regarding abnormal prototypes and avoids time-consuming machine learning over large-scale high-dimensional features. The intuition is that normal gait is nearly periodic signal and anomalies may disturb such periodicity. Hence, the time-varying ratio of width to height of a walking person is transformed to frequency domain using Fast Fourier Transform (FFT), and the standard deviation over spectrum is used as an indicator of anomalies, subject to any sudden change to break the normally periodical walking rhythm. The experimental results demonstrate its precision.

Today, machine learning and deep learning have paved the way for vital and critical applications such as abnormal detection. Despite the modernity of transfer learning, it has proved to be one of the crucial inventions in the field of deep learning because of its promising results. For the purpose of this study, transfer learning is utilized to extract human motion features from RGB video frames to improve detection accuracy. A convolutional neural network (CNN) based on Visual Geometry Group network 19 (VGGNet-19) pre-trained model is used to extract descriptive features. Next, the feature vector is passed into Binary Support Vector Machine classifier (BSVM) to construct a binary-SVM model. The performance of the proposed framework is evaluated by three parameters: accuracy, area under the curve, and equal error rate. Experiments performed on two different datasets comprising highly different context abnormalities accomplished an accuracy of 97.44% and an area under the curve (AUC) of 0.9795 for University of Minnesota (UMN) dataset and accomplished an accuracy of 86.69% and an AUC of 0.7987 for University of California, San Diego Pedistrain1 (UCSD-PED1) dataset. Moreover, the performance of the pre-trained network VGGNet-19 with handcrafted feature descriptors and with other CNN pre-trained networks, respectively, has been investigated in this study for abnormal behavior detection. The results demonstrated that VGGNet-19 has better performance than histogram of oriented gradients, background subtraction, and optical flow. In addition, the VGGNet-19 shows higher detection accuracy than other pre-trained networks: GoogleNet, ResNet50, AlexNet, and VGGNet-16.

Abnormal behaviour detection is crucial given the reliance on video surveillance for public safety. However, most vision-based methods fail to adapt to the variability and complexity of the crowd scenes inherent in real-world scenarios. This study introduced an abnormal crowd behaviour detection algorithm based on lightweight Graphic Neural Networks (GNNs) using trajectory similarities and spatial-temporal attributes. Integrating Graph Attention Networks (GATs) to encode the graph structure, creating latent embeddings that capture both local and global relationships within crowds. Also introducing an autoencoder (AE) structure to identify anomalies by comparing true movement with the decoded output. Anomaly is detected when the reconstruction error is high. Experiments on UCSD Pedestrians and ShanghaiTech campus datasets show the successful use of trajectory similarities as graph features for GNNs to generate logical embeddings of crowd movement and highlight the impact of time windows on anomaly detection precision as reflected in F1-scores. The Area Under the Curve (AUC) metric shows stability, highlighting the model’s balanced performance. Qualitative analysis of the ShanghaiTech campus dataset illustrates the model’s ability to align detection with visual anomalies. Overall, this study demonstrates the effectiveness of the proposed graph-based anomaly detection framework for crowd monitoring through a lightweight model suitable for CCTV.

Video abnormal behavior detection is an important research direction in the field of computer vision and has been widely used in surveillance video. This paper proposes an abnormal behavior detection model based on human skeletal structure and recurrent neural network, which uses dynamic skeletal features for abnormal behavior detection. The model in this paper extracts key points from multiple frames through the human pose estimation algorithm, obtains human skeleton information, and inputs it into the established autoencoder recurrent neural network to detect abnormal human behavior from video sequences. Compared with traditional appearance-based models, our method has better anomaly detection performance under multi-scene and multi-view.KeywordsAbnormal behavior detectionGRUAutoencoderSkeletal information

More than 80% of video surveillance systems are used for monitoring people. Old human detection algorithms, based on background and foreground modelling, could not even deal with a group of people, to say nothing of a crowd. Recent robust and highly effective pedestrian detection algorithms are a new milestone of video surveillance systems. Based on modern approaches in deep learning, these algorithms produce very discriminative features that can be used for getting robust inference in real visual scenes. They deal with such tasks as distinguishing different persons in a group, overcome problem with sufficient enclosures of human bodies by the foreground, detect various poses of people. In our work we use a new approach which enables to combine detection and classification tasks into one challenge using convolution neural networks. As a start point we choose YOLO CNN, whose authors propose a very efficient way of combining mentioned above tasks by learning a single neural network. This approach showed competitive results with state-of-the-art models such as FAST R-CNN, significantly overcoming them in speed, which allows us to apply it in real time video surveillance and other video monitoring systems. Despite all advantages it suffers from some known drawbacks, related to the fully-connected layers that obstruct applying the CNN to images with different resolution. Also it limits the ability to distinguish small close human figures in groups which is crucial for our tasks since we work with rather low quality images which often include dense small groups of people. In this work we gradually change network architecture to overcome mentioned above problems, train it on a complex pedestrian dataset and finally get the CNN detecting small pedestrians in real scenes.

With the development of Internet technology and multimedia technology, the application scenarios of video in people’s lives are more and more extensive. On this basis, a bidirectional convolutional network based on time domain and spatial domain is proposed. A new network structure is proposed based on convolutional neural networks. Firstly, pixel information is extracted from video images in the spatial domain and used as input features; At the same time, in the time domain, optical flow information is integrated into video behavior, and the cone algorithm is used to calculate the changes in the optical flow vector of pixels at different times, and it is transformed into convolutional features in the time domain. The system uses special software and hardware devices to accurately record the three-dimensional motion trajectories of various parts of the moving target. On this basis, we can accurately establish a model of the moving target and conduct semantic analysis of the moving behavior (such as animation). Without supervision, the wanton dissemination of bad video content will also have a bad impact on the general public, especially young people.

Modeling abnormal behavior detection in the intelligent video monitoring system for recognition or detection of special event has attracted significant research interest in recent years. In order to achieve more effective recognition of abnormal behavior detection of the intelligent video surveillance system, this paper proposes a working human abnormal operation recognition approach based on deep multi-instance sorting model. Firstly, the uncut long video is sparse to obtain normal and abnormal behavior video segments, and the RGB and optical flow features in the segment are extracted by the deep convolution network. Video feature vectors are obtained by the consensus function and feature extractor. Then, multi-instance sorting learning is used to assign abnormality scores between 0 and 1 to feature vectors. When the abnormal values of abnormal packets are higher than those of normal packets, the abnormality score is returned and the high abnormality score is determined as an abnormal behavior. The experiments on the open THUMOS14 data set and our own XAGCWD data set using CUDA GPU accelerated computing to demonstrate that our approach improves the recognition accuracy about 10.8% and high accuracy of abnormal detection. The main purpose of this research is to apply the model proposed in this paper to the intelligent workshop behavior monitoring system to effectively realize the safety management of workshop personnel.