Adversarially Learned Abnormal Trajectory Classifier

Abnormal event detection, also known as anomaly detection, is one challenging task in security video surveillance. It is important to develop effective and robust movement representation models for global and local abnormal event detection to fight against factors such as occlusion and illumination change. In this paper, a new algorithm is proposed. It can locate the abnormal events on one frame, and detect the global abnormal frame. The proposed algorithm employs a sparse measurement matrix designed to represent the movement feature based on optical flow efficiently. Then, the abnormal detection mission is constructed as a one-class classification task via merely learning from the training normal samples. Experiments demonstrate that our algorithm performs well on the benchmark abnormal detection datasets against state-of-the-art methods.

With the booming development of tourism, travel security problems are becoming more and more prominent. Congestion, stampedes, fights and other tourism emergency events occurred frequently, which should be a wake-up call for tourism security. Therefore, it is of great research value and application prospect to real-time monitor tourists and detect abnormal events in tourism surveillance video by using computer vision and video intelligent processing technology, which can realize the timely forecast and early warning of tourism emergencies. At present, although most of the video-based abnormal event detection methods work well in simple scenes, there are often problems such as low detection rate and high false positive rate in complex motion scenarios, and the detection of abnormal events can’t be processed in real time. To tackle these issues, we propose an abnormal event detection model in tourism video based on salient spatio-temporal features and sparse combination learning, which has good robustness and timeliness in complex motion scenarios and can be adapted to real-time anomaly detection in practical applications. Specifically, spatio-temporal gradient model is combined with foreground detection to extract 3D gradient features on the foreground target of video sequence as the salient spatio-temporal features, which can eliminate the interference of the background. Sparse combination learning algorithm is used to establish the abnormal event detection model, which can realize the real-time detection of abnormal events. In addition, we construct a new ScenicSpot dataset with 18 video clips (5964 frames) containing both normal and abnormal events. The experimental results on ScenicSpot dataset and two standard benchmark datasets show that our method can realize the automatic detection and recognition of tourists’ abnormal behavior, and has better performance compared with the classical methods.

In this paper we address the problem of online video abnormal event detection. A vast number of methods to automatically detect abnormal events in videos have been recently proposed. However, the majority of these recently proposed methods cannot attain online performance; in other words, they cannot detect events as soon as they occur. Thus there is a lack of methods specifically aimed to detect events in online fashion. In this paper, we propose to incorporate binary features to detect abnormal events in an online manner. This is based on the fact that binary features are well known to require short processing times, compared to double-precision features. The main contribution of this work is then at the feature extraction step. Our experiment results of our binary-based framework show that our proposed binary features help to reduce processing times for anomaly detection, while outperforming other online methods, in terms of detection accuracy.

Traditional abnormal trajectory detection algorithms mainly involve the measurement of a single feature; however, the influence of other features on abnormal trajectory is ignored, resulting in the inability to fully discover the abnormal trajectory in the trajectory database. To overcome this limitation, we propose an abnormal trajectory detection method - called TADSS - to find the hidden abnormal trajectory by using a comprehensive measurement. Firstly, we employ three kernel functions to measure the time, velocity and position feature values of trajectory data, where the kernel functions extract semantic feature of the position, time feature of trajectory, and velocity feature of object motion from each trajectory data. Secondly, we propose a feature fusion strategy to measure the similarity of trajectory data, where we assign weights to the above kernel functions and then use a linear combination approach to fuse the weighted kernel functions. Thirdly, we build a trajectory feature graph by using the above fused kernel functions, and then divide the trajectory feature graph into a plurality of subgraphs by using a conventional graph clustering technique. Last, we propose a sparse subgraph method to detect abnormal trajectory, where a novel weight coefficient concept is used to distinguish sparse subgraph. Experimental results driven by both the vehicle trajectory data of Shanghai city and the Atlantic hurricane data demonstrate the performance of our TADSS.

Abnormal event detection with the lowest latency is an indispensable function for safety-critical systems, such as cyber defense systems. However, as systems become increasingly complicated, conventional sequential event detection methods become less effective, especially when we need to define indicator metrics from complicated data manually. Although deep neural networks (DNNs) have been used to handle heterogeneous data, the theoretic assurability and explainability are still insufficient. This article provides a holistic framework for the quickest and sequential detection of abnormalities and time-dependent abnormal events. We explore the latent space characteristics of zero-bias neural networks considering the classification boundaries and abnormalities. We then provide a novel method to convert zero-bias DNN classifiers into performance-assured binary abnormality detectors. Finally, we provide a sequential quickest detection (QD) scheme that provides the theoretically assured lowest abnormal event detection delay under false alarm constraints using the converted abnormality detector. We verify the effectiveness of the framework using real massive signal records in aviation communication systems and simulation. Codes and data are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/pcwhy/AbnormalityDetectionInZbDNN</uri> .

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

Event detection plays an essential role in video content analysis. On the other hand, according to our analysis, the coding structures in new video coding standard High Efficient Video Coding (HEVC) have a high correlation with video contents. Hence there is large potential to identify events by reusing coding structures in HEVC, which can save a huge amount of computational resources. In this paper, we proposed a new compressed-domain feature for abnormal event detection, namely Motion Intensity Count (MIC), which makes use of motion vectors, coding unit and prediction unit modes in HEVC with little computational cost. MIC can well predict the normal paths of moving objects, which enables us to identify motions in unexpected locations where abnormal events are likely to happen. Our experiments show that MIC can correctly detect abnormal events at about 1250 fps.

Intelligent Video Investigation is on nice interest in trade applications because of increasing demand to scale back the force of analyzing the large-scale video information. Sleuthing the abnormal events from crowded video scenes offer varied difficulties. Initially, an oversized variety of moving persons will simply distract the native anomaly detector. Secondly it’s tough to model the abnormal events in real time. Thirdly, the inaccessibility of ample samples of coaching information for abnormal events ends up in problem in sturdy detection of abnormal events. Our planned system provides a peculiar approach to find anomaly in crowded video scenes. We are initially divide the video frame into patches and apply the Difference-of-Gaussian (DoG) filter to extract edges. Then we work out Multiscale Histogram of Optical Flow (MHOF) and Edge directed bar chart (EOH) for every patch. Then exploitation of Normalized Cuts (NCuts) and Gaussian Expectation-Maximization (GEM) techniques, and to cluster the similar patches into cluster and assign the motion context. Finally exploitation of k-Nearest neighbor (k-NN) search, and establish the abnormal activity at intervals in the crowded scenes. Our spatio-temporal anomaly search system helps to boost the accuracy and computation time for detection of irregular patterns. This technique is helpful for investigation, trade specific and market applications like public transportation, enforcement, etc.KeywordsMultiscale Histogram of Optical FlowGaussian Expectation MaximizationSpatio-temporal anomaly search

These days consciousness about public safety increases and CCTV cameras are installed at almost all public places. But generally automatic smart surveillance systems are not available. In this manuscript, emphasis is given to detect and classify abnormal events in surveillance video especially in crowd environments. Abnormal event detection is a challenging task because the definition of abnormality is subjective. A normal event in one situation can be considered an abnormal event in another case. In the surveillance video with a dense crowd, automatic anomaly detection becomes very difficult because of clutter and severe occlusion. This manuscript represents CNN (Convolutional Neural Network) and CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) based approaches for detection and classification of abnormal events. The CNN architecture is developed from scratch and can be used for spatial domains. LSTM architecture is developed for the temporal domain. Feature sequences are generated using CNN model and given as input to LSTM model. Experiments are carried out using five different publicly available benchmark datasets. The performance is measured by accuracy and area under the ROC (receiver operating characteristic) curve (AUC). CNN-LSTM approach works better than only CNN.

Existing methods have addressed the issue of detecting abnormal events at a smart home for medical care or security monitoring services extensively in the past decades. However, most of approaches use wearable sensors that require users to be equipped with the sensor devices at every moment. If the monitored users stop or pause the sensors, any abnormal events are not able to be detected. The use of non-wearable and non-intrusive sensors (e.g., IP cameras) is necessary for providing better user experiences and achieving sustainable and reliable detection model. However, it is still very challenging to analyze such non-wearable sensor data with a high accuracy. In this work, we propose an event detection model using a Bayesian Network. We first obtain the features by analyzing the daily videos and audios captured from different angles by multiple IP cameras at a smart home. These features are then used to construct a Bayesian network. We propose a probabilistic graph model where the dependence relations are defined in the graph as opposed to the naive Bayesian network. The experiments are presented to demonstrate the performance and utility of our model.

In order to recognize the abnormal event, such as emergency or panic, happened in public scenes timely, an algorithm based on features extraction and BP Adaboost to detect abnormal frame event from surveillance video of complex situation is proposed. The proposed method detects an abnormal event where people are running, and this panic situation is simulated by the frame in a video. Experiments show that the method can distinguish and detect the abnormal event effectively and efficiently, which has potentiality to be used in the real public monitoring.

This paper presents a feature extraction approach for surveillance system aimed at achieving the automatic detection and recognition of public security events. The proposed approach first generates a Gabor dictionary based on the human auditory critical frequency bands, and then uses the orthogonal matching pursuit (OMP) algorithm to sparse abnormal audio signal. We select the optimal several important atoms from the Gabor dictionary and extract the scale, frequency, and translation parameters of the atoms to form the OMP feature. The performance of OMP feature is compared with traditional acoustic features and their joint features, using support vector machine (SVM) and random forest (RF) classifiers. Experiments have been performed to evaluate the effectiveness of the OMP feature for supplementing traditional acoustic features. The results show the superior performance classifier for abnormal acoustic event detection (AAED) is RF. Furthermore, the introduction of the combined features addresses the problems of low recognition accuracy and poor robustness for the surveillance system in practical applications.

Detecting abnormal events plays an essential role in video content analysis and has received increasing attention in surveillance system. One of the major problems in abnormal event detection is the imbalanced classification issue due to the rare abnormal samples. Another problem is the difficulty of detecting anomalies within a reasonable amount of computation time. To address these problems, we propose an adaptive cascade dictionary learning framework for detecting the anomalies. The framework considers anomaly detection as an one-class classification problem with a cascade of dictionaries. Each stage of the cascade constructs an adaptive dictionary to detect the anomalies with costless least square optimization solution. The experiments on benchmark datasets demonstrate that the proposed method has a better performance while comparing with several state-of-the-art methods.

The application of abnormal event detection in video surveillance is an active research field, but due to the imbalance of positive and negative samples in surveillance video, abnormal event detection is full of challenges. In this paper, we propose a new abnormal event detection method based on appearance repair and motion consistency for detecting anomalous events. Specifically, the input image is partially masked and then fed into our proposed appearance repair autoencoder for image reconstruction, and then the motion consistency of images is constructed by our proposed optical flow network. The experimental results on the UCSD, CUHK Avenue datasets show the superiority of the detection performance of our method.

Abnormal event detection aims to identify the events that deviate from expected normal patterns. Existing methods usually extract normal spatio-temporal patterns of appearance and motion in a separate manner, which ignores low-level correlations between appearance and motion patterns and may fall short of capturing fine-grained spatio-temporal patterns. In this paper, we propose to simultaneously learn appearance and motion to obtain fine-grained spatio-temporal patterns. To this end, we present an adversarial 3D convolutional auto-encoder to learn the normal spatio-temporal patterns and then identify abnormal events by diverging them from the learned normal patterns in videos. The encoder captures the low-level correlations between spatial and temporal dimensions of videos, and generates distinctive features representing visual spatio-temporal information. The decoder reconstrucccts the original video from the encoded features representing by 3D de-convolutions and learns the normal spatio-temporal patterns in an unsupervised manner. We introduce the denoising reconstruction error and adversarial learning strategy to train the 3D convolutional auto-encoder to implicitly learn accurate data distributions that are considered normal patterns, which benefits enhancing the reconstruction ability of the auto-encoder to discriminate abnormal events. Both the theoretical analysis and the extensive experiments on four publicly available datasets demonstrate the effectiveness of our method.