Abnormal Crowd Behavior Detection Using Topological Methods

A novel framework for anomaly detection in crowded scenes is presented. Three properties are identified as important for the design of a localized video representation suitable for anomaly detection in such scenes: (1) joint modeling of appearance and dynamics of the scene, and the abilities to detect (2) temporal, and (3) spatial abnormalities. The model for normal crowd behavior is based on mixtures of dynamic textures and outliers under this model are labeled as anomalies. Temporal anomalies are equated to events of low-probability, while spatial anomalies are handled using discriminant saliency. An experimental evaluation is conducted with a new dataset of crowded scenes, composed of 100 video sequences and five well defined abnormality categories. The proposed representation is shown to outperform various state of the art anomaly detection techniques.

We present a new method for detection of abnormal behaviors in crowded scenes. Based on statistics of low-level feature—optical flow, which describes human movement efficiently, the motion energy model is proposed to represent the local motion pattern in the crowd. The model stresses the difference between normal and abnormal behaviors by considering sum of square differences (SSD) metric of motion information in the center block and its neighboring blocks. Meanwhile, data increasing rate is introduced to filter outliers to achieve boundary values between abnormal and normal motion patterns. In this model, an abnormal behavior is detected if the occurrence probability of anomaly is higher than a preset threshold, namely the motion energy value of its corresponding block is higher than that of the normal one. We evaluate the proposed method on two public available datasets, showing competitive performance with respect to state-of-the-art approaches not only in detection accuracy, but also in computational efficiency.

Real-time video event detection in crowded scenes using MPEG derived features: A multiple instance learning approach

Modelling events in densely crowded environments remains challenging, due to the diversity of events and the noise in the scene. We propose a novel approach for anomalous event detection in crowded scenes using dynamic textures described by the Local Binary Patterns from Three Orthogonal Planes (LBP-TOP) descriptor. The scene is divided into spatio-temporal patches where LBP-TOP based dynamic textures are extracted. We apply hierarchical Bayesian models to detect the patches containing unusual events. Our method is an unsupervised approach, and it does not rely on object tracking or background subtraction. We show that our approach outperforms existing state of the art algorithms for anomalous event detection in UCSD dataset.

SummaryObject detection in crowded scenes involves various difficulties, such as small objects, occluded objects, and insufficient features. Existing models for crowded object detection often focus on only one detection difficulty, and they are too large to be applied in practice. To address the diverse challenges of object detection in crowded scenes, we construct a lightweight crowded object detector called MBB‐YOLO, which contains several modules for comprehensive improvement. To improve the network's ability to extract fine‐grained features, we use SPD‐Conv and the proposed MS‐Conv to replace the strided convolution in the network. An bi‐branch multi‐scale convolution attention (BMCA) module is proposed to aggregate multi‐scale contextual information. We also propose boundary‐NMS to better identify proposal boxes from different objects, which reduces suppression errors caused by object occlusion. MBB‐YOLO achieves 87.6% AP and an inference speed of 78.8 FPS on the CrowdHuman dataset, which surpasses other mainstream lightweight object detectors.

Abnormality detection in crowded scenes plays a very important role in automatic monitoring of surveillance feeds. Here we present a novel framework for abnormality detection in crowd videos. The key idea of the approach is that rarely or sparsely occurring events correspond to abnormal activities while the commonly occurring events correspond to the normal activities. Given an input video, multiple feature matrices are computed which are decomposed into their low-rank and sparse components, out of which the sparse components correspond to the abnormal activities. The approach does not require any explicit modeling of crowd behavior or training. Localization of the anomalies is obtained as a by-product of the proposed approach by doing an inverse mapping between the entries of the matrix and the pixels in the video frames. The method is very general and can be applied for both sparsely crowded as well as densely crowded scenes and it can be used to detect both global and local abnormalities. Experimental evaluation on two widely used datasets as well as some dense crowd videos downloaded from the web shows the effectiveness of the proposed approach. Comparison with several state-of-the-art crowd abnormality detection approaches show that the proposed method compares well as compared to the other approaches.

This paper presents a new inlier-based outlier detection scheme for the analysis of abnormal behavior in crowded scenes. First, we segment the video into a set of cubes, and then extract the three feature descriptors from each cube, including the histogram of oriented gradients (HOG), the histogram of motion directions, and the motion magnitude descriptors. Thereafter, for each feature descriptor the Kullback-Leibler importance estimation procedure (KLIEP) is employed to compute the ratio of test and training densities, referred to as the importance, instead of computing these two densities separately. This allows us to avoid the difficulty in estimating complex probability density estimation. The importance denotes an inlier score which represents the degree of similarity between the test and training data. Based on the importance of each descriptor and a prespecified threshold in each cube, we can then identify if a cube contains an anomaly event. Through computer simulations, we find that the new approach provides high accuracy of localization rate based on the widespread UCSD datasets compared with previous works.

Pedestrian detection in crowded scenes is widely used in computer vision. However, it still has two difficulties: 1) eliminating repeated predictions (multiple predictions corresponding to the same object); 2) false detection and missing detection due to the high scene occlusion rate and the small visible area of detected pedestrians. This paper presents a detection framework based on DETR (detection transformer) to address the above problems, and the model is called AD-DETR (asymmetrical relation detection transformer). We find that the symmetry in a DETR framework causes synchronous prediction updates and duplicate predictions. Therefore, we propose an asymmetric relationship fusion mechanism and let each query asymmetrically fuse the relative relationships of surrounding predictions to learn to eliminate duplicate predictions. Then, we propose a decoupled cross-attention head that allows the model to learn to restrict the range of attention to focus more on visible regions and regions that contribute more to confidence. The method can reduce the noise information introduced by the occluded objects to reduce the false detection rate. Meanwhile, in our proposed asymmetric relations module, we establish a way to encode the relative relation between sets of attention points and improve the baseline. Without additional annotations, combined with the deformable-DETR with Res50 as the backbone, our method can achieve an average precision of 92.6%, MR$ ^{-2} $ of 40.0% and Jaccard index of 84.4% on the challenging CrowdHuman dataset. Our method exceeds previous methods, such as Iter-E2EDet (progressive end-to-end object detection), MIP (one proposal, multiple predictions), etc. Experiments show that our method can significantly improve the performance of the query-based model for crowded scenes, and it is highly robust for the crowded scene.

Recent Transformer-based object detectors have achieved remarkable performance on benchmark datasets, but few have addressed the real-world challenge of object detection in crowded scenes using transformers. This limitation stems from the fixed query set size of the transformer decoder, which restricts the model’s inference capacity. To overcome this challenge, we propose Recurrent Detection Transformer (Recurrent DETR), an object detector that iterates the decoder block to render more predictions with a finite number of query tokens. Recurrent DETR can adaptively control the number of decoder block iterations based on the image’s crowdedness or complexity, resulting in a variable-size prediction set. This is enabled by our novel Pondering Hungarian Loss, which helps the model to learn when additional computation is required to identify all the objects in a crowded scene. We demonstrate the effectiveness of Recurrent DETR on two datasets: COCO 2017, which represents a standard setting, and CrowdHuman, which features a crowded setting. Our experiments on both datasets show that Recurrent DETR achieves significant performance gains of 0.8 AP and 0.4 AP, respectively, over its base architectures. Moreover, we conduct comprehensive analyses under different query set size constraints to provide a thorough evaluation of our proposed method.

Different approaches have been proposed in the literature for anomaly detection in image/video area. Traditional methods such as trajectory or spatio-temporal based techniques rely on hand-crafted features. The occlusion problems and high complexity in crowded scenes are the vital drawbacks behind using these methodologies. Deep learning structures proved recently to be useful for defining effective solutions for anomaly detection where the high level features are learnt and selected automatically. However, the block-wise methods such as CNNs are computationally slow. On the other hand, they are totally supervised learning methodologies, while the video-based anomaly detection is an unsupervised problem. Auto- Encoders (Convolutional AE, vibrational AE, etc.) can be considered as an alternative option. This paper presents a stateof- the-art deep learning algorithm to be applied in such an unsupervised problem. Using the basic concepts behind the Auto-Encoders as a well-known unsupervised learning algorithm, we propose a novel methodology to detect and localize the anomalies in a video scene. The presented network is trained based on the normal patterns during training phase. The proposed structure enables the system to capture the 2D structure in image sequences during the learning process. The working hypothesis is that a deep network is able to learn normal events in videos, and, therefore, the difference of normal and anomalous frames can be used for devising an anomaly score. The simulation results on well-known data sets such as UCSD confirm that the proposed methodology achieves high performance results in case of accuracy and total processing time compared with counterpart approaches.

We propose a simple yet effective proposal-based object detector, aiming at detecting highly-overlapped instances in crowded scenes. The key of our approach is to let each proposal predict a set of correlated instances rather than a single one in previous proposal-based frameworks. Equipped with new techniques such as EMD Loss and Set NMS, our detector can effectively handle the difficulty of detecting highly overlapped objects. On a FPN-Res50 baseline, our detector can obtain 4.9\% AP gains on challenging CrowdHuman dataset and 1.0\% $\text{MR}^{-2}$ improvements on CityPersons dataset, without bells and whistles. Moreover, on less crowed datasets like COCO, our approach can still achieve moderate improvement, suggesting the proposed method is robust to crowdedness.

A crowd is a gathering of a huge number of individuals in a confined area. Early identification and detection of unusual behaviors in terms of panic occurring in crowded scenes are very important. Panic detection comprises of formulating normal scene behaviors and detecting and identifying non-matching behaviors. However, panic detection and recognition is a very difficult problem, especially when considering diverse scenes. Many methods proposed to cope with these problems have limited robustness as the density of the crowd varies. In order to handle this challenge, this paper proposes the integration of different features into a unified model. Discriminant binary patterns and neighborhood information are used to model complex and unique motion patterns in order to characterize different levels of features for diverse types of crowd scenes, focusing in particular on the detection of panic and non-pedestrian entities. The proposed method was evaluated considering two benchmark datasets and outperformed five existing methods.

In this paper, we propose a novel method for global abnormal events detection in crowded scenes. Each video is described as the set of overlapping space-time cubes. The histogram of optical flow orientation and motion magnitude are used as global feature descriptor to capture the motion magnitude and orientation of the normal and abnormal events. The motion-rich space-time cubes are selected to enhance the computational speed of the algorithm and omit background subtraction step, forms the proposed approach novel and unique. Simple and cost-effective one-class SVM classifier is used to distinguish between normal and abnormal events in a video. The one-class SVM classifier is trained with normal events from the training data. The proposed method is tested on benchmark UMN and PETS2009 dataset. The improved results are reported to evaluate the performance of the proposed method.

In this paper, we propose a fast and robust framework for anomaly detection in crowed scenes. In our method, anomaly is adaptively modeled as a deviation from the normal behavior of crowd observed in the scene. For this purpose, we extract motion features by repeatedly initializing a grid of particles over a temporal window. These features are exploited in a real-time anomaly detection system. In order to model the ordinary behavior of the people moving in the crowd, we use the Gaussian mixture model (GMM) technique, which is robust enough to capture the scene dynamics. As opposed to explicitly modeling the values of all the pixels as a mixture of Gaussians, we adopted the GMM to learn the behavior of the motion features extracted from the particles. Based on the persistence and the variance of each Gaussian distribution, we determine which Gaussians can be associated to the normal behavior of the crowd. Particles with motion features that do not fit the distributions representing normal behavior are signaled as anomaly, until there is a Gaussian able to include them with sufficient evidence supporting it. Experiments are extensively conducted on publically available benchmark dataset, and also on a challenging dataset of video sequences we captured. The experimental results revealed that the proposed method performs effectively for anomaly detection.

The complexity of human detection increases significantly with a growing density of humans populating a scene. This paper presents a Bayesian detection framework using shape and motion cues to obtain a maximum a posteriori (MAP) solution for human configurations consisting of many, possibly occluded pedestrians viewed by a stationary camera. The paper contains two novel contributions for the human detection task: 1. computationally efficient detection based on shape templates using contour integration by means of integral images which are built by oriented string scans; (2) a non-parametric approach using an approximated version of the shape context descriptor which generates informative object parts and infers the presence of humans despite occlusions. The outputs of the two detectors are used to generate a spatial configuration of hypothesized human body locations. The configuration is iteratively optimized while taking into account the depth ordering and occlusion status of the hypotheses. The method achieves fast computation times even in complex scenarios with a high density of people. Its validity is demonstrated on a substantial amount of image data using the CAVIAR and our own datasets. Evaluation results and comparison with state of the art are presented.