Anomaly Detection in Crowd Scene via Online Learning

Anomaly event detection in crowd scenes is extremely important; however, the majority of existing studies merely use hand-crafted features to detect anomalies. In this study, a novel unsupervised deep learning framework is proposed to detect anomaly events in crowded scenes. Specifically, low-level visual features, energy features, and motion map features are simultaneously extracted based on spatiotemporal energy measurements. Three convolutional restricted Boltzmann machines are trained to model the mid-level feature representation of normal patterns. Then a multimodal fusion scheme is utilized to learn the deep representation of crowd patterns. Based on the learned deep representation, a one-class support vector machine model is used to detect anomaly events. The proposed method is evaluated using two available public datasets and compared with state-of-the-art methods. The experimental results show its competitive performance for anomaly event detection in video surveillance.

Video anomaly event detection is the process of finding an abnormal event deviation compared with the majority of normal or usual events. The main challenges are the high structure redundancy and the dynamic changes in the scenes that are in surveillance videos. To address these problems, we present a framework for anomaly detection and localization in videos that is based on locality-constrained affine subspace coding (LASC) and a model updating procedure. In our algorithm, LASC attempts to reconstruct the test sample by its top-k nearest subspaces, which are obtained by segmenting the normal samples space using a clustering method. A sample with a large reconstruction cost is detected as abnormal by setting a threshold. To adapt to the scene changes over time, a model updating strategy is proposed. We experiment on two public datasets: the UCSD dataset and the Avenue dataset. The results demonstrate that our method achieves competitive performance at a 700 fps on a single desktop PC.

In this paper, we propose a new approach for recognizing group events and abnormality detection in a crowded scene. A manifold learning algorithm with temporal-constraints is proposed to embed a video of a crowded scene in a low-dimensional space. Our low dimensional representation of a video preserves the spatial temporal property of a video as well as the characteristic of the video. Recognizing video events and abnormality detection in a crowded scene is achieved by studying the video trajectory in the manifold space. We evaluate our proposed method on the state-of-the-art public data-sets containing different crowd events. Qualitative and quantitative results show the promising performance of the proposed method.

We present a new method for detection of abnormal behaviors in crowded scenes. Based on statistics of low-level feature—optical flow, which describes human movement efficiently, the motion energy model is proposed to represent the local motion pattern in the crowd. The model stresses the difference between normal and abnormal behaviors by considering sum of square differences (SSD) metric of motion information in the center block and its neighboring blocks. Meanwhile, data increasing rate is introduced to filter outliers to achieve boundary values between abnormal and normal motion patterns. In this model, an abnormal behavior is detected if the occurrence probability of anomaly is higher than a preset threshold, namely the motion energy value of its corresponding block is higher than that of the normal one. We evaluate the proposed method on two public available datasets, showing competitive performance with respect to state-of-the-art approaches not only in detection accuracy, but also in computational efficiency.

Abnormal behavior detection using hybrid agents in crowded scenes

In surveillance environments, detecting anomalies requires understanding the contextual dynamics of the environment, human behaviors, and movements within a scene. Effective anomaly detection must address both the where and what of events, but existing approaches such as unimodal action-based methods or LLM-integrated multimodal frameworks have limitations. These methods either rely on implicit scene information, making it difficult to localize where anomalies occur, or fail to adapt to surveillance specific challenges such as view changes, subtle actions, low light conditions, and crowded scenes. As a result, these challenges hinder accurate detection of what occurs. To overcome these limitations, our system takes advantage of features from a lightweight scene classification model to discern where an event occurs, acquiring explicit location-based context. To identify what events occur, it focuses on atomic actions, which remain underexplored in this field and are better suited to interpreting intricate abnormal behaviors than conventional abstract action features. To achieve robust anomaly detection, the proposed Temporal-Semantic Relationship Network (TSRN) models spatio-temporal relationships among multimodal features and employs a Segment-selective Focal Margin loss (SFML) to effectively address class imbalance, outperforming conventional MIL-based methods. Experimental results on public datasets demonstrate that the proposed system effectively reduces false alarms while maintaining robustness and practicality for real-world surveillance applications.

These days consciousness about public safety increases and CCTV cameras are installed at almost all public places. But generally automatic smart surveillance systems are not available. In this manuscript, emphasis is given to detect and classify abnormal events in surveillance video especially in crowd environments. Abnormal event detection is a challenging task because the definition of abnormality is subjective. A normal event in one situation can be considered an abnormal event in another case. In the surveillance video with a dense crowd, automatic anomaly detection becomes very difficult because of clutter and severe occlusion. This manuscript represents CNN (Convolutional Neural Network) and CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) based approaches for detection and classification of abnormal events. The CNN architecture is developed from scratch and can be used for spatial domains. LSTM architecture is developed for the temporal domain. Feature sequences are generated using CNN model and given as input to LSTM model. Experiments are carried out using five different publicly available benchmark datasets. The performance is measured by accuracy and area under the ROC (receiver operating characteristic) curve (AUC). CNN-LSTM approach works better than only CNN.

In this paper, we target on the problem of estimating the statistic of pedestrian travel time within a period from an entrance to a destination in a crowded scene. Such estimation is based on the global distributions of crowd densities and velocities instead of complete trajectories of pedestrians, which cannot be obtained in crowded scenes. The proposed method is motivated by our statistical investigation into the correlations between travel time and global properties of crowded scenes. Active regions are created for each source-destination pair to model the probable walking regions over the corresponding source-destination traffic flow. Two sets of scene features are specially designed for modeling moving and stationary persons inside the active regions and their influences on pedestrian travel time. The estimation of pedestrian travel time provides valuable information for both crowd scene understanding and pedestrian behavior analysis, but was not sufficiently studied in literature. The effectiveness of the proposed pedestrian travel time estimation model is demonstrated through several surveillance applications, including dynamic scene monitoring, localization of regions blocking traffics, and detection of abnormal pedestrian behaviors. Many more valuable applications based on our method are to be explored in the future.

Crowd scene analysis is the certification tasks in crowded scene understanding. Crowd is a same or different set of people arranged in one group. Generally crowd form in the way of pedestrians, supermarket, and marathons. In this paper introduce Convolution Neural Networks and deep learning model is used for the analysis of crowd scene. In these paper propose, findings the number of people arrived in one group and also finds the crowd density map. People counting in extremely dense crowds are an important step for video surveillance and anomaly warning. The above mentioned works, Several problems becomes especially more challenging due to the lack of training samples, severe blockages, disorder scenes, and modification of perspective. In existing methods estimating crowd count using handcrafted features such as SIFTS and HOG. In current vision most suited method is to predict the better performance of estimating crowd density and crowd count based on deep learning network. Lucas kanade optical flow can finds the displacement vector between two consecutive frames. 3D volumes video slices can be arranged in sequential manner. In this crowd scene analysis represents convolutional crowd dataset as 100 videos from 800 crowd scenes and build an attribute set with 94 attributes.

Most multi-object tracking methods have achieved good results in tracking multiple pedestrians with Kalman filter, but their tracking performance in crowded scenes is still poor due to pedestrian avoidance and frequent occlusion. In crowded scenes, the pedestrian trajectory prediction with Kalman filter alone is unreliable. In this paper, a two-dimensional field-of-view avoidance force model (AFM) is proposed to assist the Kalman filter prediction by sensing the avoidance force and then complete pedestrian tracking. In the model, each pedestrian has a two-dimensional field of view to perceive the avoidance force, which determines the next predicted trajectory. In real scenes, pedestrians tend to be more concerned about the surrounding area, so different areas are set to simulate the attention mechanisms of pedestrians in real scenes. In the FairMOT model, AFM is used to optimize the pedestrian state values of Kalman filter prediction and the optimized model is trained on the MOT16 public dataset. The experimental results on the MOT20 benchmark dataset show that compared with the mainstream tracking model FairMOT, our method respectively improves MOTA by 2.7% and IDF1 by 2.2%. Our method also achieves the good performance on MOT15, MOT16, and MOT17 tracking benchmarks.

Detecting static objects in video sequences has a high relevance in many surveillance scenarios like airports and railwaystations. In this paper we propose a system for the detection of static objects in crowded scenes that, based on the detection of two background models learning at different rates, classifies pixels with the help of a finite-state machine. The background is modelled by two mixtures of Gaussians with identical parameters except for the learning rate. The state machine provides the meaning for the interpretation of the results obtained from background subtraction and can be used to incorporate additional information cues, obtaining thus a flexible system specially suitable for real-life applications. The system was built in our surveillance application and successfully validated with several public datasets.

We present an approach for online learning of discriminative appearance models for robust multi-target tracking in a crowded scene from a single camera. Although much progress has been made in developing methods for optimal data association, there has been comparatively less work on the appearance models, which are key elements for good performance. Many previous methods either use simple features such as color histograms, or focus on the discriminability between a target and the background which does not resolve ambiguities between the different targets. We propose an algorithm for learning a discriminative appearance model for different targets. Training samples are collected online from tracklets within a time sliding window based on some spatial-temporal constraints; this allows the models to adapt to target instances. Learning uses an Ad-aBoost algorithm that combines effective image descriptors and their corresponding similarity measurements. We term the learned models as OLDAMs. Our evaluations indicate that OLDAMs have significantly higher discrimination between different targets than conventional holistic color histograms, and when integrated into a hierarchical association framework, they help improve the tracking accuracy, particularly reducing the false alarms and identity switches.

The most critical objective in security surveillance is abnormal event detection in public scenarios. A scheme is presented for detecting abnormal behaviours in the activities of human groups based on social behaviour analysis. This approach efficiently models group activities than some of the previous strategies that use independent local features. This paper presents a feature descriptor method to signify the movement by implementing the optical flow through covariance matrix coding. The multi‐RoI (region of interest) covariance matrix has some frames or patches which could represent the movement in high accuracy. Normal samples are plentiful in public surveillance videos, while there are only a few abnormal samples. For that, the model of a hybridised optical flow covariance matrix is represented in this paper. Optical flow (OF) in the temporal domain is measured as a critical feature of video streams. The logistic regression method is used to detect abnormal activities in a crowded scene. Finally, the behaviours of human crowds can be predicted using benchmark datasets such as UMN, UCSD as well as BEHAVE. The obtained experimental results show that the proposed approach can effectively detect abnormal events from the abandoned environment of surveillance videos.

This paper presents a principled way for dealing with occlusions in visual tracking which is a long-standing issue in computer vision but largely remains unsolved. As the major innovation, we develop a learning-based jump-diffusion process to jointly track object locations and estimate their visibility statuses over time. Our method employs in particular a set of jump dynamics to change object's visibility statuses and a set of diffusion dynamics to track objects in videos. Different from the traditional jump-diffusion process that stochastically generates dynamics, we utilize deep policy functions to determine the best dynamic at the present step and learn the optimal policies from raw videos using reinforcement learning methods.Our method is capable of tracking objects with severe occlusions in crowded scenes and thus recovers the complete trajectories of objects that undergo multiple interactions with others. We evaluate the proposed method on challenging video sequences and compare it to alternative methods. Significant improvements are obtained particularly for the videos including frequent interactions or occlusions.

Anomaly detection technology in crowd scene is very important in public place. Crowd detection differs from pedestrian detection which we assume no individual pedestrian can be properly segmented in the image. We propose a scheme which the scen can be treated the crowd motion patterns as the spatial-temporal domain. In the classification stage, we divide whole frame into small blocks, and motion pattern in each block is encoded by the distribution of motion bags in it. PLSA classifier is proposed to infer classification of crowed detection, and we classify motion pattern into normal or abnormal group according to the deviation between motion pattern and train model. The comprehensive implementation can detect crowd in real-time. This paper presents an approach to automatically detect abnormal behavior in crowd scene with Interest points to represent moving objects to generate word of bags, which are used to describe crowed moriment results show that the speed of detection has been greatly improved using our approach.