Integrated multiple behavior models for abnormal crowd behavior detection

Abnormal crowd behavior detection has recently attracted increasing attention due to its wide applications in computer vision research areas. However, it is still an extremely challenging task due to the great variability of abnormal behavior coupled with huge ambiguity and uncertainty of video contents. To tackle these challenges, we propose a new probabilistic framework named variational abnormal behavior detection (VABD), which can detect abnormal crowd behavior in video sequences. We make three major contributions: (1) We develop a new probabilistic latent variable model that combines the strengths of the U-Net and conditional variational auto-encoder, which also are the backbone of our model; (2) We propose a motion loss based on an optical flow network to impose the motion consistency of generated video frames and input video frames; (3) We embed a Wasserstein generative adversarial network at the end of the backbone network to enhance the framework performance. VABD can accurately discriminate abnormal video frames from video sequences. Experimental results on UCSD, CUHK Avenue, IITB-Corridor, and ShanghaiTech datasets show that VABD outperforms the state-of-the-art algorithms on abnormal crowd behavior detection. Without data augmentation, our VABD achieves 72.24% in terms of AUC on IITB-Corridor, which surpasses the state-of-the-art methods by nearly 5%.

As society develops, the demand for security is rapidly increasing. Accordingly, there is growing interest in research on methods to detect and prevent abnormal behavior using surveillance cameras in public places and private spaces such as shopping malls and airports for human safety. Many detection techniques based on deep learning models have been researched in the field of abnormal behavior detection. However, due to the lack of labeled abnormal behavior data, there are significant difficulties in developing an effective detection system. This paper surveys methods for deep learning methods to detect abnormal human behavior in surveillance video and presents recent techniques. First, I will introduce popular datasets that have often been used in previous research. After that, we categorized the existing methods for detecting abnormal behavior using deep learning into three types: supervised learning, unsupervised learning, and partially-supervised learning. We then explained the basic concepts and advantages of each method and summarized their shortcomings. We also briefly describe future research directions based on the advantages and disadvantages of each method. Based on this, it is expected that the technology of video surveillance systems that apply abnormal behavior detection will further develop.

Abnormal behavior detection using hybrid agents in crowded scenes

In this paper; we propose a novel crowd behavior representation method to detect abnormal behaviors in videos. An adaptive optical flow filtering method is proposed to utilize low-level optical flow informations. Furthermore, a simple framework is developed to detect and to localize abnormal crowd behavior using adaptive optical flow filtering result. The proposed method is more robust than other modeling methods in representing different behaviors. In this model, a normal behavior is presented by the general value. Some outliers in the temporal domain or spatial domain are presented by a higher value. Spatio-temporal cuboids are extracted from the filtering result to present the likelihood of anomaly in the frame. Experimental evaluations are performed on two public datasets with comparison to the provisos abnormal behavior detection methods in the literature. Experimental results show that the proposed methods outperform previous abnormal behavior detection techniques in the literature.

Abnormal behaviour detection is crucial given the reliance on video surveillance for public safety. However, most vision-based methods fail to adapt to the variability and complexity of the crowd scenes inherent in real-world scenarios. This study introduced an abnormal crowd behaviour detection algorithm based on lightweight Graphic Neural Networks (GNNs) using trajectory similarities and spatial-temporal attributes. Integrating Graph Attention Networks (GATs) to encode the graph structure, creating latent embeddings that capture both local and global relationships within crowds. Also introducing an autoencoder (AE) structure to identify anomalies by comparing true movement with the decoded output. Anomaly is detected when the reconstruction error is high. Experiments on UCSD Pedestrians and ShanghaiTech campus datasets show the successful use of trajectory similarities as graph features for GNNs to generate logical embeddings of crowd movement and highlight the impact of time windows on anomaly detection precision as reflected in F1-scores. The Area Under the Curve (AUC) metric shows stability, highlighting the model’s balanced performance. Qualitative analysis of the ShanghaiTech campus dataset illustrates the model’s ability to align detection with visual anomalies. Overall, this study demonstrates the effectiveness of the proposed graph-based anomaly detection framework for crowd monitoring through a lightweight model suitable for CCTV.

ABSTRACTThe detection of abnormal behaviours with fast and automatic recognising is significant in crowded areas to provide higher security to the public. The adoption of deep learning and machine learning‐based abnormal behaviour detection models enhances the influential detection and real‐time security monitoring in crowds. The researchers have remotely evaluated the heart rate based on physiological information to detect abnormal activities in various years. Over the past few years, several progress have been made, and there are still some issues concerning processing time, accuracy, and computational complexity. The developed approaches detects the activities of anomalies like traffic rule violations, riots, fighting, and stampede, in addition, several anomalous entities such as abandoned luggage and weapons at the sensitive place automatically in time. However, the identification of video anomalies methods poses several challenges because of various environmental conditions, the ambiguous nature of the anomaly, lack of proper datasets, and the complex nature of human characteristics. In recent days, there have been only a few devoted surveys associated with deep learning related video anomaly identification as the research domain is in its initial stages. In this review work, the abnormal behaviour analysis models using deep learning are reviewed in depth in for security applications. Based on the traditionally used abnormal behaviour analysis models in crowded scenes, we widely categorised the methods into classification using object tracking, classification using handcrafted extracted features, classification using non‐contact heart rate variability and blood pressure, analysing motion patterns from the visual frames, and classification using face images. We also discuss the comparative analysis of the previous methods with respect to datasets, computational infrastructure, and performance measures for both qualitative and quantitative analysis. In addition, the open and trending research challenges are analysed for future research.

This paper presents a novel video descriptor, referred to as Histogram of Oriented Track lets, for recognizing abnormal situation in crowded scenes. Unlike standard approaches that use optical flow, which estimates motion vectors only from two successive frames, we built our descriptor over long-range motion trajectories which is called track lets in the literature. Following the standard procedure, we divided video sequences in spatio-temporal cuboids within which we collected statistics on the track lets passing through them. In particular, we quantized orientation and magnitude in a 2-dimensional histogram which encodes the motion patterns expected in each cuboid. We classify frames as normal and abnormal by using Latent Dirichlet Allocation and Support Vector Machines. We evaluated the effectiveness of the proposed descriptors on three datasets: UCSD, Violence in Crowds and UMN. The experiments demonstrated (i) very promising results in abnormality detection, (ii) setting new state-of-the-art on two of them, and (iii) outperforming former descriptors based on the optical flow, dense trajectories and the social force model.

Recently, abnormal evens detection in crowds has received considerable attention in the field of public safety. Most existing studies do not account for the processing time and the continuity of abnormal behavior characteristics. In this paper, we present a new motion feature descriptor, called the sensitive movement point (SMP). Gaussian Mixture Model (GMM) is used for modeling the abnormal crowd behavior with full consideration of the characteristics of crowd abnormal behavior. First, we analyze the video with GMM, to extract sensitive movement point in certain speed by setting update threshold value of GMM. Then, analyze the sensitive movement point of video frame with temporal and spatial modeling. Identify abnormal behavior through the analysis of mutation duration occurs in temporal and spatial model, and the density, distribution and mutative acceleration of sensitive movement point in blocks. The algorithm can be implemented with automatic adapt to environmental change and online learning, without tracking individuals of crowd and large scale training in detection process. Experiments involving the UMN datasets and the videos taken by us show that the proposed algorithm can real-time effectively identify various types of anomalies and that the recognition results and processing time are better than existing algorithms.

The world population continues to grow. The size of gatherings at various venues under different circumstances is tremendously increasing. Any mass assembly has potential risk of the lethal crowd disaster. The various algorithms in computer vision techniques are being developed as a part of proactive system for crowd management. This paper presents novel approach to identify abnormal crowd behavior. The algorithm employedoptical flow method to estimate displacement vectors of moving crowd and computation of crowd motion energy. The crowd motion energy was further modified by crowd motion intensity (CMI). The peaks in the CMI characteristics were the indicators of abnormal activity and were detected more accurately by applying threshold. The algorithm has been tested on standard UMN dataset with an average accuracy of 91.66 percentages. The accuracy has been improved with application of threshold. Adaptive threshold may aid to further improve the performance of algorithm.

Amid rapid advancements in artificial intelligence, the detection of abnormal human behaviors in complex traffic environments has garnered significant attention. However, detection errors frequently occur due to interference from complex backgrounds, small targets, and other factors. Therefore, this paper proposes a research methodology that integrates the anomaly detection YOLO-SGCF algorithm with the tracking BoT-SORT-ReID algorithm. The detection module uses YOLOv8 as the baseline model, incorporating Swin Transformer to enhance global feature modeling capabilities in complex scenes. CBAM and CA attention are embedded into the Neck and backbone, respectively: CBAM enables dual-dimensional channel-spatial weighting, while CA precisely captures object location features by encoding coordinate information. The Neck layer incorporates GSConv convolutional modules to reduce computational load while expanding feature receptive fields. The loss function is replaced with Focal-EIoU to address sample imbalance issues and precisely optimize bounding box regression. For tracking, to enhance long-term tracking stability, ReID feature distances are incorporated during the BoT-SORT data association phase. This integrates behavioral category information from YOLO-SGCF, enabling the identification and tracking of abnormal pedestrian behaviors in complex environments. Evaluations on our self-built dataset (covering four abnormal behaviors: Climb, Fall, Fight, Phone) show mAP@50%, precision, and recall reaching 92.2%, 90.75%, and 86.57% respectively-improvements of 3.4%, 4.4%, and 6% over the original model-while maintaining an inference speed of 328.49 FPS. Additionally, generalization testing on the UCSD Ped1 dataset (covering six abnormal behaviors: Biker, Skater, Car, Wheelchair, Lawn, Runner) yielded an mAP score of 92.7%, representing a 1.5% improvement over the original model and outperforming existing mainstream models. Furthermore, the tracking algorithm achieved an MOTA of 90.8% and an MOTP of 92.6%, with a 47.6% reduction in IDS, demonstrating superior tracking performance compared to existing mainstream algorithms.

This paper introduced the concept, the classification and the control principal of network behaviors,discussed the concept, composition and structure, and the behavior control of the controlled network system, then proposed the detection and control model of abnormal behavior in controlled network system. This model of abnormal behavior detection was designed by combining misuse and intrusion detection mechanism, and is the specific application of the research on network behavior and controlled theory, and it controls the network behaviors especially abnormal behaviors. The theory is the basis for establishing the dynamic controllable network security system. With the development of network and information technologies, it seems that the network size, the diversity of network equipment and the complexity of network topology structure are constantlyincreasing, which leads to the probability of security problems emergeis also increasing. However, the traditional network security measures cannot meet the needs of the actual complex network activities, the existing information network has an urgent need to propose a new theory or new method which can reflect the overall network security and behavior control states. As a kind of efficient, scientific and reasonable network security theory, the controlled network is the integration of existing network security technologies and measures, and the aim of it is to construct a controlled network and solve the network security problems(1).

Monitoring videos captured by surveillance cameras is a very difficult and time consuming task. There is a need for automated analysis using computer vision methods in order to recognize abnormal human behaviors and assist authorities. On the other hand, crowd (group of people) behavior analysis is a new direction of research, which can be utilized for automatic detection of panic in crowds. Once, videos are processed using computer vision technologies, another problem is how this data is indexed for search and analysis, since cameras continuously capture videos resulting vast amounts of data. Oftentimes various techniques are used for indexing, storage and access of video surveillance information, which makes global analysis and search on this data very difficult. In this paper, we introduce a novel semantic metadata model based on multimedia standards to extract and annotate globally inter-operable data about abnormal crowd behaviors from surveillance videos. We demonstrate on UMN and PETS2009 datasets that generated semantic annotations enable detailed searching and analysis of abnormal crowd behaviors with complex queries.

Artificial features are adopted in most of the existing abnormal behavior detection. However, it is difficult to choose and design an effective behavior feature for the reason of highly computational complexity and complex scenarios. To solve this problem, temporal consistency based weak supervised abnormal behavior detection method is proposed in this paper. First, temporal gram matrices are constructed for a given video sequence. Then, a pair of behavior units (Candidate action fragment) are formed by exploiting the temporal consistency and the smoothness of human behavior, which helps to locate the start frame and end frame of the related class of abnormal behavior in the video sequence to train the corresponding classifier. Finally, sparse reconstruction is here utilized to detect abnormal behavior. Experiments conducted on common database including CAVIAR and Crossing demonstrate the effectiveness of the proposed method.

Problem of detection and localization of abnormal behaviors in crowded scenes is considered in this paper. To extract various activities of crowd, spatiotemporal Laplacian eigenmap method is proposed. Here, spatiotemporal variations of local motions are learned in an embedded space. Regular behavior of a crowd is modeled using representatives of various activities. To detect abnormal behavior in local as well as global contexts and to localize regions showing abnormality, model of regular crowd behavior is used. Computational simplicity is the feature of this method.

A novel automatic detection method for abnormal behavior of single fish using image fusion