Learning Multimodal Deep Representations for Crowd Anomaly Event Detection

Abnormal event detection has attracted great research attention in video surveillance. In this paper, the authors presented a robust method of trajectories clustering for abnormal event detection. This method is based on two layers and benefits from two well-known clustering algorithms: the agglomerative hierarchical clustering and the k-means clustering. Facing to the challenges related to the trajectories, e.g., different sizes, the authors introduce a preprocessing step to unify their sizes and reduce their dimensionality. The experimental results show the performance and accuracy of their proposed method.

The application of abnormal event detection in video surveillance is an active research field, but due to the imbalance of positive and negative samples in surveillance video, abnormal event detection is full of challenges. In this paper, we propose a new abnormal event detection method based on appearance repair and motion consistency for detecting anomalous events. Specifically, the input image is partially masked and then fed into our proposed appearance repair autoencoder for image reconstruction, and then the motion consistency of images is constructed by our proposed optical flow network. The experimental results on the UCSD, CUHK Avenue datasets show the superiority of the detection performance of our method.

Event detection in crowded surveillance videos is a challenging yet important problem. In this paper, we present our eSur (Event detection system on SURveillance video) system, which is derived from TRECVid'12 surveillance tasks. Currently, eSur attempts to detect two categories of events: 1) pair-wise events (e.g., PeopleMeet, PeopleSplitUp and Embrace); 2) action-like events (e.g., ObjectPut, CellToEar, PersonRuns and Pointing). In eSur system, we first employ people detection and tracking algorithms to locate target persons in 3D space-time domain. Then the video sequences in which target persons occur are partitioned into several spatio-temporal cubes. Visual features (i.e. cubic feature and MoSIFT) are computed over these cubes. After that, a sequence learning method, (namely SVM with dynamic time alignment kernel), is employed to infer the existence of an event for the video sequence. According to the TRECVid SED formal evaluation, eSur has yielded fairly encouraging results on TRECVid'12 dataset.

In surveillance environments, detecting anomalies requires understanding the contextual dynamics of the environment, human behaviors, and movements within a scene. Effective anomaly detection must address both the where and what of events, but existing approaches such as unimodal action-based methods or LLM-integrated multimodal frameworks have limitations. These methods either rely on implicit scene information, making it difficult to localize where anomalies occur, or fail to adapt to surveillance specific challenges such as view changes, subtle actions, low light conditions, and crowded scenes. As a result, these challenges hinder accurate detection of what occurs. To overcome these limitations, our system takes advantage of features from a lightweight scene classification model to discern where an event occurs, acquiring explicit location-based context. To identify what events occur, it focuses on atomic actions, which remain underexplored in this field and are better suited to interpreting intricate abnormal behaviors than conventional abstract action features. To achieve robust anomaly detection, the proposed Temporal-Semantic Relationship Network (TSRN) models spatio-temporal relationships among multimodal features and employs a Segment-selective Focal Margin loss (SFML) to effectively address class imbalance, outperforming conventional MIL-based methods. Experimental results on public datasets demonstrate that the proposed system effectively reduces false alarms while maintaining robustness and practicality for real-world surveillance applications.

In this paper, a novel approach for automatic anomaly detection in surveillance video is proposed. It is highly efficient for real-time detection. It can also handle multi-scale detection and can cope with both spatial and temporal anomalies. Specifically, features capturing both appearance and motion characteristic are extracted from densely sampled spatio-temporal video volume (STV). And to bridge the semantic gap between low-level visual feature and high-level event, we use the middle-level visual attributes as the intermediary. These three-level framework is modeled as an Extreme Learning Machine (ELM) which can effectively and efficiently tell whether a STV belongs to an anomalous event. We also use the Spatio-temporal Pyramid (STP) to capture the spatial and temporal continuity of an anomalous event , enabling our approach to cope with multi-scale and complicated events. Experiments on several datasets are carried out and the superior performance compared to state-of-the-art approaches verifies the effectiveness of our approach.KeywordsAutomatic anomaly detectionVisual attributesExtreme Learning MachineSpatio-temporal Pyrimid

In this paper, a contextual anomaly event detection method is presented using a new clips boundaries detection approach and Bayesian classifier. Fall event is considered as anomaly event in our experiments and reported as case study. The anomaly score at frame levels is obtained. The proposed method involves three main phases; preprocessing for video content preparing, clips boundaries detection for anomaly behavior classification and fall event detection. The anomaly behavior - based fall event detection is classified into three main types; sudden change, gradual change and normal change within video sequence. To this end, a Bayesian classifier is trained to predict the anomaly score of video clips using similarity score prediction and acceleration raw data of sensors. We state quantitative results for clips boundaries detection, anomaly score prediction, and fall event detection rate. Further, the performance of the proposed anomaly event detection is evaluated based on results of common performance metrics (precision, sensitivity, specificity and accuracy) on public fall event datasets. The performance evaluation demonstrates a superiority of fall detection rate compared with recent researches in term of frame-level.

In this paper, global abnormal events detection in the crowded scene is proposed. The proposed algorithm is based on a hierarchical approach. The method uses the histogram of optical flow orientation as a feature descriptor along with the magnitude of the optical flow to capture the motion. The nonlinear one-class support vector machine (SVM) classification algorithm is used to learn the normal events from the training data. After learning, one-class SVM detects abnormal events in the frame. The algorithm is fast with improved accuracy since it omits the background subtraction step and adopts hierarchy. The proposed method is tested on benchmark UMN unusual crowd activity dataset which contain global abnormal events. Promising results are reported which evaluates the performance of the proposed method.

Crowd Event Detection in Surveillance Video - An Approach based on Optical Flow High-frequency Feature Analysis

Several computer vision algorithms have been proposed to detect anomalous activities (robberies, murders, vandalism, among others) in videos. According to the learning approach, they can be classified into probabilistic distribution modeling, sparse coding, and deep learning based methods. The main drawbacks of these approaches are (i) extraction of low-level features that do not capture complex behaviors of instances on the scene, (ii) generation of features from irrelevant regions, (iii) overlooking of relationships among objects, and (iv) omission of long-term dependencies. To solve these issues, we propose a deep learning architecture that leverages the relationships among objects. It achieves this by using an attention mechanism and learning long-term dependencies using a multilayer recurrent neural network (multilayer LSTM). An AUC score of 0.749 on the UCF-Crime dataset confirms that the proposed algorithm competes effectively against several state-of-the-art approaches for anomaly detection in surveillance videos. It also explains the relationship between regions in the video frames and the anomaly detections.

Fast fire flame detection in surveillance video using logistic regression and temporal smoothing

For abnormal gait detection in surveillance videos, the existing methods suffer from that they are unable to recognize novel types of anomalies if the corresponding prototypes have not been included in the training data for supervised machine learning but it is impractical to foresee all types of anomalies. This research aims to solve the problem in an unsupervised manner, which does not rely on any prior knowledge regarding abnormal prototypes and avoids time-consuming machine learning over large-scale high-dimensional features. The intuition is that normal gait is nearly periodic signal and anomalies may disturb such periodicity. Hence, the time-varying ratio of width to height of a walking person is transformed to frequency domain using Fast Fourier Transform (FFT), and the standard deviation over spectrum is used as an indicator of anomalies, subject to any sudden change to break the normally periodical walking rhythm. The experimental results demonstrate its precision.

This paper presents a novel hybrid approach for real-time anomaly detection in video surveillance systems by integrating YOLOv8 object detection with advanced motion-based analysis techniques. The proposed system addresses critical limitations of existing single-modality detection methods through innovative fusion of deep learning and temporal analysis. The architecture incorporates parallel processing pipelines for YOLOv8 detection and optical flow computation, combined with an isolation forest-based anomaly decision framework that leverages historical detection patterns. Experimental evalution on a custom dataset of 5,000 surveillance video clips demonstrates superior performance with 92.3% accuracy, 89.7% precision, 94.1% recall, and 91.8% F1-score, while maintaining real-time processing at 30 FPS. The system significantly outperforms traditional approaches with 15.2% accuracy improvement over YOLO-only methods and 18.7% improvement over motion-only techniques. The proposed hybrid framework provides robust anomaly detection capabilities suitable for practical deployment in security-critical surveillance applications with reduced false positive rates and enhanced temporat consistence.

iSurveillance: Intelligent framework for multiple events detection in surveillance videos

Exploring deep learning-based content-based video retrieval with Hierarchical Navigable Small World index and ResNet-50 features for anomaly detection

This research presents a method for human detection at night in video surveillance camera. The process of detecting human at night is very challenging due to certain factors such as radiance, silhouette and low external light. A comparative study between three texture features that are Discrete Wavelet Transform (DWT), Histogram of Oriented Gradient (HOG) and Speeded Up Robust Feature (SURF) using Support Vector Machine (SVM), Naive Bayes and Adaboost classifiers are investigated using primary data extracted from a video surveillance camera at the faculty. The results show that HOG feature with Naive Bayes detect human in video surveillance better compared to DWT and SURF with SVM and AdaBoost classifiers.