Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos

Intelligent video surveillance systems with anomaly detection capabilities are indispensable for outdoor security. Video anomaly detection (VAD) is usually performed by learning patterns representing normal events and declaring an anomaly when an abnormal pattern is encountered. However, the features of normal patterns in a video often vary with time as real-world videos are non-stationary in nature, which makes its handling essential during VAD. To this end, we propose an approach for anomaly detection in videos, where a novel deep temporally recursive differencing network (DDN) diminishes the adverse effects of the non-stationary nature on VAD. The DDN consists of multiple layers of differencing operators of optimized orders, where every two consecutive layers are separated by a suitable nonlinearity. Spatial and temporal features are extracted from nonoverlapping blocks in video frames and fed to the DDN. While the spatial feature is obtained using a pretrained network, our temporal feature computation involves the use of FlowNetS with a new training strategy that does not require ground truth. The features at the output of DDN are used in a predictor based on autoregression and moving average of the regression errors. Then, the predictor's output estimates are compared to the corresponding actual values for anomaly detection, which also involves block-level selection and consistency check. Qualitative evaluation and quantitative comparison with several existing approaches on multiple standard datasets demonstrate the effectiveness of the proposed VAD approach. An ablation study highlighting the significance of the various components of our approach and a hyperparameter analysis are also provided.

Detecting abnormal activities in real-world surveillance videos is an important yet challenging task as the prior knowledge about video anomalies is usually limited or unavailable. Despite that many approaches have been developed to resolve this problem, few of them can capture the normal spatio-temporal patterns effectively and efficiently. Moreover, existing works seldom explicitly consider the local consistency at frame level and global coherence of temporal dynamics in video sequences. To this end, we propose Convolutional Transformer based Dual Discriminator Generative Adversarial Networks (CT-D2GAN) to perform unsupervised video anomaly detection. Specifically, we first present a convolutional transformer to perform future frame prediction. It contains three key components, i.e., a convolutional encoder to capture the spatial information of the input video clips, a temporal self-attention module to encode the temporal dynamics, and a convolutional decoder to integrate spatio-temporal features and predict the future frame. Next, a dual discriminator based adversarial training procedure, which jointly considers an image discriminator that can maintain the local consistency at frame-level and a video discriminator that can enforce the global coherence of temporal dynamics, is employed to enhance the future frame prediction. Finally, the prediction error is used to identify abnormal video frames. Thoroughly empirical studies on three public video anomaly detection datasets, i.e., UCSD Ped2, CUHK Avenue, and Shanghai Tech Campus, demonstrate the effectiveness of the proposed adversarial spatio-temporal modeling framework.

Anomaly detection in surveillance videos is a challenging task due to the diversity of anomalous video content and duration. In this paper, we consider video anomaly detection as a regression problem with respect to anomaly scores of video clips under weak supervision. Hence, we propose an anomaly detection framework, called Anomaly Regression Net (ARNet), which only requires video-level labels in training stage. Further, to learn discriminative features for anomaly detection, we design a dynamic multiple-instance learning loss and a center loss for the proposed AR-Net. The former is used to enlarge the inter-class distance between anomalous and normal instances, while the latter is proposed to reduce the intra-class distance of normal instances. Comprehensive experiments are performed on a challenging benchmark: ShanghaiTech. Our method yields a new state-of-the-art result for video anomaly detection on ShanghaiTech dataset.

Anomaly detection plays a critical role in intelligent video surveillance. However, real-world video data obtained always contains large numbers of normal video data, along with large numbers of unlabeled data. A promising solution with one-class classification and semi-supervised learning may not be satisfactory as they fail to make good use of unlabeled data with only normal data available. In this paper, we introduced a new framework, called Positive Unlabeled learning-based Anomaly event Detection (PU-AD), to exploit the weakly-supervised information. To the best of our knowledge, this is the first work that introduces the PU idea and achieves detecting abnormal events with a limited number of partially labeled data. Experiments on real-world surveillance videos show that the proposed method outperforms the existing state-of-the-art methods.

We propose a novel unsupervised approach based on a two-stage object-centric adversarial framework that only needs object regions for detecting frame-level local anomalies in videos. The first stage consists in learning the correspondence between the current appearance and past gradient images of objects in scenes deemed normal, allowing us to either generate the past gradient from current appearance or the reverse. The second stage extracts the partial reconstruction errors between real and generated images (appearance and past gradient) with normal object behaviour, and trains a discriminator in an adversarial fashion. In inference mode, we employ the trained image generators with the adversarially learned binary classifier for outputting region-level anomaly detection scores. We tested our method on four public benchmarks, UMN, UCSD, Avenue and ShanghaiTech and our proposed object-centric adversarial approach yields competitive or even superior results compared to state-of-the-art methods.

Anomaly detection in crowd surveillance videos is a critical task for ensuring public safety and security. In this research paper, we propose a comprehensive framework for anomaly detection using deep learning techniques, specifically Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Generative Adversarial Networks (GANs). Our objective is to narrow the gap between computational complexity and detection effectiveness while satisfying the demand for explainability in anomaly detection systems. The proposed framework leverages the UCSD Ped2 dataset and focuses on incorporating spatial constraints to enhance anomaly detection performance. We introduce novel approaches for feature extraction and anomaly detection using GANs, aiming to capture essential spatial and temporal information while reducing computational complexity. Specifically, we develop encoders within the GAN framework to map input data to a lower-dimensional latent space and perform anomaly detection based on the similarity between latent codes. Through extensive experimentation and evaluation, we compare the performance of RNNs, CNNs, and GANs in detecting anomalies in crowd surveillance videos. Various evaluation metrics, including precision, recall, F1 score, ROC-AUC, and PR-AUC, is considered to assess the effectiveness of each approach. Additionally, they studied hyperparameters such as learning rate, batch size, network architecture, and GAN-specific parameters to optimize anomaly detection performance. Our results demonstrate the usefulness of the proposed framework in achieving robust and explainable anomaly detection in crowd surveillance videos. By combining advanced deep learning techniques with spatial constraints and explainability considerations, our research contributes to the development of more reliable and interpretable anomaly detection systems for real-world applications.

Video anomaly detection is of great significance due to its wide applications in video surveillance. Recently, there is a trend of using a video prediction framework to tackle this problem. This kind of methods detects anomalies according to the difference between a predicted frame and its ground truth. However, existing prediction methods lack the consideration of multi-scale temporal constraints on generating future frames. Therefore, based on the prediction framework, this paper proposes a temporal enhanced anomaly detection approach, which designs a generative adversarial network with dual discriminator (frame discriminator and sequence discriminator) to predict future frames. In order to obtain more realistic predictions, other than commonly used spatial constraints and the adversarial penalty from the frame discriminator, we also consider both short-range and long-range motions to impose constraints. Specifically, for short-range motion modeling, we utilize the optical flow loss to ensure temporal continuity over two adjacent frames, while for long-range motion modeling, we design a sequence discriminator to identify fake contained sequences from real sequences, making frames more consistent with their previous consecutive frames. Experiments on three datasets, UCSD Ped1, UCSD Ped2 and Avenue, demonstrate the effectiveness of our method in terms of various evaluation criteria for video anomaly detection.

Anomaly detection in traffic videos has been recently gaining attention due to its importance in intelligent transportation systems. Due to several factors such as weather, viewpoint, lighting conditions, etc. affecting the video quality of a real time traffic feed, it still remains a challenging problem. Even though the performance of state-of-the-art methods on the available benchmark dataset has been competitive, they demand a massive amount of external training data combined with significant computational resources. In this paper, we propose a fast unsupervised anomaly detection system comprising of three modules: preprocessing module, candidate selection module and backtracking anomaly detection module. The preprocessing module outputs stationary objects detected in a video. Then, the candidate selection module removes the misclassified stationary objects using a nearest neighbor approach and then uses K-means clustering to identify potential anomalous regions. Finally, the backtracking anomaly detection algorithm computes a similarity statistic and decides on the onset time of the anomaly. Experimental results on the Track 4 test set of the NVIDIA AI CITY 2020 challenge show the efficacy of the proposed framework as we achieve an F1-score of 0.5926 along with 8.2386 root mean square error (RMSE) and are ranked second in the competition.

In surveillance environments, detecting anomalies requires understanding the contextual dynamics of the environment, human behaviors, and movements within a scene. Effective anomaly detection must address both the where and what of events, but existing approaches such as unimodal action-based methods or LLM-integrated multimodal frameworks have limitations. These methods either rely on implicit scene information, making it difficult to localize where anomalies occur, or fail to adapt to surveillance specific challenges such as view changes, subtle actions, low light conditions, and crowded scenes. As a result, these challenges hinder accurate detection of what occurs. To overcome these limitations, our system takes advantage of features from a lightweight scene classification model to discern where an event occurs, acquiring explicit location-based context. To identify what events occur, it focuses on atomic actions, which remain underexplored in this field and are better suited to interpreting intricate abnormal behaviors than conventional abstract action features. To achieve robust anomaly detection, the proposed Temporal-Semantic Relationship Network (TSRN) models spatio-temporal relationships among multimodal features and employs a Segment-selective Focal Margin loss (SFML) to effectively address class imbalance, outperforming conventional MIL-based methods. Experimental results on public datasets demonstrate that the proposed system effectively reduces false alarms while maintaining robustness and practicality for real-world surveillance applications.

Video anomaly detection aims to find the events in a video that do not conform to the expected behavior. The prevalent methods mainly detect anomalies by snippet reconstruction or future frame prediction error. However, the error is highly dependent on the local context of the current snippet and lacks the understanding of normality. To address this issue, we propose to detect anomalous events not only by the local context, but also according to the consistency between the testing event and the knowledge about normality from the training data. Concretely, we propose a novel two-stream framework based on context recovery and knowledge retrieval, where the two streams can complement each other. For the context recovery stream, we propose a spatiotemporal U-Net which can fully utilize the motion information to predict the future frame. Furthermore, we propose a maximum local error mechanism to alleviate the problem of large recovery errors caused by complex foreground objects. For the knowledge retrieval stream, we propose an improved learnable locality-sensitive hashing, which optimizes hash functions via a Siamese network and a mutual difference loss. The knowledge about normality is encoded and stored in hash tables, and the distance between the testing event and the knowledge representation is used to reveal the probability of anomaly. Finally, we fuse the anomaly scores from the two streams to detect anomalies. Extensive experiments demonstrate the effectiveness and complementarity of the two streams, whereby the proposed two-stream framework achieves state-of-the-art performance on ShanghaiTech, Avenue and Corridor datasets among the methods without object detection. Even if compared with the methods using object detection, our method reaches competitive or better performance on the ShanghaiTech, Avenue, and Ped2 datasets.

Anomaly detection in video assists in the resolution of a wide range of problems. A robust anomaly detection model is necessary due to the growing use of surveillance cameras in both indoor and outdoor settings. As a result, numerous strategies have been proposed in this field. Anomaly detection has already been the subject of several surveys. This survey focuses on deep learning approaches based on video anomaly detection. we categorize the various Deep Learning approaches according to their objectives like score based, future frame-based, Clasiification and reconstruction error based approaches. Additionally, it discusses evaluation criteria and commonly used datasets. We also suggest some possible directions future directions for research.

Anomaly detection in videos is challenging due to the scarcity and variance in positive samples. Current anomaly detection methods can be categorized into reconstruction models and future frame prediction-based models. However, reconstruction models might be exceptionally adapted to abnormal events due to the learning capacity and generalization ability of deep neural networks, whereas prediction-based methods can be sensitive to noise. In this study, we propose an anomaly detection model based on the latent feature space, which combines advantages from both sides. We argue that the constraints in the latent feature space can promote reconstruction; moreover, the optical flow is also considered to introduce temporal information. We use SPyNet for accurate and efficient optical flow estimation. We extensively validate our method on the UCSD Ped1, UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets. The results demonstrated the feasibility of the proposed method and the benefit of utilizing information in the latent feature space.

As surveillance technology is continuously improving, an ever-increasing number of cameras are being deployed everywhere. Relying on manual detection of anomalies through cameras may be unreliable and untimely. Therefore, the application of deep learning in video anomaly detection is being extensively studied. Anomaly Detection (AD) refers to identifying events that deviate from the desired actions. This article discusses representative unsupervised and weakly-supervised learning methods applied to various data types. In these machine learning methods, Generative Adversarial Network, Auto Encoder, Recurrent Neural Network, etc. are broadly adopted for AD. Some renowned and new datasets are reviewed. Furthermore, we also proposed several future directions of research in video anomaly detection.

Due to the limited availability of anomaly examples, video anomaly detection is often seen as one-class classification (OCC) problem. A popular way to tackle this problem is by utilizing an autoencoder (AE) trained only on normal data. At test time, the AE is then expected to reconstruct the normal input well while reconstructing the anomalies poorly. However, several studies show that, even with normal data only training, AEs can often start reconstructing anomalies as well which depletes their anomaly detection performance. To mitigate this, we propose a temporal pseudo anomaly synthesizer that generates fake-anomalies using only normal data. An AE is then trained to maximize the reconstruction loss on pseudo anomalies while minimizing this loss on normal data. This way, the AE is encouraged to produce distinguishable reconstructions for normal and anomalous frames. Extensive experiments and analysis on three challenging video anomaly datasets demonstrate the effectiveness of our approach to improve the basic AEs in achieving superiority against several existing state-of-the-art models.

For enhancing public safety, a surveillance system is essential. Specifically, video surveillance is the most popular way to maintain safety in public and private areas. The detection and recognition of abnormal activity is difficult due to a complex environment, video quality, and varying noise levels. Addressing the challenges of accuracy and video processing, the proposed study uses a cross-attention network with feature fusion to improve the recognition of abnormal activity in complex scenarios. Cross-attention helps to capture contextual information from different videos. The proposed model combines an innovative method of cross attention and feed-forward attention with latent space representation-based fusion, aiming to improve accuracy. The simulation of the study uses two benchmark datasets, UCF and UCSD and achieves remarkable performance with 97.1 % and 91.31 % accuracy. A simulation study has also demonstrated a comparative analysis with different convolution and attention networks for anomaly detection. This study proposes an effective video processing scheme with wide practical potential. The study also provides a new perspective and methodological basis for future research and applications in related fields.