Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos with deep learning

During past few years’ significant usage of video monitoring has been observed in several places. Video monitoring has always helped in improvement of security specific aspects. These systems provide major helping hand in monitoring through multiple object detection methods. However, there are certain challenges related to weather conditions which needs to be addressed. In view of this in this paper, we study multiple object detection in surveillance videos through hybrid deep gaussian mixture ensembles. This method integrates multiple deep gaussian mixture components to form ensembles which helps to develop multiple object detection solutions in surveillance videos. The computational system consists of ensemble of three major deep gaussian mixture components. The ensemble pipeline is formed with augmenting, bagging and stacking methods. The ensemble consists of hybrid object detection paradigm with steps background analysis, video pre-processing, data integration, object detection, foreground detection and post-processing. The computational pipeline is successfully experimented with ViSOR and CDnet 2014 benchmarked datasets. All results are validated with accuracy, precision, sensitivity or recall, specificity, F1-Score and RMSE metrics. Several comparative studies are performed with state-of-the-art methods as well as baselines. The experimental results demonstrate superiority of this method in comparison with other methods. Ablation studies have also been performed here with superior results. This system presents robustness in multiple object detection for real life challenges such as sudden illumination variation, shadow presence, long term occlusion and formation of dynamic backgrounds. It provides cost-effective, more profitable, efficient and sustainable real time multiple object detection solutions

Event detection in crowded surveillance videos is a challenging yet important problem. In this paper, we present our eSur (Event detection system on SURveillance video) system, which is derived from TRECVid'12 surveillance tasks. Currently, eSur attempts to detect two categories of events: 1) pair-wise events (e.g., PeopleMeet, PeopleSplitUp and Embrace); 2) action-like events (e.g., ObjectPut, CellToEar, PersonRuns and Pointing). In eSur system, we first employ people detection and tracking algorithms to locate target persons in 3D space-time domain. Then the video sequences in which target persons occur are partitioned into several spatio-temporal cubes. Visual features (i.e. cubic feature and MoSIFT) are computed over these cubes. After that, a sequence learning method, (namely SVM with dynamic time alignment kernel), is employed to infer the existence of an event for the video sequence. According to the TRECVid SED formal evaluation, eSur has yielded fairly encouraging results on TRECVid'12 dataset.

Object detection is a critical step in automated surveillance. A common approach to constructing object detectors consists of annotating large datasets and using them to train the detectors. However, due to inevitable limitations of a typical training data set, such supervised approach is unsuitable for building generic surveillance systems applicable to a wide variety of scenes and camera setups. In our previous work we proposed an unsupervised method for learning and detecting the dominant object class in a general dynamic scene observed by a static camera. In this paper, we investigate the possibilities to expand the applicability of this method to the problem of multiple dominant object classes. We propose an idea how to approach this expansion, and perform a proof-of-concept evaluation of this idea using a representative surveillance video sequence.

This paper presents a supervised learning scheme that employs key-frame extraction to enhance the performance of pre-trained deep learning models for object detection in surveillance videos. Developing supervised deep learning models requires a significant amount of annotated video frames as training data, which demands substantial human effort for preparation. Key frames, which encompass frames containing false negative or false positive objects, can introduce diversity into the training data and contribute to model improvements. Our proposed approach focuses on detecting false negatives by leveraging the motion information within video frames that contain the detected object region. Key-frame extraction significantly reduces the human effort involved in video frame extraction. We employ interactive labeling to annotate false negative video frames with accurate bounding boxes and labels. These annotated frames are then integrated with the existing training data to create a comprehensive training dataset for subsequent training cycles. Repeating the training cycles gradually improves the object detection performance of deep learning models to monitor a new environment. Experiment results demonstrate that the proposed learning approach improves the performance of the object detection model in a new operating environment, increasing the mean average precision (mAP@0.5) from 54% to 98%. Manual annotation of key frames is reduced by 81% through the proposed key-frame extraction method.

More than 80% of video surveillance systems are used for monitoring people. Old human detection algorithms, based on background and foreground modelling, could not even deal with a group of people, to say nothing of a crowd. Recent robust and highly effective pedestrian detection algorithms are a new milestone of video surveillance systems. Based on modern approaches in deep learning, these algorithms produce very discriminative features that can be used for getting robust inference in real visual scenes. They deal with such tasks as distinguishing different persons in a group, overcome problem with sufficient enclosures of human bodies by the foreground, detect various poses of people. In our work we use a new approach which enables to combine detection and classification tasks into one challenge using convolution neural networks. As a start point we choose YOLO CNN, whose authors propose a very efficient way of combining mentioned above tasks by learning a single neural network. This approach showed competitive results with state-of-the-art models such as FAST R-CNN, significantly overcoming them in speed, which allows us to apply it in real time video surveillance and other video monitoring systems. Despite all advantages it suffers from some known drawbacks, related to the fully-connected layers that obstruct applying the CNN to images with different resolution. Also it limits the ability to distinguish small close human figures in groups which is crucial for our tasks since we work with rather low quality images which often include dense small groups of people. In this work we gradually change network architecture to overcome mentioned above problems, train it on a complex pedestrian dataset and finally get the CNN detecting small pedestrians in real scenes.

This paper proposes a new algorithm for abnormal behavior detection in surveillance video. Firstly, the motion information image of each frame is constructed by calculating the optical flow size and the angle difference between the optical flow vectors between consecutive frames, and then the obtained motion image information is input into the convolutional neural network (CNN) for training, and used for video abnormal behavior detection. In the algorithm, the motion information image generated based on optical flow information can provide the motion information features in the video image more accurately, which makes it easier to distinguish the normal behavior and abnormal behavior of the video. The experiment of this algorithm is carried out on the commonly used data set PETS 2009. Experimental results show that the proposed method and other algorithms have a significant improvement in the accuracy of abnormal behavior detection.

Several computer vision algorithms have been proposed to detect anomalous activities (robberies, murders, vandalism, among others) in videos. According to the learning approach, they can be classified into probabilistic distribution modeling, sparse coding, and deep learning based methods. The main drawbacks of these approaches are (i) extraction of low-level features that do not capture complex behaviors of instances on the scene, (ii) generation of features from irrelevant regions, (iii) overlooking of relationships among objects, and (iv) omission of long-term dependencies. To solve these issues, we propose a deep learning architecture that leverages the relationships among objects. It achieves this by using an attention mechanism and learning long-term dependencies using a multilayer recurrent neural network (multilayer LSTM). An AUC score of 0.749 on the UCF-Crime dataset confirms that the proposed algorithm competes effectively against several state-of-the-art approaches for anomaly detection in surveillance videos. It also explains the relationship between regions in the video frames and the anomaly detections.

Fast fire flame detection in surveillance video using logistic regression and temporal smoothing

For abnormal gait detection in surveillance videos, the existing methods suffer from that they are unable to recognize novel types of anomalies if the corresponding prototypes have not been included in the training data for supervised machine learning but it is impractical to foresee all types of anomalies. This research aims to solve the problem in an unsupervised manner, which does not rely on any prior knowledge regarding abnormal prototypes and avoids time-consuming machine learning over large-scale high-dimensional features. The intuition is that normal gait is nearly periodic signal and anomalies may disturb such periodicity. Hence, the time-varying ratio of width to height of a walking person is transformed to frequency domain using Fast Fourier Transform (FFT), and the standard deviation over spectrum is used as an indicator of anomalies, subject to any sudden change to break the normally periodical walking rhythm. The experimental results demonstrate its precision.

In this paper, global abnormal events detection in the crowded scene is proposed. The proposed algorithm is based on a hierarchical approach. The method uses the histogram of optical flow orientation as a feature descriptor along with the magnitude of the optical flow to capture the motion. The nonlinear one-class support vector machine (SVM) classification algorithm is used to learn the normal events from the training data. After learning, one-class SVM detects abnormal events in the frame. The algorithm is fast with improved accuracy since it omits the background subtraction step and adopts hierarchy. The proposed method is tested on benchmark UMN unusual crowd activity dataset which contain global abnormal events. Promising results are reported which evaluates the performance of the proposed method.

Crowd Event Detection in Surveillance Video - An Approach based on Optical Flow High-frequency Feature Analysis

Deep Convolutional Neural Network (CNN) technology has attracted the interest of researchers in computer vision to discover objects in photos and videos and recognize faces and their characteristics because it has effective performance. This paper describes the main features of CNN's most accurate deep learning and its applications. The YOLOv3 model was used to accurately classify and predict objects in each frame of a video by embedding the object box and extracting features using the ImageAI library. This library supports pre-trained models on the coco dataset to detect pedestrians and objects using training data (video about Kirkuk University). It is possible to use the importance of this study to monitor the traffic or search for something tracked by video surveillance. In addition to detecting pedestrians in the video, visible objects, and faces. The result compares this video detection method and is evaluated using performance metrics such as a map. It achieved the best video processing performance with an accuracy of 99.52% and very high accuracy for video object detection (98.97% people, 99.12% cars, 100% bicycle, 100% clock), which proved to be higher performance compared to other detection models.

Recent advancements in intelligent surveillance systems for video analysis have been a topic of great interest in the research community due to the vast number of applications to monitor humans’ activities. The growing demand for these systems aims towards automatic violence detection (VD) systems enhancing and comforting human lives through artificial neural networks (ANN) and machine intelligence. Extremely overcrowded regions such as subways, public streets, banks, and the industries need such automatic VD system to ensure safety and security in the smart city. For this purpose, researchers have published extensive VD literature in the form of surveys, proposals, and extensive reviews. Existing VD surveys are limited to a single domain of study, i.e., coverage of VD for non-surveillance or for person-to-person data only. To deeply examine and contribute to the VD arena, we survey and analyze the VD literature into a single platform that highlights the working flow of VD in terms of machine learning strategies, neural networks (NNs)-based patterns analysis, limitations in existing VD articles, and their source details. Further, we investigate VD in terms of surveillance datasets and VD applications and debate on the challenges faced by researchers using these datasets. We comprehensively discuss the evaluation strategies and metrics for VD methods. Finally, we emphasize the recommendations in future research guidelines of VD that aid this arena with respect to trending research endeavors.

In this paper, a novel approach for automatic anomaly detection in surveillance video is proposed. It is highly efficient for real-time detection. It can also handle multi-scale detection and can cope with both spatial and temporal anomalies. Specifically, features capturing both appearance and motion characteristic are extracted from densely sampled spatio-temporal video volume (STV). And to bridge the semantic gap between low-level visual feature and high-level event, we use the middle-level visual attributes as the intermediary. These three-level framework is modeled as an Extreme Learning Machine (ELM) which can effectively and efficiently tell whether a STV belongs to an anomalous event. We also use the Spatio-temporal Pyramid (STP) to capture the spatial and temporal continuity of an anomalous event , enabling our approach to cope with multi-scale and complicated events. Experiments on several datasets are carried out and the superior performance compared to state-of-the-art approaches verifies the effectiveness of our approach.KeywordsAutomatic anomaly detectionVisual attributesExtreme Learning MachineSpatio-temporal Pyrimid

For enhancing public safety, a surveillance system is essential. Specifically, video surveillance is the most popular way to maintain safety in public and private areas. The detection and recognition of abnormal activity is difficult due to a complex environment, video quality, and varying noise levels. Addressing the challenges of accuracy and video processing, the proposed study uses a cross-attention network with feature fusion to improve the recognition of abnormal activity in complex scenarios. Cross-attention helps to capture contextual information from different videos. The proposed model combines an innovative method of cross attention and feed-forward attention with latent space representation-based fusion, aiming to improve accuracy. The simulation of the study uses two benchmark datasets, UCF and UCSD and achieves remarkable performance with 97.1 % and 91.31 % accuracy. A simulation study has also demonstrated a comparative analysis with different convolution and attention networks for anomaly detection. This study proposes an effective video processing scheme with wide practical potential. The study also provides a new perspective and methodological basis for future research and applications in related fields.