Background subtraction for surveillance videos with camera jitter

in order to solve the adverse effects of strong light and shadow on the test results, a fusion frame difference and background subtraction method in the HSV space is used in this paper. By using frame difference method to solve the effect of strong light, but frame difference method can not detect object when the object do not move, the method of background subtraction can detect it, building Gaussian background model in the HSV space can eliminate shadows. Empirical results show that the method of fusion frame difference and background subtraction in the HSV space can get overcome the effect of strong light and shadows. Fusion background subtraction and frame difference method based on establishing a Gaussian mixture model in HSV space can overcome the disadvantages of the frame difference method, at the same time it can also solve the false detection of object which result from the background subtraction method.

The analysis and understanding of video sequences is currently quite an active research field. Many applications such as video surveillance, optical motion capture or those of multimedia need to first be able to detect the objects moving in a scene filmed by a static camera. This requires the basic operation that consists of separating the moving objects called from the static information called background. Many background subtraction methods have been developed (Bouwmans et al. (2010); Bouwmans et al. (2008)). A recent survey (Bouwmans (2009)) shows that subspace learning models are well suited for background subtraction. Principal Component Analysis (PCA) has been used to model the background by significantly reducing the data's dimension. To perform PCA, different Robust Principal Components Analysis (RPCA) models have been recently developed in the literature. The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. However, authors compare their algorithm only with the PCA (Oliver et al. (1999)) or another RPCA model. Furthermore, the evaluation is not made with the datasets and the measures currently used in the field of background subtraction. Considering all of this, we propose to evaluate RPCA models in the field of video-surveillance. Contributions of this chapter can be summarized as follows: 1) A survey regarding robust principal component analysis and 2) An evaluation and comparison on different video surveillance datasets

Moving object detection is critical for automated video analysis in many vision-related tasks, such as surveillance tracking, video compression coding, etc. Robust Principal Component Analysis (RPCA), as one of the most popular moving object modelling methods, aims to separate the temporally-varying (i.e., moving) foreground objects from the static background in video, assuming the background frames to be low-rank while the foreground to be spatially sparse. Classic RPCA imposes sparsity of the foreground component using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> -norm, and minimizes the modeling error via <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _2$</tex-math></inline-formula> -norm. We show that such assumptions can be too restrictive in practice, which limits the effectiveness of the classic RPCA, especially when processing videos with dynamic background, camera jitter, camouflaged moving object, etc. In this paper, we propose a novel RPCA-based model, called Hyper RPCA, to detect moving objects on the fly. Different from classic RPCA, the proposed Hyper RPCA jointly applies the maximum correntropy criterion (MCC) for the modeling error, and Laplacian scale mixture (LSM) model for foreground objects. Extensive experiments have been conducted, and the results demonstrate that the proposed Hyper RPCA has competitive performance for foreground detection to the state-of-the-art algorithms on several well-known benchmark datasets.

To address the issue of background subtraction include shadow challenge, an online robust principal component analysis (RPCA) method with box constraint (BC-RPCA) has been proposed to detect moving object and accelerate the RPCA like method. First of all, the BC-RPCA method considers the input image sequences as low rank background, sparse foreground and moving shadow. Then the Augmented Lagrangian method is used to convert the box constraint into the objective function and rank-1 modification for thin SVD is also employed to accelerate the solver via alternating direction method of multipliers (ADMM). Finally, the experiments demonstrated the proposed method works effectively and has low computational complexity during real-time application.

Currently, background subtraction is being actively studied in many image processing applications. Nuclear Norm Minimization (NNM) and Weighted Nuclear Norm Minimization (WNNM) are commonly used background subtraction methods based on Robust Principal Component Analysis (RPCA). However, these techniques approximate the RPCA rank function and take the form of an iterative optimization algorithm. Therefore, due to the approximation, the NNM solution can not converge if the number of frames is small. In addition, the NNM and WNNM processing times are delayed because of their iterative optimization schemes. Thus, NNM and WNNM are not suitable for real-time background subtraction. In order to overcome these limitations, this paper presents a real-time background subtraction method using tensor decomposition in accordance with the recent tensor analysis research trend. In this study, we used the closed form TUCKER2 decomposition solution to omit the iterative process while retaining the L1 norm of the RPCA rank function. This proposed method allows for convergence even when the number of frames is small. Compared to NNM and WNNM, the proposed method reduces the processing time by more than 80 times and has a higher precision even when the number of frames are less than 10.

Aligning a given set of images is usually conducted in batch mode manner, which not only requires large amount of memory but also adjusts all the previous transformations to register an input image. To address this issue, we propose a novel approach to image alignment by incorporating the geometric transformation into online robust principal component analysis (PCA). Instead of calculating the warp update using noisy input samples like the conventional methods, we suggest directly linearizing the object function by performing warp update on the recovered samples, which corresponds to an efficient inverse composition algorithm. Since the basis matrix is kept constant for a given sample, both the latent vector and warp update can be very efficiently computed. Moreover, we present two basis updating methods for robust PCA, including the closed-form solution and stochastic gradient descent scheme. We have conducted the extensive experiments on the real-world tasks of background subtraction with camera motion and visual tracking on the challenging video sequences, whose promising results demonstrate the efficacy of our presented approach.

Real-time foreground object extraction is an important subject for computer vision applications. Model-based background subtraction methods have been used to extract the foreground objects. Different from previous methods, this paper introduces a hybrid codebook-based background subtraction method by combining the mixture of Gaussian (MOG) with the codebook (CB) method. We propose an ellipsoid CB model for modeling the dynamic background with highlight and shadow, and develop a modified shadow/ highlight removal method to overcome the influence of illumination change. Our method can avoid extracting the false foreground pixels (e.g., dark background) or missing the real foreground pixels (e.g., bright foreground). Finally, we have done two experiments to compare the performance of our method with the others based on [18] and the change detection benchmark dataset provided in CVPR 2011, respectively.

An increasing number of methods for background subtraction use Robust PCA to identify sparse foreground objects. While many algorithms use the $$\ell _1$$ l 1 -norm as a convex relaxation of the ideal sparsifying function, we approach the problem with a smoothed $$\ell _p$$ l p -quasi-norm and present pROST, a method for robust online subspace tracking. The algorithm is based on alternating minimization on manifolds. Implemented on a graphics processing unit, it achieves realtime performance at a resolution of $$160 \times 120$$ 160 × 120 . Experimental results on a state-of-the-art benchmark for background subtraction on real-world video data indicate that the method succeeds at a broad variety of background subtraction scenarios, and it outperforms competing approaches when video quality is deteriorated by camera jitter.

Most statistical background subtraction techniques are based on the analysis of temporal color/intensity distributions. However, learning statistics on a series of time frames can be problematic, especially when no frames absent of moving objects are available or when the available memory isn’t sufficient to store the series of frames needed for learning. In this paper, we propose a framework that allows common statistical motion detection methods to use spatial statistics gathered on one frame instead of a series of frames as is usually the case. This simple and flexible framework is suitable for various applications including the ones with a mobile background such as when a tree is shaken by wind or when the camera jitters. Three statistical background subtraction methods have been adapted to the proposed framework and tested on different synthetic and real image sequences.KeywordsBackground subtractionspatial distributions

Moving object detection in the presence of complex dynamic backgrounds such as swaying of trees, spouting of water from fountain, ripples in water, flag fluttering in the wind, camera jitters, noise, etc., is known to be very difficult and challenging task. In addition to this, illumination variation, camouflage and real-time constraint aggravate the problem further. Background subtraction (BS) is a widely used algorithm for moving object detection in the presence of static cameras. Its performance purely depends on the choice of features used for background modeling. In this paper, we have proposed a novel multi-feature and multi-modal based background subtraction using Local Binary Pattern (LBP) and Histogram of oriented Gradients (HOG) for complex dynamic scene. Each pixel is modeled as a set of multi-feature calculated from its neighborhood and multi-modal BS is performed using Gaussian mixture model (GMM). To show its efficacy, the proposed algorithm is compared with some of the state-of-the-art BS techniques. In order to evaluate the algorithm in uncontrolled environments, a collection of publicly available database has been used. Quantitative and qualitative results justify our algorithm for efficient moving object detection in the presence of swaying of trees, camouflage and ripples in the water surface.

One of the fundamental operations in computer vision applications is change detection, in which moving foreground objects are segmented from a static background. A common approach for change detection is the comparison of an image frame with the stored background model using a matching algorithm, a process known as background subtraction. However, such techniques fail in environments with dynamic backgrounds, illumination changes, or shadow and camera jitters. This study focuses on effectively detecting scene changes in complex environments. To this end, we proposed a new colour descriptor named local colour difference pattern (LCDP) that is insusceptible to shadow and is able to capture both colour and texture features at a pixel location. Furthermore, a scene change detection framework was proposed to handle dynamic scenes based on sample consensus that integrates LCDP and a novel spatial model fusion mechanism. Experiments using the CDnet benchmark dataset demonstrated the effectiveness of the proposed approach to change detection in complex environments.

In this paper, we propose a data-driven foreground object detection technique which can detect foreground objects from a moving camera. We propose to build a data-driven consensus foreground object template (CFOT) and then detect the foreground object region in each frame. The proposed foreground object detection technique is equipped with the following functions: (1) the ability to detect the foreground object captured by a fast moving camera ; (2) the ability to detect a low contrast (spatially/temporally) foreground object; and (3) the ability to detect a foreground object from a dynamic background. There are three contributions of our method: (1) a newly proposed data-driven foreground region decision process for generating the CFOT has been shown robust and efficient; (2) a foreground object probability is proposed for properly dealing with the imperfect initial foreground region estimations; and (3) a CFOT is generated for precise foreground object detection.

Moving object detection is a fundamental step in various computer vision applications. Robust Principal Component Analysis (RPCA) based methods have often been employed for this task. However, the performance of these methods deteriorates in the presence of dynamic background scenes, camera jitter, camouflaged moving objects, and/or variations in illumination. It is because of an underlying assumption that the elements in the sparse component are mutually independent, and thus the spatiotemporal structure of the moving objects is lost. To address this issue, we propose a spatiotemporal structured sparse RPCA algorithm for moving objects detection, where we impose spatial and temporal regularization on the sparse component in the form of graph Laplacians. Each Laplacian corresponds to a multi-feature graph constructed over superpixels in the input matrix. We enforce the sparse component to act as eigenvectors of the spatial and temporal graph Laplacians while minimizing the RPCA objective function. These constraints incorporate a spatiotemporal subspace structure within the sparse component. Thus, we obtain a novel objective function for separating moving objects in the presence of complex backgrounds. The proposed objective function is solved using a linearized alternating direction method of multipliers based batch optimization. Moreover, we also propose an online optimization algorithm for real-time applications. We evaluated both the batch and online solutions using six publicly available datasets that included most of the aforementioned challenges. Our experiments demonstrated the superior performance of the proposed algorithms compared with the current state-of-the-art methods.

A method is presented for extracting object information from an image sequence taken by a static monocular camera. The method was developed towards a low computational complexity in order to be used in real-time surveillance applications. Our approach makes use of both intensity and edge information of each frame and works efficiently in an indoor environment. It consists of two major parts: background processing and foreground extraction. The background estimation and updating makes the object detection robust to environment changes like illumination changes and camera jitter. The fusion of intensity and edge information allows a more precise estimation of the position of the different foreground objects in a video sequence. The result obtained are quite reliable, under a variety of environmental conditions.

Foreground detection is one of the active research topics in computer vision and a precondition for intelligent video analytics. Background subtraction is often considered as a reliable method for extracting foreground from the video sequences. The real-world challenges such as intermittent object motion, dynamic backgrounds, bad weather conditions, shadows, and illumination variations affect the performance of foreground detection. Most of the conventional background subtraction methods result in faulty detection and fragmented foreground in case of complex scenes. In recent years, deep neural networks make revolutionary advances in the area of foreground detection. The convolutional neural networks based background subtraction methods are notable for their performance but their performance drop in case of unseen videos and incur high-computational cost. In this paper, we investigate the fusion of superpixel with an adaptive background modeling for foreground detection in several challenging videos. A non-iterative clustering method is adopted for superpixel clustering because of its computational efficiency. Integration of effective superpixel algorithm with background subtraction has paved the way for improvements in unsupervised background subtraction methods. The proposed method is evaluated on different videos of changedetection.net 2014 dataset in order to validate its effectiveness on critical challenges of background subtraction.KeywordsBackground modelingForegroundMoving objectsSuperpixelsUnsupervised