Special issue on background modeling for foreground detection in real-world dynamic scenes

Cut set-based Dynamic Key frame selection and Adaptive Layer-based Background Modeling for background subtraction

Dynamic backgrounds and sudden illumination changes are two of the major problems associated with background subtraction techniques. In this paper, we present a novel approach to background subtraction that addresses both of these challenges. In particular, we present an improved codebook background modelling and subtraction technique. We utilise image segmentation on the background image and model the background with a codebook for each pixel along with a pseudo background layer. We perceive background motion as an occlusion of one background layer by a nearby background layer. In other words, sliding of one background layer over a neighbouring layer causes background motion and will hence result in false segmentation. We present our approach of codeword spreading across layer boundaries to handle background motion. Furthermore, we present a two-step update of the background codebook to handle both sudden and gradual illumination changes.

Robustness is an important factor for background modelling on various scenarios. Current pixel‐based adaptive segmentation method cannot effectively tackle diverse objects simultaneously. To address this problem, in this study, a background modelling method using discriminative motion representation is proposed. Instead of simple usage of intensity to construct the background model, the proposed method extracts a new local descriptor which uses a weighted combination of differential excitations for each pixel to enhance the discriminability of pixels. On the basis of this background model, different categories of objects can be quickly identified by a simple but effective classification rule and accurately be represented in background model by a smart selection of updating strategies. Therefore, the authors’ background modelling method can generate complete representation for static objects and decrease false detection caused by dynamic background or illumination variations. Extensive experiments have been conducted to demonstrate that the proposed method obtains more advantages of foreground detection than the state‐of‐the‐art methods. In addition, the proposed method provides a computational efficient algorithm for foreground detection tasks.

Foreground detection is one of the key steps in computer vision applications. Many foreground and background models have been proposed and achieved promising performance in static scenes. However, due to challenges such as dynamic background, irregular movement, and noise, most algorithms degrade sharply in complex scenes. To address the problem, we propose a motion-aware structured matrix factorization approach (MSMF), which integrates the structural and spatiotemporal motion information into a unified sparse-low-rank matrix factorization framework. Technologically, it has three main contributions: First, a variant of structured sparsity-inducing norm is proposed to constrain both structure and sparsity of foreground. The model is robust to the statistical variability of the underlying foreground pixels in complex scenes. Second, to capture the ambiguous pixels, a spatiotemporal cube-based motion trajectory is extracted for assisting matrix factorization. Finally, to solve the optimization problem of structured matrix factorization, we develop an augmented Lagrange multiplier method with the alternating direction strategy and Douglas-Rachford monotone operator splitting algorithm. Experiments demonstrate that the proposed approach achieves impressive performance in separating irregular moving foreground while suppressing the dynamic background and the noise, and outperforms some state-of-the-art algorithms.

This paper presents a hybrid pixel-based background (HPB) model, which is constructed by single stable record and multi-layer astable records after initial learning. The image foreground object detection must face the problems of moving background, illumination changes, chaotic, etc. in real word applications. In our approach, the HPB model can be used for background subtraction to extract objects precisely in various complex scenes. Using the multi-layer astable records, we also propose the homogeneous background subtraction that can detect the foreground object with less record memory. Based on the benchmark videos, the experimental results show that single stable and 3-layer multi-layer astable records can be enough for background model construction and then updated quickly to overcome the background variation. The proposed approach can improve the averages Error Rate of foreground object detection up to 86% when comparing with the latest works.

On the role and the importance of features for background modeling and foreground detection

Detection of video objects under bad weather and poor illumination condition is a challenging task. We address this issue using the notion of background modeling. LBP-based background modeling and model learning has been used to detect the video objects but performance degrades in the above complex background. We propose the adjacency- and reinforced adjacency-based variants of LBP for complex real-world background modeling and model learning for object detection. In this regard, we have proposed the four variants; (1) enhanced adjacent local binary pattern, (2) enhanced reinforced adjacent local binary pattern, (3) enhanced adjacent local ternary pattern, and (4) enhanced reinforced adjacent local ternary pattern. Besides, we have embedded the Gabor and LBP features to obtain an embedded feature, which is subsequently used with the notions of adjacency. Unlike the background learning approach where the model learns only the background, our model learning algorithm learns the background together with the foreground objects and hence named as mixed-mode learning strategy. These models together with the new learning strategy are tested with CD2014 (snowfall and blizzard) and PETS (2014 and 2016) data sets, and the performance of the proposed models has been compared with LBP-based methods and other state-of-the-art methods.

Background modeling and subtraction is a natural technique for object detection in videos captured by a static camera, and also a critical preprocessing step in various high-level computer vision applications. However, there have not been many studies concerning useful features and binary segmentation algorithms for this problem. We propose a pixelwise background modeling and subtraction technique using multiple features, where generative and discriminative techniques are combined for classification. In our algorithm, color, gradient, and Haar-like features are integrated to handle spatio-temporal variations for each pixel. A pixelwise generative background model is obtained for each feature efficiently and effectively by Kernel Density Approximation (KDA). Background subtraction is performed in a discriminative manner using a Support Vector Machine (SVM) over background likelihood vectors for a set of features. The proposed algorithm is robust to shadow, illumination changes, spatial variations of background. We compare the performance of the algorithm with other density-based methods using several different feature combinations and modeling techniques, both quantitatively and qualitatively.

Moving Object Detection systems have long been in use to look at security sensitive areas. The manufacture of video investigating systems sensible desires fast, reliable and durable algorithms for moving object detection classification pursuit and activity analysis. Moving object detection is that the essential step for extra analysis of video. It handles segmentation of moving objects from stationary background objects. Object classification step categorizes detected objects into preened classes like human, vehicle, animal, clutter, etc. it's a necessity to inform apart objects from each other therefore on trace and analyse their actions reliably. In previous system we have got performed background subtraction by victimization clever Edge Detection. In clever Edge Detection technique we tend to tend to square measure taking 2 photos for comparison those square measure background image and foreground image. Previous strategies for object detection square measure methodology along side object detectors image segmentation and Background subtraction. Our methodology aims to section objects supported motion info and it contains a part of background modelling. In the previous strategies we tend to tend to square measure conducting background subtraction only for photos. For this we tend to tend to projected an image component wise background modelling and subtraction technique mistreatment multiple choices. Hence, throughout this color, gradient and Hear-Like choices square measure integrated to handle the variation element. Thus, motion segmentation and background subtraction square measure the foremost connected topics to the current paper.

Background subtraction is a widely used technique for detecting moving objects in image sequences. Very often background subtraction approaches assume the availability of one or more clear frames (i.e., without foreground objects) at the beginning of the image sequence in input. This strong assumption is not always correct, especially when dealing with dynamic background. In this paper, we present the results of an on-line and real-time background initialization method, called IMBS, which generates a reliable initial background model even if no clear frames are available. The accuracy of the proposed approach is calculated on a set of seven publicly available benchmark sequences. Experimental results demonstrate that IMBS generates accurate background models with respect to eight different quality metrics.

Background Subtraction (BS) is one of the key steps in video analysis. Many background models have been proposed and achieved promising performance on public data sets. However, due to challenges such as illumination change, dynamic background etc. the resulted foreground segmentation often consists of holes as well as background noise. In this regard, we consider generalized fused lasso regularization to quest for intact structured foregrounds. Together with certain assumptions about the background, such as the low-rank assumption or the sparse-composition assumption (depending on whether pure background frames are provided), we formulate BS as a matrix decomposition problem using regularization terms for both the foreground and background matrices. Moreover, under the proposed formulation, the two generally distinctive background assumptions can be solved in a unified manner. The optimization was carried out via applying the augmented Lagrange multiplier (ALM) method in such a way that a fast parametric-flow algorithm is used for updating the foreground matrix. Experimental results on several popular BS data sets demonstrate the advantage of the proposed model compared to state-of-the-arts.

Foreground detection is one of the active research topics in computer vision and a precondition for intelligent video analytics. Background subtraction is often considered as a reliable method for extracting foreground from the video sequences. The real-world challenges such as intermittent object motion, dynamic backgrounds, bad weather conditions, shadows, and illumination variations affect the performance of foreground detection. Most of the conventional background subtraction methods result in faulty detection and fragmented foreground in case of complex scenes. In recent years, deep neural networks make revolutionary advances in the area of foreground detection. The convolutional neural networks based background subtraction methods are notable for their performance but their performance drop in case of unseen videos and incur high-computational cost. In this paper, we investigate the fusion of superpixel with an adaptive background modeling for foreground detection in several challenging videos. A non-iterative clustering method is adopted for superpixel clustering because of its computational efficiency. Integration of effective superpixel algorithm with background subtraction has paved the way for improvements in unsupervised background subtraction methods. The proposed method is evaluated on different videos of changedetection.net 2014 dataset in order to validate its effectiveness on critical challenges of background subtraction.KeywordsBackground modelingForegroundMoving objectsSuperpixelsUnsupervised

Background subtraction is an important part of various computer vision applications that can detect the foreground objects by comparing the current pixels with a background model. The general approaches gradually update the background model according to the current status, but might fail in sudden illumination changes. An illumination‐robust background modelling method is proposed to address this problem. The method provides quick illumination compensation using two background models with different adaption rates. Experimental results show that the proposed algorithm outperforms several state‐of‐art approaches and provides low computational cost.

One of the fundamental operations in computer vision applications is change detection, in which moving foreground objects are segmented from a static background. A common approach for change detection is the comparison of an image frame with the stored background model using a matching algorithm, a process known as background subtraction. However, such techniques fail in environments with dynamic backgrounds, illumination changes, or shadow and camera jitters. This study focuses on effectively detecting scene changes in complex environments. To this end, we proposed a new colour descriptor named local colour difference pattern (LCDP) that is insusceptible to shadow and is able to capture both colour and texture features at a pixel location. Furthermore, a scene change detection framework was proposed to handle dynamic scenes based on sample consensus that integrates LCDP and a novel spatial model fusion mechanism. Experiments using the CDnet benchmark dataset demonstrated the effectiveness of the proposed approach to change detection in complex environments.

Obtaining a dynamically updated background reference image is an important and challenging task for video applications using background subtraction. This paper proposes a novel algorithm for dynamic video background reconstruction with move-then-stop arbitrary-long time video object detection enabled. In addition to the adaptive mixture Gaussian background model (AMGBM), the proposed algorithm makes use of the location information of the detected objects in the previous background subtraction step as a feedback control for the current step. The background update action of a certain pixel position is not taken until a signal indicating "object starts moving" is received. The proposed algorithm uses an update counter which is updated only for pixels that are not occluded by an object. The update procedure takes place after the background subtraction when the position of all objects is known. The experimental results show that the proposed algorithm outperforms existing AMGBM by providing novel feature to detect move-then-stop video objects, even when an object stops for an arbitrarily long time.