Efficient Object Segmentation Using Digital Matting for MPEG Video Sequences

Detection of video objects under bad weather and poor illumination condition is a challenging task. We address this issue using the notion of background modeling. LBP-based background modeling and model learning has been used to detect the video objects but performance degrades in the above complex background. We propose the adjacency- and reinforced adjacency-based variants of LBP for complex real-world background modeling and model learning for object detection. In this regard, we have proposed the four variants; (1) enhanced adjacent local binary pattern, (2) enhanced reinforced adjacent local binary pattern, (3) enhanced adjacent local ternary pattern, and (4) enhanced reinforced adjacent local ternary pattern. Besides, we have embedded the Gabor and LBP features to obtain an embedded feature, which is subsequently used with the notions of adjacency. Unlike the background learning approach where the model learns only the background, our model learning algorithm learns the background together with the foreground objects and hence named as mixed-mode learning strategy. These models together with the new learning strategy are tested with CD2014 (snowfall and blizzard) and PETS (2014 and 2016) data sets, and the performance of the proposed models has been compared with LBP-based methods and other state-of-the-art methods.

This paper presents a hybrid pixel-based background (HPB) model, which is constructed by single stable record and multi-layer astable records after initial learning. The image foreground object detection must face the problems of moving background, illumination changes, chaotic, etc. in real word applications. In our approach, the HPB model can be used for background subtraction to extract objects precisely in various complex scenes. Using the multi-layer astable records, we also propose the homogeneous background subtraction that can detect the foreground object with less record memory. Based on the benchmark videos, the experimental results show that single stable and 3-layer multi-layer astable records can be enough for background model construction and then updated quickly to overcome the background variation. The proposed approach can improve the averages Error Rate of foreground object detection up to 86% when comparing with the latest works.

Background subtraction method is widely used in most of the video motion detection algorithms especially for video surveillance application. Background subtraction is used to extract moving or static foreground objects from the background scene. The efficiency of foreground-background segmentation heavily relies on background model which must be able to cope with changes in the scene and granularity of the foreground objects. The robust background model can produce good foreground segmentation results and it is still a great challenge to get accurate and high performance result today. In this paper, a video foreground-background segmentation approach is proposed. This approach is based on Codebook (CB) model with Kalman Filter. This approach can be used to extract foreground objects from the video stream. The Lab color space is used in this approach to calculate color difference between two pixels using CIEDE2000 color difference formula. Extracted foreground object from video sequence using this approach is useful for object detection in video surveillance applications. This approach has been tested with PETS and CDnet2014 datasets and segmentation results accuracy are evaluated compare with ground truth.

The segmentation of moving objects in image sequence can be formulated as a background subtraction problem—the separation of objects from the background in each image frame. The background scene is learned and modeled. A pixelwise process is employed to classify each pixel as an object or background based on its similarity with the background model. The segmentation is challenging due to the occurrence of dynamic elements such as illumination change and background motions. We propose a framework for object segmentation with a novel feature for background representation and new mechanisms for updating the background model. A ternary pattern is employed to characterize the local texture. The pattern and photometric features are used for background modeling. The classification of pixel is performed based on the perceptual similarity between the current pixel and the background model. The segmented object is refined by taking into account the spatial consistency of the image feature. For the background model update, we propose two mechanisms that are able to adapt to abrupt background change and also merge new background elements into the model. We compare our framework with various background subtraction algorithms on video datasets.

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple images for each detection, and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of images per study. The task of converting images to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g. camera malfunction or moving vegetation) or pictures of humans. We offer the first widely available computer vision tool for processing camera trap images. Our results show that the tool is accurate and results in substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps. In this dissertation, we have developed new image/video processing and computer vision algorithms for efficient and accurate object detection and sequence-level classiffication from natural scene camera-trap images. This work addresses the following five major tasks: (1) Human-animal detection. We develop a fast and accurate scheme for human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification. Specifically, first, We develop an effective background modeling and subtraction scheme to generate region proposals for the foreground objects. We then develop a cross-frame image patch verification to reduce the number of foreground object proposals. Finally, We perform complexity-accuracy analysis of deep convolutional neural networks (DCNN) to develop a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times. Our experimental results demonstrate that the proposed method outperforms existing methods on the camera-trap dataset. (2) Object segmentation from natural scene. We first design and train a fast DCNN for animal-human-background object classification, which is used to analyze the input image to generate multi-layer feature maps, representing the responses of different image regions to the animal-human-background classifier. From these feature maps, we construct the so-called deep objectness graph for accurate animal-human object segmentation with graph cut. The segmented object regions from each image in the sequence are then verfied and fused in the temporal domain using background modeling. Our experimental results demonstrate that our proposed method outperforms existing state-of-the-art methods on the camera-trap dataset with highly cluttered natural scenes. (3) DCNN domain background modeling. We replaced the background model with a new more efficient deep learning based model. The input frames are segmented into regions through the deep objectness graph then the region boundaries of the input frames are multiplied by each other to obtain the regions of movement patches. We construct the background representation using the temporal information of the co-located patches. We propose to fuse the subtraction and foreground/background pixel classiffcation of two representation : a) chromaticity and b) deep pixel information. (4) Sequence-level object classiffcation. We proposed a new method for sequence-level video recognition with application to animal species recognition from camera trap images. First, using background modeling and cross-frame patch verification, we developed a scheme to generate candidate object regions or object proposals in the spatiotemporal domain. Second, we develop a dynamic programming optimization approach to identify the best temporal subset of object proposals. Third, we aggregate and fuse the features of these selected object proposals for efficient sequence-level animal species classification.

This paper presents a moving object detection method against sudden illumination change using improved background modeling. Initially, background model is created for every pixel from the first frame. The sample values for the model of a pixel are collected from the neighborhood of that pixel. Then the new pixel values from the new frames are compared to make background foreground decision. Conventional background modeling faces problem with change in the illumination of the scene. The proposed method frequently checks whether abrupt change of illumination take place or not and then initialize the background model from the frame that is detected with changed illumination. The illumination change is detected by obtaining the images of two frames that are taken at a suitable interval in HSV color space. Then the mean change value of each channel is calculated to make a decision. This enables the background model to start over with new sample values that are obtained in the current illumination condition and the background subtraction process can successfully detect moving object with greater accuracy even in changing illumination condition. Simulation results indicate that the proposed method gives excellent results in illumination changing condition to detect moving object whereas the conventional background modeling can not detect accurately. Comparison analysis shows that our proposed method outperforms recent methods in terms of detection accuracy.

Object detection is an important task for computer vision applications. Many researchers have proposed a number of methods to detect the objects through background modeling. To adapt to “illumination changes” in the background, local feature-based background models are proposed. They assume that local features are not affected by background changes. However, “motion changes”, such as the movement of trees, affect the local features in the background significantly. Therefore, it is difficult for local feature-based models to handle motion changes in the background. To solve this problem, we propose a new background model in this paper by applying a statistical framework to a local feature-based approach. Our proposed method combines the concepts of statistical and local feature-based approaches into a single framework. In particular, we use illumination invariant local features and describe their distribution by Gaussian Mixture Models (GMMs). The local feature has the ability to tolerate the effects of “illumination changes”, and the GMM can learn the variety of “motion changes”. As a result, this method can handle both background changes. Some experimental results show that the proposed method can detect the foreground objects robustly against both illumination changes and motion changes in the background.

Adaptive bootstrapping management by keypoint clustering for background initialization

The problem of foreground detection in real‐time video surveillance applications is addressed. Proposes is a framework, which is computationally cheap and has low memory requirements. It combines two simple processing blocks, both of which are essentially background subtraction algorithms. The main novelty of the approach is a combination of an autoregressive moving average filter with two background models having different adaptation speeds. The first model, having a lower adaptation speed, models long‐term background and detects foreground objects by finding areas in the current frame which significantly differ from the proposed background model. The second model, with a higher adaptation speed, models the short‐term background and is responsible for finding regions in the scene with a high foreground object activity. The final foreground detection is built by combining the outputs from these building blocks. The foreground obtained by the long‐term modelling block is verified by the output of the short‐term modelling block, i.e. only the objects exhibiting significant motion are detected as real foreground objects. The proposed method results in a very good foreground detection performance at a low computational cost.

We propose an effective background model using mixture of Gaussians and Laplacian pyramid decomposition for foreground object segmentation from complex scene containing stationary and moving objects. The Laplacian pyramid is employed to decompose the input image into a low-frequency big scale image and a high-frequency image. We build two mixtures of Gaussians in each pixel to represent the statistical characteristics of the stationary and moving points with proper feature vectors. Big scale foreground objects are obtained by fusing the results from stationary and dynamic models. Original foreground objects are then restored using the established model, low-frequency and high-frequency images. The experiments we performed here on complex scenes containing dynamic background objects have showed better performance and less memory cost compared.

Non-parametric representation of pixel intensity distribution is quite effective to construct proper background model and to detect foreground objects accurately. However, from the viewpoint of practical application, the computation cost of the distribution estimation should be reduced. In this paper, we present fast estimation of the probability density function (PDF) of pixel value using Parzen density estimation and foreground object detection based on the estimated PDF. Here, the PDF is computed by partially updating the PDF estimated at the previous frame, and it greatly reduces the computation cost of the PDF estimation. Thus, the background model adapts quickly to changes in the scene and, therefore, foreground objects can be robustly detected. Several experiments show the effectiveness of our approach.

We propose a new method for background modeling. Our method is based on the two complementary approaches. One uses the probability density function(PDF) to approximate background model. The PDF is estimated non-parametrically by using Parzen density estimation. And foreground object is detected based on the estimated PDF. The other method is based on the evaluation of the local texture at pixel-level resolution while reducing the effects of variations in lighting. Fusing their approach realize robust object detection under varying illumination. Several experiments show the effectiveness of our approach.

Background subtraction is a widely used technique for detecting moving objects in image sequences. Very often background subtraction approaches assume the availability of one or more clear frames (i.e., without foreground objects) at the beginning of the image sequence in input. This strong assumption is not always correct, especially when dealing with dynamic background. In this paper, we present the results of an on-line and real-time background initialization method, called IMBS, which generates a reliable initial background model even if no clear frames are available. The accuracy of the proposed approach is calculated on a set of seven publicly available benchmark sequences. Experimental results demonstrate that IMBS generates accurate background models with respect to eight different quality metrics.

In this paper, we couple effective dynamic background modeling with deep learning classification to develop a fast and accurate scheme for human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification. Specifically, first, we develop an effective background modeling and subtraction scheme to generate region proposals for the foreground objects. We then develop a cross-frame image patch verification to reduce the number of foreground object proposals. Finally, we perform complexity-accuracy analysis of deep convolutional neural networks (DCNN) to develop a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times. Our experimental results demonstrate that the proposed method outperforms existing methods on the camera-trap dataset.

Background subtraction in real applications: Challenges, current models and future directions