Higher-order statistical learning for joint motion estimation and segmentation in video

In this paper, we present an image understanding system using fuzzy sets and fuzzy measures. This system is based on a symbolic object-oriented image interpretation system. We apply a simple, powerful three-dimensional (3-D) recursive filter to tracking moving objects in a dynamic image sequence. This filter has a time-varying 3-D frequency-planar passband that is adapted in a feedback system to automatically track moving objects. However, as objects in the image sequence are not well-defined and are engaged in dynamic activities, their shapes and trajectories in most cases can be described only vaguely. In order to handle these uncertainties, we use fuzzy measures to capture subtle variations and manage the uncertainties involved. This enables us to develop an image understanding system that produces a very natural output. We demonstrate the effectiveness of our system with complex real traffic scenes.

A noise robust method for 2D shape estimation of moving objects in video sequences considering a moving camera

Tracking objects in image sequences involves performing motion analysis at the object level, which is becoming an increasingly important technology in a wide range of computer video applications, including video teleconferencing, security and surveillance, video segmentation, and editing. In this chapter, we focus on sequential Bayesian estimation techniques for visual tracking. We first introduce the sequential Bayesian estimation framework, which acts as the theoretic basis for visual tracking. Then, we present approaches to constructing representation models for specific objects.

Video segmentation is different from segmentation of a single image. While several correct solutions may exist for segmenting a single image, there needs to be a consistency among segmentations of each frame for video segmentation. Previous approaches of video segmentation concentrate on motion, or combine motion and color information in a batch fashion. We propose a maximum a posteriori probability (MAP) framework that uses multiple cues, like spatial location, color and motion, for segmentation. We assign weights to color and motion terms, which are adjusted at every pixel, based on a confidence measure of each feature. We also discuss the appropriate modeling of PDFs of each feature of a region. The correct modeling of the spatial PDF imposes temporal consistency among segments in consecutive frames. This approach unifies the strengths of both color segmentation and motion segmentation in one framework, and shows good results on videos that are not suited for either of these approaches.

The segmentation of moving objects in image sequence can be formulated as a background subtraction problem—the separation of objects from the background in each image frame. The background scene is learned and modeled. A pixelwise process is employed to classify each pixel as an object or background based on its similarity with the background model. The segmentation is challenging due to the occurrence of dynamic elements such as illumination change and background motions. We propose a framework for object segmentation with a novel feature for background representation and new mechanisms for updating the background model. A ternary pattern is employed to characterize the local texture. The pattern and photometric features are used for background modeling. The classification of pixel is performed based on the perceptual similarity between the current pixel and the background model. The segmented object is refined by taking into account the spatial consistency of the image feature. For the background model update, we propose two mechanisms that are able to adapt to abrupt background change and also merge new background elements into the model. We compare our framework with various background subtraction algorithms on video datasets.

Background estimation, i.e. automatic recovery of the background image from a sequence of images containing moving foreground objects, is an important module in many applications, e.g. surveillance and video segmentation. In this paper, we present a simple, yet effective and robust approach for background estimation based on loopy belief propagation. Robustness of the proposed approach means: (i) minimal assumption on the input frames, and (ii) no need to tune parameters. Basically, the background can be recovered even when the occluding foreground objects stay still for a long time. Furthermore, no motion information needs to be known or estimated for the foreground objects, which implies that background can be recovered from a set of frames which are not consecutive temporally. Analysis and experiments are provided to compare the proposed approach to related methods. Experimental results on typical surveillance videos demonstrate the effectiveness of our approach.

The new MPEG-4 video coding standard enables content-based functions. In order to support the new standard, frames should be decomposed into Video Object Planes (VOP), each VOP representing a moving object. This paper proposes an image segmentation method to separate moving objects from image sequences. The proposed method utilizes the spatial-temporal information. Spatial segmentation is applied to divide each image into connected areas and to find precise object boundaries of moving objects. To locate moving objects in image sequences, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed with Neyman-Pearson criterion. Spatial segmentation produces a spatial segmentation mask, and temporal segmentation yields a change detection mask that indicates moving objects and the background. Then spatial-temporal merging can be used to get the final results. This method has been tested on several images. Experimental results show that this segmentation method is efficient.

A debris image tracking using optical flow algorithm

A new method for Z-direction object velocity estimation in image sequences is outlined. An extended Kalman filter is used to estimate the Z-direction velocity of an object in image sequences. After motion compensation in the Z-direction, a second 2-D Kalman filter can be used to estimate velocities of the object in the image plane.

The problems of tracking moving target of quad-rotor UAV involve target tracking in image sequence and flight control of UAV. Because Camshift algorithm is easily disturbed by background with similar color and has poor robustness to occlusion interference when tracking objects in image sequence. A new tracking algorithm based on Kalman filter and Camshift algorithm with multi-feature fusion is proposed. On this basis, the horizontal displacement between the target and the UAV is calculated as the control input, and a position-attitude outer-inner loop controller is designed to ensure the target near the center of view of the camera in order to track the target efficiently. Experimental and simulation results verify the performance of the algorithm.

We present a method for motion-based video segmentation and segment classification as a step towards video summarization. The sequential segmentation of the video is performed by detecting changes in the dominant image motion, assumed to be related to camera motion and represented by a 2D affine model. The detection is achieved by analysing the temporal variations of some coefficients of the 2D affine model (robustly) estimated. The obtained video segments supply reasonable temporal units to be further classified. For the second stage, we adopt a statistical representation of the residual motion content of the video scene, relying on the distribution of temporal co-occurrences of local motion-related measurements. Pre-identified classes of dynamic events are learned off-line from a training set of video samples of the genre of interest. Each video segment is then classified according to a Maximum Likelihood criterion. Finally, excerpts of the relevant classes can be selected for video summarization. Experiments regarding the two steps of the method are presented on different video genres leading to very encouraging results while only low-level motion information is considered.

We present a method for segmenting an arbitrary number of moving objects in image sequences using the geometry of 6 points in 2D to infer motion consistency. The method has been evaluated on the Hopkins 155 database and surpasses current state-of-the-art methods such as SSC, both in terms of overall performance on two and three motions but also in terms of maximum errors. The method works by finding initial clusters in the spatial domain, and then classifying each remaining point as belonging to the cluster that minimizes a motion consistency score. In contrast to most other motion segmentation methods that are based on an affine camera model, the proposed method is fully projective.KeywordsImage PointGeneralise Extreme ValueSpectral ClusterMotion SegmentationEpipolar ConstraintThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Three dimensional tracking of rigid objects in motion using 2D optical flows

The Bayesian reconstruction of the shape of an object in image sequences using deformable template techniques is considered. The object shape is represented by a prototype outline which is deformed under constraints of continuity and rigidity. The deformation of the template retains consistent shape characteristics whilst allowing variation and evolution of object shape over time. The reconstruction of single images using deformable templates allows incorporation of more detailed prior knowledge of expected image features than a simple local pixel dependence model. An application to a sequence of CCD animated digital images is given. >

The paper presents a particle filter for integrated detection and tracking of multiple objects in image sequences, using color as the observation feature. The color based feature has been found useful when dealing with non-rigid object deformations, partial occlusions, rapidly changing dynamics and complex backgrounds. The problem of joint detection and tracking is formulated as a hybrid valued sequential state estimation problem, incorporating a discrete variable which represents the number of existing objects. The solution is implemented in the form of a particle filter. The performance of the proposed algorithm is evaluated on various real-world video sequences with objects entering and leaving the scene.