Feature based global and local motion estimation for videoconference sequences

One important recent application of image registration has been in the estimation of global motion parameters for object-based video coding. A limitation of current global motion estimation approaches is the additional complexity of the gradient-descent optimization that is typically required to calculate the optimal set of global motion parameters. In this paper we propose a new low-complexity algorithm for global motion estimation. The complexity of the proposed algorithm is reduced by performing the majority of the operations in the gradient-descent optimization using logic operations rather than full-precision arithmetic operations. This use of logic operations means that the algorithm can be implemented much more easily in hardware platforms such as field programmable gate arrays (FPGAs). Experimental results show that the execution time for software implementations of the new algorithm is reduced by a factor of almost four when compared to existing fast implementations without any significant loss in registration accuracy.

In this paper, an efficient and fast global motion estimation method is proposed. This method is a generic one that can accommodate various global motion models, ranging from a simple translation model to an eight parameters perspective model. At the first stage, a simple filter is implemented to classify different parts of the frames. Since pixels from homogeneous regions and highly textured regions do not contribute much in global motion estimation, it is proposed that they-be excluded from processing. The remained pixels are fed into a three-level hierarchical motion estimation system and a dynamic sprite is eventually generated. From the experimental results, we can conclude that the proposed global motion estimation system manages to speed up the processing time for sprite generation while maintaining quality in the reconstructed frames.

Pixel-based and motion vector-based global motion estimation (GME) techniques are evaluated in this paper with an automatic system for camera motion characterization. First, the GME techniques are compared with a frame-by-frame PNSR measurement using five video sequences. The best motion vector-based GME method is then evaluated together with a common and a simplified pixel-based GME technique for camera motion characterization. For this, selected unedited videos from the TRECVid 2005 BBC rushes corpus are used. We evaluate how the estimation accuracy of global motion parameters affects the results for camera motion characterization in terms of retrieval measures. The results for this characterization show that the simplified pixel-based GME technique obtains results that are comparable with the common pixel-based GME method, and outperforms significantly the results of an earlier proposed motion vector-based GME approach.

An efficient motion estimator with application to medical image registration

Global motion generally describes the motion of the camera, although it may comprise large object motion. The region of support for global motion representation consists of the entire image frame. Therefore, estimating global motion parameters tends to be computationally costly due to the involvement of all the pixels in the calculation. Efficient global motion estimation (GME) techniques are sought after in many applications such as video coding, image stabilization and super-resolution. In this paper, we propose to select only a small subset of the pixels in estimating the global motion parameters, based on a combination of fixed and random subsampling patterns. Simulation results demonstrate that the proposed method was able to speed up the conventional all-pixel GME approach by up to 7 times, without significant loss in the estimation accuracy. The combined subsampling patterns were also found to provide better motion estimation accuracy/complexity tradeoffs than those achievable by using either fixed or random patterns alone.

We present a novel technique for the estimation of global nonrigid motion of an object's boundary without using point correspondences. A complete description of the motion of an object's boundary involves specifying a displacement vector at each point of the boundary. Such a description provides a large amount of information, which needs to be processed further for study of the global characteristics of the deformation. Nonrigid motion can be studied hierarchically in terms of global nonrigid motion and point-by-point local nonrigid motion. The technique presented gives a method for estimating a global affine or polynomial transformation between two object boundaries. The novelty of the technique lies in the fact that it does not use any point correspondences. Our method uses hyperquadric models to model the data and estimate the global deformation. We show that affine or polynomial transformation between two datasets can be recovered from the hyperquadric parameters. The usefulness of the technique is twofold. First, it paves the way for viewing nonrigid motion hierarchically in terms of global and local motion. Second, it can be used as a front end to other motion analysis techniques that assume small motion. For instance, most nonrigid motion analysis algorithms make some assumptions on the type of nonrigid motion (conformal motion, small motion, etc.) that are not always satisfied in practice. When the motion between two datasets is large, our algorithm can be used to estimate the affine transformation (which includes scale and shear) or a polynomial transformation between the two datasets, which can then be used to warp the first dataset closer to the second so as to satisfy the small-motion assumption. We present experimental results with real and synthetic two- and three-dimensional data.

The mosaic generation methods benefit from global motion estimation (GME) methods, which yield almost accurate estimation of motion parameters. However, the generated mosaics are usually more blurred than the original frames due to the image warping stage and errors in motion estimation. The transformed coordinates resulting from GME are generally real numbers whereas images are sampled into integer values. Although GME methods generate proper motion parameters, a slight error in motion estimation may propagate to subsequent mosaic generation steps. We propose a method to generate clearer mosaics from video. The temporal integration of images is performed using the histemporal filter based on the histogram of values within an interval. The initial frame in the video sequence is registered at a higher resolution to generate a high resolution mosaic. Instead of warping of each frame, the frames are warped into the mosaic at intervals. This reduces the blurring in the mosaic.

Sprite coding is a very efficient method for the representation and compression of background video object, which has been accepted by MPEG-4 standard. However, how to automatically generate sprite image from raw video sequence is still an open issue, because the motion of foreground video objects not only disturbs the accuracy of motion estimation but also blurs the generated sprite image. This paper presents a novel technique for the generation of background sprite image with improved global motion estimation (GME) and automatic segmentation. Since GME plays an important role in sprite generation, a hybrid scheme jointing the short-term and long-term motion estimation techniques is first developed in this paper. Automatic video segmentation based on available motion information is incorporated into sprite generation, which can mask moving foreground objects and meanwhile protect background sprite from blur. Therefore, the proposed technique is used to construct high quality sprite directly from a raw video sequence. Furthermore, with this technique the existing sprite coding can be extended to the field of frame-based coding.

This paper proposes a filtering approach based on global motion estimation (GME) and global motion compensation (GMC) as pre-processing and post-processing for video CODEC. For the pre-processing of video CODEC, group-of-pictures (GOP), i.e., basic unit for GMC and reference frames are first defined for an input video sequence. Next, GME and GMC are sequentially performed for every frame in each GOP. Finally, block-based adaptive temporal filter is applied between global motion compensated frames prior to video encoding. For the post-processing at the end of decoder, every decoded frame is inversely motion-compensated by using the transmitted global motion information. The holes which are generated during inverse motion compensation can be filled using the reference frames. Experimental results the proposed algorithm provides higher BD-PSNRs than before-use on H.264 and HEVC platforms.

Digital video stabilization is a cost-effective way to reduce the effect of camera shake in handheld video cameras. We propose several enhancements for video stabilization based on integral projection matching,1 which is a simple and efficient global motion estimation technique for translational motion. One-dimensional intensity projections along the horizontal and vertical axes provide a signature of the image. Global motion estimation aims at finding the largest similarity between shifted intensity projections between consecutive frames. The obtained shifts provide information about the global inter-frame motion. Relying upon the estimated global motion an output frame of reduced size is determined deploying motion smoothing. We propose several enhancements of prior works to improve the stabilization performance and to reduce computational complexity and memory requirements. The main enhancement is a partitioning of the projection intensities to better cope with in-scene motion. Logarithmic search is deployed to seek for a minimum matching error for selected partitions in two subsequent frames. Furthermore we propose a novel motion smoothing approach we call center-attracted motion damping. We evaluate the performance of the enhancements under various imaging conditions using real video sequences as well as synthetic video sequences with provided ground-truth motion. The stabilization accuracy is sufficient under most imaging conditions so that the effect of camera shake is eliminated or significantly reduced in the stabilized video.

This paper presents an image-based velocity control method for a VTOL UAV. For increased autonomy, a realtime embedded global motion estimation process using efficient feature tracking was developed. The VTOL vehicle is assumed to be equipped with a minimum sensor suite — i.e. a camera, altitude sensor and IMU — and to hover over a featured flat plane. This paper first presents a fast feature-based global motion estimation method, and then proposes a control strategy based on this method. Finally, experimental results using a quadrotor demonstrate the performance of the proposed control strategy and feature-based motion estimation.

In this paper, we present an improved approach to predictive video decoding based on global and local motion reliability. The framework consists of three processing stages. Global motion (GM) estimation and motion reliability analysis are the key components in the first stage, where we model global motion and refine the MV field. In the second stage, we predict local and global motion for the target frame, and determine corresponding weights based on the Linear Minimum Mean Square Error (LMMSE) criterion. Finally in the third stage, a temporal interpolator is applied to compose two future frames, which are linearly combined to form the final predicted frame. Our results indicate the proposed method achieves better visual quality compare to other state-of-the-art predictive decoding approaches, particularly in sequences involving moving camera and objects.

The growing use of cameras embedded in autonomous robotic platforms and worn by people is increasing the importance of accurate global motion estimation (GME). However, existing GME methods may degrade considerably under illumination variations. In this paper, we address this problem by proposing a biologically-inspired GME method that achieves high estimation accuracy in the presence of illumination variations. We mimic the early layers of the human visual cortex with the spatio-temporal Gabor motion energy by adopting the pioneering model of Adelson and Bergen and we provide the closed-form expressions that enable the study and adaptation of this model to different application needs. Moreover, we propose a normalisation scheme for motion energy to tackle temporal illumination variations. Finally, we provide an overall GME scheme which, to the best of our knowledge, achieves the highest accuracy on the Pose, Illumination, and Expression (PIE) database.

Most global motion estimation (GME) methods are oriented to video coding while video object segmentation methods either assume no global motion (GM) or directly adopt a coding-oriented method to compensate for GM. This paper proposes a hierarchical differential GME method oriented to video object segmentation. A scheme which combines three-step search and motion parameters prediction is proposed for initial estimation to increase efficiency. A robust estimator that uses object information to reject outliers introduced by local motion is also proposed. For the first frame, when the object information is unavailable, a robust estimator is proposed which rejects outliers by examining their distribution in local neighborhoods of the error between the current and the motion-compensated previous frame. Subjective and objective results show that the proposed method is more robust, more oriented to video object segmentation, and faster than the referenced methods.

Global motion estimation (GME) and compensation is one of the key modules in MPEG-4 Advanced Simple Profile (ASP). However, there are no hardware architectures for GME since existing algorithms are not suitable for hardware implementation. In this paper, GME in MPEG-4 ASP is analyzed, and a hardware-oriented GME algorithm is proposed according to the analysis results, which is a combination of a feature points based algorithm and a novel iterative sprite point matching algorithm. The associated hardware architecture is also proposed. Simulation results show that the performance of the proposed algorithm is the same as that of MPEG-4 Verification Model. The implementation results show the proposed hardware architecture can achieve the requirements of MPEG-4 ASP@L3 with 66K gates and 31Kb internal memory at 100 MHz working frequency.