Efficient Global Motion Estimation using Fixed and Random Subsampling Patterns

Digital video stabilization is a cost-effective way to reduce the effect of camera shake in handheld video cameras. We propose several enhancements for video stabilization based on integral projection matching,1 which is a simple and efficient global motion estimation technique for translational motion. One-dimensional intensity projections along the horizontal and vertical axes provide a signature of the image. Global motion estimation aims at finding the largest similarity between shifted intensity projections between consecutive frames. The obtained shifts provide information about the global inter-frame motion. Relying upon the estimated global motion an output frame of reduced size is determined deploying motion smoothing. We propose several enhancements of prior works to improve the stabilization performance and to reduce computational complexity and memory requirements. The main enhancement is a partitioning of the projection intensities to better cope with in-scene motion. Logarithmic search is deployed to seek for a minimum matching error for selected partitions in two subsequent frames. Furthermore we propose a novel motion smoothing approach we call center-attracted motion damping. We evaluate the performance of the enhancements under various imaging conditions using real video sequences as well as synthetic video sequences with provided ground-truth motion. The stabilization accuracy is sufficient under most imaging conditions so that the effect of camera shake is eliminated or significantly reduced in the stabilized video.

One important recent application of image registration has been in the estimation of global motion parameters for object-based video coding. A limitation of current global motion estimation approaches is the additional complexity of the gradient-descent optimization that is typically required to calculate the optimal set of global motion parameters. In this paper we propose a new low-complexity algorithm for global motion estimation. The complexity of the proposed algorithm is reduced by performing the majority of the operations in the gradient-descent optimization using logic operations rather than full-precision arithmetic operations. This use of logic operations means that the algorithm can be implemented much more easily in hardware platforms such as field programmable gate arrays (FPGAs). Experimental results show that the execution time for software implementations of the new algorithm is reduced by a factor of almost four when compared to existing fast implementations without any significant loss in registration accuracy.

Pixel-based and motion vector-based global motion estimation (GME) techniques are evaluated in this paper with an automatic system for camera motion characterization. First, the GME techniques are compared with a frame-by-frame PNSR measurement using five video sequences. The best motion vector-based GME method is then evaluated together with a common and a simplified pixel-based GME technique for camera motion characterization. For this, selected unedited videos from the TRECVid 2005 BBC rushes corpus are used. We evaluate how the estimation accuracy of global motion parameters affects the results for camera motion characterization in terms of retrieval measures. The results for this characterization show that the simplified pixel-based GME technique obtains results that are comparable with the common pixel-based GME method, and outperforms significantly the results of an earlier proposed motion vector-based GME approach.

In this paper, an efficient and fast global motion estimation method is proposed. This method is a generic one that can accommodate various global motion models, ranging from a simple translation model to an eight parameters perspective model. At the first stage, a simple filter is implemented to classify different parts of the frames. Since pixels from homogeneous regions and highly textured regions do not contribute much in global motion estimation, it is proposed that they-be excluded from processing. The remained pixels are fed into a three-level hierarchical motion estimation system and a dynamic sprite is eventually generated. From the experimental results, we can conclude that the proposed global motion estimation system manages to speed up the processing time for sprite generation while maintaining quality in the reconstructed frames.

We present an algorithm for face 3D motion estimation in videoconference scenes. The algorithm uses a modification of the CANDIDE face model and is based on feature tracking and the extended Kalman filter (EKF). Various techniques are adopted to increase the robustness of the feature tracking procedure and, in particular, a filtering technique on reference blocks. Global motion estimation is used as a starting point for local motion detection. To this purpose, we generate, by texture mapping, a synthetic image of the mouth, whose shape is changed using a set of action units (AU). The optimal AU values are determined via a gradient-based minimization procedure of the error energy between the template and the actual mouth image. The proposed scheme is quite robust and was tested with success on long sequences.

An efficient motion estimator with application to medical image registration

In this paper, we present an improved approach to predictive video decoding based on global and local motion reliability. The framework consists of three processing stages. Global motion (GM) estimation and motion reliability analysis are the key components in the first stage, where we model global motion and refine the MV field. In the second stage, we predict local and global motion for the target frame, and determine corresponding weights based on the Linear Minimum Mean Square Error (LMMSE) criterion. Finally in the third stage, a temporal interpolator is applied to compose two future frames, which are linearly combined to form the final predicted frame. Our results indicate the proposed method achieves better visual quality compare to other state-of-the-art predictive decoding approaches, particularly in sequences involving moving camera and objects.

Sprite coding is a very efficient method for the representation and compression of background video object, which has been accepted by MPEG-4 standard. However, how to automatically generate sprite image from raw video sequence is still an open issue, because the motion of foreground video objects not only disturbs the accuracy of motion estimation but also blurs the generated sprite image. This paper presents a novel technique for the generation of background sprite image with improved global motion estimation (GME) and automatic segmentation. Since GME plays an important role in sprite generation, a hybrid scheme jointing the short-term and long-term motion estimation techniques is first developed in this paper. Automatic video segmentation based on available motion information is incorporated into sprite generation, which can mask moving foreground objects and meanwhile protect background sprite from blur. Therefore, the proposed technique is used to construct high quality sprite directly from a raw video sequence. Furthermore, with this technique the existing sprite coding can be extended to the field of frame-based coding.

Current Moving Object Detectors in airborne Region of Interest (ROI) coding systems for police surveillance applications used on-board of UAVs are often based on Global Motion Estimation (GME) techniques. Since in these scenarios the camera is moving, simple background removal approaches cannot be applied without a Global Motion Compensation (GMC). Common GMC algorithms assume the ground to be planar, allowing the pixels of the previous frame to be motion compensated into the current frame by applying a projective transformation. The difference image between the compensated frame and the current frame emphasis regions containing possible motion. Such moving object detectors are great in terms of run-time efficiency but are known to lack in terms of accuracy - especially for unstructured regions of moving objects - as well as the robustness against noise. Superpixel segmentation was recently proposed to overcome the issue of the imprecise region cuts given by the difference image. It provides a greatly improved true positive detection rate, but unintentionally also increases the area of false positives. This paper proposes the use of a mesh-based GME and GMC to detect the moving object regions wherein a cluster filter eliminates errors in the optical flow by assuming a smooth vector field as the global motion model. In doing so we improve the coding efficiency of the fully automatic ROI coding system by more than 24% for moving object areas conserving the detection benefits of the integration of superpixel segmentation.

This paper proposes a filtering approach based on global motion estimation (GME) and global motion compensation (GMC) as pre-processing and post-processing for video CODEC. For the pre-processing of video CODEC, group-of-pictures (GOP), i.e., basic unit for GMC and reference frames are first defined for an input video sequence. Next, GME and GMC are sequentially performed for every frame in each GOP. Finally, block-based adaptive temporal filter is applied between global motion compensated frames prior to video encoding. For the post-processing at the end of decoder, every decoded frame is inversely motion-compensated by using the transmitted global motion information. The holes which are generated during inverse motion compensation can be filled using the reference frames. Experimental results the proposed algorithm provides higher BD-PSNRs than before-use on H.264 and HEVC platforms.

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

We present a novel technique for the estimation of global nonrigid motion of an object's boundary without using point correspondences. A complete description of the motion of an object's boundary involves specifying a displacement vector at each point of the boundary. Such a description provides a large amount of information, which needs to be processed further for study of the global characteristics of the deformation. Nonrigid motion can be studied hierarchically in terms of global nonrigid motion and point-by-point local nonrigid motion. The technique presented gives a method for estimating a global affine or polynomial transformation between two object boundaries. The novelty of the technique lies in the fact that it does not use any point correspondences. Our method uses hyperquadric models to model the data and estimate the global deformation. We show that affine or polynomial transformation between two datasets can be recovered from the hyperquadric parameters. The usefulness of the technique is twofold. First, it paves the way for viewing nonrigid motion hierarchically in terms of global and local motion. Second, it can be used as a front end to other motion analysis techniques that assume small motion. For instance, most nonrigid motion analysis algorithms make some assumptions on the type of nonrigid motion (conformal motion, small motion, etc.) that are not always satisfied in practice. When the motion between two datasets is large, our algorithm can be used to estimate the affine transformation (which includes scale and shear) or a polynomial transformation between the two datasets, which can then be used to warp the first dataset closer to the second so as to satisfy the small-motion assumption. We present experimental results with real and synthetic two- and three-dimensional data.

This was a prospective pilot study to investigate the global motion characteristics of the spondylolysis and spondylolisthesis populations. The aim of this study is to determine the influence of a spondylolisthesis or a spondylolysis on global spinal motion and to establish whether this is dependent on the cause of the slip or the degree/grade of the slip. The condition of spondylolisthesis has been extensively discussed in the literature with respect to its etiology and management. However, the mechanics and movement of the spine in relation to pathology and the effect of this condition on function have received scant attention. The motion of the lumbar spine was investigated in 31 patients (19 men, 12 women, mean age 47.7 +/- 17.8 years) who were diagnosed as having either a lumbar spondylolysis or a spondylolisthesis. These patients were compared with a preexisting database of 203 normal subjects (100 men, 103 women, mean age 39.8 +/- 13.4 years). Patients were graded according to the type of spondylolisthesis or spondylolysis they had, and the extent of slip was rated using Meyerding's classification (1932) and measured directly using methods of Boxall et al (1979) and Wiltse et al (1983). Direct measurements of the extent of slip using Boxall et al (1979) and Wiltse et al (1983) methods were found to have no significant correlation with the resultant range of motion (ROM) or the speed of movement. This study suggests that motion parameters are influenced by the grade of slip in patients with spondylolisthesis, and the type of spondylolisthesis i.e., whether isthmic or degenerative. In the A-P flexion-extension plane, the results indicate that subjects with a defect only, i.e., a spondylolysis, and thus no slip present with a spinal hypermobility (P < 0.01). Subjects with an isthmic slip tend to be either slightly hypermobile or within the anticipated range of motion, whereas those subjects with a degenerative slip tend to be hypomobile (P < 0.05). Movements into lateral flexion were restricted in both the isthmic and degenerative spondylolisthesis patients, whereas rotation was only influenced by the level at which the defect occurred. In terms of degree of displacement, in higher grades of displacement, there was a trend towards hypermobility. The findings of this study suggest that the grade and type of spondylolisthesis do influence global motion parameters. This information may be useful in the clinical assessment of this patient group.

In recent years, global motion estimation (GME) has become an important tool in the fields of video coding, video compression and computer vision. Estimating the correct global motion is often more difficult when the video scene contains large foreground objects. Previous approaches to addressing this problem have required the application of algorithm parameters that are sequence dependent or give inconsistent results for different video sequences. In this paper, we propose a fully automatic approach that can successfully estimate global motion in the presence of large foreground objects. The proposed algorithm determines an initial estimate of the foreground pixels and then reduces the effect of the remaining foreground by using a modified Lorentzian estimator. Experimental results show the proposed method produces superior and more consistent performance than some recent approaches for a wide range of sequences.

In Versatile Video Coding (VVC), local affine motion compensation (LAMC) is adopted to handle complex motions, such as rotation and zooming. However, it is inefficient to use LAMC to handle the global motion due to the following two reasons. First, the use of LAMC may lead to some extra bit cost on the affine motion model parameters. Second, the precision of LAMC is restricted by the MV precision of the control points. Therefore, in this paper, we propose a global homography motion compensation (GHMC) framework to better characterize the global motion. For each coding block, an extra mode is added to perform motion compensation based on an 8-parameter global homography motion model. In addition, an extrapolation scheme is designed to derive the parameters from reference frames to save the bit cost for signaling them. The proposed framework is implemented into the VVC reference software VTM-6.0. Experimental results show that, on average, 0.69% and 0.66% BD-rate reduction is achieved under Low Delay P and Low Delay B configurations, respectively, for sequences with rich complex global motions.