Improvement algorithm of video coding efficiency using pre-filtering and post-filtering based on global motion compensation

Current Moving Object Detectors in airborne Region of Interest (ROI) coding systems for police surveillance applications used on-board of UAVs are often based on Global Motion Estimation (GME) techniques. Since in these scenarios the camera is moving, simple background removal approaches cannot be applied without a Global Motion Compensation (GMC). Common GMC algorithms assume the ground to be planar, allowing the pixels of the previous frame to be motion compensated into the current frame by applying a projective transformation. The difference image between the compensated frame and the current frame emphasis regions containing possible motion. Such moving object detectors are great in terms of run-time efficiency but are known to lack in terms of accuracy - especially for unstructured regions of moving objects - as well as the robustness against noise. Superpixel segmentation was recently proposed to overcome the issue of the imprecise region cuts given by the difference image. It provides a greatly improved true positive detection rate, but unintentionally also increases the area of false positives. This paper proposes the use of a mesh-based GME and GMC to detect the moving object regions wherein a cluster filter eliminates errors in the optical flow by assuming a smooth vector field as the global motion model. In doing so we improve the coding efficiency of the fully automatic ROI coding system by more than 24% for moving object areas conserving the detection benefits of the integration of superpixel segmentation.

A novel motion representation and estimation scheme is presented that leads to improved coding efficiency of block-based video coders like, e.g., H.263. The proposed scheme is based on motion-compensated prediction from more than one reference frame. The reference pictures are warped or globally motion-compensated versions of the previous frame. Affine motion models are used as warping parameters approximating the motion vector field. Motion compensation is performed using standard block matching in the multiple reference frames buffer. The frame reference and the affine motion parameters are transmitted as side information. The approach is incorporated into an H.263-based video codec at minor syntax changes and embedded into a rate-constrained motion estimation and macroblock mode decision frame work. In contrast to conventional global motion compensation, where one motion model is transmitted, we show that multiple global motion models are of benefit in terms of coding efficiency. Significant coding gains in comparison to TMN-10, the test model of H.263+, are achieved that provide bit-rate savings between 20% and 35% for the various sequences tested

Global motion estimation (GME) and compensation is one of the key modules in MPEG-4 Advanced Simple Profile (ASP). However, there are no hardware architectures for GME since existing algorithms are not suitable for hardware implementation. In this paper, GME in MPEG-4 ASP is analyzed, and a hardware-oriented GME algorithm is proposed according to the analysis results, which is a combination of a feature points based algorithm and a novel iterative sprite point matching algorithm. The associated hardware architecture is also proposed. Simulation results show that the performance of the proposed algorithm is the same as that of MPEG-4 Verification Model. The implementation results show the proposed hardware architecture can achieve the requirements of MPEG-4 ASP@L3 with 66K gates and 31Kb internal memory at 100 MHz working frequency.

A motion compensated de-interlacing method with adaptive global motion estimation and compensation is proposed to recover the defects of interlaced video sequence with camera panning, rotating or zooming. GME and GMC are used to recover the change of the whole picture due to camera motions. Two local motion compensated de-interlacing methods are proposed and applied to de-interlace the interlaced video sequences with or without global motion respectively. SAD checking and global/local motion comparator can be used as a block based mode decision system for intra/MC/GMC modes. The proposed algorithm could achieve higher image quality of interlaced video sequences than any other usual de-interlacing algorithm on progressive devices.

In Versatile Video Coding (VVC), local affine motion compensation (LAMC) is adopted to handle complex motions, such as rotation and zooming. However, it is inefficient to use LAMC to handle the global motion due to the following two reasons. First, the use of LAMC may lead to some extra bit cost on the affine motion model parameters. Second, the precision of LAMC is restricted by the MV precision of the control points. Therefore, in this paper, we propose a global homography motion compensation (GHMC) framework to better characterize the global motion. For each coding block, an extra mode is added to perform motion compensation based on an 8-parameter global homography motion model. In addition, an extrapolation scheme is designed to derive the parameters from reference frames to save the bit cost for signaling them. The proposed framework is implemented into the VVC reference software VTM-6.0. Experimental results show that, on average, 0.69% and 0.66% BD-rate reduction is achieved under Low Delay P and Low Delay B configurations, respectively, for sequences with rich complex global motions.

We present a video object extraction algorithm combining image partition with motion detection. It consists of three stages: spatial image partition (SIP), temporal motion detection (TMD) and spatiotemporal projection (STP). Firstly, we partition the current video frame into a series of image regions by a modified watershed transform with double open-closing reconstruction and non-linear pixel classification, and the corresponding region boundaries are achieved. In the next stage, global motion estimation and compensation (GME & GMC) are performed, then motion detection based on Gaussianity test is applied to pixels on region boundaries, not to all pixels belonging to regions. In final stage, motion mask is projected to the current frame to segmenting video objects. Preliminary simulation results demonstrate the performance of the proposed algorithm.

A novel video salient object extraction method based on visual attention

For the defect of manual detection of Sulfur hexafluoride(SF 6 ) gas leakage in laser imaging video, this paper proposes an efficient and automatic gas leakage detection method. Firstly, global motion estimation and compensation are performed in laser imaging video, secondly, accumulative frame-difference and an improved high-order statistics(HOS) algorithm are used to detect leakage area, finally, the leakage area is located in laser imaging video. The experimental results show that the proposed method can detect SF 6 gas leakage area from the laser imaging video sequence with unobvious motion area and serious noise, thus realize accurate location of gas leakage area.

We present an algorithm for face 3D motion estimation in videoconference scenes. The algorithm uses a modification of the CANDIDE face model and is based on feature tracking and the extended Kalman filter (EKF). Various techniques are adopted to increase the robustness of the feature tracking procedure and, in particular, a filtering technique on reference blocks. Global motion estimation is used as a starting point for local motion detection. To this purpose, we generate, by texture mapping, a synthetic image of the mouth, whose shape is changed using a set of action units (AU). The optimal AU values are determined via a gradient-based minimization procedure of the error energy between the template and the actual mouth image. The proposed scheme is quite robust and was tested with success on long sequences.

Tracking a face for knowledge-based coding of videophone sequences

Pixel-based and motion vector-based global motion estimation (GME) techniques are evaluated in this paper with an automatic system for camera motion characterization. First, the GME techniques are compared with a frame-by-frame PNSR measurement using five video sequences. The best motion vector-based GME method is then evaluated together with a common and a simplified pixel-based GME technique for camera motion characterization. For this, selected unedited videos from the TRECVid 2005 BBC rushes corpus are used. We evaluate how the estimation accuracy of global motion parameters affects the results for camera motion characterization in terms of retrieval measures. The results for this characterization show that the simplified pixel-based GME technique obtains results that are comparable with the common pixel-based GME method, and outperforms significantly the results of an earlier proposed motion vector-based GME approach.

One important recent application of image registration has been in the estimation of global motion parameters for object-based video coding. A limitation of current global motion estimation approaches is the additional complexity of the gradient-descent optimization that is typically required to calculate the optimal set of global motion parameters. In this paper we propose a new low-complexity algorithm for global motion estimation. The complexity of the proposed algorithm is reduced by performing the majority of the operations in the gradient-descent optimization using logic operations rather than full-precision arithmetic operations. This use of logic operations means that the algorithm can be implemented much more easily in hardware platforms such as field programmable gate arrays (FPGAs). Experimental results show that the execution time for software implementations of the new algorithm is reduced by a factor of almost four when compared to existing fast implementations without any significant loss in registration accuracy.

Digital video stabilization is a cost-effective way to reduce the effect of camera shake in handheld video cameras. We propose several enhancements for video stabilization based on integral projection matching,1 which is a simple and efficient global motion estimation technique for translational motion. One-dimensional intensity projections along the horizontal and vertical axes provide a signature of the image. Global motion estimation aims at finding the largest similarity between shifted intensity projections between consecutive frames. The obtained shifts provide information about the global inter-frame motion. Relying upon the estimated global motion an output frame of reduced size is determined deploying motion smoothing. We propose several enhancements of prior works to improve the stabilization performance and to reduce computational complexity and memory requirements. The main enhancement is a partitioning of the projection intensities to better cope with in-scene motion. Logarithmic search is deployed to seek for a minimum matching error for selected partitions in two subsequent frames. Furthermore we propose a novel motion smoothing approach we call center-attracted motion damping. We evaluate the performance of the enhancements under various imaging conditions using real video sequences as well as synthetic video sequences with provided ground-truth motion. The stabilization accuracy is sufficient under most imaging conditions so that the effect of camera shake is eliminated or significantly reduced in the stabilized video.

Chapter 6 - Multiple-Reference Motion Estimation Techniques

In this paper, an efficient and fast global motion estimation method is proposed. This method is a generic one that can accommodate various global motion models, ranging from a simple translation model to an eight parameters perspective model. At the first stage, a simple filter is implemented to classify different parts of the frames. Since pixels from homogeneous regions and highly textured regions do not contribute much in global motion estimation, it is proposed that they-be excluded from processing. The remained pixels are fed into a three-level hierarchical motion estimation system and a dynamic sprite is eventually generated. From the experimental results, we can conclude that the proposed global motion estimation system manages to speed up the processing time for sprite generation while maintaining quality in the reconstructed frames.