Efficient Motion Re-Estimation With Rate-Distortion Optimization for MPEG-2 to H.264/AVC Transcoding

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

This paper presents an efficient joint disparity-motion estimation algorithm and a fast motion estimation algorithm in an MPEG-4 compatible stereoscopic sequence codec for 3D broadcasting. In stereoscopic sequence coding, the critical problems to be solved are double bandwidth and computational complexity. However, the previous trials to reduce coding bits have increased computational complexity. In this paper, we try to implement fast algorithms for stereoscopic sequences while maintaining high coding efficiency. Joint disparity-motion estimation can increase coding efficiency and reduce the computational complexity of the stereoscopic codec by using the relationship between the disparity and motion fields. The disparity vectors are estimated by using the left and right motion vectors and the previous disparity vectors in each time frame. In order to obtain more accurate disparity vectors, we employ the spatial prediction process after joint estimation. From joint estimation and spatial prediction, we can obtain the accurately predicted disparity vectors and then increase the coding efficiency from those predicted vectors. Moreover, we propose a fast motion estimation algorithm which utilizes correlation between the motion vectors of the neighboring blocks. We confirmed that the PSNR of the proposed method is 1.0-2.0 dB higher than ones of conventional methods. In addition, the processing time is reduced by almost 1/10.

Predictive fast Motion Estimation (ME) algorithms have been widely used in video CODECs due to their performance efficiency and low computational complexity. In this thesis, a new block-based fast motion estimation technique named Dynamic Predictive Search Algorithm (DPSA) is developed, which can be considered in predictive zonal search category. The proposed approach is based on the observation that temporally and spatially adjacent macro-blocks are not just statically correlated, but also dynamic alterations in their motion content are highly coherent. DPSA introduces a new set of six candidate predicted motion vectors. For early termination criteria, DPSA modifies termination procedure of already existing EPZS algorithm. Performance of this newly proposed algorithm has been compared to four other state-of-the-art algorithms implemented on JVT, H.264 standard software platform. Experimental results have proven that DPSA accomplishes up to 38% compression ratio enhancement achieved by a process with more 14.75% less computational complexity and up to0.47 dB higher PSNR values over the EPZS. It also manages to have up to 13% speed up over EPZS algorithm. Because of its simplicity and low computational complexity DPSA is energy efficient for portable video processing in computation- or power-constrained applications and easy to be implemented on both FPGA- and Microcontroller-based embedded systems. Also, higher compression ratio makes DPSA more compatible with limited capacity storage media, and limited band-width transmission networks.

For video compression, motion estimation is popularly employed to exploit temporal correlation existing in video sequences. If we employ the full search block matching algorithm for estimating motion vectors, it requires very heavy computational complexity. Although several fast block matching algorithms have been proposed to solve this problem, they sacrifice their reconstructed image quality. In this paper, we derive optimal search patterns for fast block matching motion estimation. By analyzing the block matching algorithm as a function of the block size and the shape, we find optimal search patterns for initial motion estimation. The proposed idea can provide an analytical ground for the current MPEG-2 proposals. In addition, we propose a new fast motion estimation algorithm using adaptive search patterns, considering matching criteria and statistical properties of object displacement. In order to select an appropriate search pattern, we exploit the relationship between the motion vector and the frame difference of each block. By changing the search pattern adaptively, we can improve the motion prediction accuracy, while reducing the required computational complexity compared to other fast block matching algorithms.

Motion in modern video coders is estimated using a block matching algorithm that calculates the distance and direction of motion on a block-by-block basis. In this paper, a novel fast block-based motion estimation algorithm is proposed. This algorithm uses an efficient projection framework that bounds the distance between a template block and candidate blocks. Fast projection is performed using a family of highly efficient filter kernels--the gray-code kernels--requiring only 2 operations per pixel per kernel. The projection framework is combined with a rejection scheme which allows rapid rejection of candidate blocks that are distant from the template block. The tradeoff between computational complexity and quality of results can be easily controlled in the proposed algorithm; thus, it enables adaptivity to image content to further improve the results. Experiments show that the proposed adaptive algorithm outperforms other popular fast motion estimation algorithms.

There is an increasing need for high quality video on low power, portable devices. Possible target applications range from entertainment and personal communications to security and health care. While H.264/AVC answers the need for high quality video at lower bit rates, it is significantly more complex than previous coding standards and thus results in greater power consumption in practical implementations. In particular, motion estimation (ME), in H.264/AVC consumes the largest power in an H.264/AVC encoder. It is therefore critical to speed-up integer ME in H.264/AVC via fast motion estimation (FME) algorithms and hardware acceleration. In this paper, we present our hardware oriented modifications to a hybrid FME algorithm, our architecture based on the modified algorithm, and our implementation and prototype on a PowerPC-based Field Programmable System on Chip (FPSoC). Our results show that the modified hybrid FME algorithm on average, outperforms previous state-of-the-art FME algorithms, while its losses when compared with FSME, in terms of PSNR performance and computation time, are insignificant. We show that although our implementation platform is FPGA-based, our implementation results compare favourably with previous architectures implemented on ASICs. Finally we also show an improvement over some existing architectures implemented on FPGAs.

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce the computation by limiting the number of searching locations. This is accomplished at the expense of less accuracy of motion estimation and gives rise to an appreciably higher mean squared error (MSE) for motion compensated images. We present a new fast and efficient search algorithm for block motion estimation that produces better quality performance and less computational time compared with a three-step search (TSS) algorithm. The proposed algorithms are based on the ideas of the dithering pattern for pixel decimation, multiple-candidate for pixel-decimation-based full search, and center-based distribution of the motion vector. From the experimental results, the proposed algorithm is superior to the TSS in both quality performance (about 0.2 dB) and computational complexity (about half).

H.264/AVC is the outstanding and significant video compression standard developed by ITU-T/ISO/IEC Joint Video Team (JVT). Motion estimation (ME) plays a key role in H.264/AVC, it concerns greatly on computational complexity especially when using the full search (FS) algorithm. Although many fast ME algorithms have been proposed to reduce the huge calculation complexity instead of FS, the ME still can not satisfy the critical real-time applicationpsilas needs. In this paper, a fast integer pixel variable block motion estimation algorithm based on JVT which accepted UMHexagonS algorithm is presented for H.264/AVC encoder. With special considerations on the motion activity of the current macro-block, several techniques, i.e., adaptive search strategies have been utilized to significantly improve the video coding performance. The simulation results analysis shows that the proposed algorithm maintains an unnoticeable quality loss in terms of PSNR on average compared with FS and reduces nearly 20% motion estimation time compared with UMHexagonS while maintaining coding efficiency.

In single viewpoint video coding, there are many fast block matching motion estimation algorithms proposed, such as three-step search, four-step search, diamond search as well as hexagon-based search and so on. However, experimental analysis show that these algorithms are not suitable for using directly in Multiview View Coding (MVC). Since the increased search range, the larger format of multiview video as well as the correlations between inter-view frames are not considered by these algorithms, they may easy led the block matching search into local minimum, the Rate-Distortion (R-D) performance will degrade dramatically. In this paper, we propose a novel multiple hexagon search algorithm to address this problem. Firstly, according to the original initial search point, four sub-search windows are constructed. Then, the hexagon based search algorithm will be performed respectively in the four sub-search windows. The final result is the best search point with the minimum R-D cost among the best points in the four sub-search windows. In order to trade off the computational complexity and R-D performance, two adaptive early termination strategies are proposed. The experimental results show that the proposed algorithm yields a quite promising coding performance in terms of R-D performance and computational complexity. Especially, the proposed algorithm can work well in multiview video sequences with various motion and disparity activities.

This paper discusses the design of a fast algorithm for motion estimation with emphasis on hardware cost considerations, real-time application, network adaptation, and flexibility. To achieve the best trade-off among hardware cost, computational complexity, and distortion performance, we propose a multi-stage pixel-subsampling motion estimation algorithm. The algorithm has a lower hardware cost than Liu's subsampling algorithm and the three-step hierarchical search algorithm (3SHS) in terms of data flow control, I/O bandwidth, and regularity. Its computational complexity is close to that of 3SHS and its distortion performance, which is better than that of Liu's algorithm and 3SHS, is close to that of full search.

Fast search motion estimation algorithms when compared to full search motion estimation algorithms often converge to a local minimum, providing a momentous reduction in computational cost. However, the motion vectors measurement process in fast search algorithms is subject to noise and matching errors. Researchers have investigated the use of mathematical tools used for stochastic estimation from noisy measurements, to seek optimal estimates. Amongst those tools, is the conventional Kalman filtering, which addresses the general problem of trying to estimate the state of a discrete-time controlled process that is governed by the linear stochastic difference. This research investigates the possible combinations of benchmark motion estimation algorithms and the Kalman filter. In this paper, the author presents an in-depth investigation and a detailed analysis on the use of the above combination, and seeks to establish conditions under which the application would be successful. Experimental results show that the above is possible only under certain conditions and constraints of certain properties of the video sequences being coded. Furthermore, a recommendation is made on when it is possible to use the adaptive Kalman filter instead of the conventional filter to enhance the motion vectors at the cost of extra complexity.

A fast and efficient hybrid block-matching motion estimation algorithm is proposed in this paper. The fact that motion characteristics are relatively stable in a GOP (group of pictures) is utilized to reduce the computation burden without quality degradation. For P-frames, a multi-resolution block-matching algorithm is used. At the coarsest level, edge information is used to reduce searching positions before an exhaustive search. At the finer levels, only the several candidates chosen at the upper level are verified. For B-frames, MAE (mean absolute error) is used not only for the matching criteria, but also for classifying the macroblocks with the motion information extracted from P-frames. Concurrently, the search window size is adjusted using the contribution of different motion range to the improvement of the picture quality with motion compensation. When the decoded I-frame and P-frames are used as the reference, experimental results show that reconstructed digital frames have even better quality than the full search algorithm in terms of PSNR, under MPEG-1 coding environment.

Most of the fast search motion estimation algorithms reduce the computational cost of motion estimation greatly by checking only a few search points inside the search area by using full distortion measure. This paper proposes a multi-layer motion estimation which employs partial distortion as its distortion measure to reduce the number of computations involved in each search point instead of reducing the number of search points. A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer. It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm.

In the new H.264/AVC video coding standard, motion estimation takes up a significant encoding time, especially when using the straightforward full search algorithm (FS). A fast flexible multi-frame motion estimation algorithm with adaptive search strategies (FMASS) is presented. With special considerations on the multiple reference frames and block modes, several techniques, i.e., adaptive search strategies for single frame and flexible multi-frame selection, have been utilized to improve significantly the speed-up performance in H.264. Extensive simulations show that it can minimize the matching points by more than 1190 times compared with FS. In addition, the output quality of the encoded sequences loses only 0.053 dB in terms of PSNR on average. Fast speed-up performance and unnoticeable quality losses make the proposed algorithm outperform most of the other well-known algorithms proposed recently, such as ARPS3, MVFAST and UMHexagonS, of which the latter two have been already accepted by MPEG-4 and JVT respectively.

A fast half pixel motion estimation algorithm based on spatio-temporal correlations is proposed to reduce the computational complexity. According to spatially and temporally correlated information, the proposed method decides whether half pixel motion estimation is skipped or not for the current block. Experimental results show that the proposed method outperforms most of current methods in computational complexity by reducing the number of search points with little degradation in image quality. When compared to full half pixel search method, the proposed algorithm achieves the search point reduction up to 95% with only 0.01 ~ 0.06 (dB) degradation of image quality.