Exploiting Reference Frame History in H.264/AVC Motion Estimation

Motion estimation using multiple reference frames is widely used as the basis for recent video coding standards (eg. H.264/AVC) to achieve increased coding efficiency. However, this increases the complexity of the encoding process. In this paper, a new technique for efficient motion estimation is proposed. A combination of multiple reference frame selection and image residue-based mode selection is used to improve motion estimation time. By dynamic selection of an initial reference frame in advance, the number of reference frames to be considered is reduced. In addition, from examination of the residue between the current block and reconstructed blocks in preceding frames, variable block size mode decisions are made. Modified initial motion vector estimation and early stop condition detection are also adopted to speed up the motion estimation procedure. Experimental results compare the performance of the proposed algorithm with a state of the art motion estimation algorithm and demonstrate significantly reduced motion estimation time while maintaining PSNR performance.

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

The complexity of motion estimation in the High Efficiency Video Coding (HEVC) standard is very high as it uses a number of prediction block sizes. The test zonal search (TZS) mechanism is used as motion estimation algorithm in the fast search mode of HM encoder, the reference software for HEVC. In this paper, we present schemes for reducing the complexity of the TZS algorithm for uni-prediction in HEVC. The proposed mechanisms help in reducing complexity of grid, raster and refinement search stages of TZS. The performance of the proposed mechanisms is tested independently and also combined in the fast search mode of HM-16.18. The motion estimation time and the number of search points in the combined algorithm are reduced by 68.14% and 77.10% with a BD-rate of 0.30% and a BD-PSNR of $$-0.007$$ dB in comparison with the original fast search mode in the low-delay P main profile. The proposed complexity reduction schemes are compared with those adopted in other recently proposed fast motion estimation algorithms and the performance of the proposed scheme is found to be superior in terms of the reduction in motion estimation time.

In H.264/AVC standard, motion estimation can be processed on multiple reference frames (MRF) to improve the video coding performance. For the VLSI real-time encoder, the heavy computation of fractional motion estimation (FME) makes the integer motion estimation (IME) and FME must be scheduled in two macro block (MB) pipeline stages, which makes many fast MRF algorithms inefficient for the computation reduction. In this paper, two algorithms are provided to reduce the computation of FME and IME. First, through analyzing the block's Hadamard transform coefficients, all-zero case after quantization can be accurately detected. The FME processing in the remaining frames for the block, detected as all-zero one, can be eliminated. Second, because the fast motion object blurs its edges in image, the effect of MRF to aliasing is weakened. The first reference frame is enough for fast motion MBs and MRF is just processed on those slow motion MBs with a small search range. The computation of IME is also highly reduced with this algorithm. Experimental results show that 61.4%-76.7% computation can be saved with the similar coding quality as the reference software. Moreover, the provided fast algorithms can be combined with fast block matching algorithms to further improve the performance.

In the new video coding standard H.264/AVC, motion estimation (ME) is allowed to search multiple reference frames. Therefore, the required computation is highly increased, and it is in proportion to the number of searched reference frames. However, the reduction in prediction residues is mostly dependent on the nature of sequences, not on the number of searched frames. Sometimes the prediction residues can be greatly reduced, but frequently a lot of computation is wasted without achieving any better coding performance. In this paper, we propose a context-based adaptive method to speed up the multiple reference frames ME. Statistical analysis is first applied to the available information for each macroblock (MB) after intra-prediction and inter-prediction from the previous frame. Context-based adaptive criteria are then derived to determine whether it is necessary to search more reference frames. The reference frame selection criteria are related to selected MB modes, inter-prediction residues, intra-prediction residues, motion vectors of subpartitioned blocks, and quantization parameters. Many available standard video sequences are tested as examples. The simulation results show that the proposed algorithm can maintain competitively the same video quality as exhaustive search of multiple reference frames. Meanwhile, 76 %-96 % of computation for searching unnecessary reference frames can be avoided. Moreover, our fast reference frame selection is orthogonal to conventional fast block matching algorithms, and they can be easily combined to achieve further efficient implementations.

Most of the fast search motion estimation algorithms reduce the computational cost of motion estimation greatly by checking only a few search points inside the search area by using full distortion measure. This paper proposes a multi-layer motion estimation which employs partial distortion as its distortion measure to reduce the number of computations involved in each search point instead of reducing the number of search points. A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer. It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm.

Motion Estimation (ME) is the most computationally intensive part of video compression systems. Multiple reference frame (MRF) ME used in H.264 standard increases the video coding efficiency at the expense of increased computational complexity and power consumption. Therefore, in this paper, we present a reconfigurable baseline H.264 video encoder hardware in which the number of reference frames used for MRF ME can be configured based on the application requirements in order to trade-off video coding efficiency and power consumption. The proposed H.264 video encoder hardware is based on an existing low cost H.264 intra frame coder hardware and it includes new reconfigurable MRF ME, mode decision and motion compensation hardware. The proposed H.264 video encoder hardware is capable of processing 55 CIF (352×288) frames per second and its power consumption ranges between 115mW and 235mW depending on the number of reference frames used for MRF ME.

In the new H.264/AVC video coding standard, motion estimation takes up a significant encoding time, especially when using the straightforward full search algorithm (FS). A fast flexible multi-frame motion estimation algorithm with adaptive search strategies (FMASS) is presented. With special considerations on the multiple reference frames and block modes, several techniques, i.e., adaptive search strategies for single frame and flexible multi-frame selection, have been utilized to improve significantly the speed-up performance in H.264. Extensive simulations show that it can minimize the matching points by more than 1190 times compared with FS. In addition, the output quality of the encoded sequences loses only 0.053 dB in terms of PSNR on average. Fast speed-up performance and unnoticeable quality losses make the proposed algorithm outperform most of the other well-known algorithms proposed recently, such as ARPS3, MVFAST and UMHexagonS, of which the latter two have been already accepted by MPEG-4 and JVT respectively.

H.264IMPEG-4 AVC allows multiple reference frames and multiple partition types to improve the coding performance at the expense of much higher computation complexity. This paper proposes a fast motion estimation (ME) algorithm to reduce search complexity for this demanding configuration. Our experimental results show that the partition type which gives the lowest matching cost for the reference frame closest to the current picture in temporal domain is likely to be the best partition type for all reference frames. Based on this finding, the proposed method performs a less complicated search effort for the other partition types on the remaining reference frames to obtain significant savings in search complexity. This approach can be easily modified to support any existing fast search algorithms. Simulation results show the proposed algorithm significantly reduces the overall motion search complexity and at the same time, maintains minimum coding performance degradation.

Recently standardized by the Joint Video Team (JVT) of ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Moving Picture Experts Group (MPEG), the High Efficiency Video Coding (HEVC) was adopted to replace the H264/AVC with better compression performance that facilitates video transmission and storage while maintaining the same reconstructed video quality. This excellent coding efficiency is performed through an exhaustive algorithm for mode decision that requires a huge computational complexity. To accelerate the encoding process, fast motion estimation (ME) algorithms are proposed in this paper. Firstly, according to statistical results of the motion vector difference prediction distribution, an adaptive search range selection algorithm based on a depth level is presented. In addition, several motion estimation search patterns were designed to reduce the calculation redundancy of motion estimation process. Experimental results show that the proposed algorithm schemes can save the ME time up to 88% while upholding almost the same rate-distortion performances.

The variable block size motion estimation (ME) and compensation (MC) using multiple reference frames is adopted in H.264/AVC to improve coding efficiency. However, the computational complexity for ME/MC increases proportional to the number of reference frames. In this paper, we propose a new efficient reference frame selection algorithm to reduce the complexity. The proposed algorithm selects suitable reference frames by employing the spatial and temporal correlation of video sequence. The experimental results show that the proposed algorithm decreases video encoding time while keeping the similar visual quality and bit rates.

With the development of fast motion estimation (ME) algorithms towards video compression standard H.264, burden on integer-pel ME is slightly reduced. However the integer-pel ME is becoming computationally complex due to demand of higher and higher resolution videos. On the other hand, there is always trade-off between the speed and performance of these search algorithms. This work improvises the performance of content awareness enabled integer-pel ME algorithm for encoding the fast and slow motion video sequences of high definition (HD) resolution category viz. 1280×720 and 1920×1080. The algorithm proposes a novel notion of 'integrity index' with focus on increasing the PSNR and reducing the bit rate. It is implemented in the frame work of x265 version 1.7 video encoder. Motion independent ME algorithm is analysed quantitatively in terms of parameters based on rate distortion and ME time. The proposed algorithm is found to be performing better on BD-Rate and BD-PSNR parameters for the videos of both resolutions under consideration. Observed increase in motion estimation time is less than four seconds compared to hexagonal search algorithm of ME which is the benchmark in fast ME algorithm.

This paper presents a novel block motion estimation scheme with edge alignment strategy on H.264/AVC video coding, whose computational complexity is linearly dependent on the number of allowed reference frames and block sizes using the full exhaustive search. Many fast block-matching algorithms reduce the computational complexity of motion estimation by carefully designing the search patterns with different shapes or sizes which have significant impact on the search speed and distortion performance. However, the search speed and the distortion performance conflict often with each other for these methods. In this paper, given a block in the current frame, we first apply a fast approximate method based on edge alignment to obtain a good initial motion vector as well as a tight initial bound of distortion measure. Then, considering the edge orientation of the block, a modified hexagon search is used to fine tune the motion vector in low computational complexity. The proposed algorithm also pays attentions to the characteristics of multiple reference frames and multiple block sizes in H.264/AVC. Computer simulation results show that the proposed method gives good performance and spans a new way to design a cost-effective real-time video coding system.

The H.264 video coding standard adopts many new coding tools, such as variable block size, multiple reference frames, quarter-pixel-accuracy motion estimation, intra prediction, loop filter, etc. Using these coding tools, H.264 achieves significant performance. Especially for reference frame, block size, and predication direction, it can adaptively do the selection. However, the encoding complexity increases tremendously. Among these tools, the macro block modes selection and the motion estimation contributes most to total encoding complexity. This paper focuses on complexity reduction in macro block modes selection. On the basis of analyzing some fast mode selection algorithms, a new algorithm is presented, in order to reduce the time of encoding, in the precondition of reducing the quality of encoding obviously.

Motion estimation (ME) for the high efficiency video coding (HEVC) occupies the largest part in the computational complexity of the HM encoder, which consists of integer ME (iME) and fractional ME (fME). We propose a fast fME algorithm that reduces its complexity substantially by reducing the number of search points to less than a half, based on statistical observations, in which four search points are selected only in the horizontal and vertical directions for half-pel search and three search points that have the smaller absolute value of the motion vector difference are selected from eight candidates for quarter-pel search. The effective number of search points in the proposed fME algorithm is reduced to about 60% and the interpolation filtering is also reduced to about 55%, compared to that of the fME in the HM encoder while experimental results show that its Bj⊘ntegaard delta (BD) rate degradation is 0.93% for the five class-B video sequences.