A Fast Sub-Pixel Motion Estimation Algorithm for H.264/AVC Video Coding

Many fast integer-pixel motion estimation algorithms have been developed for the High Efficiency Video Coding Standard, however the speed of sub-pixel motion estimation still has room for improvement. A fast sub-pixel motion estimation algorithm is proposed in this paper to speed up the sub-pixel search process. First, the proposed scheme skips sub-pixel search process in smooth prediction units. Then a fast sub-pixel search algorithm based on texture direction analysis is proposed to further reduce the computational complexity of subpixel motion estimation. The simulation results show that compared with the Full Sub-pixel Search (FSPS), the encoding complexity of the whole motion estimation process can be reduced by an average of 40.9% with negligible coding performance loss.

We studied an efficient software implementation of H.264/AVC sub-pixel motion estimation (ME) algorithm on a VLIW-SIMD digital signal processor, TMS320C6416. The sub-pixel ME algorithm demands large memory accesses while the required arithmetic operations are fairly simple. Although the CPU clock cycles for arithmetic operations can be reduced much by employing sub-word operations and applying software pipelining techniques, the limited memory bandwidth of the architecture restricts the overall performance. Moreover, aggressive VLIW-SIMD optimization results in the degradation of the performance by causing excessive CPU stalls during memory accesses. In this paper, we relieved the memory bandwidth requirements for creating quarter-pixel images by reducing the precision of image data, from 8 bits to 4 bits. As a result, the amount of memory accesses is much reduced at the cost of some increase of the arithmetic operations, which contributes to the balance of arithmetic and memory access operations. The experimental result shows that the memory stall cycles are decreased by 80% and the speed-up of 260% is obtained. The bit rate of the encoded video stream is increased slightly, about 2% on the average, due to the effects of quantization.

We propose a fast subpixel motion estimation algorithm for the H.264 advanced video coding (AVC) standard. The algorithm utilizes the correlation of the spatial interpolation effect on the full-pixel motion estimation best matches between different block sizes in order to reduce the computational cost of the overall motion-estimation process. Experimental results show that the proposed algorithm significantly reduces the CPU cycles in the various motion estimation schemes by up to 16% with similar rate-distortion performance when weighed up against the H.264/AVC standard.

H.264 is a new recommendation for moving picture coding proposed by JVT. In this standard, full pixel and sub-pixel motion estimation are the most important parts. The unequal-arm adaptive rood pattern search (APRS-3) algorithm has shown a good performance on search speed-up. In this paper, we propose an improved algorithm called APRS-4 which applies an early termination technology on APRS-3 by using adaptive threshold. This early termination technology is also applied on sub-pixel motion estimation, and a new fast sub-pixel motion estimation algorithm is proposed which uses the motion vector information of the neighbor blocks. Experimental results show that both algorithms have achieved a good speed-up ratio. Compared with full-pixel motion estimation, our new algorithm could keep image quality well with almost the same bit rate.

Motion estimation is a key step for many video process systems, but it usually suffers from low precision in complex imaging scenarios, as well as the problem of real-time processing. This paper proposes an accurate and fast motion estimation algorithm using maximum sub-image and sub-pixel phase correlation implemented on a DSP platform. Firstly, a maximum sub-image is extracted to contain the image content as much as possible, which both satisfies the requirement of Fourier precondition and the wide coverage of complete background. And then, the extracted sub-image is down-sampled with median filter to increase the signal-noise-ratio and decrease the computation load. Secondly, a sub-pixel motion estimation is used to compensate the losing precision due to down-sampling, and keep the range of motion estimation. Finally, the proposed motion estimation algorithm is implemented on a single core TMS320C6678 DSP platform, and it is accelerated by applying multistage data cache and advanced data access. Experiments demonstrate the accuracy of the proposed motion estimation algorithm in complex scenarios. Meanwhile, it can achieve 7.4 ms/frame for sub-image with 512 × 512 pixels size and 29.6 ms/frame for sub-image with 1024 × 1024 pixels size, respectively.

Fast motion estimation for H.264

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

Block matching algorithms (BMAs) are often employed for motion estimation (ME) in video coding. Most conventional BMAs treat the ME problem as an optimization problem and employ certain search schemes to find a solution. Except the time-consuming full search algorithms (FS), other fast algorithms such as the three step search (TSS), searching on a reduced search range, cannot guarantee optimal solutions, i.e., search is often trapped at local minima; the ME results are thus usually unsatisfactory. Few of them makes use of the information inherent in the images explicitly. We propose a new ME algorithm which can reduce the search range while guaranteeing global optimality in most cases. Microblock visual patterns are designed to extract edge information to guide block matching: searching is only carried out at places where the real match most likely happens; that is where similar edge features present. The proposed algorithm obtains a speed about 7 - 8 times as fast as that of FS with same search range. The prediction quality is very close to that of FS and much better than that of TSS. It can produce MPEG-1 or MPEG-2 compatible motion vectors, and can be extended to model-based ME. It is suitable for parallel implementation too. Moreover, the visual patterns are potential resources for video indexing to facilitate content-based information retrieval, which is important for applications such as VOD.

This paper presents a flexible and scalable motion estimation processor capable of supporting the processing requirements for high-definition (HD) video using the H.264 Advanced Video Codec, which is suited for FPGA implementation. Unlike most previous work, our core is optimized to execute all existing fast block matching algorithms, which we show to match or exceed the inter-frame prediction performance of traditional full-search approaches at the HD resolutions commonly in use today. Using our development tools, such algorithms can be described using a C-style syntax which is compiled into our custom instruction set. We show that different HD sequences exhibit different characteristics which necessitate a flexible and configurable solution when targeting embedded applications. This is supported in our core and toolset by allowing designers to modify the number of functional units to be instantiated. All processor instances remain binary compatible so recompilation of the motion estimation algorithm is not required. Due to this optimization process, it is possible to match the processing requirements of the selected motion estimation algorithm to the hardware microarchitecture leading to a very efficient implementation.

The scalable extension of the HEVC Video Coding Standard (H.265) offers elaborate mechanisms for motion vector prediction and estimation. S-HEVC builds on the standard by extending predictor lists for Coding Unit blocks, utilizing base-layer information in the inference of enhancement-layer Coding Units. The complex, exhaustive search schemes in use can be aided by hierarchical optimizations in subpixel motion estimation, which we propose for slow-moving CUs per frame. In this paper we implement and test an adaptive optimization of motion estimation in the standard (SHM 6.1 software release), based on a statistical analysis of the behavior of subpixel motion vector differentials in each spatial mode per Coding Unit. We propose that the least granular mode (64×64 PEL macro-block in current release) contains sufficient information at subpixel levels to decide best-mode selection, i.e., whether a complete recursion through the inner partitions (higher granularity) is required in the estimation of a CU motion vector. We further propose that subpixel motion estimation overheads can be avoided below a set threshold, given conditions set in base and enhancement layer motion estimation for priorly computed modes in the same CU. Both optimization methods are tested across a diverse set of video sequences, producing negligible quality penalties at for a sizable reduction in encoding time.

In Scalable Video Coding (SVC), motion estimation and inter-layer prediction play an important role in elimination of temporal and spatial redundancies between consecutive layers. This paper evaluates the performance of widely accepted block matching algorithms used in various video compression standards, with emphasis on the performance of the algorithms for a didactic scalable video codec. Many different implementations of Fast Motion Estimation Algorithms have been proposed to reduce motion estimation complexity. The block matching algorithms have been analyzed with emphasis on Peak Signal to Noise Ratio (PSNR) and computations using MATLAB. In addition to the above comparisons, a survey has been done on Spiral Search Motion Estimation Algorithms for Video Coding. A New Modified Spiral Search (NMSS) motion estimation algorithm has been proposed with lower computational complexity. The proposed algorithm achieves 72% reduction in computation with a minimal (<1dB) reduction in PSNR. A brief introduction to the entire flow of video compression H.264/SVC is also presented in this paper.

In an H.264 video encoder, motion estimation (ME) is the most time-consuming component. The ME process consists of two stages, integer pixel search and fractional pixel search. Since the complexity of integer pixel search has been greatly reduced by numerous fast ME algorithms, the computation overhead required by fractional pixel ME has become relatively significant. To reduce the complexity of fractional pixel ME, we propose a prediction-based directional fractional pixel ME algorithm. We utilize more accurate motion vector predictions and directional search to achieve better computation reduction. We further propose an early termination method to decrease the amount of search. Experimental results show that, compared to the full search sub-pel ME and the fast sub-pel ME proposed in H.264, the proposed method can reduce up to 84% and 74% of fractional pixel search points respectively, with a negligible degradation in quality.

Modern video codecs such as MPEG2, MPEG4-ASP and H.264 depend on computationally complex sub-pixel motion estimation (ME) to optimise rate-distortion efficiency. Sub-pixel ME is implemented within these standards using interpolated values at 1/2 or 1/4 pixel accuracy. This paper presents a novel scheme for sub-pixel ME using a model free kernel method utilising the result of the whole-pixel SAD distribution applied to an H.264 encoder. While giving an equivalent rate distortion performance, this approach approximately halves the number of quarter-pixel search positions giving an overall speed up of approximately 10% compared to the EPZS quarter-pixel method (the state of the art H.264 optimised sub-pixel motion estimator).

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

In this paper, we present a novel method for sub-pixel motion estimation. Compared to traditional methods, our technique offers a significantly reduced complexity by eliminating the explicit interpolation of pixel values at a finer sampling grid and avoiding the distortion evaluation for each candidate sub-pixel motion vector. Simulation results confirm that its performance is virtually indistinguishable from that of a traditional sub-pixel motion estimator. To illustrate applications of the new method we have developed a variable resolution motion estimator. The performance of the variable resolution motion estimator is evaluated by replacement of the fixed resolution motion estimator in Simulation Model 3 of MPEG-1 video compression standard by the variable resolution motion estimator.