Predictive search pattern selection based on edge alignment for fast motion estimation

This paper presents a novel block matching scheme with edge alignment strategy on H.264 video coding, which uses multiple references and multiple block sizes for motion estimation in order to improve the rate-distortion performance. In H.264, the computational complexity is linearly dependent on the number of allowed reference frames and block sizes using the full exhaustive search. Many fast block-matching algorithms reduce the computational complexity of motion estimation by carefully designing the search patterns with different shapes or sizes which have significant impact on the search speed and distortion performance. However, the search speed and the distortion performance conflict often with each other for these methods. In this paper, given a block in the current frame, we first apply a fast approximate method based on edge alignment to obtain a good initial motion vector as well as a tight initial bound of distortion measure. Then, considering the edge orientation of the block, a modified hexagon search is used to fine tune the motion vector in low computational complexity. The proposed algorithm also pays attentions to the characteristics of multiple reference frames and multiple block sizes in H.264. Computer simulation results show that the proposed method gives good performance and spans a new way to design a cost-effective real-time video coding system

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

Block sizes larger than the traditional 16×16 macroblock (MB) structure and extended macroblock (EMB) size motion estimation (ME) are introduced in joint model key technology areas software to improve the inter block coding efficiency. Similar to H.264, variable size motion estimation (VS-ME) is still adopted for each 16×16 MB including 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4. EMB size ME and VS-ME achieve the highest possible coding efficiency, but result in extremely large coding time which obstructs it from practical use. A fast mode decision algorithm for EMBs based on motion homogeneity is proposed to reduce the computational complexity of ME. The basic idea of the proposed method is to utilize the spatial property of motion field in prediction where EMB size ME and small size ME (including 8×8, 8×4, 4×8 and 4×4) are needed, and only in these regions EMB size ME and small size ME are enabled. The motion field is generated by the corresponding motion vectors in spatial window. Simulation results show that the proposed algorithm can save 55% average computational complexity, with negligible loss of coding efficiency.

H.264/AVC, the latest video coding standard, adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bitrate. However, the full RD cost calculation for all intra-prediction modes, the exhaustive searches for finding optimal motion vectors for all block sizes, and the multiple references frame procedure considerably increase its computational complexity with the allowed number of prediction modes. The authors have previously proposed an approach for both inter- and intra-mode decisions that takes into account the several effective parameters, image content type, quantization parameter, correlation of motion vectors and motion details of video objects. In order to reduce the complexity, here we propose a new two-step adaptive search range decision by developing a combination of previous algorithm. The first step of our adaptive search range decision determines the search range for current MB according to the motion vectors of the previous MBs. In the second step, the motion vector obtained in the current frame is used to adaptively searching the window size of the previous reference frame for multi-frame motion estimation. As such, only a subset of inter- and intra-modes is chosen for RDO calculation. Experimental results show that the proposed algorithm reduces the total encoding cost with negligible loss in PSNR and a slightly increase in the required bitrate, when compared to our previous algorithm and other algorithms reported in the literature.

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

The upcoming video coding standard, MPEG-4 AVC/JVT/H.264, motion estimation is allowed to use multiple references and multiple block sizes to improve the rate-distortion performance. However, full exhaustive search of all block sizes is computational intensive with complexity increasing linearly to the number of allowed reference frame and block size. A novel search algorithm, predict hexagon search (PHS), is proposed. The PHS pattern is applied by the comparison with the hexagon based search pattern. It can predict the hexagon search pattern in horizontal or vertical direction. Analysis shows that the speed improvement of the PHS over the diamond search (DS) and the hexagon based search (HEXBS) is about 58% and 53% respectively.

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

The growing availability and widespread use of digital multimedia information and personal wireless communication devices have motivated development of several video compression standards over the past two decades. H.264/AVC (advanced video coding) is the newest international video coding standard approved by ITU-T as Recommendation H.264 and by ISO/IEC as International Standard 14496-10 (MPEG-4 part 10). Block motion estimation is one of the most important modules in the design of any video coder. It is widely used in many motion compensated video-coding standards like MPEG and ITU-T for multimedia and teleconference applications. Recently, a hexagon-shaped search pattern was introduced in fast block motion estimation, which has exhibited faster search speed. Based on an in-depth examination of the influence of search pattern on speed performance, we propose a efficient motion estimation algorithm using a hexagon-based search pattern to achieve further improvement. The proposed algorithm is to improve the execution speed of motion estimation algorithm by incorporating the direction and angle of previously processed motion vector. The proposed algorithm is investigated in comparison with hexagon search pattern, which demonstrates significant speedup gain over the hexagon-based search. Analysis shows that a speed improvement rate of the proposed algorithm over the hexagon search algorithm can be as high as over 80% for locating some motion vectors in certain scenarios. In short, the proposed algorithm can find a same motion vector with fewer search points than the existing hexagon-search algorithm. The proposed algorithm consistently reducing the number of search points in motion estimation module of video codec.

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

In block motion estimation, a search pattern with a different shape or size has a very important impact on search speed and distortion performance. A square-shaped search pattern is adopted in many popular fast algorithms. Recently, a diamond-shaped search pattern was introduced in fast block motion estimation and has exhibited a faster search speed. Based on an in-depth examination of the influence of the search pattern on speed performance, we propose a novel algorithm using a hexagon-based search pattern to achieve further improvement. The hexagon-based search pattern is investigated in comparison with diamond search pattern and demonstrates significant speedup gain over the diamond-based search. Analysis shows that a speed improvement rate of the hexagon-based search (HEXBS) algorithm over the diamond search (DS) algorithm can be over 80% for locating some motion vectors in certain scenarios. In short, the proposed HEXBS algorithm can find the same motion vector with fewer search points than the DS algorithm. Generally speaking, the larger the motion vector, the more search points the. HEXBS algorithm can save, which is further justified by experimental results.

This paper presents a new method for frame rate up‐conversion using sub‐sampled bilateral motion estimation, motion vector smoothing, simplified bidirectional motion compensated interpolation, and complex motion detection. The proposed method produces improved image quality and requires less computational complexity compared to conventional motion estimation. The average PSNRs of the proposed method for frame rate up‐conversion outperform those of the benchmark frame rate up‐conversion technique by up to 4.39 dB for the test sequences. Furthermore, the motion estimation of the proposed FRUC method reduces the required computational complexity to 12.5% of that required for full‐search block matching motion estimation.

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

To minimize Rate-Distortion (RD) cost in video coding, RD Optimization (RDO) technique is adopted, which also brings increasingly computational complexity in Motion Estimation (ME), multiple reference selection and mode decision. In this paper, a Gaussian model based adaptive RD cost prediction method is developed to jointly speed up multiple reference selection and inter-mode decision. Besides the ability of achieving good coding efficiency with much reduced computational power, our method is also with two major advantages. First, it can achieve robust performance, including computational complexity reduction and almost intact video quality, with a wide range of video sequences resolutions and even for various scene changes; second, it could be either individually used or combined with the other efficient mode decision algorithms, and in both cases, our method could achieve remarkable performance, which is revealed by exhaustive experimental results.

H.264/AVC has better compression performance because of using motion estimation based on variable block size, but it also increases computational complexity vastly, which restrict the application range of H.264/AVC. The paper draws on ideas of MMVFAST(Multi-Motion Vector Field Adaptive Search Technology), and proposes a modified motion vector multi-template self-adaptive search algorithm of motion estimation, which can reduce the computational complexity of motion estimation effectively. Experimental results show it can retrench time cost of motion estimation and improve speed of encoding greatly on condition that the reconstructed image quality and bit rate keep invariant.

For video compression, motion estimation is popularly employed to exploit temporal correlation existing in video sequences. If we employ the full search block matching algorithm for estimating motion vectors, it requires very heavy computational complexity. Although several fast block matching algorithms have been proposed to solve this problem, they sacrifice their reconstructed image quality. In this paper, we derive optimal search patterns for fast block matching motion estimation. By analyzing the block matching algorithm as a function of the block size and the shape, we find optimal search patterns for initial motion estimation. The proposed idea can provide an analytical ground for the current MPEG-2 proposals. In addition, we propose a new fast motion estimation algorithm using adaptive search patterns, considering matching criteria and statistical properties of object displacement. In order to select an appropriate search pattern, we exploit the relationship between the motion vector and the frame difference of each block. By changing the search pattern adaptively, we can improve the motion prediction accuracy, while reducing the required computational complexity compared to other fast block matching algorithms.