Rate-computation optimized block based video coding

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

To efficiently encode data-intensive multi-view imaging content, conventional hybrid predictive coding methodologies choose to address the compression by exploiting temporal and inter-viewpoint redundancy. However, their key yet time-consuming component, motion estimation (ME), is usually not efficient in inter-viewpoint prediction because inter-viewpoint motion is quite different from temporal motion. In essence, inter-viewpoint correlation is subject to epipolar geometry, which provides constraints for multi-view image sequences. A fast inter-viewpoint ME technique is hence proposed in this paper to accelerate the encoding by employing epipolar geometry. Theoretical analysis and experimental results prove that the proposed ME algorithm can greatly reduce search region and effectively track large and irregular motion that is typical for convergent multi-view camera setups. As a result, compared with fast full search at large search size adopted in H.264, our proposed ME algorithm can obtain a similar coding efficiency while achieving a speedup ratio of 2.9.

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that offers good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications, achieving essentially the same levels of rate-distortion performance and subjective quality as that of the full search algorithm when used by the UBC H.263+ video coding reference software. However the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.

In this paper, we present an improved multi-pass true motion estimation algorithm for frame rate up-conversion. In the proposed motion estimation algorithm, the motion vectors of different objects which are parted with depth information will be refined respectively. Therefore, more accurate pixel-based motion vector can be obtained which causes the performance of Frame Rate Up-Conversion (FRUC). For overlapped regions, the motion vector can be chosen in candidates with depth information. Experimental results show that our proposed algorithm effectively enhances the overall quality of the frame rate up-converted video sequence, both subjectively and objectively, even with complex scenes and fast motion.

Motion estimation (ME) is one of the most critical tools in video coding and consumes the majority of the encoding complexity. Three types of ME are utilized in the latest video coding standards, namely integer, fractional, and affine MEs. They are implemented as three searches for the integer motion vector (IMV), fractional motion vector (FMV), and control point motion vectors (CPMVs). Many algorithms were proposed to reduce the complexity for them individually, but the overall overhead of three searches is still challenging for hardware implementations. Therefore, we propose a hardware-efficient Unified Motion Estimation (UME) to derive three types of MVs with only one search. An IME with sub-block refinement is performed to collect extra motion information while searching for the IMV. The FMV and CPMVs are then derived from the collected information using a mixed error surface and an overdetermined system. Compared to the default ME algorithms in VVC, the time cost for ME is reduced by 41.63% with a coding loss of only 0.87% under LDB configuration. For hardware implementations, the minimum required resources and corresponding latency are significantly reduced by 75.35% and 69.17%, respectively.

Motion estimation is a key step for many video process systems, but it usually suffers from low precision in complex imaging scenarios, as well as the problem of real-time processing. This paper proposes an accurate and fast motion estimation algorithm using maximum sub-image and sub-pixel phase correlation implemented on a DSP platform. Firstly, a maximum sub-image is extracted to contain the image content as much as possible, which both satisfies the requirement of Fourier precondition and the wide coverage of complete background. And then, the extracted sub-image is down-sampled with median filter to increase the signal-noise-ratio and decrease the computation load. Secondly, a sub-pixel motion estimation is used to compensate the losing precision due to down-sampling, and keep the range of motion estimation. Finally, the proposed motion estimation algorithm is implemented on a single core TMS320C6678 DSP platform, and it is accelerated by applying multistage data cache and advanced data access. Experiments demonstrate the accuracy of the proposed motion estimation algorithm in complex scenarios. Meanwhile, it can achieve 7.4 ms/frame for sub-image with 512 × 512 pixels size and 29.6 ms/frame for sub-image with 1024 × 1024 pixels size, respectively.

In this letter, we present a new motion estimation algorithm for frame rate up-conversion. The proposed dual motion estimation algorithm enhances the estimation accuracy of motion vectors by using the unidirectional and bidirectional matching ratios of blocks in the previous and current frames. In addition, the proposed motion estimation approach uses motion vector validity to evaluate the accuracy of motion vectors thereby avoiding false motion vectors. In experiments using benchmark image sequences, the proposed motion estimation algorithm improved the average peak signal-to-noise ratio of interpolated frames by up to 2.272 dB, when compared to conventional motion estimation algorithms. For the comparison of the perceptual image quality using the structural similarity, the average value of the proposed dual motion estimation was by up to 0.062 higher than those of the conventional algorithms.

The growing availability and widespread use of digital multimedia information and personal wireless communication devices have motivated development of several video compression standards over the past two decades. H.264/AVC (advanced video coding) is the newest international video coding standard approved by ITU-T as Recommendation H.264 and by ISO/IEC as International Standard 14496-10 (MPEG-4 part 10). Block motion estimation is one of the most important modules in the design of any video coder. It is widely used in many motion compensated video-coding standards like MPEG and ITU-T for multimedia and teleconference applications. Recently, a hexagon-shaped search pattern was introduced in fast block motion estimation, which has exhibited faster search speed. Based on an in-depth examination of the influence of search pattern on speed performance, we propose a efficient motion estimation algorithm using a hexagon-based search pattern to achieve further improvement. The proposed algorithm is to improve the execution speed of motion estimation algorithm by incorporating the direction and angle of previously processed motion vector. The proposed algorithm is investigated in comparison with hexagon search pattern, which demonstrates significant speedup gain over the hexagon-based search. Analysis shows that a speed improvement rate of the proposed algorithm over the hexagon search algorithm can be as high as over 80% for locating some motion vectors in certain scenarios. In short, the proposed algorithm can find a same motion vector with fewer search points than the existing hexagon-search algorithm. The proposed algorithm consistently reducing the number of search points in motion estimation module of video codec.

本文提出了一种HEVC视频编码的运动估计快速算法，该算法可以显著加速搜索过程，编码效率与HEVC参考软件(Test Model)中的Test Zone Search (TZSearch)平分秋色。较之H.264/AVC视频标准，HEVC引入了更为复杂的混合编码框架和更大尺寸的搜索窗口，从而带来了更高的计算复杂度。为了降低复杂度，我们提出了一种新的运动估计算法，该算法仅利用搜索窗中所选取的少量几个点建立二次曲面模型，通过对像素的绝对差值和(SAD)分布的迭代分析，逐步缩小搜索范围，直到得出最优的运动矢量。该算法可被用于不同的编码环境中，实验结果表明，与HM相比，该算法可以节约大约40%的运动估计算法时间，仅引起轻微的性能损失。 This paper presents a motion estimation (ME) algorithm for High Efficiency Video Coding (HEVC), which provides a strategy to speed up the search process significantly, while yielding the same quality performance as the Test Zone Search (TZSearch) scheme in HEVC Test Model (HM). Compared with the H.264/AVC, HEVC employs a more complex hybrid coding architecture and a larger size search window, leading to great computational complexity. In order to reduce the complexity, a novel motion estimation algorithm is proposed, in which some limited pixels of certain position in the current search window are utilized to build a quadratic model, and then shrink the search range repeatedly by analyzing the sum of absolute difference (SAD) distribution until the best motion vector (MV) is obtained. The proposed algorithm can be applied to various encoding conditions. Experimental results show that our method can save 40% of computations compared with the HM, with negligible decrease of coding quality.

Although variable block-size motion estimation provides significant video quality and coding efficiency improvement, it requires much higher computational complexity compared with fixed block size motion estimation. The reason is that the current motion estimation algorithms are mainly designed for fixed block size. Current variable block-size motion estimation implementation simply applies these existing motion estimation algorithms independently for different block sizes to find the best block size and the corresponding motion vector. Substantial computation is wasted because distortion data reuse among motion searches of different block sizes is not considered. In this paper, a motion estimation algorithm intrinsically designed for variable block-size video coding is presented. The proposed multiple block-size search (MBSS) algorithm unifies the motion searches for different block sizes into a single searching process instead of independently performing the search for each block size. In this unified search, the suboptimal motion vectors for different block sizes are used to determine the next search steps. Its prediction quality is comparable with that obtained by performing motion search for different block sizes independently while the computational load is substantially reduced. Experimental results show that the prediction quality of MBSS is similar to full search.

Motion estimation algorithm plays a crucial role in video coding/processing and accounts for almost one quarter of the gate count or area in many digital video system-on-a-chip (SoC) due to its computational intensive nature. In this paper, we present a complexity-reduced three dimensional (3D) motion estimator (ME) which reduces the number of search candidates and hence gate count significantly. Moreover, by virtue of its recursive architecture, hardware estimated sum of absolute differences (SAD) and motion vectors (MV) can be passed backed to the embedded DSP processor for image segmentation, resulting in efficient software/hardware partition in SoC designs. In addition, the calculated SAD's can also be further analyzed by the embedded CPU for rate control in video coding. Alleviating traditional ME artifacts such as illumination, this motion estimator with segmentation capabilities is readily applicable to video object planes (VOP) processing in levels higher than the advance simple profile in MPEG4, H.264 and motion compensated format converters such as deinterlacers and scan-rate converters.

Motion estimation (ME) is one of the most time-consuming parts in video coding. The use of multiple partition sizes in H.264/AVC makes it even more complicated when compared to ME in conventional video coding standards. It is important to develop fast and effective sub-pixel ME algorithms since: 1) the computation overhead by sub-pixel ME has become relatively significant while the complexity of integer-pixel search has been greatly reduced by fast algorithms, and 2) reducing sub-pixel search points can greatly save the computation for sub-pixel interpolation. In this letter, a novel fast sub-pixel ME algorithm is proposed which performs a “rough” sub-pixel search before the partition selection, and performs a “precise” sub-pixel search for the best partition. By reducing the searching load for the large number of non-best partitions, the computation complexity for sub-pixel search can be greatly decreased. Experimental results show that our method can reduce the sub-pixel search points by more than 50% compared to existing fast sub-pixel ME methods with negligible quality degradation.

H.264/AVC, the latest video coding standard, adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bitrate. However, the full RD cost calculation for all intra-prediction modes, the exhaustive searches for finding optimal motion vectors for all block sizes, and the multiple references frame procedure considerably increase its computational complexity with the allowed number of prediction modes. The authors have previously proposed an approach for both inter- and intra-mode decisions that takes into account the several effective parameters, image content type, quantization parameter, correlation of motion vectors and motion details of video objects. In order to reduce the complexity, here we propose a new two-step adaptive search range decision by developing a combination of previous algorithm. The first step of our adaptive search range decision determines the search range for current MB according to the motion vectors of the previous MBs. In the second step, the motion vector obtained in the current frame is used to adaptively searching the window size of the previous reference frame for multi-frame motion estimation. As such, only a subset of inter- and intra-modes is chosen for RDO calculation. Experimental results show that the proposed algorithm reduces the total encoding cost with negligible loss in PSNR and a slightly increase in the required bitrate, when compared to our previous algorithm and other algorithms reported in the literature.

Motion estimation (ME) has the highest computational complexity in motion-compensated frame rate up-conversion (MC-FRUC). For the real-time implementation of FRUC, a fast ME algorithm is required. In this paper, a new hierarchical ME algorithm for MC-FRUC is proposed. It constructs an image pyramid by dividing the frame into several sub-images according to resolution, and performs ME at the top level to reduce complexity while improving accuracy by selecting multiple motion vector candidates. These candidates are refined at the lower levels, and the final motion vector is selected at the bottom level. Thus, the proposed algorithm obtains an average peak signal-to-noise ratio gain of upto 0.85 dB compared to conventional algorithms with lower computational complexity and yields interpolated images with better visual quality than other methods.

Block matching algorithms (BMAs) are often employed for motion estimation (ME) in video coding. Most conventional BMAs treat the ME problem as an optimization problem and employ certain search schemes to find a solution. Except the time-consuming full search algorithms (FS), other fast algorithms such as the three step search (TSS), searching on a reduced search range, cannot guarantee optimal solutions, i.e., search is often trapped at local minima; the ME results are thus usually unsatisfactory. Few of them makes use of the information inherent in the images explicitly. We propose a new ME algorithm which can reduce the search range while guaranteeing global optimality in most cases. Microblock visual patterns are designed to extract edge information to guide block matching: searching is only carried out at places where the real match most likely happens; that is where similar edge features present. The proposed algorithm obtains a speed about 7 - 8 times as fast as that of FS with same search range. The prediction quality is very close to that of FS and much better than that of TSS. It can produce MPEG-1 or MPEG-2 compatible motion vectors, and can be extended to model-based ME. It is suitable for parallel implementation too. Moreover, the visual patterns are potential resources for video indexing to facilitate content-based information retrieval, which is important for applications such as VOD.