An efficient inter prediction mode selection scheme for advanced video coding based on motion homogeneity and residual complexity

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

The emerging standard of High Efficiency Video Coding (HEVC) provides a 50% improved compression performance compared to its predecessor H.264/AVC at the expense of increasing its computational complexity which is due to a large number of modes involved in the decision process where it would increase encoding time and provide hardware limitations. In this paper, the reduction of intra-coding complexity is proposed by utilization of Rate-Distortion Optimization (RDO) in preselecting a mode with the best RD cost, among all 35 intra-modes. Two cost functions – Sum of Absolute Difference (SAD) and Sum of Absolute Transform Difference (SATD, available in the HEVC Test Model (HM) Reference Software, are determined by making encoder decisions. This paper compares the performance and complexity cost of SAD only for intra-mode, with SATD-based intra-mode by modifying the HM reference encoder through forced selection of the mentioned modes. Results show that SAD consistently produces the least encoding time among all modes, along with sensible values of bitrate. Furthermore, error concealment was introduced in decoding the video sequences with the use of IEEE 802.22 Standard which only allowed power gains resulting to packet error rates (PER) ranging from 1% to 30% for the analysis on the behavior of each video sequence in an environment where error is introduced.

High Efficiency Video Coding (HEVC), the latest video coding standard, aims to provide coding performance that is much superior to that of its predecessor, H.264, especially for high definition video. To fulfill this goal, the inter-prediction unit (PU) partitions of HEVC are more complex, and the search range of motion estimation (ME) is much larger. As a result, ME becomes a bottleneck in the design of the HEVC inter predictor. In response to this challenge, we developed a hardware-oriented integer ME algorithm and the related hardware implementation. Our proposed algorithm led to a decrease in terms of the Bjontegaard Delta rate when compared with the HEVC test model 15.0. The corresponding hardware solution benefitted from 2-D data reuse supported by horizontal and vertical reference SRAMs, on-chip memory reduction supported by $4\times4$ block compression, and a low-power sum of absolute difference (SAD) tree supported by PU-level chip selection. When adopting a $32\times32$ SAD tree, the minimum and maximum required working frequency for 4K $\times2\text{K}$ at 30 frames/s videos was [375, 500] MHz. These results demonstrated that our proposed solution offered desirable improvement in both coding speed and coding performance.

本文提出了一种HEVC视频编码的运动估计快速算法，该算法可以显著加速搜索过程，编码效率与HEVC参考软件(Test Model)中的Test Zone Search (TZSearch)平分秋色。较之H.264/AVC视频标准，HEVC引入了更为复杂的混合编码框架和更大尺寸的搜索窗口，从而带来了更高的计算复杂度。为了降低复杂度，我们提出了一种新的运动估计算法，该算法仅利用搜索窗中所选取的少量几个点建立二次曲面模型，通过对像素的绝对差值和(SAD)分布的迭代分析，逐步缩小搜索范围，直到得出最优的运动矢量。该算法可被用于不同的编码环境中，实验结果表明，与HM相比，该算法可以节约大约40%的运动估计算法时间，仅引起轻微的性能损失。 This paper presents a motion estimation (ME) algorithm for High Efficiency Video Coding (HEVC), which provides a strategy to speed up the search process significantly, while yielding the same quality performance as the Test Zone Search (TZSearch) scheme in HEVC Test Model (HM). Compared with the H.264/AVC, HEVC employs a more complex hybrid coding architecture and a larger size search window, leading to great computational complexity. In order to reduce the complexity, a novel motion estimation algorithm is proposed, in which some limited pixels of certain position in the current search window are utilized to build a quadratic model, and then shrink the search range repeatedly by analyzing the sum of absolute difference (SAD) distribution until the best motion vector (MV) is obtained. The proposed algorithm can be applied to various encoding conditions. Experimental results show that our method can save 40% of computations compared with the HM, with negligible decrease of coding quality.

Currently, most video resources on line are encoded in the H.264/AVC format. More fluent video transmission can be obtained if these resources are encoded in the newest international video coding standard: high efficiency video coding (HEVC). In order to improve the video transmission and storage on line, a transcoding method from H.264/AVC to HEVC is proposed. In this transcoding algorithm, the coding information of intraprediction, interprediction, and motion vector (MV) in H.264/AVC video stream are used to accelerate the coding in HEVC. It is found through experiments that the region of interprediction in HEVC overlaps that in H.264/AVC. Therefore, the intraprediction for the region in HEVC, which is interpredicted in H.264/AVC, can be skipped to reduce coding complexity. Several macroblocks in H.264/AVC are combined into one PU in HEVC when the MV difference between two of the macroblocks in H.264/AVC is lower than a threshold. This method selects only one coding unit depth and one prediction unit (PU) mode to reduce the coding complexity. An MV interpolation method of combined PU in HEVC is proposed according to the areas and distances between the center of one macroblock in H.264/AVC and that of the PU in HEVC. The predicted MV accelerates the motion estimation for HEVC coding. The simulation results show that our proposed algorithm achieves significant coding time reduction with a little loss in bitrates distortion rate, compared to the existing transcoding algorithms and normal HEVC coding.

Scalable High efficiency Video Coding (SHVC) is the scalable extension of the latest video coding standard High Efficiency Video Coding (HEVC). One of the key novelties introduced by SHVC is that it enables hybrid codec scalability. This basically means that the video layers can be encoded with different video standards providing backward compatibility between codecs. In this paper, we propose a software parallel SHVC decoder in hybrid codec scalability configuration. The proposed design consists of an Advanced Video Coding (AVC) decoder for the Base Layer (BL) and a HEVC decoder for the Enhanced Layer (EL). In order to perform Inter Layer Prediction (ILP), a communication of decoding states and outputs is established between the two decoders. While the native frame based parallelism is still allowed within the two decoders, the proposed design also enables the use of frame based parallelism between the two decoders. The proposed software design enables a real time decoding of the HEVC EL at 2160p60 while the AVC base layer is decoded at 1080p60 for ×2 spatial scalability.

This paper presents a highly parallel motion estimation architecture for High Efficiency Video Coding (HEVC) encoder. The proposed architecture has 16 processing units operating in parallel to calculate the sum of absolute difference values of all possible variable prediction block sizes. Hence, it calculates the bit cost regarding every partition in order to find the best matching candidate in terms of bit cost. The proposed unit processes block sizes from 4×4 up to 64×64. The proposed architecture was prototyped, simulated and synthesized using 65nm TSMC CMOS technology. At 720 MHz clock frequency, the proposed architecture processes 2K (1920×1080) resolution at 30 fps with ±27 (55×55) pixel search range using full search algorithm. Moreover, the proposed architecture is a flexible one and it can be used with different search algorithms to process higher resolutions such as 4K (3840×2160) resolution with 30 fps rate. To the best of our knowledge, the proposed architecture is one of the first ASIC motion estimation architectures in the literature for HEVC.

Objective appraisal method, for instance, sum of absolute difference (SAD) or sum of the squared errors (SSE), is adopted by high efficiency video coding (HEVC), which inherits conventional objective rate distortion model. However, these methods fail to achieve high consistency with subjective visual perception. In order to improve computational efficiency of the video coding in HEVC, in this article, structural similarity (SSIM) is considered as rate distortion cost function of HEVC. MSE is introduced into SSIM, so that a new rate distortion cost function can get. This problem can be solved by using the Lagrangian multiplier method. The simulation results showed that compared our proposed algorithm with HM13.0 in AI configuration, the performance of rate distortion is average improved about 2.6%, as well as getting better coding quality. In addition, no significant runtime increases in the stage of encoding.

In the high-efficiency video coding (HEVC) standard, intra prediction has higher computational complexity compared with H.264/AVC (advanced video coding) because of increasing the number of intra prediction modes and also higher number of coding unit (CU) sizes. The HEVC encoder evaluates 35 prediction modes on five possible prediction unit (PU) sizes to find the one with the minimum rate–distortion cost. Although this approach improves coding efficiency, it is very time-consuming. In this paper, we propose a fast intra prediction method to reduce the complexity of I-frame coding. The proposed method consists of three stages which is based on smoothness spatial feature. In the first stage, a measure is introduced to estimate CU smoothness by using sum of absolute differences (SAD) among CU pixels in four directions. By considering that a smooth region can be predicted with larger CUs, when the measured smoothness parameter is lower than a predefined threshold, only the prediction modes in the current CU are evaluated. In the second stage, the number of intra prediction modes is reduced based on the calculated SADs in the previous stage. In the last stage, if the first three candidate modes resulted from rough mode decision stage in the previous PU and the current PU are similar, then the best mode prediction of the previous PU is selected as the best candidate mode. Experimental results indicate that the proposed method can reduce the coding time on average to 43 % and maintain coding video quality, whereas bitrate increases negligibly (0.5 %).

High Efficient Video Coding(HEVC) is the latest generation of video compression standard after Advanced Video Coding(AVC). Which meets the video coding requirements of high-definition and high-definition video, and its real-time performance has been improved too. Inter prediction coding is one of the most frequently used methods in HEVC coding. Whether it is at the coding side or the decoding side, the motion search takes most time in the inter prediction. In the HEVC standard software HM, Test zone (TZ) Search has been used to replace the full search algorithm, which saves a lot of time on motion search, but it is still far from enough. This article proposes a fast search method based on neighbor blocks' move vector (MV). And the relationship between the Inter_2N×2N mode PU with the other modes of PU is used to judge whether narrow the search range. The experimental results show that the proposed algorithm saves about 23.5% motion estimation(ME) time and 19.37% total encoding time compared to TZ Search, and the bit consumption is basically negligible.

High Efficiency Video Coding (HEVC) is a video coding standard that offers higher performance than previous video coding standards such as H.264/AVC. Merge mode is one of the new tools adopted in HEVC to improve the inter-frame coding efficiency. Merge mode saves the bits for the motion vector (MV) by sharing the MV with neighboring blocks. Merge mode estimation (MME) is the process of finding a merge mode candidate, which requires extensive computations and memory accesses due to the associated motion compensation. Although MME is very similar to motion estimation (ME) in many ways, previous research on ME cannot be directly applied to solve many difficulties in designing MME hardware. In this paper, the characteristics of and the computational complexity involved in MME are discussed. To improve the throughput of the MME hardware, partially increased parallelism is efficiently exploited. Furthermore, the $M$ -of- $N$ -pixel combination and flexible memory access schemes are proposed to maximize the scalability to support various block sizes of HEVC and to reduce the time for fetching reference data. The proposed schemes are applied to the MME hardware design in this paper. The proposed hardware can process 56074 of $64\times 64$ coding tree units per second with a clock frequency of 366 MHz, and its gate count is 585.4k with 2 kB of dual-port static RAM.

This paper provides a multiple-layer parallel motion estimation (ME) scheme implemented on GPU for High Efficiency Video Coding (HEVC). The scheme is hierarchically structured, including four layers: coding tree unit (CTU), prediction unit (PU), motion vector (MV) selection and instruction optimization. In PU-layer, costs of various PU sizes were obtained through a SAD (sum of absolute differences) look-up table instead of progressive cost merging. And during MV selection, GPU's comparison instruction was used to avoid branches. At the same time, concurrent CTUs processing and SIMD (Single Instruction, Multiple Data) optimization also improve the performance significantly. Experimental results show that the proposed scheme can take full advantage of GPU and achieves over 90 times speedup compared with the HM10.0 using fast ME.

Fast Interframe mode decision algorithm based on mode mapping and MB activity for MPEG-2 to H.264/AVC transcoding

High Efficiency Video Coding (HEVC) is in development and aims to revolutionize the compression world with a potential 50% bitrate saving over Advanced Video Coding (AVC, or H.264 / MPEG-4 AVC). Is history repeating itself? There was a 10 year gap between the standardization of MPEG-2 Video and AVC, with AVC eventually maturing to deliver a 50% bitrate saving over MPEG-2 Video. The ITU-T|ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) is aggressively aiming for a 2013 distribution date for the HEVC standard. Early investigations indicate there is scope for a 30–50% bitrate saving over AVC. Some may argue this is a technology without a home, but potential applications lie in IPTV over DSL where HEVC would increase service reach, or point-to-point links for newsgathering where bandwidth is at a premium. But the most applicable usages will be delivering the emerging 4Kx2K video format in the same channel capacity as HD AVC today and eventually more power-efficient coding for handheld devices. This paper explores the HEVC tool set enhancements that facilitate the compression gains over AVC, with example sequences compressed in the HM test model, and an exploration into the commercial opportunities which HEVC opens up in the television industry.

With the advent of new video standards such as MPEG-4 part-10 and H.264/H.26L, demands for advanced video coding, particularly in the area of variable block size video motion estimation (VBSME), are increasing. In this paper, we propose a new one-dimensional (1-D) very large-scale integration architecture for full-search VBSME (FSVBSME). The VBS sum of absolute differences (SAD) computation is performed by re-using the results of smaller sub-block computations. These are distributed and combined by incorporating a shuffling mechanism within each processing element. Whereas a conventional 1-D architecture can process only one motion vector (MV), this new architecture can process up to 41 MV sub-blocks (within a macroblock) in the same number of clock cycles.