Online Learning-Based Multi-Stage Complexity Control for Live Video Coding.

High efficiency video coding (HEVC) is the newest video coding standard to greatly increase the coding efficiency of its ancestor H.264/AVC with the aids of its new features such as the quadtree-based coding unit (CU) partitioning, a simple deblocking filter, and other advanced coding techniques. However, HEVC delivers a highly increased computation complexity, which is mainly due to the exhaustive rate distortion optimization search of quadtree-based CU partitioning. Firstly, a feature reduction approach is proposed on a fuzzy support vector machine (SVM) based CU size decision method. The proposed feature reduction approach with rate control (RC) can reduce computational complexity by eliminating some correlated features of a fuzzy SVM-based CU size decision method under a similar coding efficiency. According to the empirical results, our approach can achieve up to 3% of complexity reduction under the same rate distortion (RD) performance over a fuzzy SVM-based approach. Secondly, instead of machine learning (ML) based fast algorithm approach, a CU partitioning pattern optimization method based on genetic algorithm (GA) is proposed to save the computational complexity of a hierarchical quadtree-based CU partitioning. The required coding unit partitioning pattern for an exhaustive partitioning and the rate distortion cost are efficiently considered as the chromosome and the fitness function of the genetic algorithm, respectively. To reduce the computational time, CU partitioning patterns of the key frame is searched and shared to other consecutive frames by taking into account the highly temporal correlation. Our evaluation results show that the proposed method can achieve 62.5% and 16.7% computational complexity reduction on average at 8 Mbps with a negligible average quality degradation compared with HM16.5 and state-of-the-art support vector machine-based fast algorithm, respectively, under low delay P configuration with rate control while 64.1% and 15.1% under low delay configuration with rate control.

To improve compression performance, High Efficiency Video Coding (HEVC) employs a quad-tree based block representation, namely Coding Tree Unit (CTU), which can support larger partitions and more coding modes than a traditional macroblock. Despite its high compression efficiency, the number of combinations of coding modes increases dramatically, which results in high computational complexity at the encoder. Here we propose a flexible framework for HEVC coding mode selection, with a user-defined global complexity factor. Based on linear programming, a hierarchical complexity allocation scheme is developed to allocate computational complexities among frames and Coding Units (CUs) to maximize the overall Rate-Distortion (RD) performance. In each CU, with the allocated complexity factor, a mode mapping based approach is employed for coding mode selection. Extensive experiments demonstrate that, with a series of global complexity factors, the proposed model can achieve good trade-offs between computational complexity and RD performance.

Self-learning residual model for fast intra CU size decision in 3D-HEVC

The high efficiency video coding (HEVC) video coding standard under development can achieve higher compression performance than previous standards, such as MPEG-4, H.263, and H.264/AVC. To improve coding performance, a quad-tree coding structure and a robust rate-distortion (RD) optimization technique is used to select an optimum coding mode. Since the RD costs of all possible coding modes are computed to decide an optimum mode, high computational complexity is induced in the encoder. A fast learning-based coding unit (CU) size selection method is presented for HEVC intra prediction. The proposed algorithm is based on theoretical analysis that shows the non-normalized histogram of oriented gradient (n-HOG) can be used to help select CU size. A codebook is constructed offline by clustering n-HOGs of training sequences for each CU size. The optimum size is determined by comparing the n-HOG of the current CU with the learned codebooks. Experimental results show that the CU size selection scheme speeds up intra coding significantly with negligible loss of peak signal-to-noise ratio.

High Efficiency Video Coding (HEVC), a new video coding standard currently being established, adopts a quadtree-based Coding Unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity compared to previous video coding standards due to the necessity of finding the best CU partitions. In this paper, a fast CU splitting and pruning method is presented for HEVC intra coding, which allows for significant reduction in computational complexity with small degradations in rate-distortion (RD) performance. The proposed fast splitting and pruning method is performed in two complementary steps: 1) early CU split decision and 2) early CU pruning decision. For CU blocks, the early CU splitting and pruning tests are performed at each CU depth level according to a Bayes decision rule method based on low-complexity RD costs and full RD costs, respectively. The statistical parameters for the early CU split and pruning tests are periodically updated on the fly for each CU depth level to cope with varying signal characteristics. Experimental results show that our proposed fast CU splitting and pruning method reduces the computational complexity of the current HM to about 50% in encoding time with only 0.6% increases in BD rate.

The emerging high efficiency video coding standard (HEVC) adopts the quadtree-structured coding unit (CU). Each CU allows recursive splitting into four equal sub-CUs. At each depth level (CU size), the test model of HEVC (HM) performs motion estimation (ME) with different sizes including 2N × 2N, 2N × N, N × 2N and N × N. ME process in HM is performed using all the possible depth levels and prediction modes to find the one with the least rate distortion (RD) cost using Lagrange multiplier. This achieves the highest coding efficiency but requires a very high computational complexity. In this paper, we propose a fast CU size decision algorithm for HM. Since the optimal depth level is highly content-dependent, it is not efficient to use all levels. We can determine CU depth range (including the minimum depth level and the maximum depth level) and skip some specific depth levels rarely used in the previous frame and neighboring CUs. Besides, the proposed algorithm also introduces early termination methods based on motion homogeneity checking, RD cost checking and SKIP mode checking to skip ME on unnecessary CU sizes. Experimental results demonstrate that the proposed algorithm can significantly reduce computational complexity while maintaining almost the same RD performance as the original HEVC encoder.

Symmetry considerations play a key role in modern science, and any differentiable symmetry of the action of a physical system has a corresponding conservation law. Symmetry may be regarded as reduction of Entropy. This work focuses on reducing the computational complexity of modern video coding standards by using the maximum entropy principle. The high computational complexity of the coding unit (CU) size decision in modern video coding standards is a critical challenge for real-time applications. This problem is solved in a novel approach considering CU termination, skip, and normal decisions as three-class making problems. The maximum entropy model (MEM) is formulated to the CU size decision problem, which can optimize the conditional entropy; the improved iterative scaling (IIS) algorithm is used to solve this optimization problem. The classification features consist of the spatio-temporal information of the CU, including the rate–distortion (RD) cost, coded block flag (CBF), and depth. For the case analysis, the proposed method is based on High Efficiency Video Coding (H.265/HEVC) standards. The experimental results demonstrate that the proposed method can reduce the computational complexity of the H.265/HEVC encoder significantly. Compared with the H.265/HEVC reference model, the proposed method can reduce the average encoding time by 53.27% and 56.36% under low delay and random access configurations, while Bjontegaard Delta Bit Rates (BD-BRs) are 0.72% and 0.93% on average.

High Efficiency Video Coding (HEVC) is a new generation video coding standard by Joint Collaborative Team on Video Coding (JCTVC). The development of HEVC aims at the compression for ultra high definition videos. In HEVC, the encoder adopts a quad-tree based video coding structure, in which each incoming frame is split into a set of coding tree blocks (CTB) for compression. Each CTB can be recursively split into coding unit (CU), predict unit (PU) and transform unit (TU). This paper proposes an early termination algorithm of recursive partition based on the Sobel edge density of CTB in HEVC intra coding mode. The proposed method can reduce the depth of recursive partition for each CTB and thus save a lot of encoding time. Experimental results by HM10.1 show that the average time savings is about 14.2% and the resultant increase of encoded BD-Rate is only 0.02%. The proposed method can achieve significant time savings with negligible BD-Rate increase.

High Efficiency Video Coding (HEVC), the current video coding standard provides improved compression performance compared to its predecessor, AVC/H.264. It adopts a highly flexible quad-tree coding block partitioning which results in significant compression gain. However, in order to select the best Coding Unit (CU) size, every coding unit of all sizes needs to be traversed. This imposes a heavy computational burden on the encoder. In this paper, a fast early coding unit selection is therefore proposed based on the Hadamard Absolute Difference (HAD) cost. For each CU size, the splitting process is terminated if the HAD cost is below a specified threshold. In addition, for the 8 × 8 CUs, only Prediction Units (PUs) of size 2N × 2N are allowed based on the difference between the HAD cost of the best mode and the DC coefficient. Experimental results show that the proposed method provides a decrease of 41.2% computation time on average, compared to HEVC reference software, with an increase of only 0.003% in bit-rate.

The latest video coding standard High Efficiency Video Coding (HEVC) is developed by Joint Collaborative Team on Video Coding(JCTVC). HEVC can achieve better compression performance than any other coding standards, which adopts the quadtree structured coding tree unit (CTU). In encoding process, the CTU is recursively split into four equally sized coding unit (CU)s. Along with the multiple sizes of the CUs, the number of prediction unit(PU) modes has been largely increased, which raises the computational complexity in HEVC encoder greatly. In this paper, a fast inter mode decision algorithm based on motion estimation (ME) and texture feature is proposed to speed up the PU mode decision in HEVC encoder. Experimental results demonstrate that the overall algorithm that is proposed can save 41% encoding time on average while the decrease of coding quality is negligible, when compared to that of the HM10.0 reference software.

In comparison with H.264/Advanced Video Coding, the newest video coding standard, High Efficiency Video Coding (HEVC), improves video coding rate-distortion (RD) performance, but at the price of significant increase in its encoding complexity, especially, in intra-mode decision due to the adoption of more complex block partitions and more candidate intra-prediction modes (IPMs). To reduce the mode decision complexity in HEVC intra-frame coding, while maintaining its RD performance, in this paper, we first formulate the mode decision problem in intra-frame coding as a Bayesian decision problem based on the newly proposed transparent composite model (TCM) for discrete cosine transform coefficients, and then present an outlier-based fast intra-mode decision (OIMD) algorithm. The proposed OIMD algorithm reduces the complexity using outliers identified by TCM to make a fast coding unit split/nonsplit decision and reduce the number of IPMs to be compared. To further take advantage of the outlier information furnished by TCM, we also refine entropy coding in HEVC by encoding the outlier information first, and then the actual mode decision conditionally given the outlier information. The proposed OIMD algorithm can work with and without the proposed entropy coding refinement. Experiments show that for the all-intra-main test configuration of HEVC: 1) when applied alone, the proposed OIMD algorithm reduces, on average, the encoding time (ET) by 50% with 0.7% Bjontegaard distortion (BD)-rate increase and 2) when applied in conjunction with the proposed entropy coding refinement, it reduces, on average, both the ET by 50% and BD-rate by 0.15%.

The joint collaborative team on video coding (JCT-VC) is developing the next-generation video coding standard which is called high efficiency video coding (HEVC). In the HEVC, there are three units in block structure: coding unit (CU), prediction unit (PU), and transform unit (TU). The CU is the basic unit of region splitting like macroblock (MB). Each CU performs recursive splitting into four blocks with equal size, starting from the tree block. In this paper, we propose a fast CU depth decision algorithm for HEVC technology to reduce its computational complexity. In2N×2N PU, the proposed method compares the rate-distortion (RD) cost and determines the depth using the compared information. Moreover, in order to speed up the encoding time, the efficient merge SKIP detection method is developed additionally based on the contextual mode information of neighboring CUs. Experimental result shows that the proposed algorithm achieves the average time-saving factor of 44.84% in the random access (RA) at Main profile configuration with the HEVC test model (HM) 10.0 reference software. Compared to HM 10.0 encoder, a small BD-bitrate loss of 0.17% is also observed without significant loss of image quality.

In the high-efficiency video coding (HEVC) standard, intra prediction has higher computational complexity compared with H.264/AVC (advanced video coding) because of increasing the number of intra prediction modes and also higher number of coding unit (CU) sizes. The HEVC encoder evaluates 35 prediction modes on five possible prediction unit (PU) sizes to find the one with the minimum rate–distortion cost. Although this approach improves coding efficiency, it is very time-consuming. In this paper, we propose a fast intra prediction method to reduce the complexity of I-frame coding. The proposed method consists of three stages which is based on smoothness spatial feature. In the first stage, a measure is introduced to estimate CU smoothness by using sum of absolute differences (SAD) among CU pixels in four directions. By considering that a smooth region can be predicted with larger CUs, when the measured smoothness parameter is lower than a predefined threshold, only the prediction modes in the current CU are evaluated. In the second stage, the number of intra prediction modes is reduced based on the calculated SADs in the previous stage. In the last stage, if the first three candidate modes resulted from rough mode decision stage in the previous PU and the current PU are similar, then the best mode prediction of the previous PU is selected as the best candidate mode. Experimental results indicate that the proposed method can reduce the coding time on average to 43 % and maintain coding video quality, whereas bitrate increases negligibly (0.5 %).

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

To satisfy the rapidly increasing demands for high resolution video, the Joint Collaborative Team on Video Coding (JCT-VC) developed a new video coding standard, i.e. High-efficiency Video Coding (HEVC). With the same perceptual video quality as H.264, HEVC can achieve a bit rate reduction of approximately 50%. However, the quad tree structured coding units (CUs) cause high computational complexity. In this paper, a novel algorithm is proposed to decrease the computational complexity of HEVC intra coding. We mainly use the total coding bits of CUs as threshold to terminate CUs' partition. Furthermore, we also consider the spatial correlation between the current LCU and its neighbor LCUs to predict the current largest CU minimum depth so that we can skip some certain CUs' rate distortion cost checking.