Low-complexity intra coding scheme based on Bayesian and L-BFGS for VVC

High Efficiency Video Coding (HEVC), a new video coding standard currently being established, adopts a quadtree-based Coding Unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity compared to previous video coding standards due to the necessity of finding the best CU partitions. In this paper, a fast CU splitting and pruning method is presented for HEVC intra coding, which allows for significant reduction in computational complexity with small degradations in rate-distortion (RD) performance. The proposed fast splitting and pruning method is performed in two complementary steps: 1) early CU split decision and 2) early CU pruning decision. For CU blocks, the early CU splitting and pruning tests are performed at each CU depth level according to a Bayes decision rule method based on low-complexity RD costs and full RD costs, respectively. The statistical parameters for the early CU split and pruning tests are periodically updated on the fly for each CU depth level to cope with varying signal characteristics. Experimental results show that our proposed fast CU splitting and pruning method reduces the computational complexity of the current HM to about 50% in encoding time with only 0.6% increases in BD rate.

With the development of multimedia equipment and the increasing demand for high-quality video applications, the traditional video coding standard, H.265/High Efficiency Video Coding (HEVC), can no longer effectively satisfy the requirements. To promote the development of high-quality video, a new generation video coding standard, H.266/Versatile Video Coding (H.266/VVC), is established, and it is the inheritance and development of H.265/HEVC. It not only retains many mature technologies and methods in HEVC but also adds some new coding tools, such as wide-angle prediction and Multitype Tree (MTT) partition structure. The MTT partition structure brings a more flexible partition method of Coding Unit (CU), but the accompanying increase in computational complexity is unacceptable. In order to ensure an effective balance between coding efficiency and coding quality, a fast CU partition algorithm based on texture is proposed in this paper. First, the texture complexity of the neighboring CU is used as a threshold for evaluating the complexity of the current CU, so as to skip the unpromising depth. Then, the gradient features are extracted to determine whether the Quad-Tree (QT) partition is executed. Finally, the improved Canny operator is used to extract edge features, and the partition mode in the horizontal or vertical direction is excluded. The algorithm was embedded in VTM7.0, and the video sequences with different resolutions were tested under general experimental configuration. Simulation experiment results show that the average time saving of this method reached 50.56% compared with the anchor algorithm. At the same time, the average BDBR is increased by 1.31%.

In the versatile video coding (VVC) proposed by the Joint Video Exploration Team (JVET), the quad-tree with the nested multi-type tree (QTMT) partition scheme has been adopted based on the quadtree structure in the high efficiency video coding (HEVC). The video coding quality of VVC is better than the HEVC, but the algorithm complexity has also increased greatly. In this work, we present an adaptive CU split decision for intra frame with the pooling-variable convolutional neural network (CNN), targeting at various coding unit (CU) shape. The shape-adaptive CNN is realized by the variable pooling layer size where we can make the most of the pooling layer in CNN and retain the original information. Based on the proposed CNN, the CU split or not will be decided by only one trained network, same architecture and parameters for the CUs with multiple sizes. Moreover, with the proposed shape-based CNN training scheme, the various training sample size can be processed successfully. The CUbased network can avoid the full rate-distortion optimization for the CU split and the CU-level rate control can also be enabled. The experiment results show that the proposed method can save 33% coding time with only 0.99% Bjontegaard Delta bitrate (BD-rate) increase.

This paper presents a parallel motion estimation algorithm on Graphics Processing Units (GPU) with a GPU-based fast Coding Unit (CU) splitting mechanism for speeding up the execution speed of High Efficiency Video Coding (HEVC). Parallel motion estimation algorithms only offer motion vectors to HEVC encoder, but CU splitting decision in HEVC still needs more information to speed up the encoder. Therefore, a mechanism using GPU to signify encoder which CU depth can be split instantly using motion data is designed. With experiments on Kepler GK110 GPU, the proposed parallel algorithm gains over 510 times faster than the full search motion estimation in the HEVC reference software, and the proposed fast CU splitting mechanism can further save 23% total execution time when compared to the general encoder with parallel motion estimation.

3D-HEVC is the latest 3D video coding project of MPEG based on High Efficiency Video Coding (HEVC). The encoding complexity of it is remarkably high due to the coding units (CU) size decision process together with the rate distortion optimization (RDO) process. In this paper, we proposed a fast CU depth decision algorithm taking advantages of the interview correlations. With the CU depth correlation analysis between the independent view and the dependent view, the CU depth of each CU in the dependent view is automatically determined based on inter-view prediction and quad-tree structure constraints. It eliminates the brute force RDO process in determining CU depth, which provides an average of 60% saving in texture encoding complexity with a negligible loss in compression efficiency.

Intraprediction has been significantly improved in high efficiency video coding over H.264/AVC with quad-tree-based coding unit (CU) structure from size 64×64 to 8×8 and more prediction modes. However, these techniques cause a dramatic increase in computational complexity. An intracoding algorithm is proposed that consists of perceptual fast CU size decision algorithm and fast intraprediction mode decision algorithm. First, based on the visual saliency detection, an adaptive and fast CU size decision method is proposed to alleviate intraencoding complexity. Furthermore, a fast intraprediction mode decision algorithm with step halving rough mode decision method and early modes pruning algorithm is presented to selectively check the potential modes and effectively reduce the complexity of computation. Experimental results show that our proposed fast method reduces the computational complexity of the current HM to about 57% in encoding time with only 0.37% increases in BD rate. Meanwhile, the proposed fast algorithm has reasonable peak signal-to-noise ratio losses and nearly the same subjective perceptual quality.

In the high-efficiency video coding (HEVC) standard, intra prediction has higher computational complexity compared with H.264/AVC (advanced video coding) because of increasing the number of intra prediction modes and also higher number of coding unit (CU) sizes. The HEVC encoder evaluates 35 prediction modes on five possible prediction unit (PU) sizes to find the one with the minimum rate–distortion cost. Although this approach improves coding efficiency, it is very time-consuming. In this paper, we propose a fast intra prediction method to reduce the complexity of I-frame coding. The proposed method consists of three stages which is based on smoothness spatial feature. In the first stage, a measure is introduced to estimate CU smoothness by using sum of absolute differences (SAD) among CU pixels in four directions. By considering that a smooth region can be predicted with larger CUs, when the measured smoothness parameter is lower than a predefined threshold, only the prediction modes in the current CU are evaluated. In the second stage, the number of intra prediction modes is reduced based on the calculated SADs in the previous stage. In the last stage, if the first three candidate modes resulted from rough mode decision stage in the previous PU and the current PU are similar, then the best mode prediction of the previous PU is selected as the best candidate mode. Experimental results indicate that the proposed method can reduce the coding time on average to 43 % and maintain coding video quality, whereas bitrate increases negligibly (0.5 %).

High Efficiency Video Coding (HEVC) adopts a quad-tree structured coding unit (CU) and more prediction unit (PU) modes are used in all CU depth levels. These improvements bring better coding performance, but the exhaustive search process for optimal CU and PU selection causes higher computational complexity than earlier standards. To speed up the encoder, this paper proposes a fast intra coding algorithm. Firstly, a fast CU size decision method is introduced, which selects different depth decision methods for each largest coding unit. Secondly, a fast direction mode decision method is proposed. It first compares the direction modes of the parent unit and most probable modes (MPMs) list, and then the first direction mode of rate-distortion optimization (RDO) list is used to early terminate the RDO process. Experimental results show that compared with HM 10.0, the proposed algorithm achieves 47% time reduction, whereas the value of BDBR is only 0.7%.

The high efficiency video coding (HEVC) video coding standard under development can achieve higher compression performance than previous standards, such as MPEG-4, H.263, and H.264/AVC. To improve coding performance, a quad-tree coding structure and a robust rate-distortion (RD) optimization technique is used to select an optimum coding mode. Since the RD costs of all possible coding modes are computed to decide an optimum mode, high computational complexity is induced in the encoder. A fast learning-based coding unit (CU) size selection method is presented for HEVC intra prediction. The proposed algorithm is based on theoretical analysis that shows the non-normalized histogram of oriented gradient (n-HOG) can be used to help select CU size. A codebook is constructed offline by clustering n-HOGs of training sequences for each CU size. The optimum size is determined by comparing the n-HOG of the current CU with the learned codebooks. Experimental results show that the CU size selection scheme speeds up intra coding significantly with negligible loss of peak signal-to-noise ratio.

High-efficiency video coding is the most recent video compression standard. HEVC is designed to decrease the bit rate of video transmission without affecting the video quality. The intra prediction of HEVC features about 35 directional modes. The planar mode and DC modes are included into it. The decision about the most appropriate intra mode within the coding unit of high-efficiency video encoder is a vital component in video coding. The intra mode decision has been a crucial, computationally complex processing step and has a share of 85% in the overall video coding complexity. The optimal mode is selected by rough mode decision (RMD) process from all 35 modes and final decision of partitioning is taken through the rate distortion optimization (RDO) process. The brute force RD cost calculation process consumes a large portion of HEVC encoding complexity. This paper presents analysis of the spread of 35 directional modes over the video frame and the correlation between the homogeneous or non-homogeneous characteristics of video content and the spread of directional modes over the video frame. The proposed method is based on the sum of average gradient evaluated for each of the 35 directional modes which help to reduce the number of candidate modes for rough mode decision and RD cost calculation. The performance of the proposed algorithm is evaluated on three distinct classes of video sequences. The offline classification accuracy of the proposed scheme is measured to be 90%. The exhaustive analysis of mode decision carried out in the proposed method will be subsequently useful for training machine learning algorithm for early decision about coding unit depth and fast prediction of the appropriate intra mode. The early depth decision and reduction in the number of candidate coding units to be passed through iterative RD cost computation will drastically reduce the computation complexity and increase the encoding time of high-efficiency video encoder.

Versatile Video Coding (VVC) is the latest generation of video coding standards released to meet the arrival of the ultra-high-definition (UHD) video era. Compared with the quad-tree (QT) structure in High Efficiency Video Coding (HEVC), VVC obtains excellent coding performance by introducing the division structure of binary-tree (BT) and ternary-tree (TT), but at the same time, it also greatly improves the computational complexity in the coding process. In order to reduce the coding unit (CU) division traversal time in the inter-frame coding process, this paper proposes an algorithm for fast CU division using random forest for the VVC inter-frame coding process. The algorithm first calculates the eigenvalues of the current CU block, and then inputs them into the random forest model to obtain the predicted classification. At the same time, it decides whether to adopt the predicted result based on the decision rule. If adopted, the stack in the encoder is modified to jump over-redundant traversal operations. Experimental results show that the proposed algorithm reduces the encoding time by about 37.30% on average in random access mode, while the video quality loss is only 1.41%.

H.266/Versatile Video Coding (VVC) is the most recent video coding standard developed by the Joint Video Experts Team (JVET). The quad-tree with nested multi-type tree (QTMT) architecture that improves the compression performance of H.266/VVC is introduced. Moreover, H.266/VVC contains a greater number of intra-prediction modes than H.265/High Efficiency Video Coding (HEVC), totalling 67. However, these lead to extremely the coding computational complexity. To cope with the above issues, a fast intra-coding unit (CU) size decision method and a fast intra-prediction mode decision method are proposed in this paper. Specifically, the trained Support Vector Machine (SVM) classifier models are utilized for determining CU partition mode in a fast CU size decision scheme. Furthermore, the quantity of intra-prediction modes added to the RDO mode set decreases in a fast intra-prediction mode decision scheme based on the improved search step. Simulation results illustrate that the proposed overall algorithm can decrease 55.24% encoding runtime with negligible BDBR.

The joint collaborative team on video coding (JCT-VC) is developing the next-generation video coding standard which is called high efficiency video coding (HEVC). In the HEVC, there are three units in block structure: coding unit (CU), prediction unit (PU), and transform unit (TU). The CU is the basic unit of region splitting like macroblock (MB). Each CU performs recursive splitting into four blocks with equal size, starting from the tree block. In this paper, we propose a fast CU depth decision algorithm for HEVC technology to reduce its computational complexity. In2N×2N PU, the proposed method compares the rate-distortion (RD) cost and determines the depth using the compared information. Moreover, in order to speed up the encoding time, the efficient merge SKIP detection method is developed additionally based on the contextual mode information of neighboring CUs. Experimental result shows that the proposed algorithm achieves the average time-saving factor of 44.84% in the random access (RA) at Main profile configuration with the HEVC test model (HM) 10.0 reference software. Compared to HM 10.0 encoder, a small BD-bitrate loss of 0.17% is also observed without significant loss of image quality.

In High Efficiency Video Coding (HEVC), the computational complexity has been increased so that an Early SKIP mode decision method is proposed by using coded block flag of an inter prediction unit (PU) to speed up mode decision with BD-bitrate increase. In this paper, we propose a fast SKIP mode decision algorithm to speed up PU mode decision for HEVC in a rate-distortion (RD) optimization sense. It is based on an adaptive linear predictor for PU mode type and available neighboring SKIP mode units on the quad-tree partition structure in HEVC for adaptive RD optimization. Moreover, the proposed method consists of both local (neighboring PUs) and global (previous PUs in current frame) level schemes on each coding unit (CU) level. In this way, it is possible to exploit high correlations between the RD cost of SKIP mode and spatial characteristics of video sequences by using the distribution of SKIP mode and a linear RD cost predictor. Experimental results show that the encoding computational complexity can be reduced by an average of 36 % with threshold RD cost, when compared to that of the HM 12 reference software.

High-efficiency video coding (HEVC) is a successor to the H.264/AVC standard as the newest video-coding standard using a quad-tree structure with the three block types of a coding unit (CU), a pred...