New rate-complexity-quantization modeling and efficient rate control for H.264/AVC

High-Efficiency Video Coding (HEVC) is the latest video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). An effective rate control mechanism plays a critical role in making the best possible use of the network resources. The coding performance can be maximised through the appropriate allocation of bits under the constraints of a total bit rate budget and the buffer size. Existing rate control strategies generally determine the rate control parameters according to previous encoding results. However, there is no priori-knowledge available when the first frame of a video sequence is encoded or scene changes occur. Consequently, it is challenging to choose appropriate rate control parameters in such cases. In order to address this issue, we investigated deep learning oriented methods and proposed a Convolutional Neural Network (CNN)-based rate control scheme for HEVC intra-coding. As the CNN has the advantage of local perception and the Coding Tree Unit's (CTU's) Quantisation Parameter (QP) value is determined during the CTU level rate control process, we applied the CNN at the CTU level as well to achieve an accurate QP prediction. An improved QP determination model (R-QP model) involving only two model coefficients (µ and v) was suggested to simplify the QP calculation process. A CNN was used to predict the model coefficients, namely µ and v. Our CNN has four convolution layers, two pooling layers and three fully connected layers. Each of the convolution layers contains a Rectified Linear Unit (ReLU). The last fully connected layer outputs the predicted values of µ and v. The images from the UCID dataset and the RAISE dataset were used to train the CNN. In order to obtain a universal model, none of the frames in the HEVC common test sequences were employed as training images. In order to achieve a reasonable bit budget allocation, a CNN-based framework was used to predict the bit consumption for each CTU according to the entire picture content that it contains. At the training stage, the network took the original images and the QP values as inputs, and the corresponding actual bit consumptions were used as the labels. Once the CNN coefficients have been obtained through training, the bit consumption for each CTU under different QP values can be directly predicted from the original images. The predicted bit consumption was subsequently used to readjust the bit budget for each CTU to achieve a more reasonable bit allocation. We implemented the proposed CNN-based rate control scheme in the HEVC reference software HM 16.9. As the priori-knowledge of the previous encoding results had not been considered in the proposed algorithm, only the first frame of each video sequence was encoded. The experimental results showed that our proposed scheme performs better than the default rate control implementation in the HEVC reference software, namely the BDBR is reduced by 1.33% on average while maintaining the same reconstructed picture quality. Compared with the state-of-the-art CNN-based rate control algorithm proposed by Li, our schemes achieved better coding efficiency. It can be concluded that the proposed R-QP model is effective in describing the rate-distortion relationship in HEVC, and the CNN-based bit allocation scheme is appropriate for ensuring that the best use of each bit is made.

An operational rate control (RC) scheme based on two-stage encoding is studied in this research, where frame-layer rate control with a constant bit rate to achieve constant video quality is examined. In the first encoding stage, the R-D optimized mode decision and its associated motion estimation (RDO) as well as DCT/Q, IQ/IDCT and entropy coding are performed for all macroblocks (MBs) for a target frame using an initial quantization parameter (QP), which is the QP of its previous frame. In the second encoding stage, the residual signal from the first stage is encoded using several QP values around the initial QP. Given the target bits and distortion for the current frame, the residual signal is finally encoded using the QP determined by comparing target bits and distortion with actual bits and distortion. To reduce the additional coding complexity of the two-stage encoding, upper and lower bounds around target bits and distortion are employed to reduce the number of encoding required in the second stage. Experimental results are given to show the superior performance of two proposed rate control algorithms, where one targets at the constant bit rate while the other at constant quality.

A New Rate Control Scheme in HDTV Systems

In this work, a CTU level rate control algorithm is proposed for High Efficiency Video Coding (HEVC). On top of the CTU level rate control algorithm, an efficient bit allocation method considering the HEVC hierarchical coding structure is specifically designed. Instead of directly applying the rate distortion model at the CTU level, it's proposed to derive the CTU level quantization parameter (QP) based on the frame level QP with feedback of the coding status of CTUs. To further improve the coding performance of the CTU level rate control, a QP adjustment strategy is proposed and incorporated into the rate control algorithm. Experimental results show that the proposed CTU level rate control algorithm can achieve better coding performance than the state-of-the-art rate control scheme for HEVC (1.9% for RA, 3.2% for LD). Moreover, the coding quality become much smoother with the proposed algorithm.

The purpose of this paper is to improve allocation of the number of bits by accurately estimating the target bits in H.264/AVC rate control. In our scheme, we propose an enhancement method of the target unit-layer bit allocation. The enhancement uses a frame and unit complexity estimation to improve the existing mean absolute difference (MAD) complexity measurement. Using the statistical characteristics, we obtained the change in occurrence bits about Quantization Parameter (QP) to apply the bit amount by QP from the video characteristic, and applied this to the estimated bit amount of the current frame. The simulation results show that the proposed rate control scheme not only achieved time savings of more than 99% over the existing rate control algorithm, but also that the PSNR and bit rate were almost the same as the performance in all sequences.

Consistent video quality is important for video coding applications, which is also a popular optimization target for rate control. In this paper, a rate control scheme is proposed to reduce the fluctuation of video quality. First, we set up the optimization formulation of distortion and derive the distortion model by analyzing quadtree-based coding unit (CU) structure in high efficiency video coding (HEVC). Then the frame bit allocation algorithm is proposed by considering the inter-dependency. After the frame basic quantization parameter (QP) is obtained, the solution of optimization formulation finally regulates QP to maintain the consistent video quality. Experimental results show that the proposed rate control scheme can reduce the fluctuation of video quality up to 92.9% than benchmark, where the average reduction is 74.1%.

This paper presents an efficient rate control algorithm based on estimation theory. We measure a conditional mean by estimating a joint probability density function (PDF) using Parzen's window. The training data can estimate the nonlinear rate-distortion (R-D) relationship between the quantization parameter (QP) and the bits spent for each macroblock depending on the sum of absolute differences (SAD). We increase the accuracy of this joint PDF by clustering the training data depending on the QP values. This localization helps the estimation process to be very accurate. Next we apply a mean operator to simplify the conditional mean estimation of the rate given the SAD and QP values. This information is stored into three look-up tables depending on picture types. We use these tables to find the optimal QP values in least-mean-square (LMS) sense for a given bit budget of the current frame. Simulation results show that the proposed algorithm outperforms the informative MPEG-4 rate control algorithm in terms of reproduced image quality and coding efficiency while requiring much less implementation complexity. Most of all it keeps the bit rate very close to the required bit-rate due to the accuracy of the conditional mean estimator that solves the nonlinear R-D function.

Rate control plays an important role in H.264/AVC video coding standard. For the frame layer rate control, since the estimated values of quantization parameter (QP) of the initial frame vary from the image complexity, this paper firstly proposes an improved method for the initial QP estimation to make it more accurate. For the macro block (MB) layer rate control, an improved MB layer rate control algorithm based on human perception is also proposed in this paper. The proposed method classifies MBs by how visually attractive the MBs are and set perception-weights to different MBs. Then the encoder adjusts the QP according to the classified level and combines the perception-weights with structural similarity (SSIM) index to build up an improved rate-distortion optimization (RDO) model. Our experimental results show that the proposed algorithms outperform the original encoder x264 (Version 0.76.X) and significantly reduce the output bit-rate up to almost 50% without dropping down the visual quality of the video for low bit-rate applications.

This paper presents an efficient rate control scheme for the H.264/AVC video coding in low-delay environments. In our scheme, we propose an enhancement to the buffer-status based H.264/AVC bit allocation method. The enhancement is by using a PSNR-based frame complexity estimation to improve the existing mean absolute difference based (MAD-based) complexity measure. Bit allocation to each frame is not just computed by encoder buffer status but also adjusted by a combined frame complexity measure. To prevent the buffer from undesirable overflow or underflow under small buffer size constraint in low delay environment,the computed quantization parameter (QP) for the current MB is adjusted based on actual encoding results at that point. We also propose to compare the bits produced by each mode with the average target bits per MB to dynamically modify Lagrange multiplier (/spl lambda//sub MODE/) for mode decision. The objective of QP and /spl lambda//sub MODE/ adjustment is to produce bits as close to the frame target as possible, which is especially important for low delay applications. Simulation results show that the H.264 coder, using our proposed scheme, obtains significant improvement for the mismatch ratio of target bits and actual bits in all testing cases, achieves a visual quality improvement of about 0.6 dB on the average, performs better for buffer overflow and underflow,and achieves a similar or smaller PSNR deviation.

We present a region of interest (ROI) based rate control for H.263 compatible video conferencing. A face detection and tracking scheme with very low complexity is proposed for segmentation. By analyzing quadratic rate models in frame layer, video object plane (VOP) layer, and macroblock (MB) layer extracted from the test data, a quadratic rate model at the MB layer with a modified physical meaning is proposed to improve the model accuracy. The basic idea is to use a group of uncoded MBs in the current VOP instead of individual MBs to update model parameters. A joint VOP layer and MB layer rate control algorithm is proposed. The VOP layer rate control assigns target bit rate for each VOP based on the coding complexity and visual importance, and determines an average quantization parameter (QP) for each VOP. Some new features of MB layer rate control are designed to utilize both average statistics of a VOP and individual statistics of MB together. The performance is compared with conventional TMN8 and object-based VM8 rate control, better peak SNR (PSNR) for ROI, and more accurate rate control can be achieved for various video sequences. The proposed rate control algorithm can be extended for H.264 ROI-based scalable video coding.

This paper points out some problems inherent in H.264/AVC intra frame rate control (RC) and presents several new algorithms to solve them. Three main algorithms are proposed in this paper. Multiple quantization parameters (QPs) determination algorithm based on the statistics of the deviation measure is proposed. Thus, relatively accurate QP estimation can be achieved since it takes the features of the current frame content into consideration. Two slice-based RC schemes are also proposed to stabilize the output bit rate. The scene change is detected with the complexity of adjacent frames based on deviation measure, which not only generates stable bit rate output but also improves the video quality. The combination of all the above algorithms improve the average peak signal-to-noise ratio (PSNR) by 0.74 dB while keeping the mismatch of the bit rate output less than 7.18% compared to the H.264/AVC reference model, JM12.0, under various sequences.

This paper presents advanced rate control technologies for 3D-HDTV. We remodel the quadratic rate-distortion model for 3D multiview sequences based on their picture types. The proposed method consists of two levels for more accurate bits rate control. In a frame level, it sets quantization parameters based on HVS (human visual system) and reduces error bits by remodeling rate-quantization. In a MB level, rate control is activated to provide stricter buffer regulations and higher bit rate encoding based on the target bits and the quantization parameters calculated at the frame level. The proposed algorithm shows improvements in rate control as well as in PSNR compared to the conventional methods. It also provides more accurate rate control and higher image quality than TM5 and TMN8 at various bit rates.

This paper presents a novel rate control framework for H.264/Advanced Video Coding-based video coding that improves the preservation of gradient-based features like scale-invariant feature transform or speeded up robust feature compared with the default rate control algorithm in the JM reference software. First, a criterion (matching score) for feature preservation on the basis of the bag-of-features concept is proposed. Then, the matching scores are collected as a function of the quantization parameters and analyzed for different feature types. With this analysis, macroblocks are categorized into different groups before encoding. Our rate control algorithm assigns different quantization parameters to each group according to the importance of the group for feature extraction. The experimental results show that our rate control algorithm achieves the desired target bit rate, and more features are preserved compared with videos encoded using the default rate control. The proposed approach not only improves feature preservation, but also leads to a noticeable performance improvement in a real image retrieval system. The rate control framework proposed in this paper is fully standard compatible.

This paper proposes an efficient binary representation of delta Quantization Parameter (QP) for High Efficiency Video Coding (HEVC). Video encoders adapt QPs of coding blocks for visual quality optimization and rate control. Although they send only delta QPs (dQPs) obtained by causal prediction, the side information overhead is expensive. Therefore the HEVC design necessitates an efficient dQP coding. The proposed scheme converts a dQP to a binary string in which the first and second bins indicate the significance and sign of the dQP respectively and the rest represents the magnitude minus 1. Furthermore, it detects and truncates redundant bins in the binary strings by using the sign and an admissible dQP range. Thus it reduces the length of dQP binary strings and improves dQP coding efficiency. Simulation results using HEVC reference software demonstrate that the proposed scheme improves the dQP coding efficiency by 6% while reducing its bin rates by 25%.

Rate control plays an important role in video coding and transmission. In this paper, instead of using rate- distortion (R-D) model to perform rate control as many existing methods do, a rate-quality (R-Q) model of H.264/AVC has first been proposed to represent the coding characteristics of H.264, where PSNR (peak signal-to-noise ratio) is used to represent the reconstructed video quality. Then a H.264 rate control method at frame layer has been presented based on the proposed R-Q model. In this method, the quantization parameter (i.e. quantization step) of the current frame is determined by the proposed R-Q model according to the coding statistics of the previous coded frames. The experimental results show that, compared with the rate control algorithm of JM86 reference software, the output bit rate achieved by our proposed one is more close to the target bit rate while the reconstructed quality is comparable.