Rate-controlled requantization transcoding for H.264/AVC video streams

The alteration of compressed video bitstream due to embedding of digital watermark tends to produce unpredictable video bit rate variations which may in turn lead to video playback buffer overflow/underflow or transmission bandwidth violation problems. This paper presents a novel bit rate control technique for real-time MPEG watermarking applications. The proposed bit rate control scheme evaluates the combined bit length of a set of multiple watermarked VLC codewords and successively replaces watermarked VLC codewords having the largest increase in bit length with their corresponding unmarked VLC codewords until a target bit length is achieved. The proposed method offers much flexibility and scalability compared to the pioneering scheme proposed by Hartung and Girod. Experimental results show that the proposed bit rate control scheme is effective in meeting the bit rate targets and capable of improving the watermark detection robustness for different video contents compressed at different bit rates.

In this paper, a novel rate control mechanism is proposed for constant bit rate video streaming. The initial quantization parameter used for encoding a video sequence is determined using the average spatio-temporal complexity of the sequence, its resolution and the target bit rate. Simple linear estimation models are then used to predict the number of bits that would be necessary to encode a frame for a given complexity and quantization parameter. The experimental results demonstrate that our proposed rate control mechanism significantly outperforms the existing rate control scheme in the Joint Model (JM) reference software in terms of Peak Signal to Noise Ratio (PSNR) and consistent perceptual visual quality while achieving the target bit rate. Furthermore, the proposed scheme is validated through implementation on a miniature test-bed.

An operational rate control (RC) scheme based on two-stage encoding is studied in this research, where frame-layer rate control with a constant bit rate to achieve constant video quality is examined. In the first encoding stage, the R-D optimized mode decision and its associated motion estimation (RDO) as well as DCT/Q, IQ/IDCT and entropy coding are performed for all macroblocks (MBs) for a target frame using an initial quantization parameter (QP), which is the QP of its previous frame. In the second encoding stage, the residual signal from the first stage is encoded using several QP values around the initial QP. Given the target bits and distortion for the current frame, the residual signal is finally encoded using the QP determined by comparing target bits and distortion with actual bits and distortion. To reduce the additional coding complexity of the two-stage encoding, upper and lower bounds around target bits and distortion are employed to reduce the number of encoding required in the second stage. Experimental results are given to show the superior performance of two proposed rate control algorithms, where one targets at the constant bit rate while the other at constant quality.

Implementing a reliable unequal error protection (UEP) technique based on motion activity of video frames is considered as an effective method to improve performance of compressed video bitstream transmitted over the wireless network. Most of the existing works are relevant to motion activity of H.264/advanced video coding frames. These mainly determine importance of frames based on either size of partitions having motion information or motion activity of their blocks. However, in the latest coding standard High Efficiency Video Coding (HEVC), there exists an improved block partitioning method, which demands a different representation of motion activity, especially for frames comprising a large number of units. This paper proposes motion density-based scheme of unequal error protection (UEP) for HEVC-compressed video bitstream. In comparison with previously motion activity-based UEP techniques, motion density has better capability to extract importance of the small-size units and consequently determines importance of frames. This is achieved by number of simulations conducted for the inter-prediction mode of the video bitstream.

We present a region of interest (ROI) based rate control for H.263 compatible video conferencing. A face detection and tracking scheme with very low complexity is proposed for segmentation. By analyzing quadratic rate models in frame layer, video object plane (VOP) layer, and macroblock (MB) layer extracted from the test data, a quadratic rate model at the MB layer with a modified physical meaning is proposed to improve the model accuracy. The basic idea is to use a group of uncoded MBs in the current VOP instead of individual MBs to update model parameters. A joint VOP layer and MB layer rate control algorithm is proposed. The VOP layer rate control assigns target bit rate for each VOP based on the coding complexity and visual importance, and determines an average quantization parameter (QP) for each VOP. Some new features of MB layer rate control are designed to utilize both average statistics of a VOP and individual statistics of MB together. The performance is compared with conventional TMN8 and object-based VM8 rate control, better peak SNR (PSNR) for ROI, and more accurate rate control can be achieved for various video sequences. The proposed rate control algorithm can be extended for H.264 ROI-based scalable video coding.

Rate control (RC) is a critical component in learned image compression (LIC), particularly in the emerging JPEG-AI standard, which enables adaptive bitrate achievement to meet diverse bandwidth constraints. JPEG-AI default RC employs an iterative optimization process, wherein a pre-trained RC model is selected and the (generated) latent representations are adjusted based on the mismatch between actual and target bitrates. Despite satisfactory results, such a trial-and-error paradigm necessitates multiple processing cycles, resulting in inevitable computational overhead. We propose an efficient neural rate control framework for JPEG-AI to address this limitation. Our idea is to train a ResNet-based neural control (NRC) to learn the mapping from the input images and target bitrates to the optimal coding parameters. The trained NRC can then be applied to predict the coding parameters based on the new input images and target bitrates directly. Experimental results on DIV2K and MSCOCO datasets show that our NRC achieves comparable rate-distortion performance while reducing encoding time by about 5× compared to JPEG-AI default RC.

The rate control plays a very important role in video streaming applications. The current rate control scheme in the advance video coding standard H.264/AVC still results in a relatively large quality fluctuation for frame-level with higher motion activities. However, the current video coding standards remove the spatial-temporal redundancy based on the signal statistical correlation. Therefore, to reach higher compression efficiency without perceptually degrading of the reconstructed video signal, the properties of the human visual system need to be better exploited. Research effort has been dedicated to modeling the spatial saliency map based on the properties of HVS. The importance of video image varied with the regions, and human visual attention too varied with the importance of region of video image, so using the attention selectivity properties of the human visual system, the paper proposed video coding rate control method based human visual saliency map in H.264/AVC. For each frame, target bits are allocated through the correction of attention region size between the current frame and previous frame, and for each MB, MB's complexity are modified through the saliency and MAD value of MB. At last, the algorithm was implemented in reference software JM12.2. Simulation results showed that, comparing with traditional algorithm, the proposed reduced the coding bit rate, at the same time, and it also improved the subjective quality of reconstructed video in visual attention region.

Based on our rate-distortion (R-D) model, an innovative coding characteristics, texture-complexity (TC), and its adaptive linear forward-prediction model are proposed. For picture quality control in H.264, an overhead bits prediction model is also proposed to best distribute available bits. Experimental results show that the H.264 encoder (JM98), using the proposed rate control algorithm, achieves an objective quality improvement up to 0.477 dB, produces flatter bit-rate curve and meets target bit budget better than outcomes of JM98 rate control method (JVT-G012).

Rate control ensures the coded bit stream can meet channel bandwidth or storage space requirements. however, it may degrade rate-distortion (R-D) performance of encoders. Inspired by temporal dependent rate distortion optimization, this paper proposes a temporal dependency based CTU level rate control algorithm to improve R-D performance while achieving a high bitrate accuracy. First, an estimation method of CTU level R-D model is proposed to better characterize R-D relationship. Second, a formulation combining inter-CTU dependencies and R-D characteristics is constructed for CTU level bit allocation, which aims to minimize the distortion of the whole video sequence under the constraint of frame level bit budget. Finally, in the process of encoding a frame, a strict constraint of bits is used to reduce the difference between numbers of actual and target bits for the current frame. Experimental results show that, on the platform of HEVC test model (HM-16.7), the proposed algorithm achieves considerable R-D performance improvement with smaller bitrate error. Specifically, against HM-16.7 without rate control, the proposed scheme achieves bitrate savings of 4.2% and 3.7% on average under the configurations of low delay P and B frames, respectively.

A new rate control scheme is used to calculate the target bit rate for each frame based on a quadratic formulation of the rate distortion function. The distortion measure are assumed to be either the average quantization scale of a frame or mean square error. The rate distortion function is modeled as a second order function of the inverse of the distortion measure. We presented a closed form solution for the target bit allocation which applies to both the digital storage media (DSM) applications (e.g., MPEG-2) and very low bit rate (VLBR) applications (e.g., MPEG-4). We perform experiments on a wavelet-based coder to demonstrate the versatility of our scheme. Model parameters are estimated using statistical linear regression analysis. Since the estimation uses the past encoded frames of the same picture prediction type (I, P and B pictures), the proposed approach is a single pass rate control technique. For the MPEG-2 coder the fluctuations of the bit counts are significantly reduced by 20 percent to 65 percent in the standard deviation of the bit count while the picture quality remains the same. For the MPEG-4 VM coder, the PSNR is improved by almost one dB as compared to the best MPEG-4 VM rate control using exhaustive combinations of quantization scale parameters.

This paper presents advanced rate control technologies for 3D-HDTV. We remodel the quadratic rate-distortion model for 3D multiview sequences based on their picture types. The proposed method consists of two levels for more accurate bits rate control. In a frame level, it sets quantization parameters based on HVS (human visual system) and reduces error bits by remodeling rate-quantization. In a MB level, rate control is activated to provide stricter buffer regulations and higher bit rate encoding based on the target bits and the quantization parameters calculated at the frame level. The proposed algorithm shows improvements in rate control as well as in PSNR compared to the conventional methods. It also provides more accurate rate control and higher image quality than TM5 and TMN8 at various bit rates.

This paper presents a novel rate control framework for H.264/Advanced Video Coding-based video coding that improves the preservation of gradient-based features like scale-invariant feature transform or speeded up robust feature compared with the default rate control algorithm in the JM reference software. First, a criterion (matching score) for feature preservation on the basis of the bag-of-features concept is proposed. Then, the matching scores are collected as a function of the quantization parameters and analyzed for different feature types. With this analysis, macroblocks are categorized into different groups before encoding. Our rate control algorithm assigns different quantization parameters to each group according to the importance of the group for feature extraction. The experimental results show that our rate control algorithm achieves the desired target bit rate, and more features are preserved compared with videos encoded using the default rate control. The proposed approach not only improves feature preservation, but also leads to a noticeable performance improvement in a real image retrieval system. The rate control framework proposed in this paper is fully standard compatible.

Due to the limited storage capacity and network bandwidth, an efficient rate control (RC) algorithm becomes more and more critical for multi-view video coding (MVC). Based on inter-view dependency and spatio-temporal correlation, a novel bit allocation method for multi-view texture video coding is proposed in this article. Firstly, considering that the distortion in the base view (BV) is directly transmitted to the dependent view (DV) by inter-view skip mode, a joint multi-view RD model is built based on the inter-view dependency. Based on the proposed joint multi-view RD model, a precise power model is derived to represent the target bitrates relationship between the BV and the DV. Secondly, since the P frame in the DV (P-DV) is mainly predicted from the corresponding I frame in the BV (I-BV) by disparity compensated prediction (DCP), the constant proportional relationship between the ratio of the average bitrates of the P-DV to the corresponding I-BV and the ratio of the total bitrates of the DV to the BV is discovered. Based on this discovery, a novel linear model is developed to assign the target bitrates of the P-DV. Finally, considering the spatio-temporal correlation, a new parameter prediction method is proposed for the R-$\lambda $ model in coding tree unit (CTU) level. Extensive experimental results show that the proposed overall method outperforms other state-of-the-art algorithms in terms of RD performance.

Automated vehicle detection plays an essential role in the traffic video surveillance system. Video communication of these traffic cameras over real-world limited bandwidth networks can frequently suffer network congestion or unstable bandwidth, especially in regard to wireless systems. This often hinders the detection of moving vehicles in variable bit-rate video streams. This paper presents a novel approach for vehicle detection based on probabilistic neural networks through artificial neural networks which can accurately detect moving vehicles not only in high bit-rate video streams but also in low bit-rate video streams. The overall results of detection accuracy analyses demonstrate that the proposed approach has a substantially higher degree of both qualitative and quantitative efficacy than other state-of-the-art methods. For instance, the proposed method achieved Similarity and F1 accuracy rates that were up to 77.69% and 81.85% higher than the other compared methods, respectively.

Although high efficiency video coding (HEVC) has achieved high coding performance, it has been shown that dependent rate-distortion optimization (RDO) can still further improve the coding performance at the encoder side. Inspired by our previous work on temporal-dependent RDO, this paper proposes a rate control method at coding tree unit (CTU) level to obtain both higher rate-distortion (R-D) performance and lower bitrate errors. We first formulate the global optimization problem in hybrid video coding and present a general solution framework based on a weighted Lagrange multiplier, where the weighted coefficient is related to the strength of temporal dependency among coding units, and a propagation factor is introduced for quantitative measurement of the temporal dependency. Second, an estimation method of CTU level R-D model is proposed to characterize R-D relationship more accurately. Finally, under the constraint of frame level bit budget, a formulation combining inter-block dependency and R-D characteristic is developed for optimal bit allocation at the CTU level. In the process of encoding a frame, a relaxed constraint algorithm (RCA) and a strict constraint algorithm (SCA) are employed to reduce the difference between numbers of actual and target bits for the current frame, respectively. Experimental results demonstrate that, on the HEVC test model (HM-16.7), our methods including RCA and SCA achieve considerable R-D performance improvement with lower bitrate errors, outperforming the state-of-the-art schemes. It is worth mentioning that even compared with HEVC without rate control, the RCA can still achieve bitrate savings of 5.4% and 5.1% on average under configurations of low delay P and B frames, respectively.