GHEVC: An Efficient HEVC Decoder for Graphics Processing Units

Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarity

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

Scalable High efficiency Video Coding (SHVC) is the scalable extension of the latest video coding standard High Efficiency Video Coding (HEVC). One of the key novelties introduced by SHVC is that it enables hybrid codec scalability. This basically means that the video layers can be encoded with different video standards providing backward compatibility between codecs. In this paper, we propose a software parallel SHVC decoder in hybrid codec scalability configuration. The proposed design consists of an Advanced Video Coding (AVC) decoder for the Base Layer (BL) and a HEVC decoder for the Enhanced Layer (EL). In order to perform Inter Layer Prediction (ILP), a communication of decoding states and outputs is established between the two decoders. While the native frame based parallelism is still allowed within the two decoders, the proposed design also enables the use of frame based parallelism between the two decoders. The proposed software design enables a real time decoding of the HEVC EL at 2160p60 while the AVC base layer is decoded at 1080p60 for ×2 spatial scalability.

Runtime Energy versus Quality Tuning in Motion Compensation Filters for HEVC

This paper presents a parallel motion estimation algorithm on Graphics Processing Units (GPU) with a GPU-based fast Coding Unit (CU) splitting mechanism for speeding up the execution speed of High Efficiency Video Coding (HEVC). Parallel motion estimation algorithms only offer motion vectors to HEVC encoder, but CU splitting decision in HEVC still needs more information to speed up the encoder. Therefore, a mechanism using GPU to signify encoder which CU depth can be split instantly using motion data is designed. With experiments on Kepler GK110 GPU, the proposed parallel algorithm gains over 510 times faster than the full search motion estimation in the HEVC reference software, and the proposed fast CU splitting mechanism can further save 23% total execution time when compared to the general encoder with parallel motion estimation.

High Efficiency Video Coding (HEVC) is a new video coding standard created by the JCT-VC group within ISO/IEC and ITU-T. HEVC is targeted to provide the same quality as H.264 at about half of the bit-rate and will replace soon to its predecessor in multimedia consumer applications. Up to now, only a few decoder implementations have been reported, most of them oriented to carry out a complexity analysis. In this paper, a DSP-based implementation of the HEVC HM9.0 decoder is presented. Up to the best of our knowledge, it is the first DSP-based implementation shown in the scientific literature. Several tests have been carried out to measure the decoder performance and the computational load distribution among its functional blocks. These results have been compared with the ones obtained with the decoder implementations reported up to date. Finally, based on the results obtained in previous works regarding software optimization of DSP-based decoders, realtime could be achieved for SD formats with a single DSP after optimizing our HEVC decoder. For HD formats, multi-DSP technology will be needed.

Fractional motion compensation (FMC) has been widely adopted in modern video coding standards such as high-efficiency video coding (HEVC). The fractional pixels in FMC are generated by the interpolation filters, and the coefficients of the filters are designed using a Fourier decomposition of the discrete cosine transform (DCT) in HEVC. To overcome the compression deficiency caused by these handcrafted filters, deep learning-based interpolation filter approaches have been introduced recently. In this paper, we analyze the drawbacks of the mean square error (MSE) only loss function which is widely adopted in existing learned-based approaches, and introduce the rate-distortion optimization (RDO) theory into network training to formulate a joint loss function to overcome this issue. Then, a new training scheme is proposed for the detailed implementation of the joint loss function. Besides the calculated MSE loss, the residual will be sent into a rate estimation model which includes three stages: transform/quantization/rate modeling, and the output rate cost will be used as the second part of the loss function. Finally, we propose a novel network structure that utilizes the deformable convolution layer and the residual dense block to enhance the feature extraction and transmission ability to adapt to complex video contents. The proposed RDO-based interpolation filter (RDOIF) has been integrated into HEVC reference software HM-16.7, and extensive experiments are conducted to verify the efficiency of the proposed method. Experimental results show that the proposed RDOIF method achieves 3.1%, 2.4%, and 1.4% BD-rate reduction on average under the low-delay-P, low-delay-B, and random access configurations, respectively. When implemented in the versatile video coding (VVC) test model VTM-10.0, the proposed method can achieve 2.3%, 1.2%, and 0.5% BD-rate savings on average under the low-delay-P, low-delay-B, and random access configurations, respectively.

This work presents a comparison of two implementations of the last software version of The High Efficiency Video Coding (HEVC) decoder in a single low cost processor ARM Cortex-A series using NEON architecture which is a Single Input Multiple Data (SIMD). By using this technology of optimization, the whole execution time is reduced up to x4. We have analyzed separately all the blocks in the decoder and determined the consumed time percentage of each one compared to the whole time. This allows us to detect the more time consuming blocks. Then, we can find an optimal way to accelerate their functioning.

Intra prediction is an important component of video compression. The High Efficiency Video Coding (HEVC) standard supports advanced Intra prediction tools to achieve remarkable Intra coding performance. However, higher compression efficiency performance may be achieved using Combined Intra Prediction (CIP). CIP consists in combining conventional Intra reference samples, with samples extracted from within the current block being encoded. In this paper, an improvement to the original CIP approach is proposed to achieve higher compression efficiency. Due to the fact the encoder does not have access to reconstruction samples while compressing a block, a controlled drift occurs at the block level between the Intra prediction blocks generated at the encoder and decoder when using CIP. The proposed approach aims at reducing such a drift, hence improving CIP performance. This Improved CIP is able to achieve up to 1.4 % BD-rate savings with respect to the HEVC reference software. The paper also proposes the combination of Improved CIP with an additional tool for enhancing the accuracy of Intra prediction. Up to 1.9 % BD-rate savings can be achieved using this approach.

High Efficient Video Coding (HEVC) is the latest standard in the video transmission sector, and is designed to improve the compression rate and efficiency of video streaming applications. However, one of the many design constraints to improve HEVC is the reliability of transmission. HEVC introduces a sever output degradation when error propagation occurs because of its high compression rate. In this paper, the performance of HEVC is evaluated by video streaming over a Rayleigh distributed environment. The transmission over Rayleigh distributed environment will generate lost and dropped packets because of attenuation in the reception power and limited capacity of the network. The packet loss is then analyzed such that it obeys the statistical envelope of the propagation model used. The wireless simulation generates error patterns which depicts which frame and slice IDs in the video sequence have experienced a packet loss, which will be used as reference by the modified HEVC decoder for motion concealment. The results measured from the decoded video quality are calculated using the peak-signal-to-noise ratio (PSNR) which is compared to the percent error rate. These results were obtained by setting bit rates of 32 kbps and 128 kbps for three video sequences with different motion levels, as well as setting the bytes per slice to 16. In a wireless network with a Rayleigh distributed channel, increasing the bit rate and motion level of the video sequence will decrease the overall PSNR.

H.264/AVC is regarded as a popular video coding standard, and is widely used in multimedia applications. However, with an increasing demand for better quality videos, high efficiency video coding (HEVC) is all set to serve as the successor to H.264/AVC for higher resolution video applications. Since a majority of the multimedia devices have already been operating based on the H.264/AVC standard, it may not be worthwhile to completely replace the existing software and hardware components by different modules in order to adopt HEVC in such devices. Need is therefore felt to design a decoder for supporting H.264/AVC as well as HEVC, rather than attempting individual designs. This paper introduces a new dual‐standard deblocking filter architecture, which supports both H.264/AVC and HEVC standards. Algorithmic verification has been done in Matlab and then an appropriate VLSI architecture has been implemented on FPGA as well as in ASIC domain. The proposed architecture takes 26 clock cycles for H.264/AVC and 14 cycles for HEVC to complete the filtering of a 16 × 16 pixel block. It consumes 5.80 mW normalised power and occupies an area equivalent to 70.1k equivalent gate at frequency of 100 MHz. The proposed architecture takes 8.42 ms to filter the 4K ultra high definition (UHD) (3840 × 2160) frame in H.264 standard, and it takes 18 ms to filter the 8K UHD (7680 × 4320) frame in HEVC standard.

High Efficiency Video Coding (HEVC) introduced many new coding tools to gain improved coding efficiency. However, it greatly introduces computational cost, and this will give rise to the demand for the fast algorithm that can satisfy the real-time encoding of HEVC encoder for low bandwidth environment. In this paper we introduce and evaluate a novel machine learning based approach that uses multi-view clustering to reduce the complexity of intra-coding algorithm based on the analysis of original texture and its neighborhood. The proposed approach is based on the hypothesis that intra coding mode decisions in HEVC video have a correlation with the intensities of adjacent CUs within a CTU. The proposed approach consists of two steps, coding unit size decision and reduction of candidate modes by using multi-view clustering. This results in a significant reduction in encoding time. The results show that multi-view clustering has a promising application in video encoding and can reduce the complexity substantially with negligible impact on quality. The results show that the proposed method considerably reduces encoding time with a negligible loss in quality.

Interest in playing/streaming multimedia content in Full High Definition (Full HD) and the upcoming Ultra High Definition (UHD) is rapidly increasing. However, TVs with such resolutions has higher hardware and bandwidth requirements. Hence, advanced compression tools are essential to provide TV services in Full HD and UHD quality. The aim of this paper is to explore coding efficiency performance of High Efficiency Video Coding (HEVC) and VP9 compression tools for video content in Full HD and UHD resolutions. Performance of both video compression tools is evaluated and compared with objective and subjective video quality metrics. The results show that the HEVC and VP9 codecs for Full HD and UHD videos have comparable coding efficiency. Moreover, in several cases, Quality of Experience (QoE) for Full HD, depending on the video content and target bitrate, can outperform QoE for UHD.

In the present situation, applications related to multimedia are discovered to be comfortable with the use of video. The number of end consumers who use video continues to rise every day. People are presently searching for videos with better quality among the ones that are currently available there. This results in the launch and dissemination of HD (high definition) videos. Ultra high-definition (UHD) videos are becoming more and more popular as a result of this advancement and need. However, as video communication keeps expanding, there is an upsurge in network traffic because of the limited bandwidth, especially among smart cities. Different advancement codecs have been suggested to deal with the data stream to overcome this hazardous circumstance. However, the fact that modern UHD videos have huge amounts of data makes the available codecs even more complicated. UHD videos can be processed with the latest improvement codec, H.265/High-efficiency video coding (HEVC). Nevertheless, it is impacted by increased power consumption and intricate calculations. Limitations in the codec's functionality confine its use to specific applications, preventing its application in wireless, mobile, or portable settings. Hence, this research concentrates on implementing frame-level quality enhancement through a deep learning network known as FQE-Net. The deep learning convolutional neural network (DLCNN) is specifically crafted to manage films with resolutions up to 16K. Its primary objectives include reducing complexity, minimizing artifacts, enhancing the efficiency of the HEVC codec, and compacting energy consumption. To achieve superior efficiency, it is imperative to replace the DWT transforms within the HEVC codec with a DLCNN model. Additionally, incorporating the Content Block Search Algorithm for Motion Estimation and Compensation, alongside filtering techniques like Sample Adaptive Filter and Deblocking Filter, becomes essential. The simulation results showed that the suggested FQE-Net performed better than the conventional techniques.

High Efficiency Video Coding (HEVC), the latest video compression standard, will play an important role in many multimedia applications in the foreseeable future. Its superior compression performance enables HEVC to be particularly suitable for high-definition videos in consumer electronics environments; however, it comes with the price of substantially high computational complexity. The HEVC encoding process, especially the Motion Estimation (ME), is very time consuming, which makes HEVC impractical for real-time applications at the moment. In this work, a hybrid encoding architecture with a set of algorithms is proposed, exploring a Graphics Processing Unit (GPU) to perform both uni-predictive and bi-predictive ME in a highly parallel manner. This reduces the complexity of the uni- and bipredictive ME on the Central Processing Unit (CPU) by up to 99% and 95% respectively, and brings significant overall time savings of up to 57.65% and 54.16% for the low delay P and the random access coding configurations respectively. The Rate-Distortion (RD) performance is only marginally affected in both cases.