Improving SHVC performance with a block based joint layer prediction solution

Scalable High efficiency Video Coding (SHVC) is the scalable extension of the latest video coding standard High Efficiency Video Coding (HEVC). One of the key novelties introduced by SHVC is that it enables hybrid codec scalability. This basically means that the video layers can be encoded with different video standards providing backward compatibility between codecs. In this paper, we propose a software parallel SHVC decoder in hybrid codec scalability configuration. The proposed design consists of an Advanced Video Coding (AVC) decoder for the Base Layer (BL) and a HEVC decoder for the Enhanced Layer (EL). In order to perform Inter Layer Prediction (ILP), a communication of decoding states and outputs is established between the two decoders. While the native frame based parallelism is still allowed within the two decoders, the proposed design also enables the use of frame based parallelism between the two decoders. The proposed software design enables a real time decoding of the HEVC EL at 2160p60 while the AVC base layer is decoded at 1080p60 for ×2 spatial scalability.

Video communication requires sufficient resources to obtain the desired image quality. Availability of bandwidth and screen resolution from different devices on the client end, as well as compression techniques used for transmission also significantly influence to the video quality. Even though the H.264/AVC video compression standard and its scalable extension is known as Scalable Video Coding (SVC) was introduced and successfully addressed the challenge, some weaknesses persist especially for high-resolution video transmissions. The size of the bitstream generated by the SVC standard for ultra-high resolution video compression can only be met if transmitted on a network with a large bandwidth. The Scalable High-Efficiency Video Coding (SHVC) were introduced to overcome problems in the SVC standards. The new video compression standard is proven to reduce bandwidth requirements by up to 46.41 % compared to the SVC standards for the same video quality. This study introduces the performance analysis of SHVC standard video transmission as compared to SVC standard. Both compression standards are transmitted and emulated through network emulator called Mininet-Wifi to get the real transmission picture. The client-server video transmission scheme with different bandwidth and various client position are set in the experiment scenario. It is found that transmission time, packet loss, and video reconstruction quality on scalable video transmission with SHVC standard is better than scalable video transmissions with SVC standard.

In this paper, we present an efficient transmission method of multiple broadcasting services using layered division multiplexing (LDM) and scalable high efficiency video coding (SHVC) based on the next generation terrestrial digital broadcasting standard, advanced television systems committee 3.0. In a two-layer LDM system, each layer forms a physical layer pipe that carries base layer and enhancement layer video streams of SHVC, respectively. This combination of physical and presentation layers technologies can maximize channel utilization when multiple broadcasting services are delivered in a single radio frequency channel with different robustness and reception conditions. An intensive performance analysis is provided when the proposed combination of LDM and SHVC is compared with other approaches, such as time division multiplexing and/or high efficiency video coding simulcast broadcasting. Several performance measurements are also introduced, such as constant data rate measurement and constant quality measurement, which are used to evaluate the performance of video codecs in conjunction with different physical layer system parameters. A prototype hardware system with LDM and SHVC capable of selecting different parameter combinations is tested in the laboratory and in real field environments to verify the performance and feasibility of the proposed LDM and SHVC combination. Results show that the proposed LDM and SHVC combination provides significant gains on video service quality as well as reception robustness.

A fast CU depth level decision algorithm with low complexity for Scalable High Efficiency Video Coding (SHVC) is proposed to reduce the computational complexity caused by the recursive quad-tree structure of Coding Unit (CU) inherited from HEVC. The proposed scheme enables the early termination for CU sizes for intra prediction in the enhancement layer (EL) by using the property of the energy distribution in the current CTU and the optimal coding depths of the neighboring CTUs. Based on an adaptive thresholds scheme, the texture complexity can be categorized to three types and the CU depth level range is therefore narrowed correspondingly. Besides, the spatial correlation existing in the encoded neighboring CTUs is utilised to further reduce the number of Rate Distortion (RD) evaluation. The coding time is therefore significantly saved. The proposed algorithm is evaluated in the reference software of SHVC (SHM12.0). The experimental results shows that the reduction in encoding time can be up to 42.95% in comparison with the original encoding scheme in SHM.

Digital video applications cover today a very large range of multimedia application: messaging, HD video telephony and video conferencing. etc. These heterogeneity motivated some years ago the development of the scalable video coding. Therefore, the Joint Collaborative Team on Video Coding (JCT-VC) has developed a new Scalable High efficiency Video Coding standard (SHVC) as an extension of HEVC. The SHVC may compress numerous video sequences of various resolutions as a single bit-stream using the inter layer prediction tools. In order to highlight the features' coding efficiency; several analyses were made using numerous High Definition (HD) sequences with various scenarios. In this paper an analysis study has done on the efficiency of SHVC. Results show that multicast coding using SHVC standard provides an average bit rate reduction of more than 30% compared to simulcast. These results demonstrate the improvement in coding efficiency of the SHVC solution with the introduction of inter-layer prediction.

As an extension of the High Efficiency Video Coding standard, Scalable High‐efficiency Video Coding (SHVC) has been developed to satisfy the challenges posed by heterogeneous environments. A layer‐based scheme is employed to support various forms of scalability, but this results in significantly increased computational complexity. A fast Coding Unit (CU) size decision algorithm is proposed to alleviate the additional computational requirement of the Enhancement Layers (ELs). The inter‐layer dependency and the spatial correlation are jointly used to skip and early terminate the CU size decision process. In addition, the quantitive relationship between the texture homogeneity and the AC coefficient energy is investigated to further narrow the depth range to be evaluated. Experimental results show that the proposed scheme saves encoding time by up to 72% compared with the SHM 12.0 implementation, with negligible degradation in Rate‐Distortion performance.

With a new video coding standard high efficiency video coding (HEVC), the ultra high definition (UHD) TV service with robust video streaming technology is emerging in the TV industry. This paper addresses the system architecture for the UHD video streaming and proposes two main ideas: (i) picture prioritization method, (ii) error concealment mode signaling (ECMS). In the experiments using HEVC reference model conducted, the proposed picture prioritization method shows the gains in video quality from 2.2 to 7.5 dB in Y-PSNR, and the error concealment mode signaling gains from 0.2 to 2.5 dB in Y-PSNR, with corresponding subjective improvements.

Due to the diversity of terminal devices, Spatial Scalable High Efficiency Video Coding (SSHVC) is an efficient solution to meet this requirement. However, its coding process is very complex, which seriously prevents its wide applications. Therefore, it is very crucial to reduce coding complexity and improve coding speed. In this paper, we propose a Gaussian Distribution-based Mode Selection for Intra Prediction of SSHVC. We show that the rate distortion costs of Inter-layer Reference (ILR) mode and Intra mode are significantly different, and both follow a Gaussian distribution. Based on this discovery, we propose to use a Bayes decision rule to determine whether ILR is the best mode so as to skip Intra mode. Experimental results demonstrate that the proposed algorithm can significantly improve coding speed with negligible coding efficiency losses.

The Scalable High Efficiency Video Coding (SHVC) standard is a state-of-the-art technology for scalable video coding that uses an exhaustive prediction mode decision algorithm within a recursive quad-tree structure coding unit to enhance coding efficiency. However, this improvement comes at the cost of increased computational complexity during encoding, which poses a challenge. To address this challenge, a novel approach has been proposed to accelerate the SHVC encoder. The proposed technique leverages the inherent spatial correlation between the selected Prediction Unit (PU) within the enhancement layer (EL) and its corresponding PU in the base layer (BL). This strategic exploitation yields impressive results, as evidenced by significant reductions in encoding time. It is worth noting that this efficiency boost is achieved with a manageable compromise in video quality and an acceptable uptick in bit rate, which remains well within the bounds of mainstream media requirements. In practice, the proposed methodology showcases its prowess by consistently achieving an average time-saving of 81.77% and 81.59%, thereby underscoring its effectiveness in practical scenarios.

SHVC performance enhancement using superior step search algorithm

In Wyner-Ziv (WZ) video coding, decoder side information (SI) estimation takes an important role. In this paper, an approach to SI estimation for WZ video coding is proposed. It is based on universal prediction and motion compensated temporal interpolation (MCTI). In the approach, a WZ frame which is to be decoded is first divided into non-overlapping blocks. According to their motion activity, the blocks in the WZ frame are further classified as of low or high motion activity. SI for blocks with low motion activity are estimated based on universal prediction, while those of high motion activity are estimated through MCTI. As a result, the approach takes the advantages of both universal prediction and MCTI based SI estimation. It can provide SI of better quality than the first approach while is more computationally efficient than the second one. The performance of the proposed approach is tested and the simulation results are presented.

High-efficiency video coding (HEVC or H.265) is the latest video compression standard developed by the joint collaborative team on video coding (JCT-VC), finalized in 2013. HEVC can achieve an average bit rate decrease of 50% in comparison with H.264/AVC while still maintaining video quality. To upgrade the HEVC used in heterogeneous access networks, the JVT-VC has been approved scalable extension of HEVC (SHVC) in July 2014. The SHVC can achieve the highest coding efficiency but requires a very high computational complexity such that its real-time application is limited. To reduce the encoding complexity of SHVC, in this chapter, we employ the temporal-spatial and inter-layer correlations between base layer (BL) and enhancement layer (EL) to predict the best quadtree of coding tree unit (CTU) for quality SHVC. Due to exist a high correlation between layers, we utilize the coded information from the CTU quadtree in BL, including inter-layer intra/residual prediction and inter-layer motion parameter prediction, to predict the CTU quadtree in EL. Therefore, we develop an efficient CTU decision method by combing temporal-spatial searching order algorithm (TSSOA) in BL and a fast inter-layer searching algorithm (FILSA) in EL to speed up the encoding process of SHVC. The simulation results show that the proposed efficient CTU decision method can achieve an average time improving ratio (TIR) about 52–78% and 47–69% for low delay (LD) and random access (RA) configurations, respectively. It is clear that the proposed method can efficiently reduce the computational complexity of SHVC encoder with negligible loss of coding efficiency with various types of video sequences.

Fractional motion compensation (FMC) has been widely adopted in modern video coding standards such as high-efficiency video coding (HEVC). The fractional pixels in FMC are generated by the interpolation filters, and the coefficients of the filters are designed using a Fourier decomposition of the discrete cosine transform (DCT) in HEVC. To overcome the compression deficiency caused by these handcrafted filters, deep learning-based interpolation filter approaches have been introduced recently. In this paper, we analyze the drawbacks of the mean square error (MSE) only loss function which is widely adopted in existing learned-based approaches, and introduce the rate-distortion optimization (RDO) theory into network training to formulate a joint loss function to overcome this issue. Then, a new training scheme is proposed for the detailed implementation of the joint loss function. Besides the calculated MSE loss, the residual will be sent into a rate estimation model which includes three stages: transform/quantization/rate modeling, and the output rate cost will be used as the second part of the loss function. Finally, we propose a novel network structure that utilizes the deformable convolution layer and the residual dense block to enhance the feature extraction and transmission ability to adapt to complex video contents. The proposed RDO-based interpolation filter (RDOIF) has been integrated into HEVC reference software HM-16.7, and extensive experiments are conducted to verify the efficiency of the proposed method. Experimental results show that the proposed RDOIF method achieves 3.1%, 2.4%, and 1.4% BD-rate reduction on average under the low-delay-P, low-delay-B, and random access configurations, respectively. When implemented in the versatile video coding (VVC) test model VTM-10.0, the proposed method can achieve 2.3%, 1.2%, and 0.5% BD-rate savings on average under the low-delay-P, low-delay-B, and random access configurations, respectively.

3D-HEVC is the latest 3D video coding project of MPEG based on High Efficiency Video Coding (HEVC). The encoding complexity of it is remarkably high due to the coding units (CU) size decision process together with the rate distortion optimization (RDO) process. In this paper, we proposed a fast CU depth decision algorithm taking advantages of the interview correlations. With the CU depth correlation analysis between the independent view and the dependent view, the CU depth of each CU in the dependent view is automatically determined based on inter-view prediction and quad-tree structure constraints. It eliminates the brute force RDO process in determining CU depth, which provides an average of 60% saving in texture encoding complexity with a negligible loss in compression efficiency.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.