Bit-depth scalability compatible to H.264/AVC-scalable extension

This paper introduces inter-layer filters for the scalable extension of High Efficiency Video Coding (SHVC) standard, which is being developed by the Joint Collaborative Team on Video Coding (JCT-VC). The major new coding tool in SHVC is inter-layer texture prediction. It provides about 18% average BD-rate reduction compared with HEVC two-layer simulcast. In the case of spatial scalability, base layer reconstructed pictures are up-sampled to the enhancement layer resolution to generate inter-layer texture prediction. A set of 2D separable 8 taps (luma) and 4 taps (chroma) DCT based interpolation filters, which follow the design principles of HEVC motion compensation interpolation filter, are used in the up-sampling process. In the case of SNR scalability, the up-sampling process is not needed since the reference layer has the same spatial resolution as the current layer but encoded with lower quality. This paper proposes a novel inter-layer filter with denoising effect for SNR scalability to improve enhancement layer coding efficiency and equalize the number of stages in inter-layer processing between SNR and spatial scalabilities. Experimental results show that the usage of inter-layer de-noising filter in SNR scalability provides up to 7.5% BD-rate reduction and has observable improvement on subjective visual quality.

Scalable High efficiency Video Coding (SHVC) is the scalable extension of the latest video coding standard High Efficiency Video Coding (HEVC). One of the key novelties introduced by SHVC is that it enables hybrid codec scalability. This basically means that the video layers can be encoded with different video standards providing backward compatibility between codecs. In this paper, we propose a software parallel SHVC decoder in hybrid codec scalability configuration. The proposed design consists of an Advanced Video Coding (AVC) decoder for the Base Layer (BL) and a HEVC decoder for the Enhanced Layer (EL). In order to perform Inter Layer Prediction (ILP), a communication of decoding states and outputs is established between the two decoders. While the native frame based parallelism is still allowed within the two decoders, the proposed design also enables the use of frame based parallelism between the two decoders. The proposed software design enables a real time decoding of the HEVC EL at 2160p60 while the AVC base layer is decoded at 1080p60 for ×2 spatial scalability.

Scalable Video Coding (SVC) is an extension of H.264/AVC standard. The base layer of SVC is compatible with H.264/AVC standard, while the enhancement layers provide desired temporal, quality and/or spatial scalabilities. Bit-stream rewriting in SVC standard allows an SVC bit-stream to be converted to an H.264/AVC bit-stream without quality loss and preferably with low computational complexity. However, current rewriting is only supported in quality scalability rather than spatial scalability, which limits the application in many practical scenarios. In this paper, a hybrid bit-stream rewriting approach to support both quality and spatial scalability is proposed based on the principle of residue upsampling in transform domain. The computational complexity of the proposed approach is much lower than the conventional scheme of cascading transcoding. Extensive experimental results demonstrate that the loss of the rate-distortion (RD) performance of the proposed rewritable SVC bit-stream is acceptable compared with the conventional SVC bit-stream, however, the RD performance is better than that of simulcast. Furthermore, the RD performance of the H.264/AVC bit-stream rewritten from the rewritable SVC bit-stream is even better than that of the input SVC bit-stream. Compared with the cascading transcoding scheme, the proposed hybrid rewriting can achieve 0.8 dB Y-PSNR gains while saving 80% processing time on average.

Current digital video applications require delivery of digital video content to different clients over heterogeneous networks. These clients may have different system resources and bandwidth capabilities. Thus, delivering of digital video data compressed at specific spatial resolution, quality levels, frame rates and encoding those video data at specific bit rates according to the client resources and the available network is essential. The characteristics of current video transmission system pose many problems for video delivery and scalable video coding is a highly attractive solution. This thesis investigates the existing video coding standards, especially the newest H.264/advanced video coding and its scalable extension: scalable video coding. In this thesis, we address two issues related to the state-of-the-art scalable video coding: 1) computational complexity reduction; and 2) video coding efficiency enhancement. In the first part of this thesis, we address the complexity issue of spatial, signal-to-noise ratio (SNR), temporal and combined scalable video coding. Our goal is to find a good tradeoff between computational complexity and video coding efficiency. We first establish the need to reduce computational complexity for current H.264/advanced video coding and scalable video coding framework. We then evaluate the mode distribution correlations between the base layer and its enhancement layers for different video scalabilities. After the exhaustive search over all possible block partitions is performed at the base layer, the number of candidate modes for luma and chroma blocks in a macroblock that take part in rate distortion optimization calculation at enhancement layers could be greatly reduced based on the correlations. Finally, adaptive fast mode decision schemes for spatial, SNR and temporal scalable video coding are presented. Our schemes could achieve consistent and significant computational complexity reduction with negligible loss in objective and subjective video quality and insignificant increments in bit rate consumption. The second part of this thesis is to provide a solution for the problem of high computational complexity of scalable video decoders. An adaptive encoding algorithm is proposed to reduce decoder complexity in coarse grain SNR scalable video coding.

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In existing pyramid-based spatially scalable coding schemes, such as H.264/MPEG-4 SVC (scalable video coding), video frame at a certain high-resolution layer is mainly predicted either from the same frame at the next lower resolution layer, or from the temporal neighboring frames within the same resolution layer. But these schemes fail to exploit both kinds of correlation simultaneously and therefore cannot remove the redundancies among resolution layers efficiently. This paper extends the idea of spatiotemporal subband transform and proposes a general in-scale motion compensation technique for pyramid-based spatially scalable video coding. Video frame at each high-resolution layer is partitioned into two parts in frequency. Prediction for the lowpass part is derived from the next lower resolution layer, whereas prediction for the highpass part is obtained from neighboring frames within the same resolution layer, to further utilize temporal correlation. In this way, both kinds of correlation are exploited simultaneously and the cross-resolution-layer redundancy can be highly removed. Furthermore, this paper also proposes a macroblock-based adaptive in-scale technique for hybrid spatial and SNR scalability. Experimental results show that the proposed techniques can significantly improve the spatial scalability performance of H.264/MPEG-4 SVC, especially when the bit-rate ratio of lower resolution bit stream to higher resolution bit stream is considerable. </para>

To provide spatial, temporal and quality scalabilities, the scalable video coding (SVC) offers several additional techniques. The inter-layer prediction is one of them to improve the coding efficiency. However, it is only applied to the enhancement layer when the base layer is available. Because of this limited inter-layer prediction, it deteriorates the coding efficiency of enhancement layer. In this paper, we propose a new inter-layer prediction scheme for spatial scalability with different frame rates. It improves the coding efficiency using the information of the interpolation layer when the base layer is not available. The experimental results show that the proposed scheme reduces the bitrates by 2~10% and improves video quality by 0.01~0.4dB compared with the reference software, JSVM 11.0.

Video communication requires sufficient resources to obtain the desired image quality. Availability of bandwidth and screen resolution from different devices on the client end, as well as compression techniques used for transmission also significantly influence to the video quality. Even though the H.264/AVC video compression standard and its scalable extension is known as Scalable Video Coding (SVC) was introduced and successfully addressed the challenge, some weaknesses persist especially for high-resolution video transmissions. The size of the bitstream generated by the SVC standard for ultra-high resolution video compression can only be met if transmitted on a network with a large bandwidth. The Scalable High-Efficiency Video Coding (SHVC) were introduced to overcome problems in the SVC standards. The new video compression standard is proven to reduce bandwidth requirements by up to 46.41 % compared to the SVC standards for the same video quality. This study introduces the performance analysis of SHVC standard video transmission as compared to SVC standard. Both compression standards are transmitted and emulated through network emulator called Mininet-Wifi to get the real transmission picture. The client-server video transmission scheme with different bandwidth and various client position are set in the experiment scenario. It is found that transmission time, packet loss, and video reconstruction quality on scalable video transmission with SHVC standard is better than scalable video transmissions with SVC standard.

This paper develops a new adaptive scanning methodology for intra frame scalable coding framework based on a subband/wavelet(DWTSB) coding approach for MPEG-4 AVC/H.264 scalable video coding (SVC). It attempts to take advantage of the prior knowledge of the frequencies which are present in different higher frequency subbands. We propose dyadic intra frame coding method with adaptive scan (DWTSB-AS) for each subband as traditional zigzag scan is not suitable for high frequency subbands. Thus, by just modification of the scan order of the intra frame scalable coding framework of H.264, we can get better compression. The proposed algorithm has been theoretically justified and is thoroughly evaluated against the current SVC test model JSVM and DWTSB through extensive coding experiments for scalable coding of intra frame. The simulation results show the proposed scanning algorithm consistently outperforms JSVM and DWTSB in PSNR performance. This results in extra compression for intra frames, along with spatial scalability. Thus Image and video coding applications, traditionally serviced by separate coders, can be efficiently provided by an integrated coding system.

In this paper, we propose two structures for scalable video coding (SVC) based on HEVC. Several inter-layer prediction mechanisms are introduced to improve coding efficiency of the proposed HEVC-based SVC. The proposed inter-layer predictions are developed on single-loop and multi-loop decoding structures. We found that the proposed SVC is able to decrease average bitrates of enhancement layers by about 10.2% for the all-intra case, and 7.4% for the random access case, compared with single layer coding with no inter-layer prediction in multi-loop decoding. In addition, the single-loop decoding with the proposed inter-layer predictions achieves coding gains of 10.2% for the all-intra case, and 2.6% for the random access case.

In the incoming amendment of H.264/AVC for scalable video coding (SVC), inter-layer prediction is incorporated into spatial scalability. According to inter-layer motion prediction, the base-layer and enhancement-layer can use one set of motion vectors, due to the strong relationship between different layers. In this paper, an improved motion estimation scheme is proposed for gaining optimal motion vectors, aiming to balance the coding efficiency between two layers, meanwhile maintain the final coding performance. The proposed scheme performs motion estimation in both two layers simultaneously, and uses quantization step as weighting parameters, which can ensure that the coding performance can not be affected by variable QP(quantization parameters). Thus, the proposed cost function is optimized. When the cost is minimal, an optimal motion vector is obtained, and the graceful tradeoff between the two layers is also achieved. Our experimental results show that the proposed scheme can balance the coding efficiency between the two layers perfectly, and at the same time improve the total coding performance.

The objective of scalable video coding is to enable the generation of a unique bitstream that can adapt to various bitrates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. To improve encoding efficiency, the SVC scheme incorporates inter-layer prediction mechanisms which increases complexity of overall encoding. In this paper several conditional probabilities are established relating motion estimation characteristics and the mode distribution at different layers of the H.264/SVC. An evaluation of these probabilities is used to structure a low-complexity prediction algorithm for Group of Pictures (GOP) in H.264/SVC, reducing computational complexity whilst maintaining similar performance. When compared to the JSVM software, this algorithm achieves a significant reduction of encoding time, with a negligible average PSNR loss and bit-rate increase in temporal, spatial and SNR scalability. Experiments are conducted to provide a comparison between our method and a recently developed fast mode selection algorithm. These demonstrate our method achieves appreciable time savings for scalable spatial and scalable quality video coding, while maintaining similar PSNR and bit rate.

The existing standard video coding schemes support spatial scalability because of its prospective applications. Unfortunately, spatial scalable codecs produce high bit rate overhead as compared to a single layer coder. In this paper, we propose a spatio-temporal scalable video coding system based on motion compensated (MC) three-dimensional subband/discrete cosine transform (3-D SBC/DCT). This coder is proposed as a solution to improve the compression performance of spatial scalability and at the same time to reduce the high bit rate consumption of the base layer. In this system, the temporal subbands are generated by a novel approach called MC DCT temporal filtering (MCDCT-TF). The temporal subbands are further spatially decomposed using two-dimensional (2-D) DCT decimation technique . The generated spatio-temporal subbands are then quantized, entropy encoded, and transmitted along with the motion vectors. Several experiments are carried out on various image sequences to test the performance of the proposed coding scheme. The simulation results show that its performance exceeds that of H263+ single layer, spatial and spatio-temporal scalable coders.

Scalable video coding (SVC) is currently developed as an extension of H.264/AVC. In the SVC encoder, an exhaustive search technique is employed to select the best coding mode for each macro block (MB). This technique achieves an optimal trade-off between rate and distortion, but it requires an extremely large encoding time. In this paper, we propose a fast mode decision algorithm for inter-layer coding at the enhancement layer. The proposed algorithm predicts the mode of each MB at the enhancement layer using the modes of a co-located MB and its neighboring MBs at the base layer. The proposed method can achieve a time saving of up to 74% in spatial scalability and 63% in coarse grain quality scalability with negligible loss of quality and bit rate increment.

H.264 Scalable Video Coding (SVC) is an extension to the Advanced Video Coding (AVC) H.264 standard which provides efficient scalability functionalities on top of the high coding efficiency of H.264/AVC. SVC allows for temporal, spatial, and quality scalability of the output video stream, encoding the video information into an H.264/AVC base layer and a series of enhancement layers which incrementally improve the quality, increase screen resolution and/or frame rate. SVC is particularly suited for mobile TV reception, since the received video quality is adaptable to variable reception conditions and heterogeneous receiver capabilities. However, mobile TV Digital Video Broadcasting (DVB) standards such as DVB-H and DVB-SH were designed prior to the introduction of SVC, and therefore the underlying transmission protocols are not optimized for scalable video delivery. In this overview paper, we review recently proposed solutions for SVC stream adaptation on the underlying DVB-H/SH protocols, and point out novel technical solutions that are currently under consideration for the next generation mobile broadcasting standard DVB-NGH.

Scalable video coding has been an interesting research topic for a long time. Temporal, spatial, and SNR scalabilities have been studied over the years. As network video applications are taking off, there is a great need for scalable video with fine granularity that the previous scalable coding schemes cannot satisfy. This paper presents the reference technique for fine granularity scalability in MPEG-4, which will satisfy the need of many network video applications.