Achieving H.264/AVC performance using distributed video coding combined with super-resolution

Distributed video coding (DVC) is an emerging video coding paradigm for systems that require low-complexity encoders that are supported by high-complexity decoders as required, for example, in real-time video capture and streaming from one mobile phone to another. Under the assumption of an error-free transmission channel, the coding efficiency of current DVC systems is still below that of the latest video codecs, such as H.264/AVC. In order to increase the coding efficiency, we propose that every Wyner-Ziv frame be downsampled by a factor of two prior to encoding and the subsequent transmission. However, this would necessitate upsampling in conjunction with interpolation at the decoder. Simple interpolation (e.g., a bilinear or bicubic filter) would be insufficient because the high-frequency (HF) spatial image content would be missing. Instead, we propose the incorporation of a super-resolution (SR) technique based upon the example-based scene-specific method to allow this HF content to be recovered. The SR technique will add computational complexity to the decoder side of the DVC system, which is allowable within the DVC framework. Rate-distortion curves show that this novel combination of SR and DVC improves the system's peak signal-to-noise ratio (PSNR) performance by up to several decibels and can actually exceed the performance of the H.264/AVC codec when GOP = IP for some video sequences.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.

Distributed video coding (DVC) is an emerging video coding paradigm due to its flexibility to introduce much lower encoding complexity than conventional predictive codecs, which is beneficial for some applications such as wireless video surveillance, wireless sensor networks, and disposable video cameras. Although DVC systems without feedback channel address a wider range of applications, it is not commonly discussed in literatures due to its lower coding performance. In this paper, on the basis of PRISM DVC architecture, a low-complexity feedback-channel-free DVC system is proposed with a new classifier to improve the coding performance. Experimental results show that the proposed system can provide about 0.8 dB gain in PSNR to the state-of-the-art and 2 dB gain to IST-PRISM. It is also competitive regarding other feedback-channel-free DVC systems and can even achieve similar performance to DVC systems with feedback channel for low-motion sequences.

Existing distributed video coding (DVC) frameworks use manually designed and optimized modules when encoding and decoding video. Each module can set the appropriate parameters as much as possible to achieve its independent optimization. But there is no connection between the modules, and overall end-to-end optimization is not realized. Inspired by the application of neural networks to video coding, we try to implement DVC using neural networks. This article proposes an end-to-end DVC framework, which combines DVC and neural networks to perform end-to-end encoding and decoding of Wyner-Ziv (WZ) frames, while achieving variable compression rates. We employ four different kinds of side information to assist in decoding the WZ frames. Experimental results demonstrate that the proposed framework achieves excellent decoded performance on videos with varying degrees of motion intensity.

Summary form only given. Distributed video coding (DVC) has been intensively studied in recent years. This new coding paradigm substantially differs from conventional prediction-based video codecs such as MPEG and H.26x, which are characterized by a complex encoder and simple decoder. The conventional DVC codec, e.g., DISCOVER codec, uses advanced frame interpolation techniques to create SI based on adjacent decoded reference frames. The quality of SI is a well-recognized factor in the RD performance of WZ video coding. A high SI quality implies a high correlation between the created SI and the original WZ frame, which then decreases the rate required to achieve a given decoded quality. Clearly, the performance of an SI creation process based on adjacent previously decoded frames is limited by the quality of the past and the future reference frames as well as the distance and motion behavior between them. The correlation between high-motion frames is low and vice versa. That is, SI quality in the conventional codecs depends on the temporal correlation of key frames, which affects the bitrate and PSNR of the compression process. In this work, a novel DVC architecture for dealing with the cases of high-motion and large GOP-size sequences is proposed to better the rate-distortion (RD) performance. For high-motion video sequences, the proposed architecture generates SI by using subsampled spatial information instead of interpolated temporal information. the proposed approach separates the video sequence into subsampled key frames and corresponding WZ frames, which changes the creation of SI. That is, all successive frames on the encoder side are downsized to sub-frames, which are then compressed by an H.264/AVC intra encoder. Experimental results reveal that the subsampling input based DVC codec can gain up to 1.47 dB in the RD measures and maintains the most important characteristic of the DVC codec, the encoder is lightweight, as compared with the conventional WZ codec, respectively. The novel DVC architecture evaluated in this study exploits spatial relations to create SI. The experimental results confirm that the RD performance of the proposed approach is superior to that of the conventional one for high-motion and/or large GOP-size sequences. The quality of spatial interpolation based SI is higher than that of the temporal interpolation one, which leads to a high-PSNR reconstructed WZ frame. The subsampled key frames are also decoded by LDPCA decoder to recover the information lost when H.264/AVC intra coding is used to increase PSNR gain. Since many spatial domain interpolation and super resolution schemes have been proposed for use in the fields of image processing and computer vision, the performance of the proposed DVC codec can be further enhanced by using better schemes to generate even better SI.

In distributed video coding (DVC) systems, the complexity of encoders is greatly reduced by removing the motion estimation operations in encoders, since the correlation between frames is utilized in decoders. The transmission of parity bits is requested through the feedback channel, until the related errors are corrected to decode the Wyner-Ziv frames. The requirement to use the feedback channel limits the application of DVC systems. In this paper, we propose an efficient method to remove the feedback channel in DVC systems. First, a simple side information generation method is proposed to calculate the amount of parity bits in the encoder, and it is shown that the proposed method yields good performance with low complexity. Then, by calibrating the theoretical entropy with three parameters, we can calculate the amount of parity bits in the encoder and remove the feedback channel. Moreover, an adaptive method to determine quantization parameters for key frames is proposed. Extensive computer simulations show that the proposed method yields better performance than conventional methods.

Distributed video coding (DVC) is an emerging video coding paradigm designed to allow low-complexity compression by resource-constrained video sensor nodes in wireless video sensor networks (WVSN). Several DVC architectures have been proposed in literature but there are hardly any security mechanisms designed to validate the integrity and authenticity of the decoded video. In this paper, we propose a novel, low-complexity, blind, and imperceptible video watermarking scheme based on DVC for WVSNs. The watermark information is embedded into key frames under selected GOP (Group of Pictures) configurations. Considering the information-intensive yet resource-constrained WVSN environment, the proposed scheme aims to limit the impact of watermark embedding complexity by hiding the secret information using non-zero quantized AC coefficients. We implemented the proposed watermarking scheme for a DVC based codec and evaluated its processing complexity (encoding/decoding). We also evaluated the average normalized correlation between the embedded and extracted watermarks for different video sequences by varying the redundancy in embedding.

In the distributed video coding system, side-information takes an important role in reconstructing the Wyner-Ziv frames. To generate good side-information, we propose a new interpolation strategy based on motion estimation and motion compensation which exploits the video temporal redundancy. Besides, we propose a new framework based on the above strategy. In this framework, multi-estimation mode and a new refinement to side-information are added. Experimental results show that the PSNR of the new framework is improved notably compared with normal DVC system

Distributed Video Coding (DVC) is a new video coding paradigm which has the ability to shift coding complexity from encoder to decoder. State-of-art DVC schemes are mainly based on two-source correlation model and only one correlation channel between side information and wyner-ziv frame is considered. In this paper, multiple-source correlation model is introduced into a famous DVC scheme, namely Transform Domain Wyner-Ziv video coder (TDWZ). A soft-input computing algorithm based on multiple-source correlation model is firstly proposed. The correlations a mong multiple side information are separately exploited. Then a new reconstruction function based on multiple source correlation model is also proposed to improve reconstructed image's quality. Experimental results show that the proposed algorithms can effectively improve the coding performance of systems.

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Distributed Video Coding (DVC) is a new video coding scheme that mainly uses the availability of decoder side information to reduce the complexity of the encoder. The quality of the side information has an important influence on the performance of the distributed video coding system. At present, the generation of side information is mainly based on a model that uses inter-frame correlation for motion estimation. In order to make full use of the intra-frame self-similarity of the side information frame, we use the idea of image denoising, and propose an improved side information generation method. Since the initial side information frame can be viewed as a noisy version of the Wyner-Ziv (WZ) frame, improving the side information frame can be translated to denoising the frame. The experimental results show that the quality of the side information obtained by the proposed scheme has been significantly improved, thereby effectively improve the performance of distributed video coding.

Distributed video coding (DVC) architecture designs, based on distributed source coding principles, have benefitted from significant progresses lately, notably in terms of achievable rate-distortion performances. However, a significant performance gap still remains when compared to prediction-based video coding schemes such as H.264/AVC. This is mainly due to the non-ideal exploitation of the video sequence temporal correlation properties during the generation of side information (SI). In fact, the decoder side motion estimation provides only an approximation of the true motion. In this paper, a progressive DVC architecture is proposed, which exploits the spatial correlation of the video frames to improve the motion-compensated temporal interpolation (MCTI). Specifically, Wyner-Ziv (WZ) frames are divided into several spatially correlated groups that are then sent progressively to the receiver. SI refinement (SIR) is performed as long as these groups are being decoded, thus providing more accurate SI for the next groups. It is shown that the proposed progressive SIR method leads to significant improvements over the Discover DVC codec as well as other SIR schemes recently introduced in the literature.

Distributed video coding (DVC) has been featured by exploiting the video statistics, partially or totally at the decoder. Wireless sensor networks are supposed to have lesser complexity encoders at the expense of higher decoder complexity. Therefore DVC is more suitable to video transmission over wireless sensor networks compared to conventional video coding. Current research work on DVC is conducted for lossless channel, i.e, parity bit stream is not influenced by noise or distortion and further correlation noise due to the residual between input video frame and side information is not estimated effectively. In other words , noisy environment is not analyzed with DVC codec in recent research works. In this paper, DVC codec is enabled with the effect of AWGN noise and further a single wireless fading channel (SISO) is considered. The correlation noise is analyzed for Foreman and Carphone video sequences and relationship of correlation of adjacent key frames are discussed.

Distributed Video Coding (DVC) is a new video coding technique which has low encoder complexity and good error resilience. Rate control is one of the most important problems in DVC. In order to allocate an optimal number of parity bits for the current frame,most distributed video coders use a feedback channel(FC) to perform rate control at the decoder. However,using feedback channel can increase the decoding complexity and result in transmission delay. To solve these problems, we propose a novel encoder rate control (ERC) algorithm for pixel-domain distributed video (PDDV) coder without feedback channel at the cost of slightly increased encoder complexity. The experiment results show that the performance obtained using our ERC algorithm is comparable to that obtained using the decoder rate control (DRC)algorithm.

Distributed video coding (DVC) was proposed to meet the low complexity encoding requirement, and it was verified to work more efficiently than H.264/AVC intra coding on video sequences with low motion. This makes DVC suitable for surveillance application. This paper presents a background aided surveillance-oriented distributed video coding system. A high quality background frame is encoded for each group of pictures (GOP), which can provides high quality SI for the background parts of the Wyner-Ziv (WZ) frames. Consequently, bit rate for the WZ frames can be reduced. Experimental results demonstrate that the proposed system can decrease the bit rate by up to 67.4% when compared with traditional DVC codec.