Error Resilient Coding Based on Reversible Data Embedding Technique for H. 264/AVC Video

This paper presents an efficient error concealment method for multiview video plus depth (MVD) delivery over error-prone channels based on the view synthesis information. Specifically, we propose an enhanced temporal error concealment (TEC) method by estimating the probable partition and missing MVs of a corrupted macroblock (MB) using the synthesized information. Furthermore, the view synthesis error concealment (VSEC) approach that directly uses the synthesized pixels to conceal the corrupted MBs is also adaptively utilized together with the TEC approach. Experimental results have shown the effectiveness of the proposed method compared with the existing methods in terms of both objective and subjective measures.

During transmission, compressed video streams of H.264/AVC are very likely to be corrupted by channel distortion. In order to resolve the problem, researchers come out with a solution called error concealment which is done at the decoder side. In this paper we conduct a research on BMA and the motion compensated error concealment methods. Compared with BMA, the MCEC algorithm is capable of providing better performance considering various packet loss rates. And the experiment results show that the objective scores of quality are consistent with the perceptual quality.

Error concealment is a well-known technique to improve reproduced picture quality in cases which parts of the coded picture are not available at a decoder for reconstruction. The conventional error concealment methods generally make use of the correlation between a damaged marcoblock and its adjacent macroblocks in the same frame and/or the previous frame. However, performance results of concealment methods are different each other and dependent on how much damaged frames and macroblocks are correlated to surrounding ones temporally and spatially. Moreover, most of them can not conceal effectively the consecutively damaged frames or macro blocks caused by wireless bursty packet loss features, since motion vectors and correlation information between the damaged images are already destroyed. The proposed error concealment aware error resilient video coding is a new method which has an intelligent error concealment selector by using pre-estimation technique of the possibly damaged motion vectors at the encoder. The method recommends the best error concealment per macroblock by transmitting additionally its error concealment selection codes to the decoder. The chosen error concealment helps to conceal the damaged MB at the decoder even when losses of frames are bursty. The experimental results show the substantial effectiveness to improve video quality at the cost of a little bit overhead at the transmitted bit streams.

Compressed video bit-streams are extremely sensitive to packet loss over error-prone channels. Error concealment (EC) has been considered as one of error control techniques to improve the reconstructed picture quality against transmission errors. However, EC methods show poor image reconstruction performance due to unavailable scene change information of the video sequence, especially when an abrupt scene change occurs. In this paper, we propose an effective EC method based on scene change detection algorithm (SCDA), which provides information to decide whether spatial or temporal EC is better to be used for intra and inter frames respectively. The simulation results show that the proposed method highly improves the subjective quality of incorrectly decoded frames and obtains an average gain of 0.7dB compared with the H.264/AVC Joint Model reference software.

In this paper, the error concealment techniques as motion vector extrapolation (MVE), boundary matching (BM), side matching (SM), 16/spl times/16-MB-based and 8/spl times/8-block-based concealment are implemented and compared with feedback-based no waiting ARQ (NWARQ) error recovery technique. A new error concealment method using preprocessing (PREP) and search is proposed and implemented. This new method tries to use the information of surrounding MV as much as possible, and find a more precise motion vector using additional search processing in a search window centered by the estimated motion vector. Simulation results, based JM1.4 codec of JVT, show that both NWARQ and PREP can significantly improve the reconstruction video quality. And the combined PREP, SEARCH and NWARQ techniques can greatly achieve improved performance. From the simulation results, the SM and 16/spl times/16-MB-based concealment methods show better performance than BM and 8/spl times/8-block-based concealment.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.

Error concealment at a decoder is an efficient method to reduce the degradation of visual quality caused by channel errors. In this paper, we propose a novel spatio-temporal error concealment algorithm based on the spatial-temporal fading (STF) scheme which has been recently introduced. Although STF achieves good performance for the error concealment, several drawbacks including blurring still remain in the concealed blocks. To alleviate these drawbacks, in the proposed method, hybrid approaches with adaptive weights are proposed. First, the boundary matching algorithm and the decoder motion vector estimation which are well-known temporal error concealment methods are adaptively combined to compensate for the defect of each other. Then, an edge preserved method is utilized to reduce the blurring effects caused by the bilinear interpolation for spatial error concealment. Finally, two concealed results obtained by the hybrid spatial and temporal error concealment are pixel-wisely blended with adaptive weights. Experimental results exhibit that the proposed method outperforms conventional methods including STF in terms of the PSNR performance as well as subjective visual quality, and the computational complexity of the proposed method is similar to that of STF.

In this paper, we propose a novel spatial image error concealment (EC) method based on deep neural network. Considering that the natural images have local correlation and non-local self-similarity, we use the local information to predict the missing pixels and the non-local information to correct the predictions. The deep neural network we utilize can be divided into two parts: the prediction part and the auto-encoder (AE) part. The first part utilizes the local correlation among pixels to predict the missing ones. The second part extracts image features, which are used to collect similar samples from the whole image. In addition, a novel adaptive scan order based on the joint credibility of the support area and reconstruction is also proposed to alleviate the error propagation problem. The experimental results show that the proposed method can reconstruct corrupted images effectively and outperform the compared state-of-the-art methods in terms of objective and perceptual metrics.

An Adaptive Spatio-Temporal Error Concealment (ASTEC) scheme based on H.264/AVC is proposed in the paper to improve the quality of the rebuilt frame in H.264/AVC decoder side. Combining the spatial error concealment method with the temporal one, a new scheme is proposed to conceal an erroneous macro block according to its encoded mode. The temporal error concealment scheme adopts the weighted Boundary Matching Algorithm to recover the erroneous macro block. The spatial error concealment scheme recovers the erroneous macro block by exploiting the directional interpolation which subdivides the macro block according to the number and the direction of edges going through it. Experiments show that the proposed concealment scheme can restrain the diffusion of erroneous macro block and improve the quality of the rebuilt frame subjectively as well as objectively.

In this paper, we propose an error concealment technique for H.264 coded videos. The algorithm is targeted at compressed videos degraded after packet losses caused by transmission over an unreliable channel. We use a combination of watermarking and half toning (dithering) techniques. At the encoder side, a dithered version of each video frame is embedded into the video using a watermarking technique. The watermarking technique used by the proposed algorithm is a modified version of the Quantization Index Modulation (QIM) algorithm, which provides a good data hiding capacity. At the decoder side, the algorithm identifies which packets of the video were lost, extracts the corresponding mark, and applies an inverse half toning technique to estimate the original content. The algorithm is fast and has a good performance, being able to restore content with a better quality than the default H.264 error concealment algorithm.

Error concealment (EC) is one of the target applications of inpainting techniques. Some methods combine the estimated lost motion vectors (MVs) with the exemplar-based inpainting technique to recover the lost regions. Due to the erroneous motion vectors that might indicate a moving object as background object and vice versa, these methods are still showing visual artifacts in the recovered regions. In this paper, a concept of motion map that can be easily generated in the decoder side is introduced and it is combined with the exemplar-based inpainting technique. The proposed method introduces an adaptive search window size that trades-off the quality and complexity. Moreover, an optional blending technique is proposed to limit the spatio-temporal artifacts. Experiments show that the proposed method improves the visual quality with 5dB on average relative to the state-of-the-art inpainting-based EC method.

A method of error concealment for JPEG2000 images is proposed in this paper. The proposed method uses the layer structure that is a feature of JPEG2000. The most significant layer is hidden in the lowest layer of the JPEG2000 bit stream, and this embedded layer is used for error concealment. The most significant layer is duplicated because JPEG2000 uses bit-plane coding. In this coding, when the upper layers are affected by errors, the coefficients of the lower layers become meaningless. A bit stream encoded using the proposed method has the same data structure as a standard JPEG2000. Therefore, it can be decoded by a standard decoder. Our simulation results demonstrated the effectiveness of the proposed method.

Region-based error concealment of right-view frames for stereoscopic video transmission

Video transmission over wireless networks becomes a major part of multimedia studies especially after fourth-generation communication standard has been introduced. There is a growing need to improve video visual quality, while its computational complexity should remain low. One of the low-complexity Error Concealment (EC) methods for missing macroblock (MB) recovery is Boundary Matching Algorithm (BMA). Nevertheless, this method suffers from lack of accuracy. In this paper, we propose a novel modified BMA method by exploiting the correlation of the pixels on the edge of the missing MB. The performance of this method is explained in detail and extra experimental results are given to demonstrate its superiority over BMA method.

Multi-view video plus depth (MVD) is an efficient format of 3D video. MVD video can be encoded by either H.264/AVC or HEVC standard to gain higher compression ratio, which benefits their broadcasting over the Internet. However, the encoded and transferred MVD video tends to develop worse visual quality degradation caused by lossy network channel. Therefore, error concealment is here to help refrain this problem. In this paper, we propose a classification-related error concealment method for MVD video. Within our method, motion-based classification is used to judge whether the corrupted blocks are static or not. If so, the blocks from reference frames are adopted to conceal the corrupted blocks. Otherwise, view-specific based concealment procedures, which are designed in accordance with view features, are used to conceal the corrupted blocks in different views. Experimental results on AVC-based Test Model (ATM) show the superiority of this concealment scheme to several other error concealment methods in PSNR along with acceptable execution time increase.