Error Concealment for Video Transmission Based on Watermarking

H.264/AVC 비디오 압축 표준은 압축 효율을 높이기 위해 다양한 크기의 블록을 사용하여 화면 사이의 움직임 예측을 수행한다. H.264/AVC는 가변적인 블록 크기의 움직임 보상을 통해 세밀한 영역의 움직임까지 예측할 수 있어 잔여 영상을 나타내는 정보량을 효과적으로 줄일 수 있다. 복호를 위해서는 각 블록의 움직임 벡터를 전송해야 하는데, 저비트율 환경에서는 움직임 벡터 정보가 전체 비트스트림의 약 40%를 차지한다. 움직임 벡터 정보량을 줄이기 위해 비디오 부호화 전문가 그룹(VCEG)에서는 다양한 움직임 벡터 예측(Motion Vector Competition) 방법을 제안하였다. 다양한 예측 움직임 벡터를 사용하여 실제 전송해야 할 움직임 벡터 차분값(Motion Vector Difference, MVD)의 크기를 줄이기 때문에 압축 효율을 높일 수 있다. 그러나 다양한 예측 움직임 벡터를 사용하기 때문에 선택된 예측 움직임 벡터의 인덱스 정보를 복호기로 전송해야 한다. 이 논문에서는 인덱스 정보를 효율적으로 전송하기 위해 Phased-in 코드를 기반으로 한 새로운 코드워드 표를 제안했다. 실험을 통해 제안한 방법을 이용하여 동일한 화질에서 평균 약 7.24%의 비트율을 절감할 수 있었고, 동일한 비트율에서는 평균 약 0.36dB의 화질을 향상시킬 수 있었다. The H.264/AVC video coding standard performs inter prediction using variable block sizes to improve coding efficiency. Since we predict not only the motion of homogeneous regions but also the motion of non-homogeneous regions accurately using variable block sizes, we can reduce residual information effectively. However, each motion vector should be transmitted to the decoder. In low bit rate environments, motion vector information takes approximately 40% of the total bitstream. Thus, motion vector competition was proposed to reduce the amount of motion vector information. Since the size of the motion vector difference is reduced by motion vector competition, it requires only a small number of bits for motion vector information. However, we need to send the corresponding index of the best motion vector predictor for decoding. In this paper, we propose a new codeword table based on the phased-in code to encode the index of motion vector predictor efficiently. Experimental results show that the proposed algorithm reduces the average bit rate by 7.24% for similar PSNR values, and it improves the average image quality by 0.36dB at similar bit rates.

In the state-of-art video coding standard HEVC, temporal motion vector (MV) predictor is adopted in order to improve coding efficiency. However, motion vector information in reference frames, which is used by temporal MV predictor, takes significant amount of bits in memory storage. Therefore motion data needs to be compressed before storing into buffer. In this paper we propose an adaptive motion data storage reduction method. First, it divides the current 16x16 block in the reference frame into four partitions. One MV is sampled from each partition and all sampled MVs form a MV candidate set. Then it judges if one or two MVs should be stored into the MV buffer by checking the maximum distance between any two of the MVs in the candidate set. If the maximum distance is greater than a certain threshold, the motion data of the two MVs that have maximum distance are put into memory; otherwise the motion data of the upper left block is stored. The basic goal of the proposed method is to improve the accuracy of temporal MV predictor at the same time reducing motion data memory size. Simulation results show that compared to the original HEVC MV memory compression method in the 4th JCT-VC meeting, the proposed scheme achieves a coding gain of 0.5%~0.6%; and the memory size is reduced by more than 87.5% comparing to without using motion data compression.

Error concealment is a well-known technique to improve reproduced picture quality in cases which parts of the coded picture are not available at a decoder for reconstruction. The conventional error concealment methods generally make use of the correlation between a damaged marcoblock and its adjacent macroblocks in the same frame and/or the previous frame. However, performance results of concealment methods are different each other and dependent on how much damaged frames and macroblocks are correlated to surrounding ones temporally and spatially. Moreover, most of them can not conceal effectively the consecutively damaged frames or macro blocks caused by wireless bursty packet loss features, since motion vectors and correlation information between the damaged images are already destroyed. The proposed error concealment aware error resilient video coding is a new method which has an intelligent error concealment selector by using pre-estimation technique of the possibly damaged motion vectors at the encoder. The method recommends the best error concealment per macroblock by transmitting additionally its error concealment selection codes to the decoder. The chosen error concealment helps to conceal the damaged MB at the decoder even when losses of frames are bursty. The experimental results show the substantial effectiveness to improve video quality at the cost of a little bit overhead at the transmitted bit streams.

Video transmission in error-prone networks suffers bit errors and packet losses that degrade the decoded video quality. In this paper, a video error concealment scheme based on H.264/AVC is proposed to enhance the error robustness using reversible data hiding. At the encoder, motion vectors of macro blocks belonging to the region of interest (ROI) are embedded into the background region within the same frame. The histogram modification technique based on quantized DCT coefficients of coded macro blocks is used to achieve loss less recovery after data extraction. If a macro block in the ROI is found missing at the decoder, the embedded motion vector can be extracted from the corresponding host macro block for the recovery of the missing macro block. This proposed scheme makes a tradeoff between error concealment performance and compression efficiency. Experimental results show the better reconstructed video quality can be achieved compared with the related error concealment methods.

H.264 adopts new coding tools such as variable block size, quarter-pixel-accuracy motion estimation/compensation, multiple reference frames, loop filter, etc. The adoption of these tools enables a macroblock to have more information compared with previous standards. In H.264 each macroblock can have up to sixteen motion vectors, four reference frames, and a mode of the macroblock, all of which can be used for temporal error concealment. The H.264 error concealment in the informative part, however, uses a similar method to prior coding standards that consider only motion vectors of macroblocks adjacent to the lost macroblock. In this paper, we propose an effective temporal error concealment algorithm for H.264-coded video. It uses not only motion vectors and reference frames but the modes of macroblocks adjacent to the lost macroblock as well. Depending on the modes of neighboring macroblocks, each lost macroblock is concealed on the basis of different block sizes. Simulation results show the proposed method yields better video quality than conventional approaches.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.

This paper presents an evaluation of the latest MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC) for video streaming applications using best effort protocols. LCEVC is a new video standard by MPEG, which enhances any base codec through an additional low bitrate stream, improving both video compression efficiency and and transmission. However, there is an interplay between packetization, packet loss visibility, choice of codec and video quality, which implies that prior studies with other codecs may be not as relevant. The contributions of this paper is, therefore in twofold: It evaluates the compression performance of LCEVC and then the impact of packet loss on its video quality when compared to H.264 and HEVC.The results from this evaluation suggest that, regarding compression, LCEVC outperformed its base codecs, overall in terms average encoding bitrate savings when using the constant rate factor (CRF) rate control. For example at a CRF of 19, the average encoding bitrate was reduced by 18.7% and 15.8% when compared with the base H.264 and HEVC codecs respectively. Furthermore, LCEVC produced better visual quality across the packet loss range compared to its base codecs and the quality only started to decrease once packet loss exceeded 0.8-1%, and decreases at a slower pace compared to its equivalent base codecs. This suggests that the LCEVC enhancement layer also provides error concealment. The results presented in this paper will be of interest to those considering the LCEVC standard and expected video quality in error-prone environments

Motion-vector-based error concealment algorithms recover lost blocks by motion compensation with the estimated motion vectors. Traditionally, the estimated motion vectors for the lost block are derived by the direct usage of the neighboring available motion vectors. In this work, the residual signal in a block encoded in the video bitstream is used to measure the reliability of the corresponding motion vector. The residual information is applied to weight the neighboring available motion vectors differently, to further improve a newly proposed mathematical formulation between encoded motion vectors and extrapolated motion vectors. The proposed error concealment method using the residual information outperforms a state-of-the-art error concealment algorithm by up to 2.12 dB in sequence PSNR, and outperforms a classic DMVE method by up to 8.27 dB.

Confident Recovery for Lost Motion Vectors in Video Transmission

This paper presents a new scheme for robust video transmission over packet erasure wireless channels based on Wyner-Ziv coding of motion regions. The multipath fading and shading of the wireless channels usually lead to loss or erroneous video packets which on occasions result in some spontaneous drop in video quality. Existing approaches with forward error correction (FEC) and error concealment have not been able to provide the desired robustness in video transmission. We develop a new scheme with a motion-based Wyner-Ziv coding (MWZC) by leveraging distributed source coding (DSC) ideas for error robustness. This new scheme is based on the fact that motion regions of a given video frame are particularly important in both objective and perceptual video quality and hence should be given preferential Wyner-Ziv coding based embedded protection. To achieve high coding efficiency, we determine the underlining motion regions based on a rate-distortion model. Within the framework of H.264/AVC specification, motion region determination can be efficiently implemented using flexible macroblock ordering (FMO) and data partitioning (DP). The bit stream generated by the proposed scheme consists two parts: the systematic portion generated from conventional H.264/AVC bit stream and the supplementary bit stream for error robust video transmission generated by the Wyner-Ziv coding of motion regions. Experimental results demonstrate that the proposed scheme significantly outperforms both decoder-based error concealment (DBEC) and conventional FEC with DBEC approaches.

As the newest video coding standard, H.264/AVC adopts the high-efficiently predictive coding and variable length entropy coding to achieve high compression efficiency. On the other side, transmission errors become the major problem faced by video broadcasting service providers. Error concealment (EC) here is adopted to handle slices with huge conjunctive corrupted areas inside. Considering error propagation from corrupted slice to succeeding ones is the key factor affecting the video quality, this paper proposes a novel temporal EC scheme including the bi-direction motion vector (MV) retrieval method and an adaptive EC ordering basing on it. Background and motional steady shift part of slice will be given top and second priority, respectively. Combined with our proposed improved boundary matching algorithm (IBMA) which provides more accurate distortion function, experiments results show that our proposal achieves better performance under different error rate channel, compared with EC algorithm adopted in H.264 reference software.

In this paper, a video watermarking technique is proposed to hide copyright information in H.264 motion vectors. In this method, watermark is embedded in the motion vectors, and the region of motion vector is restricted to hide watermarking information. In order to obtain the good watermarked video quality, intra mode can be used to encode those macroblocks badly affected by restricting the region of the chosen motion vectors in inter mode. The watermark can be easily retrieved by the encoding mode of the macroblocks or the region of its motion vectors. The experimental results demonstrate that the video quality is almost the same as that of the original. The increase of bit rate is less than 1%. The proposed method keeps the excellences of hiding watermark by using the region of motion vectors. The capacity of watermark can be increased by choosing more macroblocks to the watermarks.

In this paper, the error concealment techniques as motion vector extrapolation (MVE), boundary matching (BM), side matching (SM), 16/spl times/16-MB-based and 8/spl times/8-block-based concealment are implemented and compared with feedback-based no waiting ARQ (NWARQ) error recovery technique. A new error concealment method using preprocessing (PREP) and search is proposed and implemented. This new method tries to use the information of surrounding MV as much as possible, and find a more precise motion vector using additional search processing in a search window centered by the estimated motion vector. Simulation results, based JM1.4 codec of JVT, show that both NWARQ and PREP can significantly improve the reconstruction video quality. And the combined PREP, SEARCH and NWARQ techniques can greatly achieve improved performance. From the simulation results, the SM and 16/spl times/16-MB-based concealment methods show better performance than BM and 8/spl times/8-block-based concealment.

Given a video sequence decompressed from a compressed video sequence associated with motion vector information for each image frame, this short note presents an efficient predictive watershed-based video segmentation algorithm using motion vectors. The proposed algorithm has much computation-saving merit since the next frame can inherit the segmented results of the current frame according to the motion vector information of the next frame. Experimental results demonstrate that with little perceptual segmentation degradation, the proposed algorithm has better execution-time performance when compared to the previous Chien et al.'s video segmentation algorithm whose input video sequence is assumed to have no motion vector information. Further, the alleviation of the over-segmentation problem existed in both algorithms is investigated.

A new error concealment algorithm based on average motion vector (AVMV) method is proposed as a post-processing tool at the decoder side for recovering the lost blocks and their motion vectors as well incurred during the video transmission. In our proposed scheme, LOG operator is used to detect the texture of the blocks around the lost block and find the more precise estimation of the motion vectors, so as to improve the AVMV method. Simulation results show that the proposed method can recover the higher quality image in different rates of lost block, comparing to the existing traditional concealment algorithms.