Recursive Bayesian Filtering-Based Error Concealment Scheme for 3D Video Communication Over Severely Lossy Wireless Channels

Confident Recovery for Lost Motion Vectors in Video Transmission

An Adaptive Spatio-Temporal Error Concealment (ASTEC) scheme based on H.264/AVC is proposed in the paper to improve the quality of the rebuilt frame in H.264/AVC decoder side. Combining the spatial error concealment method with the temporal one, a new scheme is proposed to conceal an erroneous macro block according to its encoded mode. The temporal error concealment scheme adopts the weighted Boundary Matching Algorithm to recover the erroneous macro block. The spatial error concealment scheme recovers the erroneous macro block by exploiting the directional interpolation which subdivides the macro block according to the number and the direction of edges going through it. Experiments show that the proposed concealment scheme can restrain the diffusion of erroneous macro block and improve the quality of the rebuilt frame subjectively as well as objectively.

Video transmission in error-prone networks suffers bit errors and packet losses that degrade the decoded video quality. In this paper, a video error concealment scheme based on H.264/AVC is proposed to enhance the error robustness using reversible data hiding. At the encoder, motion vectors of macro blocks belonging to the region of interest (ROI) are embedded into the background region within the same frame. The histogram modification technique based on quantized DCT coefficients of coded macro blocks is used to achieve loss less recovery after data extraction. If a macro block in the ROI is found missing at the decoder, the embedded motion vector can be extracted from the corresponding host macro block for the recovery of the missing macro block. This proposed scheme makes a tradeoff between error concealment performance and compression efficiency. Experimental results show the better reconstructed video quality can be achieved compared with the related error concealment methods.

Motion-vector-based error concealment algorithms recover lost blocks by motion compensation with the estimated motion vectors. Traditionally, the estimated motion vectors for the lost block are derived by the direct usage of the neighboring available motion vectors. In this work, the residual signal in a block encoded in the video bitstream is used to measure the reliability of the corresponding motion vector. The residual information is applied to weight the neighboring available motion vectors differently, to further improve a newly proposed mathematical formulation between encoded motion vectors and extrapolated motion vectors. The proposed error concealment method using the residual information outperforms a state-of-the-art error concealment algorithm by up to 2.12 dB in sequence PSNR, and outperforms a classic DMVE method by up to 8.27 dB.

In this paper a hybrid error concealment algorithm using linear interpolation is proposed. In the proposed hybrid algorithm, the selective motion field interpolation (SMFI) is employed to conceal the erroneous macroblock. The spatial and temporal boundary-matched errors are then used to check whether the SMFI conceals the erroneous macroblock properly. If the temporally recovered macroblock is reconstructed incorrectly, the spatial error concealment using linear interpolation is employed to conceal the damaged macroblock instead of SMFI.It can achieve better performance subjectively as well as objectively over other error concealment techniques such as AMV, FBBM and MFI. Simulation was conducted on the H.263 codec and the experimental results reveal that the proposed algorithm significantly outperforms other concealment techniques.

In the 3D video extension of H.264/AVC, namely 3D-AVC, Neighboring Based Disparity Vector (NBDV) derivation has been proposed to support multiview/stereo compatibility, therefore texture views can be decoded independently to depth views. NBDV generates a disparity vector for the current macroblock (MB) using the motion information of neighboring blocks, especially those coded with motion vectors pointing to inter-view reference pictures. In 3D-AVC, NBDV has been utilized to access minimum number of spatial and temporal neighboring blocks, therefore there is a high probability that NBDV does not derive an efficient disparity vector. This paper introduces a derived disparity vector scheme, wherein only one disparity vector derived from NBDV is maintained for the whole slice and it is used as the disparity vector of the current MB if NBDV does not derive one from neighboring blocks. Simulation results show that the proposed method provides 3.6% bit rate reduction for multiview coding.

To reduce the heavy computational complexity of uni-directional and bi-directional predictions in multiview video coding (MVC), this paper proposes a multiview video coding algorithm. Uni-directional prediction is optimized. By using the geometry relationship between disparity and motion vectors, an iterative process is preformed. In each process, a new adaptive search range is used to revise motion/disparity vector. At last, optimal motion and disparity vectors are achieved. For bi-directional prediction, TZSearch algorithm is utilized to accelerate prediction. Furthermore, an early termination strategy is devised on the basis of relationships among bi-directional prediction, model error, quantization parameter and temporal level. Experimental results show that computational complexity can be significantly reduced while maintaining video coding performance compared with JMVC full search algorithm.

In this paper, we propose a joint temporal error control (JTEC) method for H.264 which combines RDO-based macroblock (MB) classification at the encoder and adaptive partition size error concealment at the decoder. The encoder classifies the MBs by evaluating the sensitivity of the MBs as the RD cost between the concealment error and the bits needed for the additional motion information. Additional motion information can be transmitted for the error sensitive MBs such as the original motion vector or motion vector index. The decoder utilizes the additional motion information for the sensitive blocks. Non-sensitive blocks are concealed by the Adaptive Partition Size (APS) method. APS selects the best partition mode for error concealment by minimizing the Weighted Double-Side External Boundary Matching Error (WDS-EBME), which jointly measures the inter-MB boundary discontinuity, inter-partition boundary discontinuity and intra-partition block artifacts in the recovered MB. A progressive concealment method is developed for the 4x4 partition mode.

The data packets are easily damaged or lost when video streams of high compression ratio are transmitted, and error concealment technique can recover by the lost or damaged macro-blocks using temporal and spatial correlations of images. The paper proposed a spatial error concealment algorithm based on FAST algorithm. First, the proposed algorithm determines the situation of edges in the lost macro-blocks according to detecting the relationship of gray values of pixels around the detected pixels. Then, the information of the lost macro-blocks is recovered by the division of edges. The experimental results showed that under the premise of ensuring the peak signal to noise ratio of the reconstructed images, the proposed algorithm saves running time 0.15s-0.37s in comparison with the traditional error concealment algorithm.

Error concealment algorithms for robust decoding of MPEG compressed video

A new error concealment algorithm based on average motion vector (AVMV) method is proposed as a post-processing tool at the decoder side for recovering the lost blocks and their motion vectors as well incurred during the video transmission. In our proposed scheme, LOG operator is used to detect the texture of the blocks around the lost block and find the more precise estimation of the motion vectors, so as to improve the AVMV method. Simulation results show that the proposed method can recover the higher quality image in different rates of lost block, comparing to the existing traditional concealment algorithms.

The paper presents a temporal error concealment technique for compressed videostreams over wireless channels. The proposed technique is two stages temporal error concealment which are: (1) reconstruction of corrupted macroblocks (MBs) using the available motion vectors (MVs) of the corresponded MBs and (2) recalculation of the previously estimated MVs in the first stage using texture information of the surrounding MBs. The experimental results show that the proposed technique produces better visual quality in subjective measurement and higher PSNR in objective measurement when compared with other conventional techniques.

This paper proposes a temporal error concealment algorithm based on the edge directions of the pixels surrounding the lost macroblock (MB). On one hand, for each pixel on the external boundary of the restored MB, the spatial boundary distortion is calculated along the edge direction, which preserves edge alignment as well as ensures spatial continuity. On the other hand, the edge directions are utilized to identify whether the lost MB contains multiple objects, and then determine the restore mode of the four 8x8 blocks in the lost MB: simultaneously or independently. The experimental results for several test video sequences demonstrate that higher performance is achieved in both objective and subjective quality of the reconstructed frames compared with conventional error concealment algorithms.

In this paper, the error concealment techniques as motion vector extrapolation (MVE), boundary matching (BM), side matching (SM), 16/spl times/16-MB-based and 8/spl times/8-block-based concealment are implemented and compared with feedback-based no waiting ARQ (NWARQ) error recovery technique. A new error concealment method using preprocessing (PREP) and search is proposed and implemented. This new method tries to use the information of surrounding MV as much as possible, and find a more precise motion vector using additional search processing in a search window centered by the estimated motion vector. Simulation results, based JM1.4 codec of JVT, show that both NWARQ and PREP can significantly improve the reconstruction video quality. And the combined PREP, SEARCH and NWARQ techniques can greatly achieve improved performance. From the simulation results, the SM and 16/spl times/16-MB-based concealment methods show better performance than BM and 8/spl times/8-block-based concealment.

We use particle filtering for correcting the erroneous motion vectors (MVs), which are derived from the boundary matching algorithm (BMA) in packet video communications. Assuming a two-state Markov channel model for transmission, the error of the extracted MVs by BMA is shown to be modeled by the Gaussian mixture distribution. Formulating the problem in the state space, we deploy particle filtering for denoising the erroneous MVs. The main challenge of using particle filters is high computational complexity that is directly related to the number of particles. The proposed particle filtering scheme is efficient even if the number of particles is decreased. Experimental results are provided to show the efficiency of this filtering approach compared to a recent scheme based on Kalman filtering. The experiments show meaningful increase in the quality of the recovered video sequences in terms of PSNR up to 3 dB compared with the other error concealment techniques. Also, the computational complexity of the proposed scheme is discussed.