Hybrid Error Concealment Using Linear Interpolation

Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery

Compressed bitstreams are, in general, very sensitive to channel errors. For instance, a single bit error in a coded video bitstream may cause severe degradation on picture quality. When bit errors occur during transmission and cannot be corrected by an error correction scheme, error concealment is needed to conceal the corrupted image at the receiver. Error concealment algorithms attempt to repair damaged portions of the picture by exploiting both the spatial and the temporal redundancies in the received and reconstructed video signal. We discuss several temporal, spatial, and transform-domain error concealment techniques for MPEG coded pictures, and propose a new scheme based on directional interpolation. We also compare the performance of these techniques by computer simulation.

Error concealment at a decoder is an efficient method to reduce the degradation of visual quality caused by channel errors. In this paper, we propose a novel spatio-temporal error concealment algorithm based on the spatial-temporal fading (STF) scheme which has been recently introduced. Although STF achieves good performance for the error concealment, several drawbacks including blurring still remain in the concealed blocks. To alleviate these drawbacks, in the proposed method, hybrid approaches with adaptive weights are proposed. First, the boundary matching algorithm and the decoder motion vector estimation which are well-known temporal error concealment methods are adaptively combined to compensate for the defect of each other. Then, an edge preserved method is utilized to reduce the blurring effects caused by the bilinear interpolation for spatial error concealment. Finally, two concealed results obtained by the hybrid spatial and temporal error concealment are pixel-wisely blended with adaptive weights. Experimental results exhibit that the proposed method outperforms conventional methods including STF in terms of the PSNR performance as well as subjective visual quality, and the computational complexity of the proposed method is similar to that of STF.

The three-dimensional video (3DV) is composed of variable-length stream sequences captured via diversified cameras surrounding an object. Thus, it is an urgent task to accomplish sufficient encoding to be compatible with incoming bandwidth demands, while achieving a recommended 3DV reception performance. In the 3DV compression framework, the lost macro-blocks (MBs) might propagate into the following frames and the adjoining views. Therefore, it is obligatory to avoid error propagation by concealing the corrupted MBs at the decoder through the utilization of appropriate post-processing error concealment (EC) techniques. The existing EC algorithms fundamentally exploit the temporal, inter-view, and spatial matching within the 3DV frames and views to reconstruct the disparity vectors (DVs) and motion vectors (MVs) of the corrupted MBs. Unluckily, in the state of high severe corruptions and heavily erroneous MBs, these concealment algorithms are predominantly unreliable and might give unreliable 3DV quality. Thence, in this work, we suggest the utilization of the outer block boundary matching algorithm to estimate the MVs and the directional interpolation EC algorithm to estimate the DVs of the erroneous MBs. After that, the Bayesian Kalman filter (BKF) is employed because of its efficiency to filter out the inherent errors in the previously predicted DVs and MVs to accomplish better 3D video performance. Experimental results on standard 3DV sequences demonstrate that the suggested BKF-based EC scheme is more powerful with heavy losses. It subjectively and objectively outperforms the traditional concealment techniques at severely random and bursty packet loss rates (PLRs).

The H.264/AVC Multi-View Coding (MVC) compresses more than one viewpoint of a scene captured by different cameras. Redundancies between views and frames are used for inter-view and inra-view prediction ,respectively, in encoding and in Error Concealment (EC) in decoding. As MVC has high error sensitivity, EC algorithms are developed to improve the quality of decoded frames. In this paper, efficient multi-view EC algorithm is proposed for intra MBs and inters MBs exploiting the spatial, inter-view and inra-view information of neighboring MBs. For Intra MBs, Spatial-Inter View (SIV) algorithm is proposed. For inter MBs, the proposed algorithm is adaptively changed according to size of lost MB. In addition, the algorithm dynamically candidates MBs according to the current used view in EC operation. Then the algorithm self generates more candidate MBs depending on best matched MB to get higher matched MB. It is shown that the proposed algorithm leads to better subjective quality in addition to higher PSNR values. By applying the proposed algorithm, the gain is improved by up to 12.4 dB and 0.73 dB compared with the case of not applying any EC algorithms and the case of applying normal EC algorithm respectively at 22% Loss Rate (LR). Keywords-Multi view Coding, Spatial and Temporal Error Concealment, Outer Boundary Matching Algorithm.

An Adaptive Spatio-Temporal Error Concealment (ASTEC) scheme based on H.264/AVC is proposed in the paper to improve the quality of the rebuilt frame in H.264/AVC decoder side. Combining the spatial error concealment method with the temporal one, a new scheme is proposed to conceal an erroneous macro block according to its encoded mode. The temporal error concealment scheme adopts the weighted Boundary Matching Algorithm to recover the erroneous macro block. The spatial error concealment scheme recovers the erroneous macro block by exploiting the directional interpolation which subdivides the macro block according to the number and the direction of edges going through it. Experiments show that the proposed concealment scheme can restrain the diffusion of erroneous macro block and improve the quality of the rebuilt frame subjectively as well as objectively.

The data packets are easily damaged or lost when video streams of high compression ratio are transmitted, and error concealment technique can recover by the lost or damaged macro-blocks using temporal and spatial correlations of images. The paper proposed a spatial error concealment algorithm based on FAST algorithm. First, the proposed algorithm determines the situation of edges in the lost macro-blocks according to detecting the relationship of gray values of pixels around the detected pixels. Then, the information of the lost macro-blocks is recovered by the division of edges. The experimental results showed that under the premise of ensuring the peak signal to noise ratio of the reconstructed images, the proposed algorithm saves running time 0.15s-0.37s in comparison with the traditional error concealment algorithm.

Distributed Video Coding (DVC) is based on a new paradigm in coding, which has received many interests recently. Among other features, DVC is said to provide better error resiliency. This paper proposes a hybrid spatial and temporal error concealment scheme to conceal errors in Wyner-Ziv (WZ) frames. We first use a spatial concealment based on edge directed filter. This step is exploited to improve the performance of subsequent temporal concealment. An enhanced temporal concealment based on motion compensated temporal interpolation is proposed, including motion vector refinement and smoothing, optimal compensation mode selection, and a new matching criterion for motion estimation. Simulation results show that the objective qualities as well as the perceptual qualities of the corrupted sequences are significantly improved by the hybrid error concealment, outperforming both spatial and temporal concealments alone.

In spatial error concealment (SEC), methods like bilinear interpolation (BI) and directional interpolation (DI) are commonly used to estimate the missing pixel values resulting from losses occurring in video streams. Despite being able to preserve spatial smoothness, BI produces a blurring effect and is unable to preserve structural information. DI produces spurious edges in regions with no strong edges, resulting in visible artefacts. In this paper, we propose a SEC algorithm that addresses the above drawbacks by formulating a weighted sum of candidate macroblocks produced from DI and BI, with weights derived adaptively through local information. We demonstrate that the proposed algorithm offers visual improvements over both DI based algorithms and the SEC algorithm based on BI in the H.264/AVC reference software JM 12.0. Most importantly, this unique approach preserves edge information and spatial smoothness in the error concealed macroblock due to the integration of both BI and DI.

In video communication over lossy packet networks (e.g., the Internet), packet loss errors can severely damage the transmitted video. The damaged video can largely be repaired with spatial error concealment, where neighboring information was used to estimate missing information. They mask the effect of missing blocks by using the spatial or/and temporal correlation to create image or video of subjectively acceptable quality.

Confronting with high packet loss conditions in the transmission of compressed images, inadequacy of intact received data severely affects the performance of error concealment (EC) techniques. Thus, it is of great interest to determine sufficient features from the original image to be transmitted as side information to the receiver, so that in the case of missing data, the EC employs the corresponding features as a priori knowledge for the recovery procedure. In order to take the advantages of side information, the Watermark driven Decentralized Best Matching (WDBM) is proposed in this paper; in each block of the image, the position of its best matching block within the original image is embedded, so that each block not only represents itself, but also describes the best similar copy of another block. Our simulation results show that as a supplementary procedure, WDRM improves the performance of existing methods, especially in high loss ratios.

This paper proposes a novel approach for multimedia signal reconstruction based on kernel density estimation (KDE). We make use of a vector formalism in which vectors consist of a first subvector containing a set of missing samples and a second one containing a set of available context samples. The missing subvector is reconstructed by a minimum mean square error estimator which employs a probability density function (pdf) obtained by KDE. As in any kernel-based method, the main issue to deal with is the estimation of an appropriate kernel bandwidth. We propose an adaptive procedure for bandwidth estimation (BE) especially conceived for signal reconstruction. Thus, unlike general KDE or kernel-based regression, which try to obtain a general fit, the focus of this BE procedure is on the specific missing subvector. Also, in order to exploit local signal correlations, our BE proposal adopts a scaling approach in which the bandwidth is computed as the local covariance matrix scaled by two factors. These two scale factors are obtained by minimization of two different approximations to the reconstruction error. The resulting reconstruction methodology is tested on a spatial error concealment (EC) application in which intracoded images have been transmitted through an error prone channel. The experimental results show the superiority of the proposed approach over a wide range of existing EC techniques.

Error resilient coding and error concealment techniques have been introduced in scalable video coding to reduce the quality degradation impact of transmission errors. Redundant picture is one of the error resilient tools, appropriate for delay sensitive applications where re-transmission is not possible and channel loss rates such as in packet-based wireless networks, are not a priori known. In this paper, combination of redundant-key-picture error resilient coding with motion-copy error concealment at the decoder is proposed based on experimental evaluation. Experimental results demonstrate that significant PSNR gain can be achieved using key picture error concealment with motion-copy for various tested sequences under different packet loss rates compared to JVT-W049.

In this paper, we present a new spatial error concealment algorithm with dynamic texture estimation and geometric interpolation. First, we apply a variant version of dynamic programming to image pixels around a corrupted image region. It dynamically estimates the texture in a corrupted image region. With the best estimated texture, lost pixels are recovered by the geometric interpolation. We apply the texture estimation and geometric interpolation to the pixel sequences from the upper and lower of the lost region, the pixel sequences from the left and the right of the lost region, the pixel sequences from the upper and the left of the lost region, and pixel sequences from the lower and the right of the lost region. Then, the best result is selected based on the set up criterion and use it for error concealment. From experimental results, the proposed spatial error concealment provides the enhancement of the dynamic capability in capturing the image texture as well as subjective and objective quality improvement of concealment results in recovering the corrupted region of images comparing to the previous works.

Compressed bit streams are, in general, very sensitive to erroneous channel. Compressed image or video transport over error prone wireless channels may results in loss or erroneous decoding and due to that bit stream may cause severe degradation on picture quality. Various error control coding techniques like, error resilience, forward error control coding and error concealment are used to control the error. This paper proposes hybrid error concealment technique. In spatial domain three methods weighted average interpolation (WAI), Directional interpolation (DI) and transformed based concealment (TBC) are implemented and compared for various noises. For video, one more temporal concealment is implemented based on motion vector. To reduce the computational complexity and better quality, we proposed a new hybrid spatio-temporal video error concealment technique with mode selection based on boundary matching decision. It performs well compare to spatial and temporal methods separately and provides satisfactory results over a large set of conditional channels, where significant information loss is present.