Error Concealment Algorithms and Quality Evaluation for H.264 Videos

In this paper, a prescription-based error concealment (PEC) method is proposed. PEC relies on pre-analyses of the concealment error image (CEI) for I-frames and the optimal error concealment scheme for P-frames at encoder side. CEI is used to enhance the image quality at decoder side after error concealment (by spatial interpolation or zero motion) of the corrupted intra-coded MBs. A set of pre-selected error concealment methods is evaluated for each corrupted inter-coded MB to determine the optimal one for the decoder. Both the CEI and the scheme indices are considered as the prescriptions for decoder and transmitted along with the video bit stream based on a reversible data embedding technique. Experiments show that the proposed method is capable of achieving PSNR improvement of up to 1.48 dB, at a considerable bit-rate, when the packet loss rate is 20%

IPTV (Internet Protocol Television) provides interactive function, which greatly improves subjective experience of users. However, IP network is a best-effort network and does not guarantee a reliable connection, which makes video transmission based on IP network prone to problems. This paper focuses on the relationship between packet loss and video perception quality in IPTV transmission. Through subjective evaluation experiments, the influence of different packet loss rate and packet loss distributions (burst loss and random loss) are quantitatively studied. Experimental results show: When packet loss rate is the same, video with burst loss has a better quality than one with random loss. The proportion of burst loss in total packet loss increases, and subjective perception quality increases exponentially. Based on experimental results, this paper also designs an objective evaluation model of video quality based on packet loss rate and packet loss distribution.

An Adaptive Spatio-Temporal Error Concealment (ASTEC) scheme based on H.264/AVC is proposed in the paper to improve the quality of the rebuilt frame in H.264/AVC decoder side. Combining the spatial error concealment method with the temporal one, a new scheme is proposed to conceal an erroneous macro block according to its encoded mode. The temporal error concealment scheme adopts the weighted Boundary Matching Algorithm to recover the erroneous macro block. The spatial error concealment scheme recovers the erroneous macro block by exploiting the directional interpolation which subdivides the macro block according to the number and the direction of edges going through it. Experiments show that the proposed concealment scheme can restrain the diffusion of erroneous macro block and improve the quality of the rebuilt frame subjectively as well as objectively.

The three-dimensional video (3DV) is composed of variable-length stream sequences captured via diversified cameras surrounding an object. Thus, it is an urgent task to accomplish sufficient encoding to be compatible with incoming bandwidth demands, while achieving a recommended 3DV reception performance. In the 3DV compression framework, the lost macro-blocks (MBs) might propagate into the following frames and the adjoining views. Therefore, it is obligatory to avoid error propagation by concealing the corrupted MBs at the decoder through the utilization of appropriate post-processing error concealment (EC) techniques. The existing EC algorithms fundamentally exploit the temporal, inter-view, and spatial matching within the 3DV frames and views to reconstruct the disparity vectors (DVs) and motion vectors (MVs) of the corrupted MBs. Unluckily, in the state of high severe corruptions and heavily erroneous MBs, these concealment algorithms are predominantly unreliable and might give unreliable 3DV quality. Thence, in this work, we suggest the utilization of the outer block boundary matching algorithm to estimate the MVs and the directional interpolation EC algorithm to estimate the DVs of the erroneous MBs. After that, the Bayesian Kalman filter (BKF) is employed because of its efficiency to filter out the inherent errors in the previously predicted DVs and MVs to accomplish better 3D video performance. Experimental results on standard 3DV sequences demonstrate that the suggested BKF-based EC scheme is more powerful with heavy losses. It subjectively and objectively outperforms the traditional concealment techniques at severely random and bursty packet loss rates (PLRs).

Error concealment at the decoder side is an economical approach to ensuring an acceptable and stable video quality in case of packet erasure or loss, and thus, it has attracted considerable research and application interest. Relevant techniques usually employ the spatial or temporal correlation to recover the motion vectors (MVs) of the missing blocks, and interpolation, extrapolation, or boundary-matching schemes are usually effective. However, for heavily corrupted sequences, e.g., with block loss rate beyond 50 %, most methods might perform less satisfactorily. Inspired by the tracking efficiency of Kalman filter (KF), in the present work, we adopted it to predict the missing MVs, and the unpredicted ones (minority) were restored complementarily using a modified bilinear motion field interpolation (MFI) method. Since the KF prediction is independent of the loss rate, the present framework proves to be robust for heavily corrupted videos. Experimental results on typical sequences reveal that the proposed algorithm outperforms the boundary-matching algorithm embedded in the H.264/AVC reference code, the MFI and the MV extrapolation techniques in the literature.

: The research tasks are: Task 1, Scalable Coding; Task 2, Alternatives to variable length coding (VLC); and Task3, Packetizing for Error Concealment. Initial work on Task 1 built on research from a prior DARPA contract, and resulted in a scalable video coder with the unique property of scalable transmission at six resolution/frame-rate combinations. Task 2 work combined fixed and variable length coding (VLC) into a joint coding approach with distinct robustness advantages, while giving up only a slight amount of performance versus straight VLC coding in the noise-free case. Task 3 Packetzing for Error Concealment explored the new technique dispersive packetization and combined it with error concealment in a state-of-the-art subband/wavelet image coder. An ns-2 network simulation revealed substantial robustness to packet loss error, with useable images recovered under packet loss rates at high as 30%, thus implementing the graceful degradation property.

Although the smaller quantization parameter has smaller coding distortion of message source, the overall distortion rate at the receiver does not necessarily decrease as its quantization parameter decreases. The reason is that a smaller quantization parameter often means a long transmission queue, and a long transmission queue means a bigger loss rate and channel distortion. In this paper, we propose a packet loss-distortion driven cross-layer optimization of video transmission for H.264 video applications in 802.11e wireless networks. Firstly, we analyzed the relationship between quantization parameter and quantization distortion and built an estimation model of transmission distortion. Then the total distortions in the received station are estimated according to the packet loss rate of different video data partition. Secondly, a selection algorithm of optimal quantization parameter based on the total distortion is presented. Our experimental results demonstrate that, at certain loss rates, the proposed method not only outperforms the up-bottom cross-layer optimization with various queue priorities for video data partitions, but also outperforms the bottom-up cross-layer with an adaptive quantization step selection both in terms of received-end destination and video traffic.

Error concealment is a well-known technique to improve reproduced picture quality in cases which parts of the coded picture are not available at a decoder for reconstruction. The conventional error concealment methods generally make use of the correlation between a damaged marcoblock and its adjacent macroblocks in the same frame and/or the previous frame. However, performance results of concealment methods are different each other and dependent on how much damaged frames and macroblocks are correlated to surrounding ones temporally and spatially. Moreover, most of them can not conceal effectively the consecutively damaged frames or macro blocks caused by wireless bursty packet loss features, since motion vectors and correlation information between the damaged images are already destroyed. The proposed error concealment aware error resilient video coding is a new method which has an intelligent error concealment selector by using pre-estimation technique of the possibly damaged motion vectors at the encoder. The method recommends the best error concealment per macroblock by transmitting additionally its error concealment selection codes to the decoder. The chosen error concealment helps to conceal the damaged MB at the decoder even when losses of frames are bursty. The experimental results show the substantial effectiveness to improve video quality at the cost of a little bit overhead at the transmitted bit streams.

Conventional end-to-end distortion models measure the overall distortion based on independent estimation of the source distortion and channel distortion. However, they are not correlating well with perceptual characteristics in which a strong dependency exists among the source distortion, channel distortion and video content. As most compressed videos are represented to human users, perception-based end-to-end distortion model should be developed for error resilient video coding. In this paper, we propose a SSIM-based end-to-end distortion model to optimally estimate the overall perceptual distortion due to quantization, error concealment and error propagation. Experiments show that the proposed end-to-end distortion model can bring significant visual quality improvement for H.264/AVC video coding over packet-switched networks.KeywordsError resilienceend-to-end distortion modelstructural similarity

In this paper, we propose an error concealment technique for H.264 coded videos. The algorithm is targeted at compressed videos degraded after packet losses caused by transmission over an unreliable channel. We use a combination of watermarking and half toning (dithering) techniques. At the encoder side, a dithered version of each video frame is embedded into the video using a watermarking technique. The watermarking technique used by the proposed algorithm is a modified version of the Quantization Index Modulation (QIM) algorithm, which provides a good data hiding capacity. At the decoder side, the algorithm identifies which packets of the video were lost, extracts the corresponding mark, and applies an inverse half toning technique to estimate the original content. The algorithm is fast and has a good performance, being able to restore content with a better quality than the default H.264 error concealment algorithm.

Mobile Ad-hoc Networks (MANETs) usually suffer from high packet loss and high link failure rates. These challenges usually increase congestion on some links while other links are almost free. In this paper, we propose a novel mechanism to enhance QoS in multipath routing protocols in MANETs based on the InfiniBand (IB) QoS architecture. The basic idea of our approach is to enhance the load balancing and thus reduce congestion on overloaded links. This mechanism has enabled us to (1) give critical applications higher priority when routing their packets across the network, (2) effectively manage frequent connections and disconnections and thus reduce link failures and packet loss rates, and (3) reduce the overall power consumption as a consequence of the previous gains. We have tested the scheme on the (IBsim) simulator and achieved significant improvements in QoS parameters compared to two well- known routing protocols: AODV and AOMDV.

Error concealment at the decoder side is an economical approach to ensuring an acceptable and stable video quality in case of packet erasure or loss, and thus attracts lots of interest. Current techniques mainly employ the spatial or temporal correlation to predict the motion vectors of the missing blocks, and interpolation, extrapolation or boundary matching schemes are usually effective. However, for heavily corrupted sequences, e.g., with macroblock loss rate beyond 50%, most methods might perform less satisfactorily. Inspired by the tracking efficiency of Kalman filter, in the present work, we adopted it to predict the missing MVs, and the unpredicted ones (minority) were recovered complementarily by using the bilinear motion field interpolation (MFI) method. The experimental results on typical sequences reveal that the proposed algorithm outperforms the boundary matching algorithm and the MFI technique by up to about 4.77 dB and 3.66 dB respectively for heavily corrupted videos.

Error concealment (EC) is one of the target applications of inpainting techniques. Some methods combine the estimated lost motion vectors (MVs) with the exemplar-based inpainting technique to recover the lost regions. Due to the erroneous motion vectors that might indicate a moving object as background object and vice versa, these methods are still showing visual artifacts in the recovered regions. In this paper, a concept of motion map that can be easily generated in the decoder side is introduced and it is combined with the exemplar-based inpainting technique. The proposed method introduces an adaptive search window size that trades-off the quality and complexity. Moreover, an optional blending technique is proposed to limit the spatio-temporal artifacts. Experiments show that the proposed method improves the visual quality with 5dB on average relative to the state-of-the-art inpainting-based EC method.

Compressed video bit-streams are extremely sensitive to packet loss over error-prone channels. Error concealment (EC) has been considered as one of error control techniques to improve the reconstructed picture quality against transmission errors. However, EC methods show poor image reconstruction performance due to unavailable scene change information of the video sequence, especially when an abrupt scene change occurs. In this paper, we propose an effective EC method based on scene change detection algorithm (SCDA), which provides information to decide whether spatial or temporal EC is better to be used for intra and inter frames respectively. The simulation results show that the proposed method highly improves the subjective quality of incorrectly decoded frames and obtains an average gain of 0.7dB compared with the H.264/AVC Joint Model reference software.

With the exponential growth of video traffic over wireless networked and embedded devices, mechanisms are needed to predict and control perceptual video quality to meet the quality of experience (QoE) requirements in an energy-efficient way. This paper proposes an energy-efficient QoE support framework for wireless video communications. It consists of two components: 1) a perceptual video quality model that allows the prediction of video quality in real-time and with low complexity, and 2) an application layer energy-efficient and content-aware forward error correction (FEC) scheme for preventing quality degradation caused by network packet losses. The perceptual video quality model characterizes factors related to video content as well as distortion caused by compression and transmission. Prediction of perceptual quality is achieved through a decision tree using a set of observable features from the compressed bitstream and the network. The proposed model can achieve prediction accuracy of 88.9% and 90.5% on two distinct testing sets. Based on the proposed quality model, a novel FEC scheme is introduced to protect video packets from losses during transmission. Given a user-defined perceptual quality requirement, the FEC scheme adjusts the level of protection for different components in a video stream to minimize network overhead. Simulation results show that the proposed FEC scheme can enhance the perceptual quality of videos. Compared to conventional FEC methods for video communications, the proposed FEC scheme can reduce network overhead by 41% on average.