Research on feedback-based error concealment techniques for packet video transmission

Motion-vector-based error concealment algorithms recover lost blocks by motion compensation with the estimated motion vectors. Traditionally, the estimated motion vectors for the lost block are derived by the direct usage of the neighboring available motion vectors. In this work, the residual signal in a block encoded in the video bitstream is used to measure the reliability of the corresponding motion vector. The residual information is applied to weight the neighboring available motion vectors differently, to further improve a newly proposed mathematical formulation between encoded motion vectors and extrapolated motion vectors. The proposed error concealment method using the residual information outperforms a state-of-the-art error concealment algorithm by up to 2.12 dB in sequence PSNR, and outperforms a classic DMVE method by up to 8.27 dB.

Error concealment is a well-known technique to improve reproduced picture quality in cases which parts of the coded picture are not available at a decoder for reconstruction. The conventional error concealment methods generally make use of the correlation between a damaged marcoblock and its adjacent macroblocks in the same frame and/or the previous frame. However, performance results of concealment methods are different each other and dependent on how much damaged frames and macroblocks are correlated to surrounding ones temporally and spatially. Moreover, most of them can not conceal effectively the consecutively damaged frames or macro blocks caused by wireless bursty packet loss features, since motion vectors and correlation information between the damaged images are already destroyed. The proposed error concealment aware error resilient video coding is a new method which has an intelligent error concealment selector by using pre-estimation technique of the possibly damaged motion vectors at the encoder. The method recommends the best error concealment per macroblock by transmitting additionally its error concealment selection codes to the decoder. The chosen error concealment helps to conceal the damaged MB at the decoder even when losses of frames are bursty. The experimental results show the substantial effectiveness to improve video quality at the cost of a little bit overhead at the transmitted bit streams.

In this paper, we propose two new fast algorithms for motion vector (MV) estimation using spatial correlationof MVs in adjacent blocks. We select a set of MV candidates based on the MV knowledge of its neighboringblocks, and then perform further search to refine the MV result. The first algorithm is performed on each blockconsecutively. The second algorithm is a modified version of the first one by using a block subsampling techniqueto reduce computational cost. We show with experimental results that, compared with full search block matchingalgorithm (FBMA), the proposed algorithms have a speed-up factor ranging from 50 to 100 with only 2-15% MSEincrease when applied to typical test image sequences.Keywords: motion vector estimation, spatial correlation, block matching algorithm, video coding 1 INTRODUCTION Video image compression has been an active research area recently, since it plays an essential role in trans-mission and storage of digital video data. The applications includes multimedia transmission, digital TV, tele-conferencing, videophone, CD-ROM storage, etc. The main idea of video compression is to exploit both spatialand temporal redundancies inherent in image sequences. A commomly used method for reducing temporal re-dundancy is motion compensated predictive coding, which is also employed in the MPEG standard [1], [8], [9].One key ingredient in motion compensated coding is motion vector (MV) estimation.The block matching technique has been widely used for MV estimation due to its simplicity. The full searchblock matching algorithm (FBMA) performs search at all locations in a fixed search window and provides an

This paper presents a remnant error concealment (REC) technique based on generic error concealment (GEC). Two most common issues in video error recovery systems are the locations of lost blocks and error recovery techniques. By using a multistage error detection algorithm to obtain the location of lost blocks of random bit error in the MPEG-2 bit stream, the REC technique uses the spatial neighborhood information to recover the lost blocks. Firstly, estimating the DC (direct current) value of the error block from the DC values of adjacent blocks, a preliminary recovered block is obtained. Subsequently, GEC is applied by using the remnant information (the boundary variations of preliminary recovered block and adjacent blocks) as input and the object function is modified accordingly. Simulation results indicate that REC can achieve input tolerance of a bit error rate (BER) as high as 1/3000, which is significantly improved over the GEC's BER /spl ap/10/sup -5/. In terms of visual performance, the result of REC is smoother than that of GEC.

Block-matching algorithm is the most common technique applied in block-based motion estimation technique. There are several block-matching algorithm based on block-based motion estimation techniques have been developed. Full search (FS), three step search (TSS), new three step search (NTSS), diamond search (DS) and hexagon based search (HS) are the most well known block-matching algorithm. These techniques are applied to video sequences to remove the temporal redundancy for compression purposes and to gauge the motion vector estimation. In addition, the mentioned block-matching algorithms are the baseline techniques that have been used to further develop all the enhanced or improved algorithms. In order to develop the proposed methods, the baseline techniques are studied to develop the proposed algorithms. This chapter proposes modelling of fast unrestricted hexagon search (FUHS16) and unrestricted hexagon-diamond search (UHDS16) algorithms for motion vector estimation, which is based on the theory and application of block-based motion estimation. Both of these algorithms are designed using 16 × 16 block size. In particular, the motion vector estimation, quality performance, computational complexity, and elapsed processing time are emphasised. These parameters have been used to measure the experimental results. It is the aim of this study that this work provides a common framework with which to evaluate and understand block-based matching motion estimation performance. On the theoretical side, four fundamental issues are explored: (1) division of frame, (2) basic block-based matching, (3) motion vector estimation, and (4) block-matching algorithm development. Various existing block-matching motion estimation algorithms have been analysed to develop the fundamental research. Based on the theoretical and fundamental research analysis the FUHS16 and UHDS16 algorithms using 16 × 16 block-based motion estimation formulations were developed. To improve the UHDS16 algorithm, 8 × 8 block-matching technique has been tested. The 8 × 8 block-matching technique is known as UHDS8. The results show positive improvements. From an application perspective, the UHDS8 algorithm efficiently captured the motion vectors in many video sequences. For example, in video compression, the use of motion vectors on individual macro-blocks optimized the motion vector information. The UHDS8 algorithm also offers improvement in terms of image quality performance, computational complexity and elapsed processing time. Thus, this chapter offers contributions in certain areas such as reducing the mechanism of computational complexity in estimating the motion from the video sequences. In particular, the FUHS16, UHDS16 and UHDS8 algorithms were developed to estimate the motion vectors field in the video sequences. Theoretical analysis block-based matching criteria are adapted to FUHS16, UHDS16 and UHDS8 algorithms, which are based on search points technique. Basically, the proposed of FUHS16, UHDS16 and UHDS8 algorithm produces the best motion vector estimation finding based on the block-based matching criteria. Besides that, the UHDS8 algorithm also improves the image quality performances and the search points in terms of the computational complexity. Overall, the study shows that the UHDS8 algorithm produces better results compared to the FUHS16 and UHDS16 algorithm.

In this paper, a prescription-based error concealment (PEC) method is proposed. PEC relies on pre-analyses of the concealment error image (CEI) for I-frames and the optimal error concealment scheme for P-frames at encoder side. CEI is used to enhance the image quality at decoder side after error concealment (by spatial interpolation or zero motion) of the corrupted intra-coded MBs. A set of pre-selected error concealment methods is evaluated for each corrupted inter-coded MB to determine the optimal one for the decoder. Both the CEI and the scheme indices are considered as the prescriptions for decoder and transmitted along with the video bit stream based on a reversible data embedding technique. Experiments show that the proposed method is capable of achieving PSNR improvement of up to 1.48 dB, at a considerable bit-rate, when the packet loss rate is 20%

We use particle filtering for correcting the erroneous motion vectors (MVs), which are derived from the boundary matching algorithm (BMA) in packet video communications. Assuming a two-state Markov channel model for transmission, the error of the extracted MVs by BMA is shown to be modeled by the Gaussian mixture distribution. Formulating the problem in the state space, we deploy particle filtering for denoising the erroneous MVs. The main challenge of using particle filters is high computational complexity that is directly related to the number of particles. The proposed particle filtering scheme is efficient even if the number of particles is decreased. Experimental results are provided to show the efficiency of this filtering approach compared to a recent scheme based on Kalman filtering. The experiments show meaningful increase in the quality of the recovered video sequences in terms of PSNR up to 3 dB compared with the other error concealment techniques. Also, the computational complexity of the proposed scheme is discussed.

The three-dimensional video (3DV) is composed of variable-length stream sequences captured via diversified cameras surrounding an object. Thus, it is an urgent task to accomplish sufficient encoding to be compatible with incoming bandwidth demands, while achieving a recommended 3DV reception performance. In the 3DV compression framework, the lost macro-blocks (MBs) might propagate into the following frames and the adjoining views. Therefore, it is obligatory to avoid error propagation by concealing the corrupted MBs at the decoder through the utilization of appropriate post-processing error concealment (EC) techniques. The existing EC algorithms fundamentally exploit the temporal, inter-view, and spatial matching within the 3DV frames and views to reconstruct the disparity vectors (DVs) and motion vectors (MVs) of the corrupted MBs. Unluckily, in the state of high severe corruptions and heavily erroneous MBs, these concealment algorithms are predominantly unreliable and might give unreliable 3DV quality. Thence, in this work, we suggest the utilization of the outer block boundary matching algorithm to estimate the MVs and the directional interpolation EC algorithm to estimate the DVs of the erroneous MBs. After that, the Bayesian Kalman filter (BKF) is employed because of its efficiency to filter out the inherent errors in the previously predicted DVs and MVs to accomplish better 3D video performance. Experimental results on standard 3DV sequences demonstrate that the suggested BKF-based EC scheme is more powerful with heavy losses. It subjectively and objectively outperforms the traditional concealment techniques at severely random and bursty packet loss rates (PLRs).

This paper presents an efficient error concealment method for multiview video plus depth (MVD) delivery over error-prone channels based on the view synthesis information. Specifically, we propose an enhanced temporal error concealment (TEC) method by estimating the probable partition and missing MVs of a corrupted macroblock (MB) using the synthesized information. Furthermore, the view synthesis error concealment (VSEC) approach that directly uses the synthesized pixels to conceal the corrupted MBs is also adaptively utilized together with the TEC approach. Experimental results have shown the effectiveness of the proposed method compared with the existing methods in terms of both objective and subjective measures.

Compressed bit streams are, in general, very sensitive to erroneous channel. Compressed image or video transport over error prone wireless channels may results in loss or erroneous decoding and due to that bit stream may cause severe degradation on picture quality. Various error control coding techniques like, error resilience, forward error control coding and error concealment are used to control the error. This paper proposes hybrid error concealment technique. In spatial domain three methods weighted average interpolation (WAI), Directional interpolation (DI) and transformed based concealment (TBC) are implemented and compared for various noises. For video, one more temporal concealment is implemented based on motion vector. To reduce the computational complexity and better quality, we proposed a new hybrid spatio-temporal video error concealment technique with mode selection based on boundary matching decision. It performs well compare to spatial and temporal methods separately and provides satisfactory results over a large set of conditional channels, where significant information loss is present.

We propose an error concealment method for robust video transmission based on watermarking technique. In this method, the motion vector information is used as the watermark and embedded in the transform coefficients before quantization. A proper embedding algorithm is designed with the characteristics of the video coder taken into consideration, so that the embedding watermark has little influence on the video quality and the coding efficiency. At the decoder, the extracted watermark is used to effectively detect errors and restore the corrupted motion vectors. Then the restored motion vectors are utilized in error concealment. Simulation results demonstrate the advantages of the proposed method in error prone environment. The proposed method is applicable to all the block-transform based video coding schemes.

We will be able to use highly parallel processing environments. This paper proposes a method for estimating translations, rotations and scaling simultaneously from the estimated motion vectors, and represents its performance with motion estimation experiments. A sector region luminosity correlation is used for estimating motion vectors. The sector region luminosity correlation is robust about the rotation and withstands large motion environments. The proposed method makes the assumption about the scaling and estimates the motion vectors based on the assumptions. Then it randomly creates the pair of the estimated motion vectors. Next, it selects the proper pair using the scaling factor of the pair. The selected pairs are included in the set of reliable motion vector pairs. The reliable motion vector pairs decide the translation, rotation and scaling. With large scaling, it is difficult to estimate the motion using the sector region luminosity correlation. But with the assumptions about the scaling, they can work. Experiments show that the proposed method makes much better correlations between images than SIFT does.

Compressed video bit-streams are extremely sensitive to packet loss over error-prone channels. Error concealment (EC) has been considered as one of error control techniques to improve the reconstructed picture quality against transmission errors. However, EC methods show poor image reconstruction performance due to unavailable scene change information of the video sequence, especially when an abrupt scene change occurs. In this paper, we propose an effective EC method based on scene change detection algorithm (SCDA), which provides information to decide whether spatial or temporal EC is better to be used for intra and inter frames respectively. The simulation results show that the proposed method highly improves the subjective quality of incorrectly decoded frames and obtains an average gain of 0.7dB compared with the H.264/AVC Joint Model reference software.

Video transmission over wireless networks becomes a major part of multimedia studies especially after fourth-generation communication standard has been introduced. There is a growing need to improve video visual quality, while its computational complexity should remain low. One of the low-complexity Error Concealment (EC) methods for missing macroblock (MB) recovery is Boundary Matching Algorithm (BMA). Nevertheless, this method suffers from lack of accuracy. In this paper, we propose a novel modified BMA method by exploiting the correlation of the pixels on the edge of the missing MB. The performance of this method is explained in detail and extra experimental results are given to demonstrate its superiority over BMA method.

We will be able to use highly parallel processing environments. This paper proposes a method for estimating translations, rotations and scaling reaching 10 times simultaneously based on the multiple scaling assumptions, and represents its performance with motion estimation experiments. A sector region luminosity correlation is used for estimating motion vectors. The sector region luminosity correlation is robust about the rotation and withstands large motion environments. The proposed method makes the assumptions about the scaling and estimates the motion vectors based on the assumptions. Then it randomly creates the pair of the estimated motion vectors. Next, it selects the proper pair using the pre-assumed scaling factor. The selected pairs are included in the set of reliable motion vector pairs. The reliable motion vector pairs decide the translation, rotation and scaling. With large scaling, it is difficult to estimate the motion using the sector region luminosity correlation. But with the assumptions about the scaling, they can work. Experiments show that the proposed method makes much better correlations between images than SIFT does in 10 times scaling changes.