MPEG-4 compatible stereoscopic sequence codec for stereo broadcasting

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce the computation by limiting the number of searching locations. This is accomplished at the expense of less accuracy of motion estimation and gives rise to an appreciably higher mean squared error (MSE) for motion compensated images. We present a new fast and efficient search algorithm for block motion estimation that produces better quality performance and less computational time compared with a three-step search (TSS) algorithm. The proposed algorithms are based on the ideas of the dithering pattern for pixel decimation, multiple-candidate for pixel-decimation-based full search, and center-based distribution of the motion vector. From the experimental results, the proposed algorithm is superior to the TSS in both quality performance (about 0.2 dB) and computational complexity (about half).

Based on the temporal and interview redundancy of stereoscopic video coding, a fast motion and disparity estimation algorithm is proposed in this paper. Firstly, a credible start point is designed taking use of the view-temporal correlation of neighboring motion and disparity vectors. Secondly, an improved searching strategy is proposed to reduce the computational complexity, in which only 2-pixel refinement is needed around the start point. The proposed algorithm can get both motion vector and disparity vector during one estimation process. The experimental results demonstrate that the proposed algorithm can reduce up to 96% coding time compared with inter-view full search with a negligible quality drop.

To reduce the heavy computational complexity of uni-directional and bi-directional predictions in multiview video coding (MVC), this paper proposes a multiview video coding algorithm. Uni-directional prediction is optimized. By using the geometry relationship between disparity and motion vectors, an iterative process is preformed. In each process, a new adaptive search range is used to revise motion/disparity vector. At last, optimal motion and disparity vectors are achieved. For bi-directional prediction, TZSearch algorithm is utilized to accelerate prediction. Furthermore, an early termination strategy is devised on the basis of relationships among bi-directional prediction, model error, quantization parameter and temporal level. Experimental results show that computational complexity can be significantly reduced while maintaining video coding performance compared with JMVC full search algorithm.

For video compression, motion estimation is popularly employed to exploit temporal correlation existing in video sequences. If we employ the full search block matching algorithm for estimating motion vectors, it requires very heavy computational complexity. Although several fast block matching algorithms have been proposed to solve this problem, they sacrifice their reconstructed image quality. In this paper, we derive optimal search patterns for fast block matching motion estimation. By analyzing the block matching algorithm as a function of the block size and the shape, we find optimal search patterns for initial motion estimation. The proposed idea can provide an analytical ground for the current MPEG-2 proposals. In addition, we propose a new fast motion estimation algorithm using adaptive search patterns, considering matching criteria and statistical properties of object displacement. In order to select an appropriate search pattern, we exploit the relationship between the motion vector and the frame difference of each block. By changing the search pattern adaptively, we can improve the motion prediction accuracy, while reducing the required computational complexity compared to other fast block matching algorithms.

We present a novel multi-resolution block matching algorithm (BMA) for fast motion estimation. At the coarsest level, a full search BMA (FSBMA) is performed for searching complex or random motion. Concurrently, spatial correlation of motion vector (MV) field is used for searching continuous motion. Here we present an efficient method for searching full resolution MVs without MV decimation even at the coarsest level. After the coarsest level search, two or three initial MV candidates are chosen for the next level. At the further levels, the MV candidates are refined within much smaller search areas. Simulation results show that in comparison with FSBMA, the proposed BMA achieves a speed-up factor over 710 with minor PSNR degradation of 0.2 dB at most, under a normal MPEG-2 coding environment. Furthermore, our scheme is also suitable for hardware implementation due to regular data-flow.

To reduce the amount of computation of the full search (FS) algorithm for fast motion estimation, we propose a new and fast matching algorithm without degradation of predicted images as in conventional FS. The computational reduction without any degradation in the predicted image comes from fast removal of impossible motion vectors. We obtain faster removal of inappropriate motion vectors using efficient matching units from the localization of complex area in image data. In this paper, we show three properties in block matching of motion estimation. Experimentally, we reduce unnecessary computations by about 30% with our algorithm compared with the conventional fast matching scan algorithm.

Fast search motion estimation algorithms when compared to full search motion estimation algorithms often converge to a local minimum, providing a momentous reduction in computational cost. However, the motion vectors measurement process in fast search algorithms is subject to noise and matching errors. Researchers have investigated the use of mathematical tools used for stochastic estimation from noisy measurements, to seek optimal estimates. Amongst those tools, is the conventional Kalman filtering, which addresses the general problem of trying to estimate the state of a discrete-time controlled process that is governed by the linear stochastic difference. This research investigates the possible combinations of benchmark motion estimation algorithms and the Kalman filter. In this paper, the author presents an in-depth investigation and a detailed analysis on the use of the above combination, and seeks to establish conditions under which the application would be successful. Experimental results show that the above is possible only under certain conditions and constraints of certain properties of the video sequences being coded. Furthermore, a recommendation is made on when it is possible to use the adaptive Kalman filter instead of the conventional filter to enhance the motion vectors at the cost of extra complexity.

Most of the fast search motion estimation algorithms reduce the computational cost of motion estimation greatly by checking only a few search points inside the search area by using full distortion measure. This paper proposes a multi-layer motion estimation which employs partial distortion as its distortion measure to reduce the number of computations involved in each search point instead of reducing the number of search points. A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer. It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm.

Fast and efficient disparity estimation (DE) is a key technology in stereoscopic image sequence processing. On the basis of analyzing the correlation between disparity vectors and motion vectors, a new fast disparity estimation algorithm for stereo video coding is presented in this paper. Experimental results show that, compared with the full search algorithm, the proposed algorithm can significantly reduce DE complexity while the reconstructed image quality is quite close to that of full search method.

We describe a fully scalable wavelet-based 2D+t (in-band) video coding architecture. We propose new coding tools specifically designed for this framework aimed at two goals: reduce the computational complexity at the encoder without sacrificing compression; improve the coding efficiency, especially at low bitrates. To this end, we focus our attention on motion estimation and motion vector encoding. We propose a fast motion estimation algorithm that works in the wavelet domain and exploits the geometrical properties of the wavelet subbands. We show that the computational complexity grows linearly with the size of the search window, yet approaching the performance of a full search strategy. We extend the proposed motion estimation algorithm to work with blocks of variable sizes, in order to better capture local motion characteristics, thus improving in terms of rate-distortion behavior. Given this motion field representation, we propose a motion vector coding algorithm that allows to adaptively scale the motion bit budget according to the target bitrate, improving the coding efficiency at low bitrates. Finally, we show how to optimally scale the motion field when the sequence is decoded at reduced spatial resolution. Experimental results illustrate the advantages of each individual coding tool presented in this paper. Based on these simulations, we define the best configuration of coding parameters and we compare the proposed codec with MC-EZBC, a widely used reference codec implementing the t+2D framework.

One objective in MPEG-2 to H.264/advanced video coding transcoding is to improve the H.264/AVC compression ratio by using more advanced macroblock encoding modes. The motion re-estimation process is by far the most time-consuming process in this type of video transcoding. In this paper, we present an efficient H.264/AVC block size partitioning prediction algorithm for MPEG-2 to H.264/AVC transcoding applications. Our algorithm uses rate-distortion optimization techniques and predicted initial motion vectors to estimate block size partitioning. It is also shown that using block size partitioning smaller than 8 × 8 (i.e., 8 × 4, 4 × 8, and 4 × 4) results in negligible compression improvements, and thus these sizes should be avoided in transcoding. Experimental results show that, compared to the state-of-the-art transcoding scheme, our transcoder yields similar rate-distortion performance, while the computational complexity is significantly reduced, requiring an average of 29% of the computations. Compared to the full-search scheme, our proposed algorithm reduces the computational complexity by about 99.47% for standard-definition television sequences and 98.66% for common intermediate format sequences. Compared to UMHexagonS, the fast motion estimation algorithm used in H.264/AVC, the experimental results show that our proposed algorithm is a better trade-off between computational complexity and picture quality.

In this paper, we propose a variable step search fast motion estimation algorithm using local statistics of neighboring motion vectors. Using the degree of correlation between neighboring motion vectors, motion search range of each macroblock is adaptively adjusted to reduce unnecessary search points. Based on the determined search range, motion vector is obtained by variable step search step. Experimental results show that comparing to fast full spiral search, fixed step, and fast variable block size motion estimation algorithms, the motion search points and computing time of proposed algorithm are dramatically reduced with acceptable loss in coding performance.

Motion in modern video coders is estimated using a block matching algorithm that calculates the distance and direction of motion on a block-by-block basis. In this paper, a novel fast block-based motion estimation algorithm is proposed. This algorithm uses an efficient projection framework that bounds the distance between a template block and candidate blocks. Fast projection is performed using a family of highly efficient filter kernels--the gray-code kernels--requiring only 2 operations per pixel per kernel. The projection framework is combined with a rejection scheme which allows rapid rejection of candidate blocks that are distant from the template block. The tradeoff between computational complexity and quality of results can be easily controlled in the proposed algorithm; thus, it enables adaptivity to image content to further improve the results. Experiments show that the proposed adaptive algorithm outperforms other popular fast motion estimation algorithms.

Due to the high computational complexity of the full-search algorithm, fast block motion estimation algorithms are needed for real-time implementations of the current video coding standards. Recently, a three-step search algorithm for block motion estimation has been proposed in the literature. In this paper, a fast scheme for the three-step search algorithm is proposed to reduce the computational complexity of the three-step search algorithm without loss of its accuracy. By using eight-bit partial sums and the corresponding lower bound for the modified sum of absolute difference (MSAD) criterion in the proposed scheme, a large number of candidate motion vectors are discarded while still retaining the optimal motion vector of the three-step search algorithm. In this way, the computational requirement of the three-step search algorithm is significantly reduced while maintaining its accuracy. It is shown that the byte-type data-parallelism on a single instruction multiple data (SIMD) architecture can be utilized to further accelerate the execution of the proposed scheme. Simulations of the scheme are carried out for various benchmark video sequences and the results demonstrate that the new scheme can reduce the computational complexity of the three-step search algorithm by 20 to 40 percent with no loss of its accuracy.