Motion estimation using maximum sub-image and sub-pixel phase correlation on a DSP platform

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

Fast motion estimation for H.264

Motion estimation (ME) is one of the most time-consuming parts in video coding. The use of multiple partition sizes in H.264/AVC makes it even more complicated when compared to ME in conventional video coding standards. It is important to develop fast and effective sub-pixel ME algorithms since: 1) the computation overhead by sub-pixel ME has become relatively significant while the complexity of integer-pixel search has been greatly reduced by fast algorithms, and 2) reducing sub-pixel search points can greatly save the computation for sub-pixel interpolation. In this letter, a novel fast sub-pixel ME algorithm is proposed which performs a “rough” sub-pixel search before the partition selection, and performs a “precise” sub-pixel search for the best partition. By reducing the searching load for the large number of non-best partitions, the computation complexity for sub-pixel search can be greatly decreased. Experimental results show that our method can reduce the sub-pixel search points by more than 50% compared to existing fast sub-pixel ME methods with negligible quality degradation.

The constrained two-bit transform (C2BT) is recently proposed for low complexity motion estimation (ME) to reduce the high computational complexity of the typical two-bit transform (2BT) while maintaining the ME accuracy. And the bit-inverted Gray-coded bit-plane matching (BGCBPM) is also proposed to enhance the ME accuracy. In this paper, we propose a fast ME algorithm exploiting these two bit-plane matching (BPM) criteria. Using the low complexity image transformations of typical images into bit-planes and their corresponding matching criteria and another hybrid matching criterion, we can efficiently determine multiple candidate motion vectors and increase the ME accuracy substantially. Experimental results show that the peak-to-peak signal-to-noise ratio (PSNR) difference between the proposed algorithm and the typical sum of absolute differences (SAD) based full search algorithm is only 0.03 dB on average with negligible computational complexity increase.

To efficiently encode data-intensive multi-view imaging content, conventional hybrid predictive coding methodologies choose to address the compression by exploiting temporal and inter-viewpoint redundancy. However, their key yet time-consuming component, motion estimation (ME), is usually not efficient in inter-viewpoint prediction because inter-viewpoint motion is quite different from temporal motion. In essence, inter-viewpoint correlation is subject to epipolar geometry, which provides constraints for multi-view image sequences. A fast inter-viewpoint ME technique is hence proposed in this paper to accelerate the encoding by employing epipolar geometry. Theoretical analysis and experimental results prove that the proposed ME algorithm can greatly reduce search region and effectively track large and irregular motion that is typical for convergent multi-view camera setups. As a result, compared with fast full search at large search size adopted in H.264, our proposed ME algorithm can obtain a similar coding efficiency while achieving a speedup ratio of 2.9.

With the development of fast motion estimation (ME) algorithms towards video compression standard H.264, burden on integer-pel ME is slightly reduced. However the integer-pel ME is becoming computationally complex due to demand of higher and higher resolution videos. On the other hand, there is always trade-off between the speed and performance of these search algorithms. This work improvises the performance of content awareness enabled integer-pel ME algorithm for encoding the fast and slow motion video sequences of high definition (HD) resolution category viz. 1280×720 and 1920×1080. The algorithm proposes a novel notion of 'integrity index' with focus on increasing the PSNR and reducing the bit rate. It is implemented in the frame work of x265 version 1.7 video encoder. Motion independent ME algorithm is analysed quantitatively in terms of parameters based on rate distortion and ME time. The proposed algorithm is found to be performing better on BD-Rate and BD-PSNR parameters for the videos of both resolutions under consideration. Observed increase in motion estimation time is less than four seconds compared to hexagonal search algorithm of ME which is the benchmark in fast ME algorithm.

Many fast integer-pixel motion estimation algorithms have been developed for the High Efficiency Video Coding Standard, however the speed of sub-pixel motion estimation still has room for improvement. A fast sub-pixel motion estimation algorithm is proposed in this paper to speed up the sub-pixel search process. First, the proposed scheme skips sub-pixel search process in smooth prediction units. Then a fast sub-pixel search algorithm based on texture direction analysis is proposed to further reduce the computational complexity of subpixel motion estimation. The simulation results show that compared with the Full Sub-pixel Search (FSPS), the encoding complexity of the whole motion estimation process can be reduced by an average of 40.9% with negligible coding performance loss.

A new sub-pixel block matching algorithm for fast motion estimation is proposed, which is designed to reduce the search position by using the distortion information of the evaluated integer points in the small diamond search pattern. In order to use the single instruction multiple data (SIMD) technique to fast parallel implement it, the storage order of the interpolated sample data must be realigned. Experimental results show that the proposed algorithm has a significant computational speedup compared to the original, with similar accuracy.

A 90-nm CMOS motion estimation (ME) processor was developed by employing dynamic voltage and frequency scaling (DVFS) to greatly reduce the dynamic power. To make full use of the advantages of DVTS, a fast ME algorithm and a small on-chip DC/DC converter were also developed. The fast ME algorithm can adaptively predict the optimum supply voltage (V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</sub> ) and the optimum clock frequency (f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> .) before each block matching process stalls. Power dissipation of the MK processor, which contained an absolute difference accumulator as well as the on-chip DC/DC converter and DVFS controller, was reduced to 29.1 muW, which was only 3% that of a conventional ME processor.

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce the computation by limiting the number of searching locations. This is accomplished at the expense of less accuracy of motion estimation and gives rise to an appreciably higher mean squared error (MSE) for motion compensated images. We present a new fast and efficient search algorithm for block motion estimation that produces better quality performance and less computational time compared with a three-step search (TSS) algorithm. The proposed algorithms are based on the ideas of the dithering pattern for pixel decimation, multiple-candidate for pixel-decimation-based full search, and center-based distribution of the motion vector. From the experimental results, the proposed algorithm is superior to the TSS in both quality performance (about 0.2 dB) and computational complexity (about half).

In an H.264 video encoder, motion estimation (ME) is the most time-consuming component. The ME process consists of two stages, integer pixel search and fractional pixel search. Since the complexity of integer pixel search has been greatly reduced by numerous fast ME algorithms, the computation overhead required by fractional pixel ME has become relatively significant. To reduce the complexity of fractional pixel ME, we propose a prediction-based directional fractional pixel ME algorithm. We utilize more accurate motion vector predictions and directional search to achieve better computation reduction. We further propose an early termination method to decrease the amount of search. Experimental results show that, compared to the full search sub-pel ME and the fast sub-pel ME proposed in H.264, the proposed method can reduce up to 84% and 74% of fractional pixel search points respectively, with a negligible degradation in quality.

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

There is an increasing need for high quality video on low power, portable devices. Possible target applications range from entertainment and personal communications to security and health care. While H.264/AVC answers the need for high quality video at lower bit rates, it is significantly more complex than previous coding standards and thus results in greater power consumption in practical implementations. In particular, motion estimation (ME), in H.264/AVC consumes the largest power in an H.264/AVC encoder. It is therefore critical to speed-up integer ME in H.264/AVC via fast motion estimation (FME) algorithms and hardware acceleration. In this paper, we present our hardware oriented modifications to a hybrid FME algorithm, our architecture based on the modified algorithm, and our implementation and prototype on a PowerPC-based Field Programmable System on Chip (FPSoC). Our results show that the modified hybrid FME algorithm on average, outperforms previous state-of-the-art FME algorithms, while its losses when compared with FSME, in terms of PSNR performance and computation time, are insignificant. We show that although our implementation platform is FPGA-based, our implementation results compare favourably with previous architectures implemented on ASICs. Finally we also show an improvement over some existing architectures implemented on FPGAs.

We studied an efficient software implementation of H.264/AVC sub-pixel motion estimation (ME) algorithm on a VLIW-SIMD digital signal processor, TMS320C6416. The sub-pixel ME algorithm demands large memory accesses while the required arithmetic operations are fairly simple. Although the CPU clock cycles for arithmetic operations can be reduced much by employing sub-word operations and applying software pipelining techniques, the limited memory bandwidth of the architecture restricts the overall performance. Moreover, aggressive VLIW-SIMD optimization results in the degradation of the performance by causing excessive CPU stalls during memory accesses. In this paper, we relieved the memory bandwidth requirements for creating quarter-pixel images by reducing the precision of image data, from 8 bits to 4 bits. As a result, the amount of memory accesses is much reduced at the cost of some increase of the arithmetic operations, which contributes to the balance of arithmetic and memory access operations. The experimental result shows that the memory stall cycles are decreased by 80% and the speed-up of 260% is obtained. The bit rate of the encoded video stream is increased slightly, about 2% on the average, due to the effects of quantization.

Motion estimation(ME) is one of the most important modules in digital video encoding/decoding and video post processing. High-quality and fast ME algorithms are desired by many applications. In this paper, motion estimation algorithms are explored in terms of execution efficiency on the CUDA(Compute Unified Device Architecture) technology which is a parallel computing architecture developed by nVIDIA. In this paper, we implemented the full search algorithm(FS), the diamond search algorithm, and the four step algorithms in both the CPU and the CUDA platforms. Experiment results show that the CUDA-based implementations of these ME algorithms can be more than 8 times as fast as the those CPU-based ME implementations.