Activity-Based Motion Estimation Scheme for H.264 Scalable Video Coding

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

Motion estimation is the biggest part of any video encoding system since it could significantly affect the encoding time and output quality. Until now, several motion estimation algorithms that have the fast motion estimation and a good PSNR are proposed. However these are not considered an embedded system processor using SDRAM as a video frame memory. In this paper, we propose new fast predictive block matching algorithm, named predictive directional rectangular zonal search. Comparing with existing algorithms, it has not only similar output quality, but more efficient operating with SDRAM. It makes possible to design a hardware motion estimator into a compact one. A motion estimation hardware block that is based on the proposed algorithm for H.263 and MPEG-4 video encoder, is implemented by 0.3μm 1.2 V CMOS technology.

A fast full search block matching algorithm is developed. The matching criterion is the sum of absolute differences or the mean square error. The algorithm evaluates lower bounds for the matching criteria for subdivided blocks in order to reduce the number of search positions. It also uses the lower bounds for a fast calculation of the matching criterion for the remaining search positions. The computational complexity of the algorithm is evaluated and compared to the three-step search strategy. The search result of the algorithm is identical to the search result of the exhaustive search.

Motion estimation is a core process within video coding schemes, because it enables the transmission and storage of video signals, while using a lower bit rate. Various methods have been proposed for motion estimation. A full search algorithm is considered to be the optimal method, but it suffers from a high computational burden. In order to reduce the computations, this paper proposes an adaptive two-step edgebased partial distortion search algorithm. The proposed algorithm reduces computations by reducing the total search point and adjusting the search range. The proposed algorithm is 147 times faster than full search (FS), 12 times faster than normalized partial distortion search, and 1.75 times faster than a two-step edge based partial distortion search (TS-EPDS). It also shows a high video quality, in comparison with FS, in terms of PSNR. The proposed algorithm is suitable for real-time implementation of high-quality digital video applications.

Using the motion estimation process is still one of the best technologies for video coding standards such as H.26X or MPEG-X. These standards have been supplied to video compression, but this area is still open algorithms. It can provide excellent coding efficiency while maintaining a low distortion. A new motion estimation of a full search algorithm with big data analytics in this research was proposed, which uses MapReduce analysis metrics. The developed method can not only be used to reduce the stored number of motion vectors compared with the full search algorithm but also to perform superior peak signal-to-noise ratio than other recent developed scheme, and achieve the same peak signal-to-noise ratio as the full search algorithm. Simulation exemplifies that our proposed scheme obtains better improvement than the full search algorithm and other well-known methods. The comparison of our proposed algorithms with other methods is evaluated both subjectively and qualitatively and the simulation demonstrated that our proposed method could obtain the desired reconstructed performance subjectively and objectively.

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

In this paper, we propose the prediction search algorithm for block matching using the temporal/spatial correlation of the video sequence and the renter-biased property of motion vectors The proposed algorithm determines the location of a better starting point for the search of an exact motion vector using the point of the smallest SAD(Sum of Absolute Difference) value by the predicted motion vector from the same block of the previous frame and the predictor candidate pint in each search region and the predicted motion vector from the neighbour blocks of the current frame. And the searching process after moving the starting point is processed a adaptive search pattern according to the magnitude of motion vector Simulation results show that PSNR(Peak-to-Signal Noise Ratio) values are improved up to the 0.75dB as depend on the video sequences and improved about 0.05∼0.34dB on an average except the FS (Full Search) algorithm.

Block matching algorithms (BMAs) are often employed for motion estimation (ME) in video coding. Most conventional BMAs treat the ME problem as an optimization problem and employ certain search schemes to find a solution. Except the time-consuming full search algorithms (FS), other fast algorithms such as the three step search (TSS), searching on a reduced search range, cannot guarantee optimal solutions, i.e., search is often trapped at local minima; the ME results are thus usually unsatisfactory. Few of them makes use of the information inherent in the images explicitly. We propose a new ME algorithm which can reduce the search range while guaranteeing global optimality in most cases. Microblock visual patterns are designed to extract edge information to guide block matching: searching is only carried out at places where the real match most likely happens; that is where similar edge features present. The proposed algorithm obtains a speed about 7 - 8 times as fast as that of FS with same search range. The prediction quality is very close to that of FS and much better than that of TSS. It can produce MPEG-1 or MPEG-2 compatible motion vectors, and can be extended to model-based ME. It is suitable for parallel implementation too. Moreover, the visual patterns are potential resources for video indexing to facilitate content-based information retrieval, which is important for applications such as VOD.

A neural network based motion-estimation technique (for successive video frames) is developed that is applicable to purely translational as well as affine movements. In addition to yielding highly accurate results, a benefit of this technique is the elimination of fractional-pixel interpolation and corresponding search. The technique is formulated in terms of a modified Hopfield neural network. The procedure consists of two stages: estimation of the neural network parameters from the present and previous frames, followed by the estimation of the affinity weights for the candidate macroblocks. The reconstructed macroblock then consists of the sum of the candidate macroblocks weighted by their affinity values as defined. Experiments on translational motion indicate that this technique produces a 5 dB higher peak signal to noise ratio (PSNR) than the full search algorithm, 6 dB higher than the logarithmic block matching algorithms, and 2 dB higher than some of the very recent motion estimation algorithms. Preliminary experiments using the neural network affine algorithm on synthetically sheared images produce an 11 dB higher PSNR than the full search and LBM. Further, due to the neural network's fault tolerant nature and inherent parallelism, potential exists for VLSI implementation as an interconnection of small cells on a single chip.

For video compression, motion estimation is popularly employed to exploit temporal correlation existing in video sequences. If we employ the full search block matching algorithm for estimating motion vectors, it requires very heavy computational complexity. Although several fast block matching algorithms have been proposed to solve this problem, they sacrifice their reconstructed image quality. In this paper, we derive optimal search patterns for fast block matching motion estimation. By analyzing the block matching algorithm as a function of the block size and the shape, we find optimal search patterns for initial motion estimation. The proposed idea can provide an analytical ground for the current MPEG-2 proposals. In addition, we propose a new fast motion estimation algorithm using adaptive search patterns, considering matching criteria and statistical properties of object displacement. In order to select an appropriate search pattern, we exploit the relationship between the motion vector and the frame difference of each block. By changing the search pattern adaptively, we can improve the motion prediction accuracy, while reducing the required computational complexity compared to other fast block matching algorithms.

In this paper, a new fast block matching algorithm is developed for motion estimation. The motion estimation algorithm use improved 3-step block matching (ITSS) algorithm that can produces better quality performance and less computational time compared with three-step search (TSS) algorithm and correct the motion vector by using Zero Detection and vector filter. From the experimental results, the proposed algorithm is superior to TSS in both quality performance and computational complexity.

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

H.264/AVC, the latest video coding standard, adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bitrate. However, the full RD cost calculation for all intra-prediction modes, the exhaustive searches for finding optimal motion vectors for all block sizes, and the multiple references frame procedure considerably increase its computational complexity with the allowed number of prediction modes. The authors have previously proposed an approach for both inter- and intra-mode decisions that takes into account the several effective parameters, image content type, quantization parameter, correlation of motion vectors and motion details of video objects. In order to reduce the complexity, here we propose a new two-step adaptive search range decision by developing a combination of previous algorithm. The first step of our adaptive search range decision determines the search range for current MB according to the motion vectors of the previous MBs. In the second step, the motion vector obtained in the current frame is used to adaptively searching the window size of the previous reference frame for multi-frame motion estimation. As such, only a subset of inter- and intra-modes is chosen for RDO calculation. Experimental results show that the proposed algorithm reduces the total encoding cost with negligible loss in PSNR and a slightly increase in the required bitrate, when compared to our previous algorithm and other algorithms reported in the literature.

A 90-nm CMOS motion estimation (ME) processor was developed by employing dynamic voltage and frequency scaling (DVFS) to greatly reduce the dynamic power. To make full use of the advantages of DVTS, a fast ME algorithm and a small on-chip DC/DC converter were also developed. The fast ME algorithm can adaptively predict the optimum supply voltage (V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</sub> ) and the optimum clock frequency (f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> .) before each block matching process stalls. Power dissipation of the MK processor, which contained an absolute difference accumulator as well as the on-chip DC/DC converter and DVFS controller, was reduced to 29.1 muW, which was only 3% that of a conventional ME processor.

In an H.264 video encoder, motion estimation (ME) is the most time-consuming component. The ME process consists of two stages, integer pixel search and fractional pixel search. Since the complexity of integer pixel search has been greatly reduced by numerous fast ME algorithms, the computation overhead required by fractional pixel ME has become relatively significant. To reduce the complexity of fractional pixel ME, we propose a prediction-based directional fractional pixel ME algorithm. We utilize more accurate motion vector predictions and directional search to achieve better computation reduction. We further propose an early termination method to decrease the amount of search. Experimental results show that, compared to the full search sub-pel ME and the fast sub-pel ME proposed in H.264, the proposed method can reduce up to 84% and 74% of fractional pixel search points respectively, with a negligible degradation in quality.