Block based parallel motion estimation using programmable graphics hardware

Ever since the introduction of freely programmable hardware components into modern graphics hardware, graphics processing units (GPUs) have become increasingly popular for general purpose computations. Especially when applied to computer vision algorithms where a Single set of Instructions has to be executed on Multiple Data (SIMD), GPU-based algorithms can provide a major increase in processing speed compared to their CPU counterparts. This paper presents methods that take full advantage of modern graphics card hardware for real-time scale invariant feature detection and matching. The focus lies on the extraction of feature locations and the generation of feature descriptors from natural images. The generation of these featurevectors is based on the Speeded Up Robust Features (SURF) method [1] due to its high stability against rotation, scale and changes in lighting condition of the processed images. With the presented methods feature detection and matching can be performed at framerates exceeding 100 frames per second for 640 × 480 images. The remaining time can then be spent on fast matching against large feature databases on the GPU while the CPU can be used for other tasks. © 2008 IEEE.

This paper proposes a fast motion estimation algorithm for variable block-sizes by utilizing motion vector bottom-up procedure for H.264. The refined motion vectors of adjacent small blocks are merged to predict the motion vectors of larger blocks for reducing the computation. Experimental results show that our proposed method has lower computational complexity than full search, fast full search and fast motion estimation of the H.264 reference software JM93 with slight quality decrease and little bit-rate increase.

This paper presents a parallel motion estimation algorithm on Graphics Processing Units (GPU) with a GPU-based fast Coding Unit (CU) splitting mechanism for speeding up the execution speed of High Efficiency Video Coding (HEVC). Parallel motion estimation algorithms only offer motion vectors to HEVC encoder, but CU splitting decision in HEVC still needs more information to speed up the encoder. Therefore, a mechanism using GPU to signify encoder which CU depth can be split instantly using motion data is designed. With experiments on Kepler GK110 GPU, the proposed parallel algorithm gains over 510 times faster than the full search motion estimation in the HEVC reference software, and the proposed fast CU splitting mechanism can further save 23% total execution time when compared to the general encoder with parallel motion estimation.

The full search block-matching algorithm is the simplest, but computationally very intensive approach. In recent years there was substantial progress in block motion estimation algorithms. Important milestones on this path were such algorithms as two-dimensional logarithmic search, three-step search, four step search, and diamond search. All these methods try to minimize the amount of search points applying sum of absolute differences (SAD) or equivalent metrics for each point. There were some works that studied partial SAD (PSAD) but mostly concentrated on a constant decimation factor. What we tried to study in this work is the performance of one of the best block matching search algorithms in combination with adaptive PSAD as a matching metric. The idea is that we use original motion estimation based on spatio-temporal correlation method, but instead of using SAD as a matching metric we use PSAD with adaptively chosen decimation factor. Our simulation results show that for high motion sequences PSNR degradation between full search and the proposed method was around 0.1-0.7 dB. The computational complexity reduction of 650-1700 times (compared with the full search) and 9 times (compared to the original method) is pretty big and may be well worth this video quality decrease. In the case of more static sequences PSNR degradation between full search and the original motion estimation method was around 0 dB. When we compare the original method with the proposed one the degradation increases to 0.1 dB. The computational complexity reduction was around 1600-1700 times (compared with the full search) and 5-7 times (compared to the original method).

We present a novel hardware architecture supporting diamond search and fast full search for block matching motion estimation. It can handle irregular data flow of these fast algorithms without pipeline bubbles, and reduce computation of duplicated search positions. The proposed architecture needs preprocessing with a small amount of computational power while performing fast full search and is suitable for the platform-based video coding system. While the diamond search mode can be applied for real-time requirements, we can choose the fast full search mode, which adapts the processing cycles to picture contents and preserves the same quality of full search block matching (FSBM), for applications of high picture quality or compression ratio. It needs only 9K gates and one additional memory of search range size and is more cost-effective than the conventional systolic array architecture.

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce the computation by limiting the number of searching locations. This is accomplished at the expense of less accuracy of motion estimation and gives rise to an appreciably higher mean squared error (MSE) for motion compensated images. We present a new fast and efficient search algorithm for block motion estimation that produces better quality performance and less computational time compared with a three-step search (TSS) algorithm. The proposed algorithms are based on the ideas of the dithering pattern for pixel decimation, multiple-candidate for pixel-decimation-based full search, and center-based distribution of the motion vector. From the experimental results, the proposed algorithm is superior to the TSS in both quality performance (about 0.2 dB) and computational complexity (about half).

Motion estimation (ME) is a time-consuming algorithm to find a matching block in the search area for video applications, such as video compression. Motion estimation algorithm includes full search algorithm and fast search algorithm. If the width of the motion estimation algorithm search area is twice the size of the block, half of the search area for adjacent blocks overlaps. In view of this, this paper proposes a search area data reuse method for fast search motion estimation. With the proposed data reuse implementation, only the first block needs to read the data of the entire search area from the off-chip memory. The other blocks only need to read half the original search area. In this way, time of access to external memory is reduced, and running time of the algorithm is also decreased. Experimental results of diamond search show that the search area data reuse methods can reduce the running time by 40% to 60% compared with the algorithm of no use data reuse, and it can also reduce the power consumption by 62% to 73%. Compared with other methods in the literature, the proposed method also performs better on running time and power consumption.

Real-time programmable graphics hardware has resource constraints that prevent complex shaders from rendering in a single pass. One way to virtualize these resources is to partition shading computations into multiple passes, each of which satisfies the given constraints. Many such partitions exist for a shader, but it is important to find one that renders efficiently. We present Recursive Dominator Split (RDS), a polynomial-time algorithm that uses a cost model to find near-optimal partitions of arbitrarily complex shaders. Using a simulator, we analyze partitions for architectures with different resource constraints and show that RDS performs well on different graphics architectures. We also demonstrate that shader partitions computed by RDS can run efficiently on programmable graphics hardware available today.

움직임 추정은 동영상 압축에서 영상 화질과 인코더 속도에 대하여 중요한 역할을 하지만, 많은 수행 시간을 요구한다. H.264/AVC에서 움직임 추정에 소요되는 수행 시간을 줄이면서 화질을 유지하기 위하여 본 논문에서는 정화소와 부화소 움직임 추정 기법을 제안하였다. 본 논문에서는 정화소 움직임 추정을 위하여 계층적인 탐색 기법을 사용하였고, 정확한 움직임 추정을 위하여 블록 크기에 따라 부화소 움직임 추정 패턴을 적응적으로 결정하였다. 제안한 정화소 움직임 추정 탐색 기법은 대칭적 십자가-엑스 탐색 패턴, 다중 사각형 탐색 패턴, 다이아몬드 탐색 패턴들로 이루어져 있다. 이 탐색 패턴들은 블록 움직임이 수직으로 크거나 블록 움직임이 크면서 규칙적인 영상에서 국부적 최소화 문제를 해결하고 움직임 추정에 소요되는 시간을 줄이기 위하여 탐색 영역 내에 탐색 점들을 규칙적, 대칭적으로 배치하였다. 제안한 부화소 움직임 추정 탐색 기법은 부화소 움직임 추정을 위하여 기존의 전역 부화소 탐색 패턴, 중앙 편향적 부화소 탐색 패턴과 제안한 부화소 움직임 탐색패턴들을 사용한다. 그리고 블록의 크기에 따라 3가지의 부화소 탐색 패턴들 중 한 패턴이 부화소 움직임 추정을 위해 적응적으로 결정된다. 블록의 크기에 따라 적응적으로 부화소 탐색 패턴이 결정되므로 보다 정확하게 부화소 움직임 추정을 수행할 수 있다. 제안한 기법을 전역 탐색 기법과 비교하였을 때 약 5.2배의 속도 향상을 가져왔으며, 영상 화질에 있어서 약 0.01 (dB)정도 성능 저하를 보였다. 반면에, 비대칭 다중육각형 탐색 기법과 비교하였을 때 움직임 추정 속도와 화질에 있어서 각각 약 1.2배와 약 0.02 (dB)정도 향상을 보였다. Motion estimation (ME) plays an important role in digital video compression. But it limits the performance of image quality and encoding speed and is computational demanding part of the encoder. To reduce computational time and maintain the image quality, integer pel and fractional pel ME methods are proposed in this paper. The proposed method for integer pel ME uses a hierarchical search strategy. This strategy method consists of symmetrical cross-X pattern, multi square grid pattern, diamond patterns. These search patterns places search points symmetrically and evenly that can cover the overall search area not to fall into the local minimum and to reduce the computational time. The proposed method for fractional pel uses full search pattern, center biased fractional pel search pattern and the proposed search pattern. According to block sizes, the proposed method for fractional pel decides the search pattern adaptively. Experiment results show that the speedup improvement of the proposed method over Unsymmetrical cross Multi Hexagon grid Search (UMHexagonS) and Full Search (FS) can be up to around <TEX>$1.2{\sim}5.2$</TEX> times faster. Compared to image quality of FS, the proposed method shows an average PSNR drop of 0.01 dB while showing an average PSNR gain of 0.02 dB in comparison to that of UMHexagonS.

The recent video compression standard, HEVC (high efficiency video coding), will most likely be used in various applications in the near future. However, the encoding process is far too slow for real-time applications. At the same time, computing capabilities of GPUs (graphics processing units) have become more powerful in these days. In this paper, we have proposed a GPU-based parallel motion estimation (ME) algorithm to enhance the performance of an HEVC encoder. A frame is partitioned into two subframes for pipelined execution to improve GPU utilization. The flow chart is redetermined to solve data hazards in the pipelined execution. Two new methods are introduced in the proposed ME: decision of a representative search center position (RSCP) and warp-based concurrent parallel reduction (WCPR). A RSCP employs motion vectors of a co-located CTU in a previously encoded frame to solve a dependency problem in parallel computation with negligible coding loss. WCPR concurrently executes several parallel reduction operations, which increases the thread utilization from 20 to 89 % without any thread synchronization. The proposed encoder can make the portion of ME in the encoder negligible with 2.2 % bitrate increase against the HEVC test model (HM) encoder. In terms of ME, the proposed ME is 130.7 times faster than that of the HM encoder.

A new hardware-oriented algorithm called the one-dimensional full search (1DFS) is presented for block-matching motion estimation in video compression. The simulation for this algorithm follows H.261 and MPEG international standards. In MPEG simulation, structures with 1-, 2- and 3-frame interpolation are compared. The performance of 1DFS is superior to that of other fast search algorithms. And it has more regular data flow, data reuse and less control overhead. It is an alternative for 2D full search block matching and achieves a good compromise between computational complexity and performance. With competent performance and reasonable computation complexity, the proposed method is more suitable for real-time hardware realization of a VLSI motion estimator for video applications.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Block-matching algorithm is the most common technique applied in block-based motion estimation technique. There are several block-matching algorithm based on block-based motion estimation techniques have been developed. Full search (FS), three step search (TSS), new three step search (NTSS), diamond search (DS) and hexagon based search (HS) are the most well known block-matching algorithm. These techniques are applied to video sequences to remove the temporal redundancy for compression purposes and to gauge the motion vector estimation. In addition, the mentioned block-matching algorithms are the baseline techniques that have been used to further develop all the enhanced or improved algorithms. In order to develop the proposed methods, the baseline techniques are studied to develop the proposed algorithms. This chapter proposes modelling of fast unrestricted hexagon search (FUHS16) and unrestricted hexagon-diamond search (UHDS16) algorithms for motion vector estimation, which is based on the theory and application of block-based motion estimation. Both of these algorithms are designed using 16 × 16 block size. In particular, the motion vector estimation, quality performance, computational complexity, and elapsed processing time are emphasised. These parameters have been used to measure the experimental results. It is the aim of this study that this work provides a common framework with which to evaluate and understand block-based matching motion estimation performance. On the theoretical side, four fundamental issues are explored: (1) division of frame, (2) basic block-based matching, (3) motion vector estimation, and (4) block-matching algorithm development. Various existing block-matching motion estimation algorithms have been analysed to develop the fundamental research. Based on the theoretical and fundamental research analysis the FUHS16 and UHDS16 algorithms using 16 × 16 block-based motion estimation formulations were developed. To improve the UHDS16 algorithm, 8 × 8 block-matching technique has been tested. The 8 × 8 block-matching technique is known as UHDS8. The results show positive improvements. From an application perspective, the UHDS8 algorithm efficiently captured the motion vectors in many video sequences. For example, in video compression, the use of motion vectors on individual macro-blocks optimized the motion vector information. The UHDS8 algorithm also offers improvement in terms of image quality performance, computational complexity and elapsed processing time. Thus, this chapter offers contributions in certain areas such as reducing the mechanism of computational complexity in estimating the motion from the video sequences. In particular, the FUHS16, UHDS16 and UHDS8 algorithms were developed to estimate the motion vectors field in the video sequences. Theoretical analysis block-based matching criteria are adapted to FUHS16, UHDS16 and UHDS8 algorithms, which are based on search points technique. Basically, the proposed of FUHS16, UHDS16 and UHDS8 algorithm produces the best motion vector estimation finding based on the block-based matching criteria. Besides that, the UHDS8 algorithm also improves the image quality performances and the search points in terms of the computational complexity. Overall, the study shows that the UHDS8 algorithm produces better results compared to the FUHS16 and UHDS16 algorithm.

In Motion estimation (ME), the block matching algorithms have a great potential of parallelism. This process of the best match is performed by computing the similarity for each block position inside the search area, using a similarity metric, such as Sum of Absolute Differences (SAD). It is used in the various steps of motion estimation algorithms. Moreover, it can be parallelized using Graphics Processing Unit (GPU) since the computation algorithm of each block pixels is similar, thus offering better results. In this work a fixed OpenCL code was performed firstly on several architectures as CPU and GPU, secondly a parallel GPU-implementation was proposed with CUDA and OpenCL for the SAD process using block of sizes from 4x4 to 64x64. A comparative study established between execution time on GPU on the same video sequence. The experimental results indicated that GPU OpenCL execution time was better than that of CUDA times with performance ratio that reached the double.

The effciency of video compression methods mainly depends on the motion compensation stage, and the design of effcient motion estimation techniques is still an important issue. An highly accurate motion estimation can significantly reduce the bit-rate, but involves a high computational complexity. This is particularly true for new generations of video compression standards, MPEG AVC and HEVC, which involves techniques such as different reference frames, sub-pixel estimation, variable block sizes. In this context, the design of fast motion estimation solutions is necessary, and can concerned two linked aspects: a high quality algorithm and its effcient implementation. This paper summarizes our main contributions in this domain. In particular, we first present the HME (Hierarchical Motion Estimation) technique. It is based on a multi-level refinement process where the motion estimation vectors are first estimated on a sub-sampled image. The multi-levels decomposition provides robust predictions and is particularly suited for variable block sizes motion estimations. The HME method has been integrated in a AVC encoder, and we propose a parallel implementation of this technique, with the motion estimation at pixel level performed by a DSP processor, and the sub-pixel refinement realized in an FPGA. The second technique that we present is called HDS for Hierarchical Diamond Search. It combines the multi-level refinement of HME, with a fast search at pixel-accuracy inspired by the EPZS method. This paper also presents its parallel implementation onto a multi-DSP platform and the its use in the HEVC context.

Hardware-accelerated direct volume rendering of unstructured volumetric meshes is often based on tetrahedral cell projection, in particular, the projected tetrahedra (PT) algorithm and its variants. Unfortunately, even implementations of the most advanced variants of the PT algorithm are very prone to rendering artifacts. In this work, we identify linear interpolation in screen coordinates as a cause for significant rendering artifacts and implement the correct perspective interpolation for the PT algorithm with programmable graphics hardware. We also demonstrate how to use features of modern graphics hardware to improve the accuracy of the coloring of individual tetrahedra and the compositing of the resulting colors, in particular, by employing a logarithmic scale for the preintegrated color lookup table, using textures with high color resolution, rendering to floating-point color buffers, and alpha dithering. Combined with a correct visibility ordering, these techniques result in the first implementation of the PT algorithm without objectionable rendering artifacts. Apart from the important improvement in rendering quality, our approach also provides a test bed for different implementations of the PT algorithm that allows us to study the particular rendering artifacts introduced by these variants.