Adaptive Search Range Algorithms for Variable Block Size Motion Estimation in H.264/AVC

본 논문은 High Efficiency Video Coding (HEVC) GPU 기반 정수화소(integer-pel) 움직임 추정(Motion Estimation)을 고속화하기 위한 적응적인 탐색영역 결정 방법을 제안한다. 적응적인 탐색영역은 Motion Vector Difference (MVD)를 이용하여 결정한다. 먼저, 입력 영상의 MVD를 분석하여 입력 영상을 두 모델로 분류한다. 이후 분류된 각 모델의 MVD 특성에 따라 적응적인 탐색영역을 결정한다. 제안하는 알고리즘을 GPU 기반 정수화소 움직임 추정에 적용하기 위해 움직임 추정의 시작점은 이전 프레임의 Motion Vector (MV)로 결정한다. 위 과정은 CPU에서 이뤄지며, CPU는 움직임 추정의 시작점과 적응적인 탐색영역을 GPU에 전송한다. 이후 GPU는 정수화소 움직임 추정을 병렬로 수행한다. 제안하는 알고리즘은 참조 모델 대비 1.1%의 BD-rate 상승과 전체 부호화 시간의 37.9% 감소 및 951.2배 빠른 정수화소 움직임 추정 수행 시간을 얻는다. 또한, 적응적인 탐색영역이 적용되지 않은 단순 병렬화 알고리즘 대비 57.5%의 정수화소 움직임 추정 시간 감소와 0.6% BD-rate 상승을 얻는다. In this paper, we propose a new Adaptive Search Range (ASR) decision algorithm for accelerating GPU-based Integer-pel Motion Estimation (IME) of High Efficiency Video Coding (HEVC). For deciding the ASR, we classify a frame into two models using Motion Vector Differences (MVDs) then adaptively decide the search ranges of each model. In order to apply the proposed algorithm to the GPU-based ME process, starting points of the ME are decided using only temporal Motion Vectors (MVs). The CPU decides the ASR as well as the starting points and transfers them to the GPU. Then, the GPU performs the integer-pel ME. The proposed algorithm reduces the total encoding time by 37.9% with BD-rate increase of 1.1% and yields 951.2 times faster ME against the CPU-based anchor. In addition, the proposed algorithm achieves the time reduction of 57.5% in the ME running time with the negligible coding loss of 0.6%, compared with the simple GPU-based ME without ASR decision.

본 논문은 움직임벡터차의 확률을 이용하여 움직임 탐색 범위를 결정하는 새로운 고속 움직임 추정 방법을 제안한다. 제안된 방법은 일종의 적응적 탐색영역 방법이다. 움직임벡터차의 분포에 대한 조사하고 그 분포의 파라미터를 최대우도추정방법으로 추정한다. 이 추정된 분포를 이용하여 탐색 범위는 움직임벡터차의 지정된 확률에 의해 효과적으로 제한될 수 있음을 보인다. 실험적으로 제안된 방법의 성능이 화질면에서 전역탐색법과 유사하며 복잡도 면에서는 중대한 감소를 입증하였다. 뿐만 아니라, 제안된 방법이 기존의 적응적 탐색영역 방법에 비해 훨씬 더 효과적으로 탐색영역을 결정함을 보여주었다. This paper presents a new fast motion estimation method where search ranges are determined by the probabilities of motion vector differences (MVDs), which is an adaptive/dynamic search range (ASR) method. The MVDs' distribution is investigated and its parameter is estimated by the maximum likelihood estimator. With the estimated distribution, we show that the search ranges can be efficiently restricted by a prefixed probability for MVDs. Experimental results showed that the performance of the proposed method is very similar to that of the full search algorithm in PSNR but it enables significant reduction in the computational complexity. In addition, they revealed that the proposed method determine the search ranges much more efficiently than the conventional ASR methods.

In recent years, the variable block size (VBS) motion estimation technique has been widely employed to improve the performance of the block matching algorithm (BMA). In VBS, the block size is varied according to the type of motion. The VBS is known to be very efficient for areas containing complex motions. But, it requires a large amount of computation. The computation required is 2 or 5 times compared with the full search BMA. In this paper, a new VBS motion estimation algorithm, called the classified variable block size (CVBS), is proposed to overcome this problem. The algorithm classifies the input blocks into 3 categories: the background, the shade-motion, and the edge-motion. According the characteristics of the classified blocks, various motion estimation techniques are then used to improve the coding performance. The performance of VBS and CVBS under the coding efficiency is investigated. It is shown that the CVBS algorithm requires about 1/6 to 1/10 computations needed by the conventional VBS algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Although variable block-size motion estimation provides significant video quality and coding efficiency improvement, it requires much higher computational complexity compared with fixed block size motion estimation. The reason is that the current motion estimation algorithms are mainly designed for fixed block size. Current variable block-size motion estimation implementation simply applies these existing motion estimation algorithms independently for different block sizes to find the best block size and the corresponding motion vector. Substantial computation is wasted because distortion data reuse among motion searches of different block sizes is not considered. In this paper, a motion estimation algorithm intrinsically designed for variable block-size video coding is presented. The proposed multiple block-size search (MBSS) algorithm unifies the motion searches for different block sizes into a single searching process instead of independently performing the search for each block size. In this unified search, the suboptimal motion vectors for different block sizes are used to determine the next search steps. Its prediction quality is comparable with that obtained by performing motion search for different block sizes independently while the computational load is substantially reduced. Experimental results show that the prediction quality of MBSS is similar to full search.

In this paper, we propose a variable step search fast motion estimation algorithm using local statistics of neighboring motion vectors. Using the degree of correlation between neighboring motion vectors, motion search range of each macroblock is adaptively adjusted to reduce unnecessary search points. Based on the determined search range, motion vector is obtained by variable step search step. Experimental results show that comparing to fast full spiral search, fixed step, and fast variable block size motion estimation algorithms, the motion search points and computing time of proposed algorithm are dramatically reduced with acceptable loss in coding performance.

High external memory bandwidth requirement is one major challenge for efficient hardware motion estimation (ME) implementation. Large double-buffered on-chip search window (SW) buffer is usually employed to increase the throughput. In this paper, we focus on SW buffer structure optimization in a systematic viewpoint. An efficient buffer share mechanism is proposed to minimize the memory consumption, simultaneously alleviate the external memory access bandwidth. Moreover, variable block size ME (VBSME) and large SW in high definition (HD) video encoder are both supported with good trade off among throughput, data regularity, and rate distortion performance. The simplified algorithm can achieve nearly 50% SW buffer saving with less than 0.15dB PSNR degradation at the worst case compared with full search VBSME.

High Efficiency Video Coding achieves twofold coding efficiency improvement compared with its predecessor H.264/MPEG-4 Advanced Video Coding. However, it suffers from high computational complexity due to its quad-tree structure in motion estimation (ME). This paper exposes the use of depth maps in the multiview video plus depth format for relieving the computational burden. The depth map provides an intimation of the objects' distance from the projected screen in a 3D scene, which is explored in adaptive search range determination in this paper. The proposed algorithm exploits the high temporal correlation between the depth map and the motion in texture. By utilizing this correlation, a depth/motion relationship map is built for a mapping process. For each block, this forms a tailor-made search range with a motion-aware asymmetric shape to skip unnecessary search points in ME. The obtained search range can be further adjusted by taking the influence of 3D-to-2D projection into consideration. Simulation results reveal that, compared to the full search approach, the proposed algorithm can reduce the complexity by 93% on average, whereas the coding efficiency can be maintained. Besides, the proposed search range determination can work well with other fast search ME algorithms in the literature.

Fast motion estimation and Inter-mode decision for H.264/MPEG-4 AVC encoding

The variable block size motion estimation (VBSME) presented in the video coding standard H.264/AVC significantly improves coding efficiency, but it requires much more considerable computational complexity than motion estimation using fixed macroblocks. To solve this problem, this paper proposes a pipelined hardware architecture for full-search VBSME aiming for high performance, simple structure, and small controls. Our architecture consists of 1-D arrays with 64 processing elements, an adder tree to produce motion vectors (MVs) for variable block sizes, and comparators to determine the minimum of MVs. This can produce all 41 MVs for variable blocks of one macroblock in the same clock cycles to other conventional 1-D arrays of 64 PEs. In addition, this can be easily controlled by a 2-bit counter. Implementation results show that our architecture can estimate MVs in CIF video sequence at a rate of 106 frames/s for the 32×32 search range.KeywordsMotion estimationvariable block sizefull searcharray architectureH.264/AVCvideo coding

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

本文提出了一种HEVC视频编码的运动估计快速算法，该算法可以显著加速搜索过程，编码效率与HEVC参考软件(Test Model)中的Test Zone Search (TZSearch)平分秋色。较之H.264/AVC视频标准，HEVC引入了更为复杂的混合编码框架和更大尺寸的搜索窗口，从而带来了更高的计算复杂度。为了降低复杂度，我们提出了一种新的运动估计算法，该算法仅利用搜索窗中所选取的少量几个点建立二次曲面模型，通过对像素的绝对差值和(SAD)分布的迭代分析，逐步缩小搜索范围，直到得出最优的运动矢量。该算法可被用于不同的编码环境中，实验结果表明，与HM相比，该算法可以节约大约40%的运动估计算法时间，仅引起轻微的性能损失。 This paper presents a motion estimation (ME) algorithm for High Efficiency Video Coding (HEVC), which provides a strategy to speed up the search process significantly, while yielding the same quality performance as the Test Zone Search (TZSearch) scheme in HEVC Test Model (HM). Compared with the H.264/AVC, HEVC employs a more complex hybrid coding architecture and a larger size search window, leading to great computational complexity. In order to reduce the complexity, a novel motion estimation algorithm is proposed, in which some limited pixels of certain position in the current search window are utilized to build a quadratic model, and then shrink the search range repeatedly by analyzing the sum of absolute difference (SAD) distribution until the best motion vector (MV) is obtained. The proposed algorithm can be applied to various encoding conditions. Experimental results show that our method can save 40% of computations compared with the HM, with negligible decrease of coding quality.

H.264/AVC, the latest video coding standard, adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bitrate. However, the full RD cost calculation for all intra-prediction modes, the exhaustive searches for finding optimal motion vectors for all block sizes, and the multiple references frame procedure considerably increase its computational complexity with the allowed number of prediction modes. The authors have previously proposed an approach for both inter- and intra-mode decisions that takes into account the several effective parameters, image content type, quantization parameter, correlation of motion vectors and motion details of video objects. In order to reduce the complexity, here we propose a new two-step adaptive search range decision by developing a combination of previous algorithm. The first step of our adaptive search range decision determines the search range for current MB according to the motion vectors of the previous MBs. In the second step, the motion vector obtained in the current frame is used to adaptively searching the window size of the previous reference frame for multi-frame motion estimation. As such, only a subset of inter- and intra-modes is chosen for RDO calculation. Experimental results show that the proposed algorithm reduces the total encoding cost with negligible loss in PSNR and a slightly increase in the required bitrate, when compared to our previous algorithm and other algorithms reported in the literature.

Multiview video coding (MVC) is an ongoing standard. In the working draft, motion estimation and disparity estimation are both employed in the encoding procedure. It achieves the highest possible coding efficiency, but results in extremely large encoding time, which obstructs it from practical applications. We propose a macroblock (MB) level adaptive search range algorithm utilizing inter-view correlation for motion estimation in MVC to reduce the complexity of the coder. For multi-view sequences, the motion vectors of the corresponding MBs in previously coded view are first extracted to analyze motion homogeneity. On the basis of motion homogeneity, MBs are classified into three types (MB in the region with homogeneous motion, with medium homogeneous motion, or with complex motion), and search range is adaptively determined for each type MB. Experimental results show that our algorithm can save 75% average computational complexity of motion estimation, with negligible loss of coding efficiency.

This paper presents a new fast motion estimation method where search ranges are determined by the probabilities of motion vector differences (MVDs). The MVDs' distribution is investigated and its parameter is estimated by the maximum likelihood estimator. With the estimated distribution, the search ranges is efficiently restricted by a prefixed probability for MVDs. Experimental results show that the performance of the proposed method is very similar to that of the full search algorithm in PSNR but it enables significant reduction in the computational complexity.