깊이 카메라를 이용한 움직임 추정 방법

Motion estimation (ME) is one of the most critical tools in video coding and consumes the majority of the encoding complexity. Three types of ME are utilized in the latest video coding standards, namely integer, fractional, and affine MEs. They are implemented as three searches for the integer motion vector (IMV), fractional motion vector (FMV), and control point motion vectors (CPMVs). Many algorithms were proposed to reduce the complexity for them individually, but the overall overhead of three searches is still challenging for hardware implementations. Therefore, we propose a hardware-efficient Unified Motion Estimation (UME) to derive three types of MVs with only one search. An IME with sub-block refinement is performed to collect extra motion information while searching for the IMV. The FMV and CPMVs are then derived from the collected information using a mixed error surface and an overdetermined system. Compared to the default ME algorithms in VVC, the time cost for ME is reduced by 41.63% with a coding loss of only 0.87% under LDB configuration. For hardware implementations, the minimum required resources and corresponding latency are significantly reduced by 75.35% and 69.17%, respectively.

Block sizes larger than the traditional 16×16 macroblock (MB) structure and extended macroblock (EMB) size motion estimation (ME) are introduced in joint model key technology areas software to improve the inter block coding efficiency. Similar to H.264, variable size motion estimation (VS-ME) is still adopted for each 16×16 MB including 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4. EMB size ME and VS-ME achieve the highest possible coding efficiency, but result in extremely large coding time which obstructs it from practical use. A fast mode decision algorithm for EMBs based on motion homogeneity is proposed to reduce the computational complexity of ME. The basic idea of the proposed method is to utilize the spatial property of motion field in prediction where EMB size ME and small size ME (including 8×8, 8×4, 4×8 and 4×4) are needed, and only in these regions EMB size ME and small size ME are enabled. The motion field is generated by the corresponding motion vectors in spatial window. Simulation results show that the proposed algorithm can save 55% average computational complexity, with negligible loss of coding efficiency.

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

This thesis presents the rate distortion analysis of conditional motion estimation, a process in which motion computation is restricted to only active pixels in the video. We model active pixels as independent and identically distributed Gaussian process and inactive pixels as Gaussian-Markov process and derive the rate distortion function based on conditional motion estimation. Rate-Distortion curves for the conditional motion estimation scheme are also presented. In addition this thesis also presents the hardware implementation of a block based motion estimation algorithm. Block matching algorithms are difficult to implement on FPGA chip due to its complexity. We implement 2D-Logarithmic search algorithm to estimate the motion vectors for the image. The matching criterion used in the algorithm is Sum of Absolute Differences (SAD). VHDL code for the motion estimation algorithm is verified using ISim and is implemented using Xilinx ISE Design tool. Synthesis results for the algorithm are also presented.

The H.264 provides various and useful features for improved coding efficiency and error robustness, but its encoder complexity is extremely high due to the computational time of ME (motion estimation) and mode decision of inter coding. In this paper, we propose a fast and simple class-based scheme to reduce the computational complexity of ME. The image in block base is classified to k classifications with SQCM (spatial-domain quickly classified method). Referring to the classified indices, we discriminate two methods of ME: one is pattern search for stationary macroblock, synchronous motion and unchanged object; other is fast inter prediction for instable or variant content of object. We can save the dispensable motion detection for different contents of video, named CQME (class-based quickly ME). Experimental results shows that the proposed method can achieve a reduction of 67% computational time on average, with an average PSNR loss of only 0.30 dB compared with the original H.264 reference software jm12.1.

The complexity of motion estimation in the High Efficiency Video Coding (HEVC) standard is very high as it uses a number of prediction block sizes. The test zonal search (TZS) mechanism is used as motion estimation algorithm in the fast search mode of HM encoder, the reference software for HEVC. In this paper, we present schemes for reducing the complexity of the TZS algorithm for uni-prediction in HEVC. The proposed mechanisms help in reducing complexity of grid, raster and refinement search stages of TZS. The performance of the proposed mechanisms is tested independently and also combined in the fast search mode of HM-16.18. The motion estimation time and the number of search points in the combined algorithm are reduced by 68.14% and 77.10% with a BD-rate of 0.30% and a BD-PSNR of $$-0.007$$ dB in comparison with the original fast search mode in the low-delay P main profile. The proposed complexity reduction schemes are compared with those adopted in other recently proposed fast motion estimation algorithms and the performance of the proposed scheme is found to be superior in terms of the reduction in motion estimation time.

In video communication systems motion compensation techniques allow one to remove temporal redundancy among frames in order to achieve reductions in the bit rate. Block based approaches are widely used for such a task, since they offer a good balance between performance and complexity; however, the resulting motion fields still present spatial and temporal redundancies, the exploitation of which would allow a greater bit rate reduction. The motion estimation method proposed attains such a goal, maintaining the block based partitioning of the resulting field. It uses a spatio-temporal prediction followed by a lossy coding of the residual field by vector quantization. The prediction is based on autocompensation, enhanced with the aid of some additional parameters so that it provides a stationary error field. The statistical distribution of such a residual field allows vector quantization to perform efficiently, even when the method is applied to video sequences with extremely varying motion activities. Both the prediction and the vector quantization simply use an index for each block to code its motion vector. Finally, the indexes are spatially organized in two single images that are coded with a region based technique.

In video coding system, block matching algorithms are used to identify the matching blocks in digital video sequence for motion estimation. Motion estimation is used to discover temporal redundancy in the digital video sequence. This paper shows the analysis of popular existing algorithms for motion estimation. The analysis is based on the number of search points/macroblocks and speed-up ratio. The endeavor of this revision is to present the successive researchers with some productive information in design of block matching algorithms.

With reference to H.264/AVC standard the paper proposes a new control algorithm, which dynamically configures the search parameters of the motion estimation (ME) task: used reference frames, valid block modes and search size. The aim is remarkably reducing the ME complexity with unaltered coding efficiency for a wide range of bit-rates from Kbps to tens of Mbps and video formats (QCIF to CCIR). The technique has been successfully applied to full search (FS) and fast ME engines.

The limited energy resources in portable multimedia devices require the reduction of encoding complexity. The complex Motion Estimation (ME) scheme of H.264/MPEG-4 AVC accounts for a major part of the encoder energy. In this paper we present a Run-Time Adaptive Predictive Energy Budgeting (enBudget) scheme for energy-aware ME that predicts the energy budget for different video frames and different Macroblocks (MBs) in an adaptive manner considering the run-time changing scenarios of available energy, video frame characteristics, and user-defined coding constraints while keeping a good video quality. It assigns different Energy-Quality Classes to different video frames and fine-tunes at MB level depending upon the predictive energy quota in order to cope with above-mentioned run-time unpredictable scenarios. Compared to UMHexagonS, EPZS, and FastME, our enBudget scheme for energy-aware ME achieves an energy saving of up to 93%, 90%, 88% (average 88%, 77%, 66%), respectively. It suffers from an average Peak Signal to Noise Ratio (PSNR) loss of 0.29 dB compared to Full Search. We also demonstrate that enBudget is equally beneficial to various other state-of-the-art fast adaptive MEs (e.g.). We have evaluated our scheme for ASIC and various FPGAs.

This paper presents a new algorithm which outperforms full search block matching algorithm (FSBMA) and other fast search block-matching motion estimation (FBMME) algorithms in terms of improved subjective and objective quality, reduction in computational complexity, increase in speed up factor and low bit rate. Here, unlike FSBMA, a coarse-to-fine search approach is chosen to find the motion vector whereby complexity of motion estimation is greatly reduced. The coarse search sees to that the search will never be caught at a local minimum, as that was the case in many FBMME. The fine search assures the exact location of the global minimum. Pixel sub-sampling is performed in the selected candidate blocks which significantly reduces computational cost with low quality degradation. A new evaluation function is introduced which further reduces the computational complexity. Simulation results have proved the effectiveness of the proposed algorithm for low, medium and high motion sequences.

Several applications in robotics require complex sensor arrangements that must be carefully calibrated, both intrinsically and extrinsically, to allow information fusion and enable the system to function as a whole. These arrangements can combine different sensing modalities--such as color cameras, laser-rangefinders, and depth cameras--in an attempt to obtain richer descriptions of the environment. Finding the location of multi-modal sensors in a common world reference frame is a difficult problem that is largely unsolved whenever sensors observe distinct, disjoint parts of the scene. This article builds on recent results in object pose estimation using mirror reflections to provide an accurate and practical solution for the extrinsic calibration of mixtures of color cameras, LRFs, and depth cameras with non-overlapping field-of-view. The method is able to calibrate any possible sensor combination as far as the setup includes at least one color camera. The technique is tested in challenging situations not covered by the current state-of-the-art, proving to be practical and effective. The calibration software is made available to be freely used by the research community.

Motion estimation typically consumes 50% to 70% of total power in video encode application. Optimizing the power consumption of motion estimation process is of great importance to low power video applications. Power dissipation increases with computational complexity. Reduction in motion estimation complexity is usually associated with increase in bit rate and a loss of quality. We explore a set of algorithms that reduce the complexity of motion estimation by adaptively changing the matching complexity based on macro-block features yet have only a modest cost in terms of bit rate increase and quality loss. The adaptive techniques are applied to the global elimination algorithm, which is a well known motion estimation algorithm. The global elimination algorithm uses fixed partition sizes and shapes irrespective of the nature of the macro-block. We show that by adapting the partition sizes and shapes according to the macro-block features such as variance and Hadamard coefficients, the computational complexity of global elimination algorithm can be significantly reduced with only a small increase in bit rate. We also propose a novel center-biased search order that uses early termination method designed to work with the global elimination algorithm. The adaptive match and center-biased search together result in around 57% reduction in computational complexity and 50% reduction in power dissipation compared to the original global elimination algorithm.

Motion estimation using multiple reference frames is widely used as the basis for recent video coding standards (eg. H.264/AVC) to achieve increased coding efficiency. However, this increases the complexity of the encoding process. In this paper, a new technique for efficient motion estimation is proposed. A combination of multiple reference frame selection and image residue-based mode selection is used to improve motion estimation time. By dynamic selection of an initial reference frame in advance, the number of reference frames to be considered is reduced. In addition, from examination of the residue between the current block and reconstructed blocks in preceding frames, variable block size mode decisions are made. Modified initial motion vector estimation and early stop condition detection are also adopted to speed up the motion estimation procedure. Experimental results compare the performance of the proposed algorithm with a state of the art motion estimation algorithm and demonstrate significantly reduced motion estimation time while maintaining PSNR performance.

Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies, a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame difference was proposed. Differential motion detection was employed to image sequences and proper threshold was adopted to identify the connected region. Then the motion region was extracted to carry out motion estimation and motion compensation on it. The experiment results show that the encoding efficiency of motion vector is promoted, the complexity of motion estimation is reduced and the quality of the reconstruction image at the same bit-rate as Multi-Resolution Motion Estimation (MRME) is improved.