Scalable FPGA Hardware Acceleration for H.264 Motion Estimation

High throughput, heavy bandwidth requirement, huge on-chip memory consumption, and complex data flow control are major challenges in high definition integer motion estimation hardware implementation. This paper proposes an efficient very large scale integration architecture for integer multi-resolution motion estimation based on optimized algorithm. There are three major contributions in this paper. First, this paper proposes a hardware friendly multi-resolution motion estimation algorithm well-suited for high definition video encoder. Second, parallel processing element (PE) array structure is proposed to implement three-level hierarchical motion estimation, only 256PEs are enough for one reference frame real-time high definition motion estimation by efficient PE reuse. Third, efficient on-chip reference pixel buffer sharing mechanism between integer and fractional motion estimation is proposed with almost 50% SRAM saving and memory bandwidth reduction. The proposed multi-resolution motion estimation algorithm reached a good balance between complexity and performance with rate distortion optimized variable block size motion estimation support. Also, we have achieved moderate logic circuit and on-chip SRAM consumption. The proposed architecture is well-suited for all MPEG-like video coding standards such as H.264, audio video coding standard, and VC-1.

In recent years, the variable block size (VBS) motion estimation technique has been widely employed to improve the performance of the block matching algorithm (BMA). In VBS, the block size is varied according to the type of motion. The VBS is known to be very efficient for areas containing complex motions. But, it requires a large amount of computation. The computation required is 2 or 5 times compared with the full search BMA. In this paper, a new VBS motion estimation algorithm, called the classified variable block size (CVBS), is proposed to overcome this problem. The algorithm classifies the input blocks into 3 categories: the background, the shade-motion, and the edge-motion. According the characteristics of the classified blocks, various motion estimation techniques are then used to improve the coding performance. The performance of VBS and CVBS under the coding efficiency is investigated. It is shown that the CVBS algorithm requires about 1/6 to 1/10 computations needed by the conventional VBS algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Although variable block-size motion estimation provides significant video quality and coding efficiency improvement, it requires much higher computational complexity compared with fixed block size motion estimation. The reason is that the current motion estimation algorithms are mainly designed for fixed block size. Current variable block-size motion estimation implementation simply applies these existing motion estimation algorithms independently for different block sizes to find the best block size and the corresponding motion vector. Substantial computation is wasted because distortion data reuse among motion searches of different block sizes is not considered. In this paper, a motion estimation algorithm intrinsically designed for variable block-size video coding is presented. The proposed multiple block-size search (MBSS) algorithm unifies the motion searches for different block sizes into a single searching process instead of independently performing the search for each block size. In this unified search, the suboptimal motion vectors for different block sizes are used to determine the next search steps. Its prediction quality is comparable with that obtained by performing motion search for different block sizes independently while the computational load is substantially reduced. Experimental results show that the prediction quality of MBSS is similar to full search.

High external memory bandwidth requirement is one major challenge for efficient hardware motion estimation (ME) implementation. Large double-buffered on-chip search window (SW) buffer is usually employed to increase the throughput. In this paper, we focus on SW buffer structure optimization in a systematic viewpoint. An efficient buffer share mechanism is proposed to minimize the memory consumption, simultaneously alleviate the external memory access bandwidth. Moreover, variable block size ME (VBSME) and large SW in high definition (HD) video encoder are both supported with good trade off among throughput, data regularity, and rate distortion performance. The simplified algorithm can achieve nearly 50% SW buffer saving with less than 0.15dB PSNR degradation at the worst case compared with full search VBSME.

With the development of fast motion estimation (ME) algorithms towards video compression standard H.264, burden on integer-pel ME is slightly reduced. However the integer-pel ME is becoming computationally complex due to demand of higher and higher resolution videos. On the other hand, there is always trade-off between the speed and performance of these search algorithms. This work improvises the performance of content awareness enabled integer-pel ME algorithm for encoding the fast and slow motion video sequences of high definition (HD) resolution category viz. 1280×720 and 1920×1080. The algorithm proposes a novel notion of 'integrity index' with focus on increasing the PSNR and reducing the bit rate. It is implemented in the frame work of x265 version 1.7 video encoder. Motion independent ME algorithm is analysed quantitatively in terms of parameters based on rate distortion and ME time. The proposed algorithm is found to be performing better on BD-Rate and BD-PSNR parameters for the videos of both resolutions under consideration. Observed increase in motion estimation time is less than four seconds compared to hexagonal search algorithm of ME which is the benchmark in fast ME algorithm.

The variable block size motion estimation (ME) and compensation (MC) using multiple reference frames is adopted in H.264/AVC to improve coding efficiency. However, the computational complexity for ME/MC increases proportional to the number of reference frames. In this paper, we propose a new efficient reference frame selection algorithm to reduce the complexity. The proposed algorithm selects suitable reference frames by employing the spatial and temporal correlation of video sequence. The experimental results show that the proposed algorithm decreases video encoding time while keeping the similar visual quality and bit rates.

The H.264/AVC inter-prediction is performed for variable block-size motion estimation (VBSME) such as 16times16, 16times8, 8times16, 8times8, 8times4, 4times8 and 4times4, it cause the high complexity for H.264 motion estimation (ME). This investigation develops an architecture for a combined fast motion estimation algorithm with edge information mode decision (EIMD) and predict hexagon search (PHS). Compared with other popular ME architecture, the proposed architecture has a large search range and low processing frequency. For the general specification of SDTV (720times480) with 4 reference frames, search range 256times256, the proposed architecture needs only 18.66 MHz. For the very high specification of QFHD (3840times2160) with 1 reference frame, search range 256times256, the proposed architecture only requires 112 MHz. The gate count of the proposed architecture is 300 K, and the memory usage is 12.6 KB.

In the new video coding standard H.264/AVC, motion estimation (ME) is allowed to search multiple reference frames. Therefore, the required computation is highly increased, and it is in proportion to the number of searched reference frames. However, the reduction in prediction residues is mostly dependent on the nature of sequences, not on the number of searched frames. Sometimes the prediction residues can be greatly reduced, but frequently a lot of computation is wasted without achieving any better coding performance. In this paper, we propose a context-based adaptive method to speed up the multiple reference frames ME. Statistical analysis is first applied to the available information for each macroblock (MB) after intra-prediction and inter-prediction from the previous frame. Context-based adaptive criteria are then derived to determine whether it is necessary to search more reference frames. The reference frame selection criteria are related to selected MB modes, inter-prediction residues, intra-prediction residues, motion vectors of subpartitioned blocks, and quantization parameters. Many available standard video sequences are tested as examples. The simulation results show that the proposed algorithm can maintain competitively the same video quality as exhaustive search of multiple reference frames. Meanwhile, 76 %-96 % of computation for searching unnecessary reference frames can be avoided. Moreover, our fast reference frame selection is orthogonal to conventional fast block matching algorithms, and they can be easily combined to achieve further efficient implementations.

This paper presents a flexible and scalable motion estimation processor capable of supporting the processing requirements for high-definition (HD) video using the H.264 Advanced Video Codec, which is suited for FPGA implementation. Unlike most previous work, our core is optimized to execute all existing fast block matching algorithms, which we show to match or exceed the inter-frame prediction performance of traditional full-search approaches at the HD resolutions commonly in use today. Using our development tools, such algorithms can be described using a C-style syntax which is compiled into our custom instruction set. We show that different HD sequences exhibit different characteristics which necessitate a flexible and configurable solution when targeting embedded applications. This is supported in our core and toolset by allowing designers to modify the number of functional units to be instantiated. All processor instances remain binary compatible so recompilation of the motion estimation algorithm is not required. Due to this optimization process, it is possible to match the processing requirements of the selected motion estimation algorithm to the hardware microarchitecture leading to a very efficient implementation.

In this paper, energy-efficient architecture for Variable Block Size Motion Estimation (VBSME) is proposed to fully utilize dynamic partial reconfiguration capability of programmable hardware fabric in the heterogeneous computing environment. Dynamic Partial Reconfiguration is a unique feature of FPGAs that makes best use of hardware resources and power by allowing adaptive algorithm to be implemented on the reconfigurable hardware fabric during runtime. We exploit this feature of FPGA to support run-time reconfiguration of the proposed modular hardware architecture for motion estimation (ME). The implemented scalable SAD array can provide different resolutions and frame rates for real time applications with multiple reconfigurable regions at the operating frequency of 90MHz.

Motion Estimation (ME) is the most computationally intensive part of video compression systems. Multiple reference frame (MRF) ME used in H.264 standard increases the video coding efficiency at the expense of increased computational complexity and power consumption. Therefore, in this paper, we present a reconfigurable baseline H.264 video encoder hardware in which the number of reference frames used for MRF ME can be configured based on the application requirements in order to trade-off video coding efficiency and power consumption. The proposed H.264 video encoder hardware is based on an existing low cost H.264 intra frame coder hardware and it includes new reconfigurable MRF ME, mode decision and motion compensation hardware. The proposed H.264 video encoder hardware is capable of processing 55 CIF (352×288) frames per second and its power consumption ranges between 115mW and 235mW depending on the number of reference frames used for MRF ME.

Comparing with search pattern motion estimation (ME) algorithms, adaptive search range (ASR) algorithms are more fundamental, regular and flexible. In variable block size motion estimation (VBSME), ASR algorithms can be applied whether on a whole frame (frame level), or on an entire macroblock which includes up to forty-one blocks (macroblock level), or just on a single block (block level). In the other hand, in H.264/AVC, not the motion vectors (MVs) but the motion vector differences (MVDs) are coded and the median motion vector predictors (median-MVPs) are used to place the search centers. In this sense, it can be thought that the search windows (SWs) are centered at the positions pointed by median-MVPs, the search ranges (SRs) play the role of limiting MVDs. Thus it is reasonable for considering using MVDs to predict SRs. In this paper, one of the MB level and two of the block level, at all three MVD based SR prediction algorithms are proposed. VBSME based experiments are carried out to assess the proposed algorithms. Comparisons between the proposed three algorithms and the previously proposed one given in [8] are done in terms of encoding quality and computational complexity.

Data access usually leads to more than 50% of the power cost in a modern signal processing system. To realize a low-power design, how to reduce the memory access power is a critical issue. Data reuse (DR) is a technique that recycles the data read from memory and can be used to reduce memory access power. In this paper, a systematic method of DR exploration for low-power architecture design is presented. For a start, the signal processing algorithms should be formulated as the nested loops structures, and data locality is explored by use of loop analysis. Then, corresponding DR techniques are applied to reduce memory access power. The proposed design methodology is applied to the motion estimation (ME) algorithms of H.264 video coding standard. After analyzing the ME algorithms, suitable parallel architectures and processing flows of the integer ME (IME) and fractional ME (FME) are proposed to achieve efficient DR. The amount of memory access is respectively reduced to 0.91 and 4.37% in the proposed IME and FME designs, and thus lots of memory access power is saved. Finally, the design methodology is also beneficial for other signal processing systems with a low-power consideration.

Energy efficiency has become a primary concern in the design of multimedia digital systems, particularly when targeting mobile devices. Approximate computing is a highly promising approach to address this challenge. This paper presents an architectural exploration in a variable block size motion estimation (VBSME) architecture using imprecise Lower-Part-OR Adders (LOA). These adders were applied to Sum of Absolute Differences units (SAD) in order to reduce the energy consumption while introducing a minimum impact on the coding efficiency. Three VBSME architectures with LOA operators were developed by considering different imprecision levels. The conducted evaluations, performed using the High-Efficiency Video Coding standard (HEVC) reference software, showed that this technique introduces a negligible impact on the coding efficiency (between 0.6% and 2.5% increase of the BD-Rate). Nevertheless, when the designed architectures were synthesized for a 45nm standard cells technology, significant power savings were observed (between 7% and 11.5%, depending on the used LOA version), demonstrating the viability and significant gains of the proposed approach.

A low-bandwidth Integer Motion Estimation (IME) module is proposed for MPEG-2 to H.264 transcoding. Based on bandwidth reduction method proposed in Ref. 1), a ping-pang memory control scheme combined with Partial Sum of Absolute Differences (SAD) Variable Block Size Motion Estimation (VBSME) architecture are realized. Experiment results show bandwidth of the proposed architecture is 70.6% of H.264 regular IME (Level C+ scheme, 2 Macro Block (MB) stitched vertically), while the on-chip memory size is 11.7% of that.