Improved motion estimation time using a combination of dynamic reference frame selection and residue-based mode decision

In the new video coding standard H.264/AVC, motion estimation (ME) is allowed to search multiple reference frames. Therefore, the required computation is highly increased, and it is in proportion to the number of searched reference frames. However, the reduction in prediction residues is mostly dependent on the nature of sequences, not on the number of searched frames. Sometimes the prediction residues can be greatly reduced, but frequently a lot of computation is wasted without achieving any better coding performance. In this paper, we propose a context-based adaptive method to speed up the multiple reference frames ME. Statistical analysis is first applied to the available information for each macroblock (MB) after intra-prediction and inter-prediction from the previous frame. Context-based adaptive criteria are then derived to determine whether it is necessary to search more reference frames. The reference frame selection criteria are related to selected MB modes, inter-prediction residues, intra-prediction residues, motion vectors of subpartitioned blocks, and quantization parameters. Many available standard video sequences are tested as examples. The simulation results show that the proposed algorithm can maintain competitively the same video quality as exhaustive search of multiple reference frames. Meanwhile, 76 %-96 % of computation for searching unnecessary reference frames can be avoided. Moreover, our fast reference frame selection is orthogonal to conventional fast block matching algorithms, and they can be easily combined to achieve further efficient implementations.

Motion Estimation (ME) is the most computationally intensive part of video compression systems. Multiple reference frame (MRF) ME used in H.264 standard increases the video coding efficiency at the expense of increased computational complexity and power consumption. Therefore, in this paper, we present a reconfigurable baseline H.264 video encoder hardware in which the number of reference frames used for MRF ME can be configured based on the application requirements in order to trade-off video coding efficiency and power consumption. The proposed H.264 video encoder hardware is based on an existing low cost H.264 intra frame coder hardware and it includes new reconfigurable MRF ME, mode decision and motion compensation hardware. The proposed H.264 video encoder hardware is capable of processing 55 CIF (352×288) frames per second and its power consumption ranges between 115mW and 235mW depending on the number of reference frames used for MRF ME.

Motion Estimation is the most crucial and time-consuming part of the H.264/AVC video compression standard. The introduction of motion search of variable block sizes in multiple reference frames has significantly increased the computational complexity. This paper proposes a fast motion estimation algorithm, Most Used Reference First (MURF), based on the usage history of reference frames. The algorithm rearranges the search order of the reference frames based on the selection probability of the reference frames in coding the current frame. The experimental results show that the proposed algorithm, when compared to the best algorithm in H.264 reference software, achieves on average 60% reduction in search points and 52% reduction in motion estimation time with comparable video quality and negligible increase in bit-rate and memory.

In H.264/AVC standard, motion estimation can be processed on multiple reference frames (MRF) to improve the video coding performance. For the VLSI real-time encoder, the heavy computation of fractional motion estimation (FME) makes the integer motion estimation (IME) and FME must be scheduled in two macro block (MB) pipeline stages, which makes many fast MRF algorithms inefficient for the computation reduction. In this paper, two algorithms are provided to reduce the computation of FME and IME. First, through analyzing the block's Hadamard transform coefficients, all-zero case after quantization can be accurately detected. The FME processing in the remaining frames for the block, detected as all-zero one, can be eliminated. Second, because the fast motion object blurs its edges in image, the effect of MRF to aliasing is weakened. The first reference frame is enough for fast motion MBs and MRF is just processed on those slow motion MBs with a small search range. The computation of IME is also highly reduced with this algorithm. Experimental results show that 61.4%-76.7% computation can be saved with the similar coding quality as the reference software. Moreover, the provided fast algorithms can be combined with fast block matching algorithms to further improve the performance.

The variable block size motion estimation (ME) and compensation (MC) using multiple reference frames is adopted in H.264/AVC to improve coding efficiency. However, the computational complexity for ME/MC increases proportional to the number of reference frames. In this paper, we propose a new efficient reference frame selection algorithm to reduce the complexity. The proposed algorithm selects suitable reference frames by employing the spatial and temporal correlation of video sequence. The experimental results show that the proposed algorithm decreases video encoding time while keeping the similar visual quality and bit rates.

The H.264 video coding standard adopts many new coding tools, such as variable block size, multiple reference frames, quarter-pixel-accuracy motion estimation, intra prediction, loop filter, etc. Using these coding tools, H.264 achieves significant performance. Especially for reference frame, block size, and predication direction, it can adaptively do the selection. However, the encoding complexity increases tremendously. Among these tools, the macro block modes selection and the motion estimation contributes most to total encoding complexity. This paper focuses on complexity reduction in macro block modes selection. On the basis of analyzing some fast mode selection algorithms, a new algorithm is presented, in order to reduce the time of encoding, in the precondition of reducing the quality of encoding obviously.

The key to high performance in image sequence coding lies in an efficient reduction of the temporal redundancies. For this purpose, motion estimation and compensation techniques have been successfully applied. This paper studies motion estimation algorithms in the context of first generation coding techniques commonly used in digital TV. In this framework, estimating the motion in the scene is not an intrinsic goal. Motion estimation should indeed provide good temporal prediction and simultaneously require low overhead information. More specifically the aim is to minimize globally the bandwidth corresponding to both the prediction error information and the motion parameters. This paper first clarifies the notion of motion, reviews classical motion estimation techniques, and outlines new perspectives. Block matching techniques are shown to be the most appropriate in the framework of first generation coding. To overcome the drawbacks characteristic of most block matching techniques, this paper proposes a new locally adaptive multigrid block matching motion estimation technique. This algorithm has been designed taking into account the above aims. It leads to a robust motion field estimation precise prediction along moving edges and a decreased amount of side information in uniform areas. Furthermore, the algorithm controls the accuracy of the motion estimation procedure in order to optimally balance the amount of information corresponding to the prediction error and to the motion parameters. Experimental results show that the technique results in greatly enhanced visual quality and significant saving in terms of bit rate when compared to classical block matching techniques. >

Rate-constrained coding is one of the many coding-efficiency oriented tools of H.264/AVC, however, the mode decision process of Rate Distortion Optimization (RDO), is characterized by high computational com- plexity. Many fast mode decision algorithms have been proposed to reduce the computational complexity of mode decision. In this paper, two algorithms are proposed for reduction of mode decision in H.264/AVC, fast reference frame selection and selective intra prediction mode decision. Fast reference frame selection is efficient for inter predication, and selective intra prediction mode decision can effectively reduce excessive calculation load of intra prediction mode decision. The simulation results demonstrated that the proposed two methods applied together could reduce the encoding time of the overall sequences by an average of 44.63%, without any noticeable degradation in coding efficiency.

Fast reference frame selection based on content similarity for low complexity HEVC encoder

Previous studies have shown that multiple reference frames are available and compete for selection during the use of spatial terms such as "above." However, the mechanisms that underlie the selection process are poorly understood. In the current paper we present two experiments and a comparison of three computational models of selection to shed further light on the nature of reference frame selection. The three models are drawn from different areas of human cognition, and we assess whether they may be applied to a reference frame selection by examining their ability to account for both existing and new empirical data comprising acceptance rates, response times, and response time distributions. These three models are the competitive shunting model (Schultheis, ), the leaky competing accumulator (LCA) model (Usher & McClelland, ), and a lexical selection model (Howard, Nickels, Coltheart, & Cole-Virtue, ). Model simulations show that only the LCA model satisfactorily accounts for the empirical observations. The key properties of this model that seem to drive its success are its bounded linear activation function, its number and type of processing stages, and its use of decay. Uncovering these critical properties has important implications for our understanding not only of spatial term use, in particular, but also of conflict and selection in human cognition more generally.

The complexity of motion estimation in the High Efficiency Video Coding (HEVC) standard is very high as it uses a number of prediction block sizes. The test zonal search (TZS) mechanism is used as motion estimation algorithm in the fast search mode of HM encoder, the reference software for HEVC. In this paper, we present schemes for reducing the complexity of the TZS algorithm for uni-prediction in HEVC. The proposed mechanisms help in reducing complexity of grid, raster and refinement search stages of TZS. The performance of the proposed mechanisms is tested independently and also combined in the fast search mode of HM-16.18. The motion estimation time and the number of search points in the combined algorithm are reduced by 68.14% and 77.10% with a BD-rate of 0.30% and a BD-PSNR of $$-0.007$$ dB in comparison with the original fast search mode in the low-delay P main profile. The proposed complexity reduction schemes are compared with those adopted in other recently proposed fast motion estimation algorithms and the performance of the proposed scheme is found to be superior in terms of the reduction in motion estimation time.

The H.264 provides various and useful features for improved coding efficiency and error robustness, but its encoder complexity is extremely high due to the computational time of ME (motion estimation) and mode decision of inter coding. In this paper, we propose a fast and simple class-based scheme to reduce the computational complexity of ME. The image in block base is classified to k classifications with SQCM (spatial-domain quickly classified method). Referring to the classified indices, we discriminate two methods of ME: one is pattern search for stationary macroblock, synchronous motion and unchanged object; other is fast inter prediction for instable or variant content of object. We can save the dispensable motion detection for different contents of video, named CQME (class-based quickly ME). Experimental results shows that the proposed method can achieve a reduction of 67% computational time on average, with an average PSNR loss of only 0.30 dB compared with the original H.264 reference software jm12.1.

With the development of fast motion estimation (ME) algorithms towards video compression standard H.264, burden on integer-pel ME is slightly reduced. However the integer-pel ME is becoming computationally complex due to demand of higher and higher resolution videos. On the other hand, there is always trade-off between the speed and performance of these search algorithms. This work improvises the performance of content awareness enabled integer-pel ME algorithm for encoding the fast and slow motion video sequences of high definition (HD) resolution category viz. 1280×720 and 1920×1080. The algorithm proposes a novel notion of 'integrity index' with focus on increasing the PSNR and reducing the bit rate. It is implemented in the frame work of x265 version 1.7 video encoder. Motion independent ME algorithm is analysed quantitatively in terms of parameters based on rate distortion and ME time. The proposed algorithm is found to be performing better on BD-Rate and BD-PSNR parameters for the videos of both resolutions under consideration. Observed increase in motion estimation time is less than four seconds compared to hexagonal search algorithm of ME which is the benchmark in fast ME algorithm.

H.264IMPEG-4 AVC allows multiple reference frames and multiple partition types to improve the coding performance at the expense of much higher computation complexity. This paper proposes a fast motion estimation (ME) algorithm to reduce search complexity for this demanding configuration. Our experimental results show that the partition type which gives the lowest matching cost for the reference frame closest to the current picture in temporal domain is likely to be the best partition type for all reference frames. Based on this finding, the proposed method performs a less complicated search effort for the other partition types on the remaining reference frames to obtain significant savings in search complexity. This approach can be easily modified to support any existing fast search algorithms. Simulation results show the proposed algorithm significantly reduces the overall motion search complexity and at the same time, maintains minimum coding performance degradation.

PurposeTo develop an automated reference frame selection (ARFS) algorithm to replace the subjective approach of manually selecting reference frames for processing adaptive optics scanning light ophthalmoscope (AOSLO) videos of cone photoreceptors.MethodsRelative distortion was measured within individual frames before conducting image-based motion tracking and sorting of frames into distinct spatial clusters. AOSLO images from nine healthy subjects were processed using ARFS and human-derived reference frames, then aligned to undistorted AO-flood images by nonlinear registration and the registration transformations were compared. The frequency at which humans selected reference frames that were rejected by ARFS was calculated in 35 datasets from healthy subjects, and subjects with achromatopsia, albinism, or retinitis pigmentosa. The level of distortion in this set of human-derived reference frames was assessed.ResultsThe average transformation vector magnitude required for registration of AOSLO images to AO-flood images was significantly reduced from 3.33 ± 1.61 pixels when using manual reference frame selection to 2.75 ± 1.60 pixels (mean ± SD) when using ARFS (P = 0.0016). Between 5.16% and 39.22% of human-derived frames were rejected by ARFS. Only 2.71% to 7.73% of human-derived frames were ranked in the top 5% of least distorted frames.ConclusionARFS outperforms expert observers in selecting minimally distorted reference frames in AOSLO image sequences. The low success rate in human frame choice illustrates the difficulty in subjectively assessing image distortion.Translational RelevanceManual reference frame selection represented a significant barrier to a fully automated image-processing pipeline (including montaging, cone identification, and metric extraction). The approach presented here will aid in the clinical translation of AOSLO imaging.