Enhanced Computation Time for Fast Block Matching Algorithm

The new video coding standard H.264/AVC achieves higher coding efficiency than previous standards. However, the efficiency arises at the cost of significant complexity. In this paper, an efficient intermode decision algorithm is proposed to cut down the complexity of an exhaustively full mode decision algorithm in the reference software. Statistical variables including RD (Rate Distortion) costs and frequencies of various modes, correlation of modes between current MB (Macroblock), and their spatial and temporal neighbor MBs are utilized to achieve fast mode decision in the algorithm. The proposed algorithm is composed of three primary steps: The characteristics information of video content, RD cost, and frequency of each mode, are obtained by statistics to assist the decision of the best coding mode for the MB. Because the accuracy of the statistical results will affect the succedent mode decision process, FMD (Full Mode Decision) algorithm are used to get the statistical values from several training frames. In addition, considering the above statistical variables change with video content's changes, they are updated at regular intervals. The best coding modes of current MBs and their neighbor MBs (both spatial and temporal) are highly correlated, so the coding mode of these neighbor MBs are used as an indication to that of current MBs. The difference of RD costs between MB mode (16×16, 16×8 and 8×16) and non-MB modes is used to decide the possible modes class for current MB. Furthermore, the possible modes are prioritized based on their occurring probabilities such that the highest probable mode will be tried first. During this process, the computed RD cost is checked against a content adaptive RD cost threshold to decide if the mode decision process should be terminated before trying the remaining modes in the class. In this way many unlikely coding modes are skipped, and the computational time is significantly reduced. Experimental results show that the proposed algorithm performs well on both the low motion sequences and the high motion sequences due to utilizing the sequence-dependent statistical variables as adaptive threshold to decide the best coding modes. On average, the algorithm reduces the total encoding time by 66.3% while image quality (PSNR) degradation is only 0.154dB with bitrate increase about 0.21%.

In this paper, we propose a neural network-based scalable fast intra prediction algorithm in H.264 in order to reduce redundant calculation time by selecting the best mode of 4 × 4 and 16 × 16 intra prediction. In this reason, it is possible to encode compulsively by 4 × 4 intra prediction mode for current MB(macro block)'s best prediction mode without redundant mode decision calculation in accordance with neural network's output resulted from corelation of adjacent encoded four left, up-left, up and up-right blocks. If there is any one of MBs encoded by 16 × 16 intra prediction among four MBs adjacent to current MB, the probability of re-prediction into 16 × 16 intra prediction will become high. We can apply neural networks in order to decide whether to force into 4 × 4 intra prediction mode or not. We can also control both the bit rates and calculation time by modulating refresh factors and weights of neural network's output depend on error back-propagation, which is called refreshing. In case of encoding several video sequences by the proposed algorithm, the total encoding time of 30 input I frames are reduced by 20% ∼ 65% depending upon the test vector compared with JM 8.4 by using neural networks and by modulating scalable refreshing factor. On the other hand, total encoding bits are increased by 0.8% ∼ 2.0% at the cost of reduced SNR of 0.01 dB.

In this paper, a new technique called Mean Predictive Block Matching (MPBM) is proposed. This technique is based on adaptive rood pattern search algorithm, in which the size is dynamically determined for each macroblock, based on the mean of two motion vectors of the neighboring macroblocks instead of one as is the case with adaptive rood patter search. The proposed technique attempts to decrease the computation time between the current macroblock and the macroblocks in the reference frame, by stopping the calculation of the sum absolute different between the pixels in the current and the reference frame macroblocks when the current uncompleted sum absolute value is greater than the previous calculated one. Experimental results showed that the proposed technique reduces the search time of the macroblock matching, as well as achieves better resolution for fast block motion estimation in comparison to the standard block matching algorithms.

For video compression, motion estimation is popularly employed to exploit temporal correlation existing in video sequences. If we employ the full search block matching algorithm for estimating motion vectors, it requires very heavy computational complexity. Although several fast block matching algorithms have been proposed to solve this problem, they sacrifice their reconstructed image quality. In this paper, we derive optimal search patterns for fast block matching motion estimation. By analyzing the block matching algorithm as a function of the block size and the shape, we find optimal search patterns for initial motion estimation. The proposed idea can provide an analytical ground for the current MPEG-2 proposals. In addition, we propose a new fast motion estimation algorithm using adaptive search patterns, considering matching criteria and statistical properties of object displacement. In order to select an appropriate search pattern, we exploit the relationship between the motion vector and the frame difference of each block. By changing the search pattern adaptively, we can improve the motion prediction accuracy, while reducing the required computational complexity compared to other fast block matching algorithms.

In order to fit the small cross-center-biased characteristic of the real world video sequences, an improved version of the well-known cross diamond search algorithm is proposed in this paper. The new algorithm uses a small cross-shaped search patterns in the first two steps to speedup the motion estimation of stationary and quasi-stationary blocks. Experimental results show that this new cross-diamond search algorithm could achieve much higher computational reduction as compared with Diamond Search (DS) and Cross Diamond Search (CDS) while similar prediction accuracy is maintained, and it is especially suitable for videoconferencing sequences.

In single viewpoint video coding, there are many fast block matching motion estimation algorithms proposed, such as three-step search, four-step search, diamond search as well as hexagon-based search and so on. However, experimental analysis show that these algorithms are not suitable for using directly in Multiview View Coding (MVC). Since the increased search range, the larger format of multiview video as well as the correlations between inter-view frames are not considered by these algorithms, they may easy led the block matching search into local minimum, the Rate-Distortion (R-D) performance will degrade dramatically. In this paper, we propose a novel multiple hexagon search algorithm to address this problem. Firstly, according to the original initial search point, four sub-search windows are constructed. Then, the hexagon based search algorithm will be performed respectively in the four sub-search windows. The final result is the best search point with the minimum R-D cost among the best points in the four sub-search windows. In order to trade off the computational complexity and R-D performance, two adaptive early termination strategies are proposed. The experimental results show that the proposed algorithm yields a quite promising coding performance in terms of R-D performance and computational complexity. Especially, the proposed algorithm can work well in multiview video sequences with various motion and disparity activities.

Efficient multi-view video coding using inter-view information

One of the new features in the H.264/AVC encoder is the use of Lagrangian Rate-Distortion Optimised (RDO) mode decision at the macroblock (MB) level. This brute-force algorithm searches through the 10 different MB coding modes to obtain the best one for encoding that MB, and is hence computationally expensive. This paper proposes a novel algorithm where the RDO can be reorganized in a better order such that the most likely MB modes will be tried first, and an early termination of the RDO process will be used once the calculated rate-distortion cost (RDcost) is below a preset threshold. The preset threshold is dependent on the RDcost of previous and neighbouring MBs that have been coded. This is based on the observation that the RDcost of an MB in the current frame is highly correlated to a co-located MB in the previous frame. Experimental results have shown that the new algorithm has dramatically reduced the encoding times of up to 61.8%, with negligible increases in bit-rates or PSNR.

Publication in the conference proceedings of EUSIPCO, Rhodes, Greece, 1998

Block matching motion estimation (BMME) plays a very important role in video coding. The performance of BMME greatly affects the quality of the encoded bit stream. In order to reduce the high computational demands of H.264 video encoder, a fast block matching motion estimation algorithm based on temporal and spatial motion correlation is proposed by the deep research of the characteristics of motion estimation. This algorithm well exploits the correlation between video sequence motion activity and macro-block coding pattern and motion vector statistic characteristics to employ different patterns. Experimental results show that the algorithm, without function losing of PNSR, can not only reduce the search complexity of motion estimation and the high computational demands but also raise the real-time performance of encoder.

Multiple reference frame prediction technology is adopted in Multiview Video Coding (MVC) to explore temporal and inter-view redundancies of multiview videos, resulting in extremely high encoding complexity by searching the best reference frame indices and the best reference directions for each macroblock (MB). In order to reduce MVC coding computations while keeping the coding efficiency and thus to advance MVC in real-time multimedia broadcasting applications, we propose a Fast Multi-reference Frame Selection Algorithm (FMFSA) for hierarchical B picture prediction structure in this paper. Due to high spatial correlations within a MB, there is a high probability for smaller MB partition modes selecting the same reference frame and direction as B16 16 does. Therefore, the reference information of latter checked MB partition modes can be directly set according to the reference information of previous examined mode. Experimental results on MVC show that the proposed FMFSA can achieve 68.34%~79.01% total encoding time reduction while the average bit rate increase and peak signal-to-noise ratio degradation are within 0.54% and 0.04 dB, respectively for test multiview sequences with various motion properties and camera arrangements.

We present a region of interest (ROI) based rate control for H.263 compatible video conferencing. A face detection and tracking scheme with very low complexity is proposed for segmentation. By analyzing quadratic rate models in frame layer, video object plane (VOP) layer, and macroblock (MB) layer extracted from the test data, a quadratic rate model at the MB layer with a modified physical meaning is proposed to improve the model accuracy. The basic idea is to use a group of uncoded MBs in the current VOP instead of individual MBs to update model parameters. A joint VOP layer and MB layer rate control algorithm is proposed. The VOP layer rate control assigns target bit rate for each VOP based on the coding complexity and visual importance, and determines an average quantization parameter (QP) for each VOP. Some new features of MB layer rate control are designed to utilize both average statistics of a VOP and individual statistics of MB together. The performance is compared with conventional TMN8 and object-based VM8 rate control, better peak SNR (PSNR) for ROI, and more accurate rate control can be achieved for various video sequences. The proposed rate control algorithm can be extended for H.264 ROI-based scalable video coding.

This paper presents an algorithm to predict interpolative motion compensation (MC) mode without computing interpolative SADs (Sum of Absolute Difference) for B-picture coding in an MPEG-2 video encoder. Firstly, an initial MC mode is selected among forward frame/field MC and backward frame/field MC for each macroblock (MB) in a Bpicture. This is accomplished by comparing four minimum SADs corresponding to the above-mentioned MC modes, and choosing a minimum SAD among them. Secondly, if the SAD corresponding to the selected mode is less than a specific threshold, we set the selected MC mode to the final MC mode. Otherwise, an MC mode of the current MB is determined between interpolative frame MC and interpolative field MC. If the sum of SADs of forward frame MC mode and backward frame MC mode is less than that of SADs of forward field MC mode and backward field MC mode, the final MC mode is set to an interpolative frame MC. Otherwise, the final MC mode is set to an interpolative field MC mode. Experimental results show that the proposed algorithm is comparable with as brute-force MC mode selection method as in conventional MPEG-2 video encoding. In addition, the proposed algorithm may noticeably reduce the memory bandwidth in implementation of MPEG-2 video encoder.

A novel Cooperative Motion Estimation Algorithm based on Particle Swarm Optimization and its multicore implementation

In the new video coding standard H.264/AVC, motion estimation (ME) is allowed to search multiple reference frames. Therefore, the required computation is highly increased, and it is in proportion to the number of searched reference frames. However, the reduction in prediction residues is mostly dependent on the nature of sequences, not on the number of searched frames. Sometimes the prediction residues can be greatly reduced, but frequently a lot of computation is wasted without achieving any better coding performance. In this paper, we propose a context-based adaptive method to speed up the multiple reference frames ME. Statistical analysis is first applied to the available information for each macroblock (MB) after intra-prediction and inter-prediction from the previous frame. Context-based adaptive criteria are then derived to determine whether it is necessary to search more reference frames. The reference frame selection criteria are related to selected MB modes, inter-prediction residues, intra-prediction residues, motion vectors of subpartitioned blocks, and quantization parameters. Many available standard video sequences are tested as examples. The simulation results show that the proposed algorithm can maintain competitively the same video quality as exhaustive search of multiple reference frames. Meanwhile, 76 %-96 % of computation for searching unnecessary reference frames can be avoided. Moreover, our fast reference frame selection is orthogonal to conventional fast block matching algorithms, and they can be easily combined to achieve further efficient implementations.