Overlapped block disparity compensation with adaptive windows for stereo image coding

This article presents a complete video codec which supports both block-based and mesh-based motion compensation modes. While the mesh-based modes are suitable for spatially continuous representation of in-plane rotations, zooms, and other motions that can be represented by parametric mappings, the block-based modes (especially overlapped block matching) perform better around motion boundaries. Therefore, a hybrid codec with block-based and mesh-based modes, which nicely complement each other, yields the best results over a wide range of video at low bit rates. Experimental results with MPEG-4 test sequences are shown to demonstrate the performance of the proposed hybrid codec. © 1998 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 9, 248–256, 1998

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

Video coding standards use the block-matching algorithm (BMA) and motion-compensated prediction to reduce temporal redundancies present in image sequences. Block matching is used since it is computationally efficient and produces a minimal representation of the motion field that is transmitted as side information. In order to build a robust coder the motion-estimation technique must be able to track motion in a noisy source. The approach presented in this paper uses spatio-temporal motion prediction, providing an accurate motion estimate even in the presence of noise. With this approach, noisy sources can be compressed efficiently and robustly in standard video coders (e.g., MPEG-1, MPEG-2, H.261, and H.263) with little increase in complexity.

Motion estimation (ME) and motion compensation (MC) based deinterlacing has been become the mainstream technology of industry. The existing ME/MC algorithms are mostly based on block matching (BM) strategy, in which various BM algorithms are proposed with different tradeoff between the matching accuracy and the CPU time. Aiming at the dilemma between the performance and the efficiency of the available deinterlacing algorithms, two adaptive ME/MC algorithms are presented in this paper. Compared with the full search BM algorithm, the proposed algorithms greatly reduce the computational amount with keeping the performance approximately. However, the inherent drawbacks of the BM limit the farther improvement of the deinterlacing performance. For this purpose, a new deinterlacing algorithm based on motion object is developed, in which it is natural motion object rather than contrived block that is taken as the basic cell for ME/MC. In the new algorithm, a more reliable technique is adopted for accurate motion region detection, and immune clonal selection algorithm is employed to accelerate the matched object searching. Meanwhile, many advanced deinterlacing methods, such as motion compensation, median filtering, Weave and Bob, are integrated into the new algorithm, so it is more adaptive to the various video sequences. The experimental results with several available test databases illustrate that the new algorithm based on motion object is better than those based on motion block on the whole.

In this contribution, a novel spatio-temporal prediction algorithm for video coding is introduced. This algorithm exploits temporal as well as spatial redundancies for effectively predicting the signal to be encoded. To achieve this, the algorithm operates in two stages. Initially, motion compensated prediction is applied on the block being encoded. Afterwards this preliminary temporal prediction is refined by forming a joint model of the initial predictor and the spatially adjacent already transmitted blocks. The novel algorithm is able to outperform earlier refinement algorithms in speed and prediction quality. Compared to pure motion compensated prediction, the mean data rate can be reduced by up to 15% and up to 1.16 dB gain in PSNR can be achieved for the considered sequences.

Distributed video coding (DVC) is an emerging video coding paradigm due to its flexibility to introduce much lower encoding complexity than conventional predictive codecs, which is beneficial for some applications such as wireless video surveillance, wireless sensor networks, and disposable video cameras. Although DVC systems without feedback channel address a wider range of applications, it is not commonly discussed in literatures due to its lower coding performance. In this paper, on the basis of PRISM DVC architecture, a low-complexity feedback-channel-free DVC system is proposed with a new classifier to improve the coding performance. Experimental results show that the proposed system can provide about 0.8 dB gain in PSNR to the state-of-the-art and 2 dB gain to IST-PRISM. It is also competitive regarding other feedback-channel-free DVC systems and can even achieve similar performance to DVC systems with feedback channel for low-motion sequences.

Motion estimation is the process which generates the motion vectors that determine how each motion compensated prediction frame is created from the previous frame. But its computational complexity poses great challenges for real time implementations. Among the different motion Estimation algorithms block matching motion estimation is the most common method of motion estimation. To minimize the search time on block matching, a simplified and efficient algorithm is very crucial for fast motion estimation process. In this paper the comparative performance of 6 different block matching algorithms are presented and compared based on the parameters of Peak Signal Noise Ratio (PSNR), Mean Absolute Difference (MAD) and Search Points per Macro block.

Error-resilient scalable compression based on distributed video coding

Complexity reduction is a commonly used method to deal with complicated video coding standards, such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). But the unstable performance under different video contents and Quantization Parameters (QPs) makes it difficult to precisely specify the target encoding time of every single sequence, which limits the practical use of the encoder. Inspired by rate control, in this paper, encoding time budget is regarded as a resource. We incorporate the hierarchical Group of Picture (GOP) encoding structure, the non-accelerated proportion, and block content variation within the frame, and design a top-down allocation and bottom-up feedback scheme, to achieve precise control of encoding complexity by controlling the maximum depth of QuadTree with nested Multi-type Tree (QTMT). In the scheme, Temporal ID (Tid)-based, Sum of Absolute Transformed Difference (SATD)-based methods and Linear (L) Model are designed to facilitate weighted allocation and feedback. The relationship between Planar Cost and encoding time is exploited as a Time-Cost (T-C) model, which guides the selection of Largest Coding Unit (LCU) encoding strategies in I-frames. For B-frames, we investigate a switching suppression assisted status-based method, to efficiently decide the encoding strategy of each LCU. Through the collaboration of the proposed technologies in the scheme, given any encoding time target achievable, encoding strategies will be automatically switched to help the actual encoding time gradually approach the target. To the best of our knowledge, this work is the first work on VVC-based encoding complexity control. The proposed scheme supports directly specifying the target encoding time or target Frame Per Second (FPS), and accurately realizing it within one pass. According to experimental results, under the target encoding time ratio of 80%, 60% and 40%, the average encoding time error is kept under 0.24%, 0.03% and 0.02% by our method, outperforming state-of-the-art works base on HEVC. These results prove the advantage of the proposed scheme.

In this paper, we propose a fast block motion estimation scheme by exploiting the correlations of motion vectors (MV) existing in spatially- and temporally-adjacent as well as hierarchically-related blocks. The basic idea is to use the information obtained from the corresponding block at a coarser resolution level and spatio-temporal neighboring blocks at the same level in order to select a good set of initial MV candidates and then perform further local search to find the best matching MV. In order to further reduce computational complexity, the sign pattern vector (SPV) is defined and used for block matching, as opposed to the pixel intensity values used in the conventional block matching methods. By using the SPV definition, a data block can be presented by a mean and a set of binary sign patterns with a much reduced data set. The block matching motion estimation is then divided into mean matching and binary phase matching. The proposed technique enables a significant reduction in computational complexity compared with the conventional block matching motion estimation (ME) because binary phase matching only involves Boolean logic operations. This scheme also significantly reduces the data transfer time between the frame buffer and motion estimator. Our test results indicate that the performance of the proposed scheme is comparable with the full-search block matching under the same search ranges, however, the proposed scheme has a speed-up factor ranging from 250 to 370 in comparison with full search.

This paper presents a novel block matching scheme with edge alignment strategy on H.264 video coding, which uses multiple references and multiple block sizes for motion estimation in order to improve the rate-distortion performance. In H.264, the computational complexity is linearly dependent on the number of allowed reference frames and block sizes using the full exhaustive search. Many fast block-matching algorithms reduce the computational complexity of motion estimation by carefully designing the search patterns with different shapes or sizes which have significant impact on the search speed and distortion performance. However, the search speed and the distortion performance conflict often with each other for these methods. In this paper, given a block in the current frame, we first apply a fast approximate method based on edge alignment to obtain a good initial motion vector as well as a tight initial bound of distortion measure. Then, considering the edge orientation of the block, a modified hexagon search is used to fine tune the motion vector in low computational complexity. The proposed algorithm also pays attentions to the characteristics of multiple reference frames and multiple block sizes in H.264. Computer simulation results show that the proposed method gives good performance and spans a new way to design a cost-effective real-time video coding system

Although frame-based MPEG-4 video services have been successfully deployed since 2000, MPEG-4 video coding is now facing great competition in becoming a dominant player in the market. Object-based coding is one of the key functionalities of MPEG-4 video coding. Realtime object-based video encoding is also important for multimedia broadcasting for the near future. Object-based video services using MPEG-4 have not yet made a successful debut due to several reasons. One of the critical problems is the coding complexity of object-based video coding over frame-based video coding. Since a video object is described with an arbitrary shape, the bitstream contains not only motion and texture data but also shape data. This has introduced additional complexity to the decoder side as well as to the encoder side. In this paper, we have analyzed the current MPEG-4 video encoding tools and proposed efficient coding technologies that reduce the complexity of the encoder. Using the proposed coding schemes, we have obtained a 56 percent reduction in shape-coding complexity over the MPEG-4 video reference software (Microsoft version, 2000 edition).

Current digital video applications require delivery of digital video content to different clients over heterogeneous networks. These clients may have different system resources and bandwidth capabilities. Thus, delivering of digital video data compressed at specific spatial resolution, quality levels, frame rates and encoding those video data at specific bit rates according to the client resources and the available network is essential. The characteristics of current video transmission system pose many problems for video delivery and scalable video coding is a highly attractive solution. This thesis investigates the existing video coding standards, especially the newest H.264/advanced video coding and its scalable extension: scalable video coding. In this thesis, we address two issues related to the state-of-the-art scalable video coding: 1) computational complexity reduction; and 2) video coding efficiency enhancement. In the first part of this thesis, we address the complexity issue of spatial, signal-to-noise ratio (SNR), temporal and combined scalable video coding. Our goal is to find a good tradeoff between computational complexity and video coding efficiency. We first establish the need to reduce computational complexity for current H.264/advanced video coding and scalable video coding framework. We then evaluate the mode distribution correlations between the base layer and its enhancement layers for different video scalabilities. After the exhaustive search over all possible block partitions is performed at the base layer, the number of candidate modes for luma and chroma blocks in a macroblock that take part in rate distortion optimization calculation at enhancement layers could be greatly reduced based on the correlations. Finally, adaptive fast mode decision schemes for spatial, SNR and temporal scalable video coding are presented. Our schemes could achieve consistent and significant computational complexity reduction with negligible loss in objective and subjective video quality and insignificant increments in bit rate consumption. The second part of this thesis is to provide a solution for the problem of high computational complexity of scalable video decoders. An adaptive encoding algorithm is proposed to reduce decoder complexity in coarse grain SNR scalable video coding.

Block matching algorithms (BMAs) are often employed for motion estimation (ME) in video coding. Most conventional BMAs treat the ME problem as an optimization problem and employ certain search schemes to find a solution. Except the time-consuming full search algorithms (FS), other fast algorithms such as the three step search (TSS), searching on a reduced search range, cannot guarantee optimal solutions, i.e., search is often trapped at local minima; the ME results are thus usually unsatisfactory. Few of them makes use of the information inherent in the images explicitly. We propose a new ME algorithm which can reduce the search range while guaranteeing global optimality in most cases. Microblock visual patterns are designed to extract edge information to guide block matching: searching is only carried out at places where the real match most likely happens; that is where similar edge features present. The proposed algorithm obtains a speed about 7 - 8 times as fast as that of FS with same search range. The prediction quality is very close to that of FS and much better than that of TSS. It can produce MPEG-1 or MPEG-2 compatible motion vectors, and can be extended to model-based ME. It is suitable for parallel implementation too. Moreover, the visual patterns are potential resources for video indexing to facilitate content-based information retrieval, which is important for applications such as VOD.

This work tackles the problem of side information generation for the case of large-duration GOPs in distributed video coding. Based on a previously developed technique for side-information enhancement, we develop a genetic algorithm particularly designed for large GOPs, taking into account the GOP size, the additional bitrate incurred by encoding hash information, as well as the decoding complexity. The proposed algorithm makes use of different interpolation methods available in the literature in a fusion-based approach. A significant gain in the average PSNR that can reach 2 dB is observed with respect to the best performing interpolation technique, while the algorithm is run for no more than 18% of the total number of blocks in a given video sequence. On the other hand, while the encoding complexity is a main concern in distributed video coding, the proposed solution incurs no additional complexity at the encoder side in the case of hash-based Wyner-Ziv video coding.