Rate-distortion optimal wavelet packet transform for low bit rate video coding

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

Three-dimensional (3-D) subband/ wavelet coding via a Motion Compensated Temporal Filter (MCTF) is emerging as a very effective structure for highly scalable video coding. In this paper, a low bit rate video coding system based on three dimensional (3-D) wavelet coding using longer filters (Le Gall 5/3, 9/7) is implemented for Telemedicine imaging applications. In addition, a new scheme for finding the motion vectors (MV) is proposed and through several experimental results, it has been observed that it outperforms H.264 base line profile in terms of PSNR.

We propose an adaptive motion-compensated temporal filtering (MCTF) structure to provide efficient temporal scalability within the H.264/AVC video compression standard. MCTF has traditionally been considered within fully scalable wavelet video coders. However, motion-compensated simple 5/3 lifted temporal wavelet filtering suffers at scene changes, as well as occlusion regions. We note that the bi-directional motion compensation mode in the H.264 standard is best equipped with the state of the art adaptive features such as adaptive block size, mode switching between forward, backward and bidirectional prediction and in-loop deblocking filter. Hence, we propose a GOP structure to implement block-based adaptive MCTF within the H.264 syntax using stored B-pictures, similar to the motion-compensated 5/3 wavelet filtering. We provide experimental results to compare the results of our proposed codec with those of other scalable wavelet video coders which use MCTF. It is also possible to employ the proposed adaptive MCTF structure within fully scalable wavelet video codecs.

Rate-distortion optimization (RDO) plays an essential role in substantially enhancing the coding efficiency. Currently, rate-distortion optimized mode decision is widely used in scalable video coding (SVC). Among all the possible coding modes, it aims to select the one which has the best trade-off between bitrate and compression distortion. Specifically, this tradeoff is tuned through the choice of the Lagrange multiplier. Despite the prevalence of conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to 3-D wavelet-based SVC where the explicit values of the quantization step are not available, with on consideration of the content features of input signal. In this paper, an efficient content adaptive Lagrange multiplier selection algorithm is proposed in the context of RDO for 3-D wavelet-based SVC targeting quality scalability. Our contributions are two-fold. First, we introduce a novel weighting method, which takes account of the mutual information, gradient per pixel, and texture homogeneity to measure the temporal subband characteristics after applying the motion-compensated temporal filtering (MCTF) technique. Second, based on the proposed subband weighting factor model, we derive the optimal Lagrange multiplier. Experimental results demonstrate that the proposed algorithm enables more satisfactory video quality with negligible additional computational complexity.

Motion Compensated Temporal Filtering (MCTF) has proved to be an efficient coding tool in the design of open-loop scalable video codecs. In this paper we propose a MCTF video coding scheme based on lifting where the prediction step is implemented using PRISM (Power efficient, Robust, hIgh compression Syndrome-based Multimedia coding), a video coding framework built on distributed source coding principles. We study the effect of integrating the update step at the encoder or at the decoder side. We show that the latter approach allows to improve the quality of the side information exploited during decoding. We present the analytical results obtained by modeling the video signal along the motion trajectories as a first order auto-regressive process. We show that the update step at the decoder allows to half the contribution of the quantization noise. We also include experimental results with real video data that demonstrate the potential of this approach when the video sequences are coded at low bitrates.

Computer networks and the internet have taken an important role in modern society. Together with their development, the need for digital video transmission over these networks has grown. To cope with the user demands and limitations of the network, compression of the video material has become an important issue. Additionally, many video-applications require flexibility in terms of scalability and complexity (e.g. HD/SD-TV, video-surveillance). Current ITU-T and ISO/IEC video compression standards (MPEG-x, H.26-x) lack efficient support for these types of scalability. Wavelet-based compression techniques have been proposed to tackle this problem, of which the Motion Compensated Temporal Filtering (MCTF)-based architectures couple state-of-the-art performance with full (quality, resolution, and frame-rate) scalability. However, a significant drawback of these architectures is their high complexity. The computational and memory complexity of both spatial domain (SD) MCTF and in-band (IB) MCTF video codec instantiations are examined in this study. Comparisons in terms of complexity versus performance are presented for both types of codecs. The paper indicates how complexity scalability can be achieved in such video-codecs, and analyses some of the trade-offs between complexity and coding performance. Finally, guidelines on how to implement a fully scalable video-codec that incorporates quality, temporal, resolution and complexity scalability are proposed.

Chapter two - Digital Image and Video Compression and Processing

Forecasts predict that Internet traffic will continue to grow in the near future. A huge share of this traffic is caused by multimedia streaming. The quality of experience (QoE) of such streaming services is an important aspect and in most cases the goal is to maximize the bit rate which-in some cases-conflicts with the requirements of both consumers and providers. For example, in mobile environments users may prefer a lower bit rate to come along with their data plan. Likewise, providers aim at minimizing bandwidth usage in order to reduce costs by transmitting less data to users while maintaining a high QoE. Today's adaptive video streaming services try to serve users with the highest bit rates that consequently results in high QoE. In practice, however, some of these high bit rate representations may not differ significantly in terms of perceived video quality compared to lower bit rate representations. In this paper, we present a novel approach to determine the statistically indifferent quality variation of adjacent video representations for adaptive video streaming services by adopting standard objective quality metrics and existing QoE models. In particular, whenever the quality variation between adjacent representations is imperceptible from a statistical point of view, the representation with higher bit rate can be substituted with a lower bit rate representation. As expected, this approach results in savings with respect to bandwidth consumption while still providing a high QoE for users. The approach is evaluated subjectively with a crowdsourcing study. Additionally, we highlight the benefits of our approach, by providing a case study that extrapolates possible savings for providers.

A new multiview just-noticeable-depth-difference(MJNDD) Model is presented and applied to compress the joint multiview video plus depth. Many video coding algorithms remove spatial and temporal redundancies and statistical redundancies but they are not capable of removing the perceptual redundancies. Since the final receptor of video is the human eyes, we can remove the perception redundancy to gain higher compression efficiency according to the properties of human visual system (HVS). Traditional just-noticeable-distortion (JND) model in pixel domain contains luminance contrast and spatial-temporal masking effects, which describes the perception redundancy quantitatively. Whereas HVS is very sensitive to depth information, a new multiview-just-noticeable-depth-difference(MJNDD) model is proposed by combining traditional JND model with just-noticeable-depth-difference (JNDD) model. The texture video is divided into background and foreground areas using depth information. Then different JND threshold values are assigned to these two parts. Later the MJNDD model is utilized to encode the texture video on JMVC. When encoding the depth video, JNDD model is applied to remove the block artifacts and protect the edges. Then we use VSRS3.5 (View Synthesis Reference Software) to generate the intermediate views. Experimental results show that our model can endure more noise and the compression efficiency is improved by 25.29 percent at average and by 54.06 percent at most compared to JMVC while maintaining the subject quality. Hence it can gain high compress ratio and low bit rate.

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that offers good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications, achieving essentially the same levels of rate-distortion performance and subjective quality as that of the full search algorithm when used by the UBC H.263+ video coding reference software. However the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.

This paper proposes a subband adaptive motion compensated temporal filtering (MCTF) technique for scalable video coding and introduces a revised synthesis gain model for the quantization in this adaptive MCTF scheme. In scalable video coding, hierarchical MCTF is extensively adopted to exploit the temporal correlation across video frames. In this hierarchical MCTF structure, the strength of temporal correlation varies with the level of temporal transform and varies with the various spatial frequency components in a frame. The reconstruction noises also have diverse strength at various subbands. According to the correlation and noise characteristics of various subbands, we can adjust the strength of motion compensated prediction step in MCTF to maximally take the advantage of temporal correlation but restrict the propagation of reconstruction noise. The quantization step of each subband is also adjusted according to synthesis gain determined by the MCTF structure. In this way an adaptive MCTF scheme is formed and the proposed technique improves the coding performance of scalable video coding

An efficient MPEG-2 to MPEG-4 video transcoder is presented in this paper. We consider the transcoding from high quality and bit rate MPEG-2 video with larger image size (e.g. 4CIF/4SIF, CIF) to lower quality and bit rate MPEG-4 video with smaller image size (e.g. CIF, QCIF). First, the transcoder needs to down-sample the input MPEG-2 video. Since the motion vectors carried by the MPEG-2 stream will be reused in the transcoding process, they are sub-sampled besides the frame pixels, and the coding mode for each down-sampled macroblock is examined. A new rate control method is proposed to convert the high bit rate MPEG-2 video to the low bit rate MPEG-4 counterpart. The proposed rate control scheme adjusts the frame rate and the frame quantization step size simultaneously according to the channel bandwidth to achieve a good temporal-spatial quality tradeoff. Due to the reuse of motion vectors, key frames (i.e. I and P frames) cannot be skipped to maintain the prediction sequential order, while some B frames containing less temporal information may be skipped in transcoding to save the bit rate. Skipped B frames can be reconstructed at the decoder to ensure the full frame rate playback. The TM7 quadratic Rate-Qtz model is adopted in the proposed rate control scheme to calculate the re-quantization step size from the given target bit. Simulations show that the proposed MPEG-2 to MPEG-4 video transcoder with rate control can out-perform the basic MPEG transcoder that adjusts the re-quantization step size at a constant frame rate. The complexity of the proposed transcoder is low so that it can be used in real-time applications.

The exploitation of spatial and temporal correlation within video signals is a cornerstone of video coding, which is extensively employed to achieve high coding efficiency. The presence of noise in video signal deteriorates the correlations, leading to a degradation in coding efficiency. Motion Compensated Temporal Filter (MCTF) is a pre-processing tool designed to remove noise from the source signal, thereby enhancing temporal correlations among adjacent frames. In this paper, a Content-Adaptive MCTF method (CAMCTF) is proposed to enhance the filtering adaptability of MCTF for diverse video contents. The proposed CAMCTF method consists of Quadtree Block Partitioning Scheme (QBPS), Offset Block Weighted Compensation (OBWC) and Structural Similarity (SSIM) based Filtering Weight Adjustment (SSFWA). Specifically, QBPS generates size-adaptive Motion Compensation Block (MCB) to better cater to the video content characteristics. Subsequently, OBWC is employed to address the uneven prediction quality of MCBs. Furthermore, the filtering weights of each MCB are fine-tuned by SSFWA based on the SSIM index. The simulation results show that the proposed CAMCTF method can achieve 1.23% and 0.94% BD-rate gains for RA and LD configurations, respectively, on top of Versatile Video Coding (VVC) reference software VTM-23.4.

A chaotic image encryption algorithm based on bit level is proposed. The 8-bit grayscale matrix image is divided into a higher 4-bit matrix image and a lower 4-bit matrix image, which named higher bit image and lower bit image, respectively. Then the encrypted operations are performed on the higher bit and lower bit image. The operations include the mutual disturbing and element exchange operations between them, the scrambling of higher bit image and the diffusion of lower bit images. Simulations and performance analyses show that the proposed scheme has the advantages of large key space, strong system sensitivity, and excellent encryption security, especially good resistance to salt and pepper noise attacks.

A high performance full scalable video codec called embedded multiresolution motion-compensated three-dimensional SPIHT (EM-MC-3DSPIHT) is presented. By combining the bit allocation of motion vector and the scalable spatio-temporal orientation wavelet tree with the EM-MC-3DSPIHT algorithm, it not only provides a full scalable decoding, but also significantly improves the rate-distortion (RD) performance compared with that of Kim et al's codec (see IEEE Trans. on Circuit and Syst. Video Tech., vol.10, p.1374-87, Dec. 2000), particularly at low bit rates. Simulations show that the proposed video codec provides performance not only better than that of MPEG-2 at medium to high bit rates but also comparable to H.263 at low bit rates.