Real-Time Adaptive Image Compression

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that offers good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications, achieving essentially the same levels of rate-distortion performance and subjective quality as that of the full search algorithm when used by the UBC H.263+ video coding reference software. However the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.

We present a semi-fixed-length motion vector coding method for H.263-based low bit rate video compression. The method exploits structural constraints within the motion field. The motion vectors are encoded using semi-fixed-length codes, yielding essentially the same levels of rate-distortion performance and subjective quality achieved by H.263's Huffman-based variable length codes in a noiseless environment. However, such codes provide substantially higher error resilience in a noisy environment.

Volumetric video, which is six-degrees-of-freedom (6D0F), serves an immersive viewing experience. However, the high bandwidth required for streaming volumetric video over networks presents challenges for its flourishing. To address this issue, we propose a novel streaming system that leverages depth image to convert 3D volumetric videos into 2D videos with low bitrate and high fps. Our system addresses two technical challenges when using depth image. Specifically, we use nonlinear regression to solve the virtual camera placement problem and the problem of how to allocate bitrate between color image and depth map when encoding. Evaluations show that our system outperforms existing volumetric video streaming methods and saves 50%-99% bandwidth under the same level of quality.

In hierarchical coding, image data are encoded in such a way that it is possible to access a given image at different quality levels or resolutions. For example, in searching an image database, hierarchical coding allows the user to initially access a low-quality version of an image (at a correspondingly low bit rate) in order to determine if the image is the desired one. Additional data can then be transmitted in stages to further refine the image. This type of scheme is termed progressive transmission. As another example, an image database might be used to support a number of output devices, each having a different resolution. A hierarchical coding scheme allows for each device to efficiently access a version of the image with the appropriate resolution. This is an example of a multiuse environment. We now discuss in more detail the concepts and desired characteristics of schemes for progressive transmission and multiuse environments. This is followed by descriptions of various image hierarchies that are useful for such applications.

Image compression, which is a type of data compression applied to digital images, has been a fundamental research topic for many decades. Recent image techniques produce very large amounts of data, which may make it prohibitive to storage and communications of image data without the use of compression. However, the traditional compression methods, such as JPEG, may introduce the compression artefact problems. Recently, deep learning has achieved great success in many computer vision tasks and is gradually being used in image compression. To solve the compression atrefact problem, in this paper, we present a lossy image compression architecture, which utilizes the advantages of the existing deep learning methods to achieve a high coding efficiency. We design a densely connected autoencoder structure for lossy image compression. Firstly, we design a densely autoencoder structure to get richer feature information from image which can be helpful for compression. Secondly, we design a U-net like network to decrease the distortion caused by compression. Finally, an improved binarizer is adopted to quantize the output of encoder. In low bit rate image compression, experiments show that our method significantly outperforms JPEG and JPEG2000 and can produce a better visual result with sharp edges, rich textures, and fewer artifacts.

Lossy image compression is generally formulated as a joint rate-distortion optimization problem to learn encoder, quantizer, and decoder. Due to the non-differentiable quantizer and discrete entropy estimation, it is very challenging to develop a convolutional network (CNN)-based image compression system. In this paper, motivated by that the local information content is spatially variant in an image, we suggest that: (i) the bit rate of the different parts of the image is adapted to local content, and (ii) the content-aware bit rate is allocated under the guidance of a content-weighted importance map. The sum of the importance map can thus serve as a continuous alternative of discrete entropy estimation to control compression rate. The binarizer is adopted to quantize the output of encoder and a proxy function is introduced for approximating binary operation in backward propagation to make it differentiable. The encoder, decoder, binarizer and importance map can be jointly optimized in an end-to-end manner. And a convolutional entropy encoder is further presented for lossless compression of importance map and binary codes. In low bit rate image compression, experiments show that our system significantly outperforms JPEG and JPEG 2000 by structural similarity (SSIM) index, and can produce the much better visual result with sharp edges, rich textures, and fewer artifacts.

Hyperspectral image compression has become increasingly important in data exploitation because of enormous data volumes and high redundancy provided by hundreds of contiguous spectral channels. Since a hyperspectral image can be viewed as a 3-dimensional (3D) image cube, many efforts have been devoted to extending 2D image compression techniques to perform 3D image compression on hyperspectral image cubes. Unfortunately, some major issues generally encountered in hyperspectral data exploitation at low or very low-bit rate compression, for example, subpixels and mixed pixels which do not occur in traditional pure pixel-based image compression are often overlooked in such a 2D-to-3D compression. Accordingly, a direct application of 2D-to-3D compression techniques to hyperspectral image cubes without taking precaution may result in significant loss of crucial spectral information provided by subtle substances such as small objects, anomalies during low bit-rate lossy compression. This paper takes a rather different view by investigating lossy hyperspectral compression from a perspective of exploring spectral information, referred to as exploitation-based lossy compression and further develops spectral/spatial hyperspectral image compression to effectively preserve crucial and vital spectral information of objects which are generally missed by commonly used mean-squared error (MSE) or signal-to-noise ratio (SNR)-based compression techniques when lossy compression is performed at low bit rates. In order to demonstrate advantages of the proposed spectral/spatial compression approach applications of subpixel target detection and mixed pixel analysis are used for experiments for performance evaluation.

As the demand for higher quality and higher resolution video increases, many applications fail to meet this demand due to low bandwidth restrictions. One factor contributing to this problem is the high bitrate requirement of the intra-coded Instantaneous Decoding Refresh (IDR) frames featuring in all video coding standards. Frequent coding of IDR frames is essential for error resilience in order to prevent the occurrence of error propagation. However, as each one consumes a huge portion of the available bitrate, the quality of future coded frames is hindered by high levels of compression. This work presents a new technique, known as Spatial Resampling of IDR Frames (SRIF), and shows how it can increase the rate distortion performance by providing a higher and more consistent level of video quality at low bitrates.

In the era of intelligent media, we all need high quality image or video information. However, due to the limitation of bandwidth or storage resource, we usually use low bit rate coding technology for image compression. But the image distortion caused will bring in a visually inferior experience, and the actual information obtained is often not sufficient. Therefore, the super-resolution reconstructed image coding technology is researched to provide a higher dynamic bit rate variation range. Based on image super-resolution reconstruction, this paper proposes two targeted image compression coding frameworks. Firstly, we compress the DCT down-sampled image as the input of super-resolution reconstruction, and use image completion as the pretreatment process of image super-resolution reconstruction, which effectively improves the quality of reconstructed image at low bit rate and, hence improves the coding efficiency. Secondly, a new image compression technique is proposed here which is based on residual compensation and super-resolution reconstruction. In the encoder, the super-resolution reconstructed image is subtracted from the original image to obtain the residual image which is compressed. In the decoder, the residual image is decompressed and added to the reconstructed image to complete the decoding process which achieves flexible rate control.

This paper proposes a new vector quantization based (VQ-based) technique for very low bit rate encoding of multispectral images. We rely on the assumption that the shape of a generic spatial block does not change significantly from band to band, as is the case for high spectral-resolution imagery. In such a hypothesis, it is possible to accurately quantize a three-dimensional (3-D) block-composed of homologous two-dimensional (2-D) blocks drawn from several bands-as the Kronecker-product of a spatial-shape codevector and a spectral-gain codevector, with significant computation saving with respect to straight VQ. An even higher complexity reduction is obtained by representing each 3-D block in its minimum-square-error Kronecker-product form and by quantizing the component shape and gain vectors. For the block sizes considered, this encoding strategy is over 100 times more computationally efficient than unconstrained VQ, and over ten times more computationally efficient than direct gain-shape VQ. The proposed technique is obviously suboptimal with respect to VQ, but the huge complexity reduction allows one to use much larger blocks than usual and to better exploit both the statistical and psychovisual redundancy of the image. Numerical experiments show fully satisfactory results whenever the shape-invariance hypothesis turns out to be accurate enough, as in the case of hyperspectral images. In particular, for a given level of complexity and image quality, the compression ratio is up to five times larger than that provided by ordinary VQ, and also larger than that provided by other techniques specifically designed for multispectral image coding.

A novel Adaptive Lossy Image Compression (ALIC) technique is proposed to achieve high compression ratio by reducing the number of source symbols through the application of an efficient technique. The proposed algorithm is based on processing the discrete cosine transform (DCT) of the image to extract the highest energy coefficients in addition to applying one of the novel quantization schemes proposed in the present work. This method is straightforward and simple. It does not need complicated calculation; therefore the hardware implementation is easy to attach. Experimental comparisons are carried out to compare the performance of the proposed technique with those of other standard techniques such as the JPEG. The experimental results show that the proposed compression technique achieves high compression ratio with higher peak signal to noise ratio than that of JPEG at low bit rate without the visual degradation that appears in case of JPEG.

With the growing importance of low-bandwidth applications, such as wireless access to the Internet, images are often sent or received at low bit rates. At these bit rates, they suffer from significant distortion and artifacts, making it difficult for those viewing the images to understand them. We present two progressive compression algorithms that focus on preserving the clarity of important image features, such as edges, at compression ratios of 80:1 and more. Both algorithms capture and encode the locations of important edges in the images. The first algorithm then transmits a standard SPIHT (set partitioning in hierarchical trees) bit stream, and at the decoder applies a nonlinear edge-enhancement procedure to improve the clarity of the encoded edges. The second approach uses a modified wavelet transform to "remove" the edges, and encodes the remaining texture information using SPIHT. With both approaches, features in the images that may be important for recognition are well preserved, even at low bit rates.

Image and video compression become very popular and intense field in recent decades, due to fast and high quality communication demand of visual information. Currently, JPEG and JPEG2000 are the most popularly used codec's for achieving image compression. Practically, on the low bit rate compression, it has been observed that the JPEG compressed images suffer from multifarious visual distortions. To address this problem effectively, this paper proposed an innovative Type-2 fuzzy directed hybrid post filtering technique, which is framed to suppress the artifacts generated in JPEG compressed images at low bit rate compression. The proposed technique addresses all three types of JPEG image artifacts: blocking, edges blurring, and aliasing. Furthermore, the proposed technique is structured with two stages to enhance the quality of JPEG compressed images. The first stage removes blocking artifacts using boundary smoothing and guided filtering techniques. The second stage reduces blurring and aliasing around the edges through Type-2 fuzzy directed local edge regeneration. To prove higher artifacts reduction efficiency of proposed work, a complete comparative performance evaluation with other existing JPEG artifact removal techniques has been also presented. This extensive performance analysis includes visual quality assessment of post-filtered images along with a subjective quality assessment on the basis of Peak Signal to Noise Ratio (PSNR) parameter. Performance evaluation illustrates that the proposed approach is efficient; provides PSNR improvement in the range of 0.2 to 1.1 dB as compared to the state of the art algorithms for all bitrates.

The goal of this work is twofold. First, we demonstrate that an advantage can be gained by using local cosine bases over wavelets to encode images that contain periodic textures. We designed a coder that outperforms one of the best wavelet coders on a large number of images. The coder finds the optimal segmentation of the image in terms of local cosine bases. The coefficients are encoded using a scalar quantizer optimized for Laplacian distributions. This new coder constitutes the first concrete contribution of the paper. Second, we used our coder to perform an extensive comparison of several optimized bells in terms of rate-distortion and visual quality for a large collection of images. This study provides for the first time a rigorous evaluation in realistic conditions of these bells. Our experiments show that bells that are designed to reproduce exactly polynomials of degree 1 resulted in the worst performance in terms of the PSNR. However, a visual inspection of the compressed images indicates that these bells often provide reconstructed images with very few visual artifacts, even at low bit rates. The bell with the most narrow Fourier transform gave the best results in terms of the PSNR on most images. This bell tends however to create annoying visual artifacts in very smooth regions at low bit rate.

The past few years have seen an increasing interest in using reversible integer wavelets in image compression. Reversible integer wavelet image coders facilitate decompression from low bit rates all the way up to lossless reconstruction. However, in the past, specific implementations of such techniques, like S+P, could not match the lossless compression performance of state-of-the-art predictive coding techniques like CALIC. We demonstrate for the first time that reversible integer wavelets together with proper context modeling can match the lossless compression performance of CALIC. This can be done without increase in the essential complexity over S+P. Our findings present a strong argument for using subband coding as a unified, elegant approach for both lossy and lossless image compression. Specifically, in this paper we outline how to obtain significantly higher coding efficiency over S+P by utilizing better filters and better context modeling and entropy coding of wavelet coefficients.