Macroblock type selection for compressed domain down-sampling of MPEG video

Digital video databases in compressed form are becoming widely available. In applications such as video browsing, and picture in picture, for a lower bitrate, there is a need to down-sample the video before transmission. The conventional approach to downscale a compressed video sequence is to decompress it at the video server, perform the down-sampling in the pixel domain and then recompress it for efficient delivery. This process is computationally intensive due to the motion estimation process required during the recompression phase. In the alternative compressed domain approach, the motion vectors of the downscaled video sequence are computed directly from the motion vectors of the original full size stream. In this paper we propose a compressed domain technique that generates a better estimate for the downscaled motion vectors. Simulations suggest that the performance achieved with the proposed method is superior by up to 1 dB to the current compressed domain techniques.

Huge number of images is available on the internet. Efficient and effective retrieval system is needed to retrieve these images by the contents or features of the images like color, texture and shape. This system is called content based image retrieval (CBIR). Conventionally features are extracted from images in pixel domain. But at present almost all images are represented in compressed form using DCT (Discrete Cosine Transformation) blocks transformation. Some critical information is removed in compression and only perceptual information is left which has significant attraction for information retrieval in compressed domain. In this paper we study the problem that how to retrieve perceptual information in compressed domain JPEG such that to improve image retrieval. Our approach is based on quantized histogram statistical texture features in DCT blocks. We show that to get best image retrieval performance by extracting the statistical texture features of quantized histogram in DCT blocks using JPEG compressed format images. Experiments on the Corel animal database using the proposed approach, give results which show that the statistical texture features of histogram are robust in retrieval of images. This shows that texture features in local compression is a significant step for effective image retrieval.

The problem of generating textual descriptions for the visual data has gained research attention in the recent years. In contrast to that the problem of generating visual data from textual descriptions is still very challenging, because it requires the combination of both Natural Language Processing (NLP) and Computer Vision techniques. The existing methods utilize the Generative Adversarial Networks (GANs) and generate the uncompressed images from textual description. However, in practice, most of the visual data are processed and transmitted in the compressed representation. Hence, the proposed work attempts to generate the visual data directly in the compressed representation form using Deep Convolutional GANs (DCGANs) to achieve the storage and computational efficiency. We propose GAN models for compressed image generation from text. The first model is directly trained with JPEG compressed DCT images (compressed domain) to generate the compressed images from text descriptions. The second model is trained with RGB images (pixel domain) to generate JPEG compressed DCT representation from text descriptions. The proposed models are tested on an open source benchmark dataset Oxford-102 Flower images using both RGB and JPEG compressed versions, and accomplished the state-of-the-art performance in the JPEG compressed domain. The code will be publicly released at GitHub after acceptance of paper.

A critical evaluation of image and video indexing techniques in the compressed domain

Visual database systems require efficient indexing to enable fast access to the images and video sequences in the database. Since image and video data are voluminous, compression is essential for storage and transmission. The International Standards Organization has proposed the JPEG and MPEG standards for image and video compression, respectively. In order to avoid the unnecessary decompression and recompression operations in the searching process, it is efficient to index the image/video in compressed form. In this paper, we present image and video indexing techniques in the compressed domain.

We propose a fast, yet simple algorithm to find the region of interest (ROI) from a compressed MPEG video bitstream, with partial decoding. We have used the properties of the human visual system (HVS), applied in the DCT domain, to extract the ROIs. Though a lot of work has been done to obtain ROIs in images based on the HVS, all these systems work in the pixel domain. However, finding the ROIs in the compressed domain has wide applications such as emphasizing perceptually important regions while transrating, defining descriptors for encoded video (MPEG-7), changing image size to adapt to heterogeneous client displays, etc.

Accurate and efficient video quality assessment (VQA) methods are important guidelines for optimizing network video quality, improving video compression performance, and recommending compression coding parameters. No-reference VQA methods are mainly divided into bitstream-based methods, which are less time-consuming but less accurate, and pixel-based methods, which are more accurate but more time-consuming and require high hardware requirements. Both methods are not applicable to the quality assessment of a large number of network videos. To address the above problem, we propose an efficient and accurate cross-domain no-reference network video quality assessment method (CDNRVQA) based on bitstream. CDNRVQA takes I-frames as representative frames and uses deep neural network to obtain video high semantic features and distortion features from the representative frames, then obtains temporal features and some spatial features of the video based on the macroblock and motion vector in the compressed videos, and uses feature fusion strategy to fuse features from pixel domain and compressed domain, and finally obtains comprehensive cross-domain features which can represent the video distortion brought by content, motion and compression. CDNRVQA performs well on large VQA datasets. CDNRVQA achieves similar performance to the state-of-the-art VQA models, but greatly improves prediction time, and greatly reduces hardware consumption on the device. CDNRVQA makes it possible to apply VQA methods to video platforms.

The video object mechanism is based on the appearance representation of the object. Tracking objects in a video has applications in video surveillance and other some applications. In previous method does not handle the case of full occlusion. The case of full occlusion can be handled by setting an LSK similarity threshold, which stops tracking when the object is lost. The objects can be tracked based on the compression domain and pixel domain. The spatial information of the object that is being tracked is retrieved using Local Steering Kernels. The Color features are extracted using color histograms. The features similarities between a candidate object ROI and the object ROI in the previous frame and the last stored object instance in the object model are measured. For extracting the foreground extended Kalman filter is used. This increases the tracking efficiency of the algorithm. Index Terms: Color histograms, localsteeringkernels, and visualobjecttracking.

A procedure for combined image watermarking and compression, based on the Hermite projection method is proposed. The Hermite coefficients obtained by using the Hermite expansion are used for watermark embedding. The image can be efficiently reconstructed by using a set of Hermite coefficients that is quite smaller than the number of original ones. Hence, the watermark embedding is actually done in the compressed domain, while maintaining still high image quality (measured by high PSNR). The efficiency of the proposed procedure is proven experimentally, showing high robustness even for very strong standard attacks. Moreover, the method is robust not only to the standard attacks, but to the geometrical attacks, as well. The proposed approach can be suitable for different copyright and ownership protection purposes, especially in real-applications that require image compression, such as multimedia and Internet applications, remote sensing and satellite imaging.

We present a new compressed domain method for tracking objects in airborne videos. In the proposed scheme, a statistical snake is used for object segmentation in I-frames, and motion vectors extracted from P-frames are used for tracking the object detected in I-frames. It is shown that the energy function of the statistical snake can be obtained directly from the compressed DCT coefficients without the need of full decompression. The number of snake deformation iterations can be also significantly reduced in compressed domain implementation. The computational cost is significantly reduced by using compressed domain processing while the performance is competitive to that of pixel domain processing. The proposed method is tested using several UAV video sequences, and experiments show that the tracking results are satisfactory.

While much progress has been achieved in the field of content-based image retrieval (CBIR), almost all CBIR techniques operate on pixel data although virtually all images are stored in compressed form. In this invited paper, we present efficient and effective CBIR techniques that operate directly in the compressed domain and thus do not require full decompression for feature extraction. In particular, we explore compressed domain techniques for JPEG images and show how CBIR features can be extracted from DCT coefficients, from differentially coded DC data, and from optimised Huffman and quantisation tables that are stored in the JPEG headers.

Many display devices now allow a number of colors, called color palette, to be displayed simultaneously. Besides, images and videos in most world wide web (WWW) databases are in compressed formats. Therefore, it becomes an important issue to retrieve a suitable color palette from compressed domain in order to have fast and faithful color reproduction for these devices. In this paper, the color palette design methods for compressed videos are presented. The proposed approaches use the reduced image rather than the whole image for the color palette design to avoid the heavy computation in video decompression. Experimental results show that output image quality of proposed methods is acceptable, to human eyes. In addition, empirical results show that the proposed shifting window scheme can reduce the main problem of displaying quantized image sequences and screen flicker.

Storing and processing Remote Sensing (RS) images require large amounts of memory space and computing resources. Consequently, RS images are compressed and stored in various compression formats, such as JPEG2000. However, the processing of RS images for machine interpretation and understanding still necessitates the deployment of an image decompression stage in its entirety, followed by a computationally demanding image analysis pipeline. The image analysis stage is commonly composed of machine learning techniques, such as Deep Convolutional Neural Network (DCNN) models. Classification of remote sensing images is among the most common image analysis tasks. In the scope of this paper, we propose a sub-band image based classification method for the Remote Sensing Scene Classification (RSSC) task in the JPEG2000 compressed domain. The proposed approach exploits the already available sub-band image coefficients to classify RS images without needing for full decompression. Our study shows that our method increases the high frequency information in the LL sub-band and allows the image to contain more detail, leading to improved classifier performance while taking advantage of the partial decompression method.

Background models have been widely used for video surveillance and other applications. Methods for constructing background models and associated application algorithms are mainly studied in spatial domain (pixel level). Many video sources, however, are in a compressed format before processing. In this paper, we propose an approach to construct background models directly from compressed video data. The proposed approach utilizes the information exploited from DCT coefficients at block level to construct accurate background models at pixel level. We implemented three representative algorithms of background models in the compressed domain, and theoretically explored their properties and the relationship with their counterparts in the spatial domain. We also present some general technical improvements to make the proposed approach more capable for a wide range of applications. The proposed method can achieve the same accuracy as the methods that construct background models from the spatial domain with much lower computational cost (26 % on average) and more compact storages. 1.

In this paper, we propose a video extraction algorithm directly in the compressed domain for low cost and fast content access to those compressed video data via MPEG. Extensive experiments show that such extracted images and videos not only maintain well-preserved content features, but also illustrate reasonable quality in terms of both PSNR values and visual inspection. In cases where video processing tasks do not necessarily require full resolution pixel data such as browsing, pattern recognition, and object tracking in surveillance applications, the proposed algorithm will provide superior performance in terms of computing efficiency, under the context that millions of video frames need to be accessed, yet they are stored in compressed format.