Image/video indexing in the compressed domain

A critical evaluation of image and video indexing techniques in the compressed domain

In the paper the features of protocols and the basic methods of organization of multimedia information transfer in telecommunication networks and distributed corporate safety networks are represented. The analysis of formats of basic elements of signals used in standards of multimedia information transfer is given.

In the technological era, the primary source of information is in the form of digital data, which has to be secured while storing or transmitting during communication over an unsecured network. Different approaches are used to provide security to digital data, viz. text, audio, image, and video. This paper initially explains the security system such as cryptography, watermarking, and steganography and their comparative analysis based on different characteristics, viz. satisfaction level of objective, type of carrier object and secret information to be used, dependency of security level, and quality assessment parameters. This review article focuses more on steganography methods applied over video. The various methods implemented for video steganography in compressed domain, viz. inter-frame and intra-frame prediction, motion vector estimation, entropy coding (CAVLC and CABAC), and transformed and quantized coefficients of DCT, DST, and DWT, etc. and the methods based on spatial and transform domain for uncompressed video are briefly described. It is followed by the detailed analysis of related work done by various researchers in video steganography and the obtained experimental results. Furthermore, the confidential data hiding in compressed videos are explained using Moving Picture Expert Group (MPEG—1, MPEG—2, MPEG—4), Advanced Video Coding (AVC)/H.264, and High-Efficiency Video Coding (HEVC)/H.265 that includes both spatial and transform domain. This paper summarizes and explains the detailed investigations of numerous techniques of video steganography based on the comprehensive literature survey. The methods used to assess the performance of video steganography are analyzed based on the quality assessment parameters such as imperceptibility; measured by peak signal to noise ratio (PSNR), mean square error (MSE), and structural similarity (SSIM), robustness; measured by bit error rate (BER) and similarity (Sim), and embedding capacity; measured by hiding ratio. The overall review of past literature facilitates to have in-depth knowledge for upgrading the video steganography.

Digital video databases in compressed form are becoming widely available. In applications such as video browsing, and picture in picture, for a lower bitrate, there is a need to down-sample the video before transmission. The conventional approach to downscale a compressed video sequence is to decompress it at the video server, perform the down-sampling in the pixel domain and then recompress it for efficient delivery. This process is computationally intensive due to the motion estimation process required during the recompression phase. In the alternative compressed domain approach, the motion vectors of the downscaled video sequence are computed directly from the motion vectors of the original full size stream. In this paper we propose a compressed domain technique that generates a better estimate for the downscaled motion vectors. Simulations suggest that the performance achieved with the proposed method is superior by up to 1 dB to the current compressed domain techniques.

Chapter 1 - Introduction to compression

Digital video databases in compressed form are becoming widely available. In applications such as video browsing, and picture in picture, for a lower bitrate there is a need to down-sample the video before transmission. The conventional approach to downscale a compressed video sequence is to decompress it at the video server, perform the down-sampling in the pixel domain and recompress it again for efficient delivery. This process is computationally expensive due to the motion estimation process needed during the recompression phase. In the alternative compressed domain based approach the downscaled motion vectors are computed directly from the original ones. We propose a method to improve the process of downscaling in the compressed domain by better selecting the motion prediction type of P and B frame macroblocks. Simulations suggest that the proposed scheme improves the compressed domain down-sampling quality by up to 2.2 dB without any substantial increase in computation.

Chapter 16 - Video Communication Networks

Current video compression standards compress video sequences at NTSC quality with factors in the range of 10-100, like in MPEG-1 and MPEC-2. To operate beyond this range, that is, MPEG-4, radically new techniques are needed. We discuss one such technique called generative video (GV). Video compression is realized in GV in two steps. First, the video sequence is reduced to constructs. Constructs are world images, corresponding to augmented images containing the non-redundant information on the sequence, and window, figure, motion, and signal processing operators, representing video sequence properties. Second, the world images are spatially compressed. The video sequence is reconstructed by applying the various operators to the decompressed world images. We apply GV to a 10 sec video sequence of a real 3-D scene and obtain compression ratios of about 2260 and 4520 for two experiments done with different quantization codebooks. The reconstructed video sequence exhibits very good perceptual quality.

Due to the perceived inadequacy of current standards for lossless image compression, the JPEG committee of the International Standards Organization (ISO) has been developing a new standard. A baseline algorithm, called JPEG-LS, has already been completed and is awaiting approval by national bodies. The JPEG-LS baseline algorithm despite being simple is surprisingly efficient, and provides compression performance that is within a few percent of the best and more sophisticated techniques reported in the literature. Extensive experimentations performed by the authors seem to indicate that an overall improvement by more than 10 percent in compression performance will be difficult to obtain even at the cost of great complexity; at least not with traditional approaches to lossless image compression. However, if we allow inter-band decorrelation and modeling in the baseline algorithm, nearly 30 percent improvement in compression gains for specific images in the test set become possible with a modest computational cost. In this paper we propose and investigate a few techniques for exploiting inter-band correlations in multi-band images. These techniques have been designed within the framework of the baseline algorithm, and require minimal changes to the basic architecture of the baseline, retaining its essential simplicity.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The compressed domain video watermarking has been less expensive since complete decoding and then re-encoding is not required for watermark embedding. Handling synchronization error due to embedding is a challenging task, especially, due to motion compensation. In this paper, we propose an embedding scheme that can minimize desynchronization in the watermark extraction process at the decoder for embedding in compressed videos with better visual quality. Experimental results show the effectiveness of the proposed scheme.

While much progress has been achieved in the field of content-based image retrieval (CBIR), almost all CBIR techniques operate on pixel data although virtually all images are stored in compressed form. In this invited paper, we present efficient and effective CBIR techniques that operate directly in the compressed domain and thus do not require full decompression for feature extraction. In particular, we explore compressed domain techniques for JPEG images and show how CBIR features can be extracted from DCT coefficients, from differentially coded DC data, and from optimised Huffman and quantisation tables that are stored in the JPEG headers.

Video communication requires sufficient resources to obtain the desired image quality. Availability of bandwidth and screen resolution from different devices on the client end, as well as compression techniques used for transmission also significantly influence to the video quality. Even though the H.264/AVC video compression standard and its scalable extension is known as Scalable Video Coding (SVC) was introduced and successfully addressed the challenge, some weaknesses persist especially for high-resolution video transmissions. The size of the bitstream generated by the SVC standard for ultra-high resolution video compression can only be met if transmitted on a network with a large bandwidth. The Scalable High-Efficiency Video Coding (SHVC) were introduced to overcome problems in the SVC standards. The new video compression standard is proven to reduce bandwidth requirements by up to 46.41 % compared to the SVC standards for the same video quality. This study introduces the performance analysis of SHVC standard video transmission as compared to SVC standard. Both compression standards are transmitted and emulated through network emulator called Mininet-Wifi to get the real transmission picture. The client-server video transmission scheme with different bandwidth and various client position are set in the experiment scenario. It is found that transmission time, packet loss, and video reconstruction quality on scalable video transmission with SHVC standard is better than scalable video transmissions with SVC standard.

Practically, action recognition using deep learning approaches are slow because of high temporal redundancy and large size of the raw video data. One of the solutions for boosting accuracy is calculating optical flows. Generally, extracting motion features are too time-consuming. Therefore, traditional action recognition methods are not suitable for real time applications. On the other hand, compressed videos are available in many situations especially when using mobile devices. We proposed a method that extracts residuals directly from compressed videos by partially decoding the video and feed them to a deep neural network. In general, exploiting the compressed domain features as available information provides a slight reduction in accuracy while the low complexity of this method makes it appropriate for real time applications. The experimental results on multiple first and third person datasets exhibit that while the proposed method provides low computational complexity, the results are highly competitive with traditional approaches in accuracy.

In spite of the recent advances in telecommunication standards, communication networks still have limited bandwidths and storage capacity. Therefore, video compression has drawn increasing importance since high-resolution video contents have become more and more used in various fields. These requirements raise the need for high-performance video-compression technologies able to reduce the amount of data to be transmitted or stored by compressing the input video signal into a bitstream file. Improving the coding efficiency was always one of the crucial issues of various compression standards that aim to get the most compact representation of the reconstructed video, with a high subjective quality. The high-efficiency video coding (HEVC) comes to respond to these requirements. However, the increased consumption of high-quality multimedia content has pushed the international communication companies to put much effort to better enhance video-coding techniques. In this perspective, an upcoming video-coding standard to be known as versatile video coding (VVC) has emerged aiming to improve the coding efficiency of the current HEVC codec. Improvements on rate distortion (RD) performance that came with both HEVC and VVC have brought an increased complexity in the majority of the coding modules, which makes it difficult to implement on hardware systems with real-time encoding. This chapter focuses on the transform coding stage as one of the most computationally demanding modules. For HEVC, efficient approximation algorithms in addition to reconfigurable and scalable architecture have been developed in order to decrease the computational complexity of the transform module. The main objectives are to meet low power and real-time processing constraints while maintaining a compression gain and a satisfying video quality. Field programmable gate array (FPGA) implementation results and comparisons with existing works confirm the efficiency of the proposed approximations since they contribute in reducing time and power consumption, optimizing the hardware resources and bringing peak signal-to noise ratio (PSNR) improvement as well. Similarly, for the adaptive multiple transform (AMT) introduced in the transform module of the VVC, approximations were done for discrete cosine transform (DCT)-II and discrete sine transform (DST)-VII transforms since they are statistically the most used ones among the five predefined types. Bitrate (BR) reduction with a slightly degradation of video quality and a less use of hardware resources are the main contributions of the proposed approximations.

Usually, it is difficult to perform compression in video files as it significantly affects the perceptible quality. This paper has reviewed various works of literature toward finding unique approaches to video compression to find that the existing techniques offer a sophisticated algorithm for implementation with less focus on the visual quality of the decoded contents. Hence, this paper presents a model that uses graph theory to perform video compression compatible with High-Efficiency Video Coding (HEVC) standards of compression. By adoption of the Lagrangian theorem, the proposed algorithm finds the best solution for compression and offers better compression standards. The outcomes are compared with existing H.264 and existing research work to find that the proposed system excels better with respect to PSNR and bit rate and thereby offers better compatibility to compress High Definition (HD) videos.