Towards Practical Compressed Video Action Recognition: A Temporal Enhanced Multi-Stream Network

The compressed domain video watermarking has been less expensive since complete decoding and then re-encoding is not required for watermark embedding. Handling synchronization error due to embedding is a challenging task, especially, due to motion compensation. In this paper, we propose an embedding scheme that can minimize desynchronization in the watermark extraction process at the decoder for embedding in compressed videos with better visual quality. Experimental results show the effectiveness of the proposed scheme.

Video compression has great significance in the communication of motion pictures. Video compression techniques try to remove the different types of redundancy within or between video sequences. In the temporal domain, the video compression techniques remove the redundancies between the highly correlated consequence frames of the video. In the spatial domain, the video compression techniques remove the redundancies between the highly correlated consequence pixels (samples) in the same frame. Evolving neural-networks based video coding research efforts are focused on improving existing video codecs by performing better predictions that are incorporated within the same codec framework or holistic methods of end-to-end video compression schemes. Current neural network-based video compression adapts static codebook to achieve compression that leads to learning inability from new samples. This paper proposes a modified video compression model that adapts the genetic algorithm to build an optimal codebook for adaptive vector quantization that is used as an activation function inside the neural network’s hidden layer. Background subtraction algorithm is employed to extract motion objects within frames to generate the context-based initial codebook. Furthermore, Differential Pulse Code Modulation (DPCM) is utilized for lossless compression of significant wavelet coefficients; whereas low energy coefficients are lossy compressed using Learning Vector Quantization (LVQ) neural networks. Finally, Run Length Encoding (RLE) is engaged to encode the quantized coefficients to achieve a higher compression ratio. Experiments have proven the system’s ability to achieve higher compression ratio with acceptable efficiency measured by PSNR.

Practically, action recognition using deep learning approaches are slow because of high temporal redundancy and large size of the raw video data. One of the solutions for boosting accuracy is calculating optical flows. Generally, extracting motion features are too time-consuming. Therefore, traditional action recognition methods are not suitable for real time applications. On the other hand, compressed videos are available in many situations especially when using mobile devices. We proposed a method that extracts residuals directly from compressed videos by partially decoding the video and feed them to a deep neural network. In general, exploiting the compressed domain features as available information provides a slight reduction in accuracy while the low complexity of this method makes it appropriate for real time applications. The experimental results on multiple first and third person datasets exhibit that while the proposed method provides low computational complexity, the results are highly competitive with traditional approaches in accuracy.

Many advanced video applications require manipulations of compressed video signals. Popular video manipulation functions include overlap (opaque or semitransparent), translation, scaling, linear filtering, rotation, and pixel multiplication. We propose algorithms to manipulate compressed video in the compressed domain. Specifically, we focus on compression algorithms using the discrete cosine transform (DCT) with or without motion compensation (MC). Such compression systems include JPEG, motion JPEG, MPEG, and H.261. We derive a complete set of algorithms for all aforementioned manipulation functions in the transform domain, in which video signals are represented by quantized transform coefficients. Due to a much lower data rate and the elimination of decompression/compression conversion, the transform-domain approach has great potential in reducing the computational complexity. The actual computational speedup depends on the specific manipulation functions and the compression characteristics of the input video, such as the compression rate and the nonzero motion vector percentage. The proposed techniques can be applied to general orthogonal transforms, such as the discrete trigonometric transform. For compression systems incorporating MC (such as MPEG), we propose a new decoding algorithm to reconstruct the video in the transform domain and then perform the desired manipulations in the transform domain. The same technique can be applied to efficient video transcoding (e.g., from MPEG to JPEG) with minimal decoding. >

Visual database systems require efficient indexing to enable fast access to the images and video sequences in the database. Since image and video data are voluminous, compression is essential for storage and transmission. The International Standards Organization has proposed the JPEG and MPEG standards for image and video compression, respectively. In order to avoid the unnecessary decompression and recompression operations in the searching process, it is efficient to index the image/video in compressed form. In this paper, we present image and video indexing techniques in the compressed domain.

In the technological era, the primary source of information is in the form of digital data, which has to be secured while storing or transmitting during communication over an unsecured network. Different approaches are used to provide security to digital data, viz. text, audio, image, and video. This paper initially explains the security system such as cryptography, watermarking, and steganography and their comparative analysis based on different characteristics, viz. satisfaction level of objective, type of carrier object and secret information to be used, dependency of security level, and quality assessment parameters. This review article focuses more on steganography methods applied over video. The various methods implemented for video steganography in compressed domain, viz. inter-frame and intra-frame prediction, motion vector estimation, entropy coding (CAVLC and CABAC), and transformed and quantized coefficients of DCT, DST, and DWT, etc. and the methods based on spatial and transform domain for uncompressed video are briefly described. It is followed by the detailed analysis of related work done by various researchers in video steganography and the obtained experimental results. Furthermore, the confidential data hiding in compressed videos are explained using Moving Picture Expert Group (MPEG—1, MPEG—2, MPEG—4), Advanced Video Coding (AVC)/H.264, and High-Efficiency Video Coding (HEVC)/H.265 that includes both spatial and transform domain. This paper summarizes and explains the detailed investigations of numerous techniques of video steganography based on the comprehensive literature survey. The methods used to assess the performance of video steganography are analyzed based on the quality assessment parameters such as imperceptibility; measured by peak signal to noise ratio (PSNR), mean square error (MSE), and structural similarity (SSIM), robustness; measured by bit error rate (BER) and similarity (Sim), and embedding capacity; measured by hiding ratio. The overall review of past literature facilitates to have in-depth knowledge for upgrading the video steganography.

Fractal compression is the lossy compression technique in the field of gray/color image and video compression. It gives high compression ratio, better image quality with fast decoding time but improvement in encoding time is a challenge. This review paper/article presents the analysis of most significant existing approaches in the field of fractal based gray/color images and video compression, different block matching motion estimation approaches for finding out the motion vectors in a frame based on inter-frame coding and intra-frame coding i.e. individual frame coding and automata theory based coding approaches to represent an image/sequence of images. Though different review papers exist related to fractal coding, this paper is different in many sense. One can develop the new shape pattern for motion estimation and modify the existing block matching motion estimation with automata coding to explore the fractal compression technique with specific focus on reducing the encoding time and achieving better image/video reconstruction quality. This paper is useful for the beginners in the domain of video compression.

Color correction needs to be performed to improve the quality of image/video production. The typical methods realize the color correction mainly in the spatial domain of RGB color space. In this paper, a linear color correction method in JPEG/MPEG-2 compressed domain is proposed. The correction is realized in the DCT domain of YUV color space without full-frame decompression. Experimental results show that the visual quality of the corrected images/videos in the compressed domain is identical to the quality of the images/videos corrected in the uncompressed domain.

Accurate and efficient video quality assessment (VQA) methods are important guidelines for optimizing network video quality, improving video compression performance, and recommending compression coding parameters. No-reference VQA methods are mainly divided into bitstream-based methods, which are less time-consuming but less accurate, and pixel-based methods, which are more accurate but more time-consuming and require high hardware requirements. Both methods are not applicable to the quality assessment of a large number of network videos. To address the above problem, we propose an efficient and accurate cross-domain no-reference network video quality assessment method (CDNRVQA) based on bitstream. CDNRVQA takes I-frames as representative frames and uses deep neural network to obtain video high semantic features and distortion features from the representative frames, then obtains temporal features and some spatial features of the video based on the macroblock and motion vector in the compressed videos, and uses feature fusion strategy to fuse features from pixel domain and compressed domain, and finally obtains comprehensive cross-domain features which can represent the video distortion brought by content, motion and compression. CDNRVQA performs well on large VQA datasets. CDNRVQA achieves similar performance to the state-of-the-art VQA models, but greatly improves prediction time, and greatly reduces hardware consumption on the device. CDNRVQA makes it possible to apply VQA methods to video platforms.

We study error control for an MPEG2 digital video compression scheme. The compressed video is transmitted over an asynchronous transfer mode (ATM) network. We propose to combine error control coding with error concealment. Specifically we use classical error detection to locate errors in the compressed domain, translate this information to the uncompressed domain and then use standard error concealment in the error locations. The solving of the error location problem in this way is the main contribution of this work.

Due to the broad bandwidth nature of the direct sequence spread spectrum (DSSS) signal, the application of narrowband interference (NBI) cancelation algorithms based on the Nyquist‐Shannon sampling theory are limited by the high sampling rate. To solve the sampling and processing difficulty in the traditional DSSS communication interference cancelation method, we used the compressive sensing (CS) to reduce the sampling rate of the DSSS communication system, and further proposed a compressed domain NBI elimination method in this study. We proved that NBI with a certain bandwidth and DSSS signal are both sparse in specific dictionaries and proposed the corresponding sparse dictionary construction method. Detailed analysis of the compressed domain features of NBI and the DSSS signal demonstrated that it is possible to extract the strong NBI components from the compressed measurements. Two compressed domain NBI cancelation algorithms were proposed based on this idea. The extracted NBI components were filtered out from the compressed measurements directly in the first algorithm, and then we achieved the DSSS signal demodulation in compressed domain using the separated DSSS signal components with the orthogonal matching pursuit (OMP) algorithm. A proposed extraction filter block sparse Bayesian learning (EFBSBL) framework algorithm was utilized to estimate the NBI from the compressed measurements in the second algorithm. The estimated NBI was eliminated from the received signal, and then we recompressed the DSSS signal after NBI cancelation and realized the DSSS signal demodulation in compressed domain with the OMP algorithm. Simulation results support our theoretical analysis results and reveal that the two proposed algorithms are both effective for NBI cancelation and significant interference cancelation performance are achieved.

Objectives. The widespread use of systems for capturing light fields is due to the high quality of the reproduced image. This type of capture, although qualitatively superior to traditional methods to capturing volumetric images, generates a huge amount of data needed to reconstruct the original captured 4D light field. The purpose of the work is to consider traditional and extended to four-dimensional image compression algorithms, to perform a comparative analysis and determine the most suitable.Methods. Mathematical methods of signal processing and methods of statistical analysis are used.Results. Algorithms are compared and analyzed in relation to the compression of four-dimensional light fields using the PSNR metric. The selected evaluation criterion is affected not only by the dimension of the compression algorithm, but also by the distance of the baseline of the capture setting, since the difference between images increases with the distance between the optical centers of each camera matrix. Thus, for installations consisting of an array of machine vision cameras located on racks and placed in a room, the obvious choice would be to use conventional image compression methods. Furthermore, based on the assessment of the arbitrariness of video compression methods, it should be noted that the XVC algorithm remains undervalued, although its results are higher. Algorithm AV1 can be considered the next in order of importance. It has been established that the latest compression algorithms show higher performance if compared to their predecessors. It has also been shown that with a small distance between the optical centers of the captured images, the use of video compression algorithms is preferable to the use of image compression algorithms, since they show better results in both three-dimensional and four-dimensional versions.Conclusions. A comparison of the results obtained shows the need to use algorithms from the video compression family (XVC, AV1) on installations with a long baseline (mounted on camera stands). When working with integrated light field cameras (Lytro) and setting the capture with a short baseline, it is recommended to use image compression algorithms (JPEG). In general, video compression algorithms are recommended, in particular XVC, since on average it shows an acceptable level of PSNR in both the case of a short and long installation baseline.

We present a new compressed domain method for tracking objects in airborne videos. In the proposed scheme, a statistical snake is used for object segmentation in I-frames, and motion vectors extracted from P-frames are used for tracking the object detected in I-frames. It is shown that the energy function of the statistical snake can be obtained directly from the compressed DCT coefficients without the need of full decompression. The number of snake deformation iterations can be also significantly reduced in compressed domain implementation. The computational cost is significantly reduced by using compressed domain processing while the performance is competitive to that of pixel domain processing. The proposed method is tested using several UAV video sequences, and experiments show that the tracking results are satisfactory.

Compressed domain speech enhancement method based on ITU-T G.722.2

Chapter 6 - How to Use Texture Analysis and Synthesis Methods for Video Compression