Minimizing Synchronization Error in Compressed Domain Watermarking

Deep learning methods have achieved impressive performance in compressed video quality enhancement tasks. However, these methods rely excessively on practical experience by manually designing the network structure and do not fully exploit the potential of the feature information contained in the video sequences, i.e., not taking full advantage of the multiscale similarity of the compressed artifact information and not seriously considering the impact of the partition boundaries in the compressed video on the overall video quality. In this article, we propose a novel Mixed Difference Equation inspired Transformer (MDEformer) for compressed video quality enhancement, which provides a relatively reliable principle to guide the network design and yields a new insight into the interpretable transformer. Specifically, drawing on the graphical concept of the mixed difference equation (MDE), we utilize multiple cross-layer cross-attention aggregation (CCA) modules to establish long-range dependencies between encoders and decoders of the transformer, where partition boundary smoothing (PBS) modules are inserted as feedforward networks. The CCA module can make full use of the multiscale similarity of compression artifacts to effectively remove compression artifacts, and recover the texture and detail information of the frame. The PBS module leverages the sensitivity of smoothing convolution to partition boundaries to eliminate the impact of partition boundaries on the quality of compressed video and improve its overall quality, while not having too much impacts on non-boundary pixels. Extensive experiments on the MFQE 2.0 dataset demonstrate that the proposed MDEformer can eliminate compression artifacts for improving the quality of the compressed video, and surpasses the state-of-the-arts (SOTAs) in terms of both objective metrics and visual quality.

Practically, action recognition using deep learning approaches are slow because of high temporal redundancy and large size of the raw video data. One of the solutions for boosting accuracy is calculating optical flows. Generally, extracting motion features are too time-consuming. Therefore, traditional action recognition methods are not suitable for real time applications. On the other hand, compressed videos are available in many situations especially when using mobile devices. We proposed a method that extracts residuals directly from compressed videos by partially decoding the video and feed them to a deep neural network. In general, exploiting the compressed domain features as available information provides a slight reduction in accuracy while the low complexity of this method makes it appropriate for real time applications. The experimental results on multiple first and third person datasets exhibit that while the proposed method provides low computational complexity, the results are highly competitive with traditional approaches in accuracy.

Many advanced video applications require manipulations of compressed video signals. Popular video manipulation functions include overlap (opaque or semitransparent), translation, scaling, linear filtering, rotation, and pixel multiplication. We propose algorithms to manipulate compressed video in the compressed domain. Specifically, we focus on compression algorithms using the discrete cosine transform (DCT) with or without motion compensation (MC). Such compression systems include JPEG, motion JPEG, MPEG, and H.261. We derive a complete set of algorithms for all aforementioned manipulation functions in the transform domain, in which video signals are represented by quantized transform coefficients. Due to a much lower data rate and the elimination of decompression/compression conversion, the transform-domain approach has great potential in reducing the computational complexity. The actual computational speedup depends on the specific manipulation functions and the compression characteristics of the input video, such as the compression rate and the nonzero motion vector percentage. The proposed techniques can be applied to general orthogonal transforms, such as the discrete trigonometric transform. For compression systems incorporating MC (such as MPEG), we propose a new decoding algorithm to reconstruct the video in the transform domain and then perform the desired manipulations in the transform domain. The same technique can be applied to efficient video transcoding (e.g., from MPEG to JPEG) with minimal decoding. >

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multistream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.

Visual database systems require efficient indexing to enable fast access to the images and video sequences in the database. Since image and video data are voluminous, compression is essential for storage and transmission. The International Standards Organization has proposed the JPEG and MPEG standards for image and video compression, respectively. In order to avoid the unnecessary decompression and recompression operations in the searching process, it is efficient to index the image/video in compressed form. In this paper, we present image and video indexing techniques in the compressed domain.

In the technological era, the primary source of information is in the form of digital data, which has to be secured while storing or transmitting during communication over an unsecured network. Different approaches are used to provide security to digital data, viz. text, audio, image, and video. This paper initially explains the security system such as cryptography, watermarking, and steganography and their comparative analysis based on different characteristics, viz. satisfaction level of objective, type of carrier object and secret information to be used, dependency of security level, and quality assessment parameters. This review article focuses more on steganography methods applied over video. The various methods implemented for video steganography in compressed domain, viz. inter-frame and intra-frame prediction, motion vector estimation, entropy coding (CAVLC and CABAC), and transformed and quantized coefficients of DCT, DST, and DWT, etc. and the methods based on spatial and transform domain for uncompressed video are briefly described. It is followed by the detailed analysis of related work done by various researchers in video steganography and the obtained experimental results. Furthermore, the confidential data hiding in compressed videos are explained using Moving Picture Expert Group (MPEG—1, MPEG—2, MPEG—4), Advanced Video Coding (AVC)/H.264, and High-Efficiency Video Coding (HEVC)/H.265 that includes both spatial and transform domain. This paper summarizes and explains the detailed investigations of numerous techniques of video steganography based on the comprehensive literature survey. The methods used to assess the performance of video steganography are analyzed based on the quality assessment parameters such as imperceptibility; measured by peak signal to noise ratio (PSNR), mean square error (MSE), and structural similarity (SSIM), robustness; measured by bit error rate (BER) and similarity (Sim), and embedding capacity; measured by hiding ratio. The overall review of past literature facilitates to have in-depth knowledge for upgrading the video steganography.

Fractal compression is the lossy compression technique in the field of gray/color image and video compression. It gives high compression ratio, better image quality with fast decoding time but improvement in encoding time is a challenge. This review paper/article presents the analysis of most significant existing approaches in the field of fractal based gray/color images and video compression, different block matching motion estimation approaches for finding out the motion vectors in a frame based on inter-frame coding and intra-frame coding i.e. individual frame coding and automata theory based coding approaches to represent an image/sequence of images. Though different review papers exist related to fractal coding, this paper is different in many sense. One can develop the new shape pattern for motion estimation and modify the existing block matching motion estimation with automata coding to explore the fractal compression technique with specific focus on reducing the encoding time and achieving better image/video reconstruction quality. This paper is useful for the beginners in the domain of video compression.

In modern world more image and video compression technologies improving gradually. But major barrier of the new technologies concepts are repeated from authors. Because authors can’t find in depth of the papers from various technologies. So Survey of Video Compression Techniques very helpful for these types problems in video compression areas. Video compression techniques such as DCT coding, Quantization, Entropy coding, Motion estimation are widely used in video compression techniques. The main focus of this paper is to analyse video compression techniques required for video processing especially to discover how much amount of data to compressed, which techniques is faster and visual quality better and so on. We evaluate the video compression techniques for finding compression ratio in terms of performance, speed and accuracy. We also compare video compression techniques with conventional methods.

Solving the video compression problem requires a multi-faceted approach, balancing quality, efficiency, and computational demands. By leveraging advancements in technology and adapting to the evolving needs of video applications, it is possible to develop compression methods that meet the challenges of the present and future digital landscape. To address these objectives, machine learning and AI approaches can be utilized to predict and remove redundancies more effectively, optimizing compression algorithms dynamically based on content. Still, state-of-the art neural network-based video compression models need large and diverse datasets to generalize well across different types of video content. Wavelets can provide both time (spatial) and frequency localization, making them highly effective for video compression. This dual localization allows wavelet transforms to handle both rapid changes in video content and slow-moving scenes efficiently, leading to better compression ratios. Yet, some wavelet coefficients may be more critical for maintaining visual quality than others. Inaccurate quantization can lead to noticeable degradation. For the first time, the suggested model combine Quantum Wavelet Transform (QWT) and Neural Networks (NN) for video compression. This fusion model aims to achieve higher compression ratios, maintain video quality, and reduce computational complexity by utilizing QWT’s efficient data representation and NN’s powerful pattern recognition and predictive capabilities. Quantum bits (qubits) can encode large amounts of information in their quantum states, enabling more efficient data representation. This is especially useful for encoding large video files. Furthermore, quantum entanglement allows for correlated data representation across qubits, which can be exploited to capture intricate details and redundancies in video data more effectively than classical methods. The experimental results reveal that QWT achieves a compression ratio of almost twice that of traditional WT for the same video, maintaining superior visual quality due to more efficient redundancy elimination.

In this paper we propose a novel saliency-based computational model for visual attention. This model processes both top-down (goal directed) and bottom-up information. Processing in the top-down channel creates the so called skin conspicuity map and emulates the visual search for human faces performed by humans. This is clearly a goal directed task but is generic enough to be context independent. Processing in the bottom-up information channel follows the principles set by Itti et al. but it deviates from them by computing the orientation, intensity and color conspicuity maps within a unified multi-resolution framework based on wavelet subband analysis. In particular, we apply a wavelet based approach for efficient computation of the topographic feature maps. Given that wavelets and multiresolution theory are naturally connected the usage of wavelet decomposition for mimicking the center surround process in humans is an obvious choice. However, our implementation goes further. We utilize the wavelet decomposition for inline computation of the features (such as orientation angles) that are used to create the topographic feature maps. The bottom-up topographic feature maps and the top-down skin conspicuity map are then combined through a sigmoid function to produce the final saliency map. A prototype of the proposed model was realized through the TMDSDMK642-0E DSP platform as an embedded system allowing real-time operation. For evaluation purposes, in terms of perceived visual quality and video compression improvement, a ROI-based video compression setup was followed. Extended experiments concerning both MPEG-1 as well as low bit-rate MPEG-4 video encoding were conducted showing significant improvement in video compression efficiency without perceived deterioration in visual quality.

A variety of approaches have been proposed in the literature for region-of-interest (ROI) estimation. In most of them the ROI definition is highly subjective, i.e., lacks scientific evidence in supporting the claim that the areas defined as ROIs are indeed perceptually interesting. In this paper we attempt to model ROIs as the visually attended areas indicated by a saliency map in order to lower as much as possible the subjectivity of selection. For evaluation purposes we follow a ROI-based video compression setup and present comparisons with state-of-the-art algorithms in terms of perceived visual quality and video compression improvement. Extended experiments concerning both MPEG-1 as well as low bit-rate MPEG-4 video encoding were conducted showing significant improvement in video compression efficiency without perceived deterioration in visual quality.

In this work we propose a new algorithm for fragile, high capacity yet file-size preserving watermarking of MPEG-2 streams. Watermarking is done entirely in the compressed domain, with no need for full or even partial decompression. The algorithm is based on a previously developed concept of VLC mapping for compressed domain watermarking. The entropy-coded segment of the video is first parsed out and then analyzed in pairs. It is recognized that there are VLC pairs that never appear together in any intra-coded block. The list of unused pairs is systematically generated by the intersection of trees. One of the trees is generated from the main VLC table given in ISO/IEC 13818-2:2000 standard. The other trees are dynamically generated for each intra coded blocks. Forcing one VLC pairs in a block to one of the unused ones generates a watermark block. The change is done while maintaining run/level change to a minimum. At the decoder, the main pair tree is created offline using publicly available VLC tables. Through a secure key exchange, the indices to unused code pairs are communicated to the receiver. We show that the watermarked video is reasonably resistant to forgery attacks and remains secure to watermark detection attempts.

In the latest years, videoconferencing has taken a fundamental role in interpersonal relations, both for personal and business purposes. Lossy video compression algorithms are the enabling technology for videoconferencing, as they reduce the bandwidth required for real-time video streaming. However, lossy video compression decreases the perceived visual quality. Thus, many techniques for reducing compression artifacts and improving video visual quality have been proposed in recent years. In this work, we propose a novel GAN-based method for compression artifacts reduction in videoconferencing. Given that, in this context, the speaker is typically in front of the camera and remains the same for the entire duration of the transmission, we can maintain a set of reference keyframes of the person from the higher-quality I-frames that are transmitted within the video stream and exploit them to guide the visual quality improvement; a novel aspect of this approach is the update policy that maintains and updates a compact and effective set of reference keyframes. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks. This allows the restoration of the high-frequency details lost after the video compression. Experiments show that the proposed approach improves visual quality and generates photo-realistic results even with high compression rates. Code and pre-trained networks are publicly available at https://github.com/LorenzoAgnolucci/Keyframes-GAN.

Color correction needs to be performed to improve the quality of image/video production. The typical methods realize the color correction mainly in the spatial domain of RGB color space. In this paper, a linear color correction method in JPEG/MPEG-2 compressed domain is proposed. The correction is realized in the DCT domain of YUV color space without full-frame decompression. Experimental results show that the visual quality of the corrected images/videos in the compressed domain is identical to the quality of the images/videos corrected in the uncompressed domain.

In recent years video watermarking has become an important research issue due to increase in threats of video piracy, content authentication, and ownership integrity. In this paper, we propose a robust watermarking method with the blind extraction process for compressed video streams like H.264/AVC. The method embeds invisible watermark bits into the P-frames. Appropriate block selection and the embedding algorithm restrict the increase in video bit rate. The security of the method lies in selecting candidate blocks using a pseudo random key from set of previously selected blocks. The simulation results show that the proposed method is robust against common image and video watermarking attacks. A robust watermark is generated in order to withstand frame drop and insert attacks, which increases the overall robustness of the proposed method.