BL-JUNIPER: A CNN-Assisted Framework for Perceptual Video Coding Leveraging Block-Level JND

Fast synthesized and predicted just noticeable distortion maps for perceptual multiview video coding

Just Noticeable Distortion (JND) based on human visual system (HVS) is widely used in the transparent watermarking. But the computation of JND is very complex, which makes it difficult to embed it into integrated circuits. To solve this problem, Haar Wavelet based JND model is exploited in this paper. Furthermore, the fast pipeline architecture based on haar-wavelet for JND computation is developed. The architecture is modeled with hardware description language, and implemented on Altera EP2C35 FPGA device in order to evaluate the its performance. The hardware cost of JND core is 1460 Logic Cell combinations and 330 registers, which is significantly smaller than the Full Band JND based architecture. From the experiment results, the system goes on well with a 120MHz clock. Compared with the full band JND based architecture, ours achieves 75% time saving.

A distortion on an image can be invisible by the Human Visual System (HVS), if the distortion is under a threshold called Just Noticeable Distortion (JND), which is determined by several sensibility factors of the HVS, such as luminance and frequency sensibility. Until now, the JND is widely used for invisible watermarking schemes; however, in visible watermarking schemes this concept is not explored sufficiently. In this paper, a novel compressed domain visible watermarking method based on the JND profile is presented. The JND is estimated by texture and intensity of each 8×8 block of the Discrete Cosine Transform (DCT). Considering that the JPEG is the most widely used digital image format and almost all photographs captured by digital camera are stored in this format; the visible watermarking scheme in compressed domain without inverse DCT operation to obtain spatial image provides important advantage from security and complexity points of view.

In this paper, a perceptual JPEG2000 compliant coder based on the just noticeable distortion (JND) for the color images is presented. Since the human visual perception to color visual signals has a limited sensitivity, any change below the visibility threshold cannot be detected by human eyes. We therefore propose a new color JND estimator to incorporate into the design of the perceptual JPEG2000 coder. Without introducing the perceptual distortion into input signals, the dynamic range of input signals in each color component of the color image is reduced to increase the compression performance. In the simulation results, the bit rate required by the perceptual JPEG2000 compliant coder is lower than that required by the standard JPEG2000 coder at the perceptually loss less visual quality of the reconstructed color image.

We propose a new spatial just noticeable distortion (JND) profile for color image processing. The JND threshold depends on various masking effects underlying existing in the human vision system (HVS). How to efficiently integrate different masking effects together is the key issue of modelling the JND profile. Based on recent vision research results, we model the masking effects in different stimulus dimensions as a nonlinear additivity model for masking (NAMM). It applies to all color components and accounts for the compound impact of luminance masking and texture masking to estimate the JND threshold in images. In our PSNR and subjective comparison to the related work, the proposed NAMM scheme provides a more accurate JND profile towards the actual JND bound in the HVS.

An adaptive steganographic method based on the measurement of just noticeable distortion profile

We explore a new perceptually-adaptive video coding (PVC) scheme in the motion-compensated prediction loop for hybrid video compression, aiming at better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color images/video is first devised in image-domain based on our introduced nonlinear additivity model for masking (NAMM). Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation (ME) and residue filtering. The scheme works with any prevalent video coding standards and various ME strategies. As an example of implementation, it is applied to the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in ME efficiency, 0.6 dB in perceptual coding quality and most remarkably, 0.17 dB in the objective coding quality measure (PSNR).

Publication in the conference proceedings of EUSIPCO, Barcelona, Spain, 2011

A spatiotemporal saliency-modulated JND profile applied to video watermarking

Existing pixel‐based just noticeable distortion (JND) models only take into account luminance adaptation and texture masking (TM). Similarly, existing discrete cosine transform (DCT) based models do not take into account foveal vision effects and do not estimate TM efficiently. As human visual system (HVS) is not sensitive to distortion below the JND threshold, estimation of the perceptual visibility threshold is widely used in digital and video processing applications. The authors propose a comprehensive and efficient pixel‐based JND model incorporating all major factors which contribute to the JND estimation. The evaluation of contrast masking (CM) is done by distinguishing the edge and TM with respect to the entropy masking properties of the HVS. Similarly, the foveal vision effects are also taken into account for the comprehensive estimation of contrast sensitivity function (CSF). Hence, the proposed pixel‐based JND model incorporates the spatio‐temporal CSF, foveal vision effects, influence of eye‐movement, luminance adaptation and CM to be more consistent with human perception. The incorporation of these important factors makes the proposed model the most comprehensive and efficient in the current literature. Psychophysical experiments were performed to test the proposed model. The results show the proposed model comprehensively outperforms other existing models proving its efficiency.

This paper proposes a novel scheme combining Discrete Wavelet Transform (DWT) and Singular Value Decomposition (SVD) for image watermarking. The watermarking image is separated by two parts randomly, then they are embedded into the sub-bands obtained from the original image in DWT domain by modifying the singular value. SVD is applied twice in this scheme.The use of the threshold of Just Noticeable Distortion (JND) based on human visual model determines the embedding strength and position of the watermark, the digital adaptive watermarking algorithm in a combinational domain is realized. The experimental results show that the proposed scheme has a good invisibility and performance of robustness for common attacks, simultaneously enhances the embedding capacity and security.

Medical images stored in a hospital system are generally confidential and integrated and require strict security. However, medical information stored on digital medical imaging systems, as well as picture archiving and communication systems (PACS), are vulnerable to attack when the data are transferred over wireless or wired communication networks. To solve this problem, a watermarking algorithm for medical images is proposed using a bit threshold map based on just noticeable distortion (JND) in the discrete cosine transform (DCT) method. The low-frequency component comprises a considerable amount of the signal energy for most images. As a result, it has a crucial effect on the image quality. Therefore, in this paper, the proposed algorithm embeds watermarks based on the low-frequency components of the image, such as the DC coefficient of the DCT. When watermarks are embedded in a low frequency area, the subjective image quality is often degraded. To compensate for the degradation of the imperceptibility of the watermarking system, which results from embedding watermarks in the low-frequency component, this research considers the human visual system. In addition, the embedding strength of the JND value is used to improve the watermarking imperceptibility. We applied the proposed watermarking algorithm to a variety of medical images using a computer simulation. The algorithm's performance was verified using a variety of attacks for eliminating watermarks. The simulation results show that the proposed algorithm robustly provides protection against a variety of possible attacks.

The video quality degradation in wireless network is much changeable for complex packet loss scenes of wireless channels. However, few Video Quality Metrics (VQM) analyze the quality degradation under different wireless packet loss conditions. This paper proposes a novel Universal Video Quality Metric (UVQM), which is suitable for both distributed loss situation and burst loss situation. The major contributions of this paper can be concluded as follows. Firstly, this metric is an end-to-end measurement which considers the impacts of encoding parameter (quantization parameter, q-scale) and transmission parameters (Packet Loss Rate, PLR and Maximum Transmission Unit, MTU) to make a complete VQM. Then, parameters used in UVQM are extracted from bitstreams which reduces the use of computation resource for decoding at the receiver. Finally, UVQM coalesces the impact of content-aware features by introducing a developed Just Noticeable Distortion (JND) which makes the prediction much accuracy. The verification results show that the predicted video quality measured by Mean Opinion Score (MOS) has a high Pearson correlation (higher than 0.74 in distributed loss situation and 0.65 in burst loss situation).

Image compression is a well-established and extensively researched field. The huge interest in it has been aroused by the rapid enhancements introduced in imaging techniques and the various applications that use high-resolution images (e.g. medical, astronomical, Internet applications). The image compression algorithms should not only give state-of-art performance, they should also provide other features and functionalities such as progressive transmission. Often, a rough approximation (thumbnail) of an image is sufficient for the user to decide whether to continue the image transmission or to abort; which accordingly helps to reduce time and bandwidth. That in turn necessitated the development of multi-resolution image compression schemes. The existed multi-resolution schemes (e.g., Multi-Level Progressive method) have shown high computational efficiency, but with a lack of the compression performance, in general. In this thesis, a LOw Complexity Multi-resolution Image Compression (LOCMIC) based on the Hierarchical INTerpolation (HINT) framework is presented. Moreover, a novel integration of the Just Noticeable Distortion (JND) for perceptual coding with the HINT framework to achieve a visual-lossless multi-resolution scheme has been proposed. In addition, various prediction formulas, a context-based prediction correction model and a multi-level Golomb parameter adaption approach have been investigated. The proposed LOCMIC (the lossless and the visual lossless) has contributed to the compression performance. The lossless LOCMIC has achieved a 3% reduced bit rate over LOCO-I, about 1% over JPEG2000, 3% over SPIHT, and 2% over CALIC. The Perceptual LOCMIC has been better in terms of bit rate than near-lossless JPEG-LS (at NEAR=2) with about 4.7%. Moreover, the decorrelation efficiency of the LOCMIC in terms of entropy has shown an advance of 2.8%, 4.5% than the MED and the conventional HINT respectively.

In this paper, a perceptual blocking distortion metric for block-based transform coded digital video is proposed. The metric is based on a spatio-temporal multichannel vision model to calculate the Just Noticeable Distortion (JND) map. The blocking dominant regions are segmented after the spatio-temporal decomposition and the JNDs in these regions are summed up to form an objective measure of the blocking artifact. Subjective and objective tests have been conducted and the results show a strong correlation between the objective blocking rating and the mean opinion score.