FPGA implementation of a fast pipeline architecture for JND computation

Fast synthesized and predicted just noticeable distortion maps for perceptual multiview video coding

A distortion on an image can be invisible by the Human Visual System (HVS), if the distortion is under a threshold called Just Noticeable Distortion (JND), which is determined by several sensibility factors of the HVS, such as luminance and frequency sensibility. Until now, the JND is widely used for invisible watermarking schemes; however, in visible watermarking schemes this concept is not explored sufficiently. In this paper, a novel compressed domain visible watermarking method based on the JND profile is presented. The JND is estimated by texture and intensity of each 8×8 block of the Discrete Cosine Transform (DCT). Considering that the JPEG is the most widely used digital image format and almost all photographs captured by digital camera are stored in this format; the visible watermarking scheme in compressed domain without inverse DCT operation to obtain spatial image provides important advantage from security and complexity points of view.

We propose a new spatial just noticeable distortion (JND) profile for color image processing. The JND threshold depends on various masking effects underlying existing in the human vision system (HVS). How to efficiently integrate different masking effects together is the key issue of modelling the JND profile. Based on recent vision research results, we model the masking effects in different stimulus dimensions as a nonlinear additivity model for masking (NAMM). It applies to all color components and accounts for the compound impact of luminance masking and texture masking to estimate the JND threshold in images. In our PSNR and subjective comparison to the related work, the proposed NAMM scheme provides a more accurate JND profile towards the actual JND bound in the HVS.

Existing pixel‐based just noticeable distortion (JND) models only take into account luminance adaptation and texture masking (TM). Similarly, existing discrete cosine transform (DCT) based models do not take into account foveal vision effects and do not estimate TM efficiently. As human visual system (HVS) is not sensitive to distortion below the JND threshold, estimation of the perceptual visibility threshold is widely used in digital and video processing applications. The authors propose a comprehensive and efficient pixel‐based JND model incorporating all major factors which contribute to the JND estimation. The evaluation of contrast masking (CM) is done by distinguishing the edge and TM with respect to the entropy masking properties of the HVS. Similarly, the foveal vision effects are also taken into account for the comprehensive estimation of contrast sensitivity function (CSF). Hence, the proposed pixel‐based JND model incorporates the spatio‐temporal CSF, foveal vision effects, influence of eye‐movement, luminance adaptation and CM to be more consistent with human perception. The incorporation of these important factors makes the proposed model the most comprehensive and efficient in the current literature. Psychophysical experiments were performed to test the proposed model. The results show the proposed model comprehensively outperforms other existing models proving its efficiency.

An adaptive steganographic method based on the measurement of just noticeable distortion profile

Just Noticeable Distortion (JND) finds the minimum distortion level perceivable by humans. This can be a natural solution for setting the compression for each video region in perceptual video coding. However, existing JND-based solutions estimate JND levels for each video frame and ignore the fact that different video regions have different perceptual importance. To address this issue, we propose a Block-Level Just Noticeable Distortion-based Perceptual (BL-JUNIPER) framework for video coding. The proposed four-stage framework combines different perceptual information to further improve the prediction accuracy. The JND mapping in the first stage derives block-level JNDs from frame-level information without the need to collect a new bock-level JND dataset. In the second stage, an efficient CNN-based model is proposed to predict JND levels for each block according to spatial and temporal characteristics. Unlike existing methods, BL-JUNIPER works on raw video frames and avoids re-encoding each frame several times, making it computationally practical. Third, the visual importance of each block is measured using a visual attention model. Finally, a proposed quantization control algorithm uses both JND levels and visual importance to adjust the Quantization Parameter (QP) for each block. The specific algorithm for each stage of the proposed framework can be changed, as long as the input and output formats of each block are followed, without the need to change other stages, based on any current or future methods, providing a flexible and robust solution. Extensive experimental results demonstrate that BL-JUNIPER achieves a mean bitrate reduction of 27.75% with a Delta Mean Opinion Score (DMOS) close to zero and BD-Rate gains of 25.44% based on MOS, compared to the baseline encoding, and also gains a better performance compared to competing methods.

Medical images stored in a hospital system are generally confidential and integrated and require strict security. However, medical information stored on digital medical imaging systems, as well as picture archiving and communication systems (PACS), are vulnerable to attack when the data are transferred over wireless or wired communication networks. To solve this problem, a watermarking algorithm for medical images is proposed using a bit threshold map based on just noticeable distortion (JND) in the discrete cosine transform (DCT) method. The low-frequency component comprises a considerable amount of the signal energy for most images. As a result, it has a crucial effect on the image quality. Therefore, in this paper, the proposed algorithm embeds watermarks based on the low-frequency components of the image, such as the DC coefficient of the DCT. When watermarks are embedded in a low frequency area, the subjective image quality is often degraded. To compensate for the degradation of the imperceptibility of the watermarking system, which results from embedding watermarks in the low-frequency component, this research considers the human visual system. In addition, the embedding strength of the JND value is used to improve the watermarking imperceptibility. We applied the proposed watermarking algorithm to a variety of medical images using a computer simulation. The algorithm's performance was verified using a variety of attacks for eliminating watermarks. The simulation results show that the proposed algorithm robustly provides protection against a variety of possible attacks.

Many conventional DCT based Just Noticeable Distortion (JND) models incorporate luminance adaptation (LA) effect of the human visual system (HVS). The conventional LA-JND models exploit only background luminance to estimate JND values. In this letter, we reveal that the LA effect of HVS depends not only on background luminance but also on frequency in DCT domain. In addition, we first propose a novel DCT-based LA-JND model that takes into account its frequency characteristics. From our psychophysical experiment results, we found that the LA-JND threshold exhibits quasi-parabolic shapes with lower and higher curvatures in lower and higher frequency ranges, respectively. Our subjective evaluation shows that the proposed LA-JND model yields almost invisible distortions for test images with average PSNR of 29.11 dB, which is 2.63 dB lower than the other models for comparison, showing higher consistency with real visual perception for the LA in DCT domain.

A spatiotemporal saliency-modulated JND profile applied to video watermarking

Publication in the conference proceedings of EUSIPCO, Barcelona, Spain, 2011

This paper proposes a novel scheme combining Discrete Wavelet Transform (DWT) and Singular Value Decomposition (SVD) for image watermarking. The watermarking image is separated by two parts randomly, then they are embedded into the sub-bands obtained from the original image in DWT domain by modifying the singular value. SVD is applied twice in this scheme.The use of the threshold of Just Noticeable Distortion (JND) based on human visual model determines the embedding strength and position of the watermark, the digital adaptive watermarking algorithm in a combinational domain is realized. The experimental results show that the proposed scheme has a good invisibility and performance of robustness for common attacks, simultaneously enhances the embedding capacity and security.

In this paper, a perceptual blocking distortion metric for block-based transform coded digital video is proposed. The metric is based on a spatio-temporal multichannel vision model to calculate the Just Noticeable Distortion (JND) map. The blocking dominant regions are segmented after the spatio-temporal decomposition and the JNDs in these regions are summed up to form an objective measure of the blocking artifact. Subjective and objective tests have been conducted and the results show a strong correlation between the objective blocking rating and the mean opinion score.

In this paper, a perceptual JPEG2000 compliant coder based on the just noticeable distortion (JND) for the color images is presented. Since the human visual perception to color visual signals has a limited sensitivity, any change below the visibility threshold cannot be detected by human eyes. We therefore propose a new color JND estimator to incorporate into the design of the perceptual JPEG2000 coder. Without introducing the perceptual distortion into input signals, the dynamic range of input signals in each color component of the color image is reduced to increase the compression performance. In the simulation results, the bit rate required by the perceptual JPEG2000 compliant coder is lower than that required by the standard JPEG2000 coder at the perceptually loss less visual quality of the reconstructed color image.

We explore a new perceptually-adaptive video coding (PVC) scheme in the motion-compensated prediction loop for hybrid video compression, aiming at better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color images/video is first devised in image-domain based on our introduced nonlinear additivity model for masking (NAMM). Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation (ME) and residue filtering. The scheme works with any prevalent video coding standards and various ME strategies. As an example of implementation, it is applied to the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in ME efficiency, 0.6 dB in perceptual coding quality and most remarkably, 0.17 dB in the objective coding quality measure (PSNR).

Image compression is a well-established and extensively researched field. The huge interest in it has been aroused by the rapid enhancements introduced in imaging techniques and the various applications that use high-resolution images (e.g. medical, astronomical, Internet applications). The image compression algorithms should not only give state-of-art performance, they should also provide other features and functionalities such as progressive transmission. Often, a rough approximation (thumbnail) of an image is sufficient for the user to decide whether to continue the image transmission or to abort; which accordingly helps to reduce time and bandwidth. That in turn necessitated the development of multi-resolution image compression schemes. The existed multi-resolution schemes (e.g., Multi-Level Progressive method) have shown high computational efficiency, but with a lack of the compression performance, in general. In this thesis, a LOw Complexity Multi-resolution Image Compression (LOCMIC) based on the Hierarchical INTerpolation (HINT) framework is presented. Moreover, a novel integration of the Just Noticeable Distortion (JND) for perceptual coding with the HINT framework to achieve a visual-lossless multi-resolution scheme has been proposed. In addition, various prediction formulas, a context-based prediction correction model and a multi-level Golomb parameter adaption approach have been investigated. The proposed LOCMIC (the lossless and the visual lossless) has contributed to the compression performance. The lossless LOCMIC has achieved a 3% reduced bit rate over LOCO-I, about 1% over JPEG2000, 3% over SPIHT, and 2% over CALIC. The Perceptual LOCMIC has been better in terms of bit rate than near-lossless JPEG-LS (at NEAR=2) with about 4.7%. Moreover, the decorrelation efficiency of the LOCMIC in terms of entropy has shown an advance of 2.8%, 4.5% than the MED and the conventional HINT respectively.