Towards perceptual image watermarking with robust texture measurement

Watermarking is an effective solution for copyright protection and forensics tracking via hiding information into the image signal. Recently, spatial-embedding watermarking methods from different transform domain have been proposed rapidly and effectively protect the copyright of the color image. Compared to the individual frequency domain method, it has both advantages of spatial domain and frequency domain. Here, we proposed a novel spatial-embedding watermarking method based on an attended just noticeable difference (JND) model with color complexity, to achieve a good tradeoff between robustness and perceptual quality. In particular, at the spatial embedding level, the direct current (DC) coefficients are selected for embedding and the perceptual JND model is used to guide the amount of pixel modification to improve visual quality. Different from the previous JND model, we proposed an attended JND model that considers color complexity, which is more consistent of the human visual perception model. Compared with the other JND models, the proposed JND model is more suitable for the watermarking framework. Experimental results show that the proposed watermarking scheme has better performance than other existing watermarking schemes. This greatly benefits the practical implementations of the spatial-embedding watermarking methods.

Just noticeable difference (JND), which reveals the visibility of our human visual system (HVS), is useful for image/video coding. Due to the content complexity, it is hard to accurately estimate the JND thresholds for different image blocks (e.g., edge and texture). Research on cognitive science indicates that the HVS is adaptive to extract the visual regularities for scene perception and understanding. Inspired by this, we analyze the effect of content complexity on visual masking, and calculate the content complexity with the distribution of orientation. Then, by considering the effect from content complexity, contrast sensitivity function, and luminance adaption, a novel JND model in Discrete Cosine Transform (DCT) domain is proposed. Experimental results demonstrate the effectiveness of the proposed model for JND estimation.

With the rapid access convenience of content brought by 5G technology, the integrity protection of content becomes more important. The reversible visible watermarking algorithm has attracted more attention due to its effective content protection. In this paper, a novel improved reversible visible image watermarking scheme based on gradient-weighted class activation mapping (Grad-CAM) and the just noticeable difference (JND) model has been presented. The proposed region of interest (ROI) selection strategy is used to locate the main protected body of images for watermark embedding. Divide the watermark and ROI into nonoverlapping blocks in the same way and then embed the classified two types of watermark blocks into corresponding ROI blocks with the JND model. The optimal bit positions for watermark embedding can be selected adaptively with JND threshold and achieve the tradeoff between the watermark visibility and watermarked image quality. For lossless image recovery and watermark extraction, the recovery information is reversibly hidden into watermarked image. In the experiments, the same process of grayscale images is used to each channel separately for color images watermarking. Besides, there are six aspects in this paper to estimate the proposed scheme; with the comparison to other reversible visible watermarking schemes, experimental results demonstrate the effectiveness of our proposed scheme.

Just noticeable difference (JND) reflecting visual redundancy of human visual system (HVS) has been widely adopted in image/video processing. In this work, we introduce a novel JND model in pixel domain for screen content (SC) images. Due to the obvious different impacts on HVS between pictorial region and computer-generated texture, we decompose a SC image into screen content set (SCS) and non-screen content set (non-SCS) for JND modeling. For SCS, edge masking effect is mainly considered by using a parametric model with an adaptive edge representation. For non-SCS, both the edge masking effect and pattern masking effect are taken into account to improve distortion tolerance by injecting more noise to pattern complexity region without perceptual quality degradation. Furthermore, visual saliency is employed to adjust pattern masking effect in the view of HVS. Experimental results show that compared with state-of-the-arts, the proposed JND model can tolerate more distortion, and provide better perceptual quality at the same noise level.

Macrophotography Images (MPIs) have recently emerged as an active topic due to the development of mobile phone camera technology. A large number of MPIs have been rapidly increasing in many rich visual services, such as smartphones or high-definition monitors. MPIs are often composed of sharp macroimage and blur background, which exhibit different perceptual properties that often lead to different just noticeable difference (JND) estimation. Inspired by this, we formulate the blur concealment (BC) as another factor to determine the total masking effect: the interaction is relatively straightforward with a limited masking effect in the sharp regions, and is complicated with a strong masking effect in the blur parts. Furthermore, texture and orientation adaption and color information weighting are separately incorporated into the contrast masking and color masking. Finally, considering both BC and masking effects, a novel robust watermarking framework based on the proposed blur-guided JND model for MPIs, targeting at further improving the MPIs copyright protection performance. Extensive experiments on MP2020 and Blur Detection data sets show that the applicability of the proposed JND model in the scenario of perceptually MPIs watermarking, and our proposed scheme can outperform the state-of-the-art watermarking schemes by providing better robustness performance at the uniform visual quality.

A new pyramidal JND (just noticeable difference) model is presented. The idea is to use this JND to determine the optimum strength for embedding the watermark providing an invisible and robust watermarking scheme. The image is first decomposed into a multiresolution representation using the pyramidal decomposition. Then, a perceptual model is proposed to compute the JND value for each pixel at each Laplacian level. This model takes into account three main characteristics of the human visual system (HVS), namely: contrast sensitivity, luminance adaptation and contrast masking. The performance of the proposed technique is evaluated in terms of transparency, using subjective and objective tests, and robustness to different common attacks.

A novel block-level perceptual image watermarking framework is proposed in this study, including tri-directional correlation and a block-level just noticeable difference (JND) model. Specifically, the difference in the discrete cosine transform (DCT) coefficients of two blocks is calculated based on three directions in the neighborhood, called the tri-directional correlation (TriDC). Additionally, the representative alternating current (AC) coefficients along horizontal, vertical, and diagonal directions, which can describe structural patterns, are projected and merged for TriDC differences. Then, the difference of the DCT coefficient is modulated to a predefined zone depending on the JND-based offset. Finally, the extent of the watermarked AC coefficients is determined with perceptual JND adjustment. The experimental results demonstrate that the proposed scheme can protect most common image processing attacks; and has better robustness compared with recent zone modulation watermarking schemes and traditional watermarking methods.

Human perception of the face involves the observation of both coarse (global) and detailed (local) features of the face to identify and categorize a person. Face categorization involves finding common visual cues, such as gender, race and age, which could be used as a precursor to a face recognition system to improve recognition rates. In this paper, we investigate the fusion of both global and local features for gender classification. Global features are obtained using the principal component analysis (PCA) and discrete cosine transformation (DCT) approaches. A spatial local binary pattern (LBP) approach augmented with a two-dimensional DCT approach has been used to find the local features. The performance of the proposed approach has been investigated through extensive experiments performed on FERET database. The proposed approach gives a recognition accuracy of 98.16% on FERET database. Comparisons with some of the existing techniques have shown a marked reduction in number of features used per image to produce results more efficiently and without loss of accuracy for gender classification.

In the robust image watermarking framework, watermarks are usually embedded in the direct current (DC) coefficients in discrete cosine transform (DCT) domain, since the DC coefficients have a larger perceptual capacity than any alternating current (AC) coefficients. However, DC coefficients are also excluded from watermark embedding with the consideration of avoiding block artifacts in watermarked images. Studies on human vision suggest that perceptual characteristics can achieve better image fidelity. With this perspective, we propose a novel spatial–perceptual embedding for a color image watermarking algorithm that includes the robust just-noticeable difference (JND) guidance. The logarithmic transform function is used for quantization embedding. Meanwhile, an adaptive quantization step is modeled by incorporating the partial AC coefficients. The novelty and effectiveness of the proposed framework are supported by JND perceptual guidance for spatial pixels. Experiments validate that the proposed watermarking algorithm produces a significantly better performance.

We investigated the application of multiresolution global and local texture features to reduce false-positive detection in a computerized mass detection program. One hundred and sixty-eight digitized mammograms were randomly and equally divided into training and test groups. From these mammograms, two datasets were formed. The first dataset (manual) contained four regions of interest (ROIs) selected manually from each of the mammograms. One of the four ROIs contained a biopsy-proven mass and the other three contained normal parenchyma, including dense, mixed dense/fatty, and fatty tissues. The second dataset (hybrid) contained the manually extracted mass ROIs, along with normal tissue ROIs extracted by an automated Density-Weighted Contrast Enhancement (DWCE) algorithm as false-positive detections. A wavelet transform was used to decompose an ROI into several scales. Global texture features were derived from the low-pass coefficients in the wavelet transformed images. Local texture features were calculated from the suspicious object and the peripheral subregions. Linear discriminant models using effective features selected from the global, local, or combined feature spaces were established to maximize the separation between masses and normal tissue. Receiver Operating Characteristic (ROC) analysis was conducted to evaluate the classifier performance. The classification accuracy using global features were comparable to that using local features. With both global and local features, the average area, Az, under the test ROC curve, reached 0.92 for the manual dataset and 0.96 for the hybrid dataset, demonstrating statistically significant improvement over those obtained with global or local features alone. The results indicated the effectiveness of the combined global and local features in the classification of masses and normal tissue for false-positive reduction.

In this study, we investigated how shapes are classified based on local and global features by four representative convolutional neural networks (CNNs), i.e., AlexNet, VGG, ResNet and Inception. While the local features are based on simple components, such as orientation of line segment, the global features are based on the whole object, such as whether an object has a hole. For example, solid triangles and solid squares are differentiated by local features, solid circles and rings are differentiated by a global feature. Two sets of experiments were performed in this research. In the first experiment, we examined how the four CNNs pre-trained on ImageNet (with transfer learning) learned to differentiate the regular shapes (equilateral-triangles, squares, circles and rings). Our results showed that the pre-trained CNNs exhibited faster learning rates in the tasks discriminating the local features than in the tasks discriminating the global feature. However, the transfer learning of discriminating the global feature in regular shapes were better generalized to irregular shapes than the transfer learning of discriminating local features. In the second experiment, the CNNs were trained from scratch (with random weights initialization) to discriminate local and global features in regular and irregular shapes. Different from the transfer learning, the CNNs exhibited faster learning rates in discriminating the global feature than the local features. Similar to transfer learning, the CNNs exhibited excellent learning generalization to discriminating the global feature of irregular shapes, but poor learning generalization to discriminating the local features in the irregular shapes. The overarching goal of this research is to create a paradigm and benchmark to directly compare how the CNNs and primate visual systems process geometrical invariants. In contrast to the ImageNet approach which employs natural images to train CNNs, we employed the “ShapeNet” approach which features geometrical shapes with well-defined properties. The ShapeNet approach will not only help elucidate the strengths and limitations of CNN computation, but also provide insights into visual information processing in the primates.

This study analyzes the effectiveness of the global (the whole face) and local (regions of eyes, nose, and mouth) features for face recognition. Features describing human faces are encoded in local ternary patterns. The two-class support vector machine is used as the supervised learning algorithm for training recognition models. In the recognition process, recognition modes based on the global features and local features are cascaded. For identifying a face image, the local features are used iteratively for filtering out candidates that can not be clearly identified by the global features, until the one with highest possibility is concluded. The experimental results show that cascading the recognition models of global and local features obtains better classification accuracy than the single classification process.

In colonoscopy examinations, the dark imaging environment of the intestine may result in low-quality scene acquisition and pose challenges for subsequent disease diagnosis. Despite the development of numerous low-light image enhancement (LIE) methods to improve the quality of enhanced colonoscopy images (ECIs), various artifacts such as abnormal brightness and low contrast are still induced. To fairly compare various LIE algorithms and accurately measure the visual quality of ECIs, we develop a novel blind quality evaluator for ECIs by integrating local and global statistical features. For global features, we first extract multiscale gradient similarity features in the multiscale space domain and multi-resolution information entropy features in the discrete cosine transform (DCT) domain to capture their sharpness degradation. Second, we utilize asymmetric generalized Gaussian distribution (AGGD) models to extract fitting parameters in color-opponent components to measure color distortion in ECIs. In addition, a set of intermediate ECIs are generated by adjusting the brightness information, from which perceptual features are computed to capture global brightness variations. For local features, we utilize local binary pattern (LBP) descriptors in the gradient domain and combine them with contrast energy features and local color-aware features to capture local quality degradation. Finally, by integrating both local and global perceptual features, a regression function is employed to build a connection between quality-aware features and subjective ratings. Extensive experiments conducted on publicly available ECIQAD dataset demonstrate that our method outperforms current mainstream blind quality metrics.

A Tone Mapping Operator (TMO) is required to display High Dynamic Range (HDR) images on conventional displays. This thesis proposes a TMO that employs a quantization technique integrated with a Just Noticeable Difference (JND) model, which represents the human visual system's capability to discern variations in brightness. The performance of the proposed algorithm is evaluated by comparing it with that of five state-of-the-art TMOs. The evaluation is divided into two manners. First is an objective evaluation with the Tone Mapping Quality Index (TMQI) method, conducted with two HDR datasets: Reinhard including nine scenes, and Funt including 224 scenes. TMQI revealed that the proposed operator achieved the second-highest score. The second method is a subjective quality evaluation, which involves a paired comparison experiment. The experiments are divided into two parts with different preference criteria: beauty and naturalness, conducted with a Reinhard dataset. The result shows that the proposed operator produced the most preferred images among all six operators, both in terms of beauty and naturalness. The proposed operator is also able to preserve image details in dark and bright areas.

At low sound frequencies, spikes in the auditory nerve (AN) are phase-locked to the stimulus waveform. For sinusoidal stimuli, spike occurrences are restricted to the positive half-waves of the sound cycles, which constraints possible values of spike rates and their variability. As the spike rates are used for representation of sound intensity, the just noticeable differences (JNDs) of sound level and of inter-aural level difference (ILD) will be influenced. Due to the lack of appropriate AN data, we explore the topic using a computational model [1]. Three possible types of spike trains are considered: a synchronous, an asynchronous spike train with variable intensity, and a periodically restarting (with sound period) asynchronous process. Generally, the synchronous process leads to several times lower JNDs than the other processes. The JNDs depend on further parameters: longer counting windows or higher number of converging fibers result in smaller JNDs, larger spike timing jitter (random displacement of spikes from their ideal position) slightly increases the JNDs. Notably, the dependence of JNDs on mean spike rate is in contradiction to psychophysical observations that report monotonic decrease of JND of intensity with increasing sound pressure level, the so called near-miss to Weber's law [2]. Even with ideal phase-lock and no jitter, the simulated JND of the synchronous algorithm has a bell shape, the other two algorithms give monotonically increasing function (Figure ​(Figure1A,1A, colored lines). Recruitment of more AN fibers with rising intensity overcomes this discrepancy; a simulation of this mechanism yields results well matching the psychophysical results (Figure ​(Figure1A,1A, black line). Figure 1 Just-noticeable differences of sound level depending on spike rate (A) and sound frequency (B). Another discrepancy of the modeling results with psychophysics is the simulated JND increasing with frequency, given a constant spike rate (Figure ​(Figure1B),1B), while psychophysical JNDs decrease with frequency. However, if we assume that maximum spike rate is in some way limited by the frequency (e.g., at most one spike can occur in each sound cycle) and that a given intensity range (e.g., 30 dB) is always mapped to the available range of spike rates (e.g., between zero and sound frequency), then the slope of the rate-intensity function increases and the JND decreases with frequency. The results show that intensity coding has specific properties at low sound frequencies, deserving more detailed electrophysiological studies than what is available in experimental literature today.