Improved bi-lateral filter in high dynamic range compression

High Dynamic Range (HDR) image and video technology aims at conveying the full range of perceptible shadow and highlight details with sufficient precision. HDR is regarded by many experts as the next evolution in digital media. However, industrial broadcasters have concerns regarding the bandwidth overhead that this new technology entails. While many consider that broadcasting HDR content would increase bandwidth requirements by around 20%, this number is based on studies where, in addition to the SDR main stream, HDR-related side information is conveyed. A recent subjective evaluation reported that encoding HDR video content in a single layer might require less bandwidth than its associated SDR version. Similar results were discussed in the MPEG ad-hoc group on High Dynamic Range and Wide Color Gamut. In this article, we explain how having more information can result in lower bandwidth requirements. To this end, we describe several limitations of the human vision system that, when exploited, optimize the HDR distribution pipeline for a human observer. Our theoretical assumption about the higher efficiency of HDR is backed up by a statistical analysis of pixel distribution in real images. The Spatial Index objective metric also reconfirms our assumption.

Evaluating the quality of high dynamic range (HDR) images has emerged as a challenging and contemporary topic with the proliferation of HDR content. In this work, a new HDR compression (HDRC) database is proposed, aiming to provide a benchmark for the development of full-reference HDR image quality assessment (IQA) algorithms when facing the latest HDR compression distortions. In particular, the proposed HDRC database is the first HDR-IQA database to incorporate Versatile Video Coding (VVC) compression distortions, closely associated with practical application scenarios. Furthermore, the proposed HDRC database is currently the largest HDR-IQA database, including 80 reference images and 400 distorted images. Extensive experiments are conducted by studying the performance compared to existing HDR-IQA databases when evaluating three HDR-specific IQA models and nine IQA models prevalent for low dynamic range (LDR) content, revealing the challenges the proposed HDRC database brings. The results indicate that the existing IQA models demonstrate noticeable decreases in accuracy when assessing new compression distortions, underscoring the need for the development of novel HDR-IQA models. Consequently, the suggested HDRC database can serve as a potential database for HDR-IQA research, fostering a comprehensive exploration of the associated fields.

In this paper, we propose a tone mapping (TM) method using color correction function (CCF) and image decomposition in high dynamic range (HDR) imaging. The CCF in the proposed TM is derived from the luminance compression function with the color constraint under which the color ratios, between the three color channels of the radiance map and dynamic range compression term, are preserved and color saturation is controlled. The proposed CCF is developed to locally perform the luminance compression and color saturation control in local TM. For image decomposition, we use a bilateral filter and apply the adaptive weight to the base layer of the luminance. Computer simulations with various sets of real, low dynamic range images show the effectiveness of the proposed TM algorithm in terms of the visual quality as well the local contrast. It can be used for contrast and color enhancement in various display and acquisition devices.

Recently, visual representations using high dynamic range (HDR) images become increasingly popular, with advancement of technologies for increasing the dynamic range of image. HDR image is expected to be used in wide-ranging applications such as digital cinema, digital photography and next generation broadcast, because of its high quality and its powerful expression ability. HDR imaging technologies will spread its sphere of influence in imaging industry. In this paper, we review the state-of-the-art studies and the trends of the HDR imaging, in terms of the following three points: (1) HDR imaging sensor and HDR image generation techniques as image acquisition technologies, (2) encode method of HDR images for efficient transmission and storage, (3) human visual system issues associated with reproduction of HDR image.

Medical endoscopic video processing requires real-time execution of color component acquisition, color filter array (CFA) demosaicing, and high dynamic range (HDR) compression under low-light conditions, while adhering to strict thermal constraints within the surgical handpiece. Traditional hardware-aware neural architecture search (NAS) relies on fixed hardware design spaces, making it difficult to balance accuracy, power consumption, and real-time performance. A collaborative “power-accuracy” optimization method is proposed for hardware-aware NAS. Firstly, we proposed a novel hardware modeling framework by abstracting FPGA heterogeneous resources into unified cell units and establishing a power–temperature closed-loop model to ensure that the handpiece surface temperature does not exceed clinical thresholds. In this framework, we constrained the interstage latency balance in pipelines to avoid routing congestion and frequency degradation caused by deep pipelines. Then, we optimized the NAS strategy by using pipeline blocks and combined with a hardware efficiency reward function. Finally, color component acquisition, CFA demosaicing, dynamic range compression, dynamic precision quantization, and streaming architecture are integrated into our framework. Experiments demonstrate that the proposed method achieves 2.8 W power consumption at 47 °C on a Xilinx ZCU102 platform, with a 54% improvement in throughput (vs. hardware-aware NAS), providing an engineer-ready lightweight network for medical edge devices such as endoscopes.

High Dynamic Range (HDR) imaging captures a wide range of luminance existing in real-world scenes. Due to large luminance levels and higher brightness of HDR displays, artefacts can be more noticeable to the Human Visual System (HVS). In a first attempt to experimentally quantify those noticeable levels for HDR images, we pioneered in conducting an exhaustive and comprehensive Just Noticeable Difference (JND) subjective experiment of which the outcome is presented in this paper. Six distortions including JPEG, JPEG2000, noise, blur, contrast change, and quantization artefacts have been considered in the test. The distortions were applied to 10 HDR images in 100 distortion levels resulting a database of 6000 HDR test images. The subjects were asked to find the image JND location on each set of 100 images they had the freedom to explore. The effect of content features on the noticeable threshold selection is investigated per distortion type. Our results in some cases show a significant correlation between content features and JNDs. We are hoping that our results can contribute to further exploitation of a precise HVS model for HDR quality assessment and optimization of the coding and bit allocation in HDR compression.

A number of successful tone mapping operators for contrast compression have been proposed due to the need to visualize high dynamic range (HDR) images on low dynamic range devices. They were inspired by fields as diverse as image processing, photographic practice, and modeling of the human visual systems (HVS). The variety of approaches calls for a systematic perceptual evaluation of their performance. We conduct a psychophysical experiment based on a direct comparison between the appearance of real-world scenes and HDR images of these scenes displayed on a low dynamic range monitor. In our experiment, HDR images are tone mapped by seven existing tone mapping operators. The primary interest of this psychophysical experiment is to assess the differences in how tone mapped images are perceived by human observers and to find out which attributes of image appearance account for these differences when tone mapped images are compared directly with their corresponding real-world scenes rather than with each other. The human subjects rate image naturalness, overall contrast, overall brightness, and detail reproduction in dark and bright image regions with respect to the corresponding real-world scene. The results indicate substantial differences in perception of images produced by individual tone mapping operators. We observe a clear distinction between global and local operators in favor of the latter, and we classify the tone mapping operators according to naturalness and appearance attributes.

Typical digital cameras exhibit a limitation regarding the dynamic range of the scene radiance they can capture. High Dynamic Range (HDR) imaging refers to methods and systems that aim to generate images that exhibit higher dynamic range between the lightest and the darkest parts of the an image. A typical approach for generating HDR images is exposure bracketing where multiple frames, each one with a different exposure setting, are captured and combined to a HDR image of the scene. The large number of images that exposure bracketing requires often leads to motion artefacts that limit the visual quality of the resulting HDR image. In this work, we propose a novel approach in HDR imaging that significantly reduces the necessary number of images. In our proposed system, we employ the notion of random exposure where each pixel of a single frame collects light for a random amount of time. By collecting a small number of such images, the full sequence of low dynamic range images can be reconstructed and subsequently used for HDR generation. The problem is solved by casting the reconstruction of the sequence as a nuclear norm minimization problem following the premises of the recently proposed theory of Matrix Completion. Experimental results suggest that the proposed method is able to reconstruct the sequence from as low as 20% of the images that traditional techniques require with minimal reduction in image quality.

In this paper we present a new technique for the display of High Dynamic Range (HDR) images on Low Dynamic Range (LDR) displays. The described process has three stages. First, the input image is segmented into luminance zones. Second, the tone mapping operator (TMO) that performs better in each zone is automatically selected. Finally, the resulting tone mapping (TM) outputs for each zone are merged, generating the final LDR output image. To establish the TMO that performs better in each luminance zone we conducted a preliminary psychophysical experiment using a set of HDR images and six different TMOs. We validated our composite technique on several (new) HDR images and conducted a further psychophysical experiment, using an HDR display as reference, that establishes the advantages of our hybrid three-stage approach over a traditional individual TMO.

High dynamic range (HDR) video compression has until now been approached by using the high profile of existing state-of-the-art H.264/AVC (Advanced Video Coding) codec or by separately encoding low dynamic range (LDR) video and the residue resulted from the estimation of HDR video from LDR video. Although the latter approach has a distinctive advantage of providing backward compatibility to 8-bit LDR displays, the superiority of one approach to the other in terms of the rate distortion trade-off has not been verified yet. In this paper, we first give a detailed overview of the methods in these two approaches. Then, we experimentally compare two approaches with respect to different objective and perceptual metrics, such as HDR mean square error (HDR MSE), perceptually uniform peak signal to noise ratio (PU PSNR) and HDR visible difference predictor (HDR VDP). We first conclude that the optimized methods for backward compatibility to 8-bit LDR displays are superior to the method designed for high profile encoder both for 8-bit and 12-bit mappings in terms of all metrics. Second, using higher bit-depths with a high profile encoder is giving better rate-distortion performances than employing an 8-bit mapping with an 8-bit encoder for the same method, in particular when the dynamic range of the video sequence is high. Third, rather than encoding of the residue signal in backward compatible methods, changing the quantization step size of the LDR layer encoder would be sufficient to achieve a required quality. In other words, the quality of tone mapping is more important than residue encoding for the performance of HDR image and video coding.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

High dynamic range (HDR) images are becoming pervasive due to capturing or rendering of a wider range of luminance, but their special display equipment is difficult to be popularized because of high cost and technological problem. Thus, HDR images must be adapted to the conventional display devices by applying tone mapping (TM) operation, which puts forward higher requirements for intellectual property protection of HDR images. As the robustness presents regional diversity in the low dynamic range (LDR) watermarked image after TM, which is different from the traditional watermarking technologies, a concept of watermarking activity is defined and used to distinguish the essential distinction of watermarking between LDR image and HDR image in this paper. Then, a novel robust HDR image watermarking algorithm is proposed against TM operations. Firstly, based on the hybrid processing of redundant discrete wavelet transform and singular value decomposition, the watermark is embedded by modifying the structure information of the HDR image. Distinguished from LDR image watermarking, the high embedding strength can cause more obvious distortion in the high brightness regions of HDR image than the low brightness regions. Thus, a perceptual brightness mask with low complexity is designed to improve the imperceptibility further. Experimental results show that the proposed algorithm is robust to the existing TM operations, with taking into account the imperceptibility and embedded capacity, which is superior to the current state-of-art HDR image watermarking algorithms.

The present paper proposes a blind multi-bit watermarking method for High Dynamic Range (HDR) images. The proposed approach is designed in order to guarantee the watermark imperceptibility both in the HDR marked image and in its Low Dynamic Range (LDR) counterpart, being thus robust against significant non-linear distortions such as those performed by tone-mapping operators (TMOs). In order to do so, the wavelet transform of the Just Noticeable Difference (JND)- scaled space of the original HDR image is employed as embedding domain. Moreover, a visual mask taking into account specific aspects of the Human Visual System (HVS) is exploited to improve the quality of the resulting watermarked image. Specifically, bilateral filtering is used to locate information on the detail part of the HDR image, where the watermark should be preferably embedded. A contrast sensitivity function is also employed to modulate the watermark intensity in each wavelet decomposition subband according to its scale and orientation. An extensive set of experimental results testifies the effectiveness of the proposed scheme in embedding multi-bit watermarks into HDR images without affecting the visual quality of the original image, while being robust against TMOs.

A deep learning based relative clarity classification method for infrared and visible image fusion

A novel compressing method of high dynamic range image based on fast integrated surround Retinex model is proposed in this paper. The proposed method has two novelties. First, multiscale surround images are integrated to a single surround field, which is applied to center/surround single-scale Retinex (SSR) model. The method reduces the "banding artifact" seen in normal SSR and simplifies the complicated computational steps in conventional multiscale Retinex. Second, the Gaussian pyramid method is introduced to cut the computation time for generating a large-scale surround by tracing a "reduction" and "expansion" sequences using down and up sampling followed by linear interpolation. The computational expense is dramatically saved less than 1/100 for getting a surround by Gaussian convolution with large kernel size. The proposed model worked well in compressing the dynamic range and improving the visibility in heavy shadow areas of natural color images while preserving pleasing contrast.