Real-Time High Dynamic Range Texture Mapping

This paper presents a novel High Dynamic Range (HDR) tone mapping (TM) system based on sub-band architecture. Standard wavelet filters of Daubechies, Symlets, Coiflets and Biorthogonal were used to estimate the proposed system performance in terms of Low Dynamic Range (LDR) image quality and reconstructed HDR image fidelity. During TM stage, the HDR image is firstly decomposed in sub-bands using symmetrical analysis-synthesis filter bank. The transform coefficients are then rescaled using a predefined gain map. The inverse Tone Mapping (iTM) stage is straightforward. Indeed, the LDR image passes through the same sub-band architecture. But, instead of reducing the dynamic range, the LDR content is boosted to an HDR representation. Moreover, in our TM sheme, we included an optimization module to select the gain map components that minimize the reconstruction error, and consequently resulting in high fidelity HDR content. Comparisons with recent state-of-the-art methods have shown that our method provides better results in terms of visual quality and HDR reconstruction fidelity using objective and subjective evaluations.

Typical digital cameras exhibit a limitation regarding the dynamic range of the scene radiance they can capture. High Dynamic Range (HDR) imaging refers to methods and systems that aim to generate images that exhibit higher dynamic range between the lightest and the darkest parts of the an image. A typical approach for generating HDR images is exposure bracketing where multiple frames, each one with a different exposure setting, are captured and combined to a HDR image of the scene. The large number of images that exposure bracketing requires often leads to motion artefacts that limit the visual quality of the resulting HDR image. In this work, we propose a novel approach in HDR imaging that significantly reduces the necessary number of images. In our proposed system, we employ the notion of random exposure where each pixel of a single frame collects light for a random amount of time. By collecting a small number of such images, the full sequence of low dynamic range images can be reconstructed and subsequently used for HDR generation. The problem is solved by casting the reconstruction of the sequence as a nuclear norm minimization problem following the premises of the recently proposed theory of Matrix Completion. Experimental results suggest that the proposed method is able to reconstruct the sequence from as low as 20% of the images that traditional techniques require with minimal reduction in image quality.

International audience

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

High dynamic range (HDR) images are becoming pervasive due to capturing or rendering of a wider range of luminance, but their special display equipment is difficult to be popularized because of high cost and technological problem. Thus, HDR images must be adapted to the conventional display devices by applying tone mapping (TM) operation, which puts forward higher requirements for intellectual property protection of HDR images. As the robustness presents regional diversity in the low dynamic range (LDR) watermarked image after TM, which is different from the traditional watermarking technologies, a concept of watermarking activity is defined and used to distinguish the essential distinction of watermarking between LDR image and HDR image in this paper. Then, a novel robust HDR image watermarking algorithm is proposed against TM operations. Firstly, based on the hybrid processing of redundant discrete wavelet transform and singular value decomposition, the watermark is embedded by modifying the structure information of the HDR image. Distinguished from LDR image watermarking, the high embedding strength can cause more obvious distortion in the high brightness regions of HDR image than the low brightness regions. Thus, a perceptual brightness mask with low complexity is designed to improve the imperceptibility further. Experimental results show that the proposed algorithm is robust to the existing TM operations, with taking into account the imperceptibility and embedded capacity, which is superior to the current state-of-art HDR image watermarking algorithms.

Tone-mapping-operators (TMOs) provide a useful means for converting high dynamic range (HDR) images to low dynamic range (LDR) images so that they can be viewed on standard displays, but this may influence the visual quality of experience (QoE) of the end-user. There is a need to understand the impact of TMOs to inform the choice of TMO algorithms for different displays, especially for small-screen-devices (SSDs) such as those used in mobile phones. This is important, as mobile devices are becoming the primary means of consuming multimedia contents. However, few studies have been undertaken to assess the impact of TMOs and viewing devices (especially SSDs) on the visual QoE of the user when using. In this chapter, we evaluate subjectively and objectively, the commonly used TMOs in different displays and resolutions for colour and grey-scale HDR images. Our results show that viewing devices have an influence on the TMOs performance, suggesting the need for a careful choice of TMO to enhance the viewing-QoE of the end-user. As expected, the higher resolution, the better HDR-image quality. Surprisingly, there was no significant difference between the Mean of Opinion Score (MOS) scores for colour and grey-scale images in SSDs. The device and TMOs affect QoE for colour and grey HDR-image equally. We found Shannon entropy (SE) to be a good objective measure of quality for colour and grey HDR images, suggesting that entropy may find use in automated HDR quality control assessment schemes, while; HDR-VDP-2 is a good objective measure for colour HDR image only.

Recently, visual representations using high dynamic range (HDR) images become increasingly popular, with advancement of technologies for increasing the dynamic range of image. HDR image is expected to be used in wide-ranging applications such as digital cinema, digital photography and next generation broadcast, because of its high quality and its powerful expression ability. HDR imaging technologies will spread its sphere of influence in imaging industry. In this paper, we review the state-of-the-art studies and the trends of the HDR imaging, in terms of the following three points: (1) HDR imaging sensor and HDR image generation techniques as image acquisition technologies, (2) encode method of HDR images for efficient transmission and storage, (3) human visual system issues associated with reproduction of HDR image.

In this paper, we propose a tone mapping (TM) method using color correction function (CCF) and image decomposition in high dynamic range (HDR) imaging. The CCF in the proposed TM is derived from the luminance compression function with the color constraint under which the color ratios, between the three color channels of the radiance map and dynamic range compression term, are preserved and color saturation is controlled. The proposed CCF is developed to locally perform the luminance compression and color saturation control in local TM. For image decomposition, we use a bilateral filter and apply the adaptive weight to the base layer of the luminance. Computer simulations with various sets of real, low dynamic range images show the effectiveness of the proposed TM algorithm in terms of the visual quality as well the local contrast. It can be used for contrast and color enhancement in various display and acquisition devices.

This paper presents an end-to-end workflow for high dynamic range (HDR) content acquisition and HDR representation that aims to reproduce in a perceptually realistic manner the appearance of the captured scene. The proposed workflow includes a camera-independent colour calibration method for accurate HDR content acquisition and a perceptually optimized HDR signal representation method that takes into account display and viewing conditions. Subjective evaluation of the proposed and other methods indicate that the proposed workflow offers statistically significant improvements in the quality of the HDR image/video displayed to the viewers relative to the anchor methods.

We conducted a large-scale study of human perceptual quality judgments of High Dynamic Range (HDR) and Standard Dynamic Range (SDR) videos subjected to scaling and compression levels and viewed on three different display devices. While conventional expectations are that HDR quality is better than SDR quality, we have found subject preference of HDR versus SDR depends heavily on the display device, as well as on resolution scaling and bitrate. To study this question, we collected more than 23,000 quality ratings from 67 volunteers who watched 356 videos on OLED, QLED, and LCD televisions, and among many other findings, observed that HDR videos were often rated as lower quality than SDR videos at lower bitrates, particularly when viewed on LCD and QLED displays. Since it is of interest to be able to measure the quality of videos under these scenarios, e.g. to inform decisions regarding scaling, compression, and SDR vs HDR, we tested several well-known full-reference and no-reference video quality models on the new database. Towards advancing progress on this problem, we also developed a novel no-reference model called HDRPatchMAX, that uses a contrast-based analysis of classical and bit-depth features to predict quality more accurately than existing metrics.

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.

Tone mapping (TM) algorithms reproduce the high dynamic range (HDR) images on low dynamic range (LDR) display devices such as monitors or printers. In this paper, we propose a local TM algorithm, in which the HDR input is segmented using the K-means algorithm and a display gamma parameter is set automatically for each segmented region. The proposed TM algorithm computes the luminance of an input that is the radiance map generated from a set of LDR images acquired with varying exposure settings. Then, according to the bilateral filtered luminance, an image is divided into a number of regions using the K-means algorithm. The display gamma value is set automatically according to the mean value of each region. Then, the tone of HDR image is reproduced by a local TM method with adaptive gamma value. We generate the tone-mapped image using the proposed local TM. Computer simulation with real LDR images shows the effectiveness of the proposed local TM algorithm in terms of the visual quality as well as the local contrast. It can be used for contrast and color enhancement in various display and acquisition devices.

High dynamic range imaging techniques involve capturing and storing real world radiance values that span many orders of magnitude. However, common display devices can usually reproduce intensity ranges only up to two to three orders of magnitude. Therefore, in order to display a high dynamic range image on a low dynamic range screen, the dynamic range of the image needs to be compressed without losing details or introducing artefacts, and this process is called tone mapping. A good tone mapping operator must be able to produce a low dynamic range image that matches as much as possible the perception of the real world scene. We propose a two stage tone mapping approach, in which the first stage is a global method for range compression based on a gamma curve that equalizes the lightness histogram the best, and the second stage performs local contrast enhancement and color induction using neural activity models for the visual cortex. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

Due to limited camera capacities, digital images usually have a narrower dynamic illumination range than real-world scene radiance. To resolve this problem, High Dynamic Range (HDR) reconstruction is proposed to recover the dynamic range to better represent real-world scenes. However, due to different physical imaging parameters, the tone-mapping functions between images and real radiance are highly diverse, which makes HDR reconstruction extremely challenging. Existing solutions can not explicitly clarify a corresponding relationship between the tone-mapping function and the generated HDR image, but this relationship is vital when guiding the reconstruction of HDR images. To address this problem, we propose a method to explicitly estimate the tone mapping function and its corresponding HDR image in one network. Firstly, based on the characteristics of the tone mapping function, we construct a model by a polynomial to describe the trend of the tone curve. To fit this curve, we use a learnable network to estimate the coefficients of the polynomial. This curve will be automatically adjusted according to the tone space of the Low Dynamic Range (LDR) image, and reconstruct the real HDR image. Besides, since all current datasets do not provide the corresponding relationship between the tone mapping function and the LDR image, we construct a new dataset with both synthetic and real images. Extensive experiments show that our method generalizes well under different tone-mapping functions and achieves SOTA performance. The code/dataset is available at https://github.com/jqtangust/EPCE-HDR.git.

This paper proposes a High Dynamic Range (HDR) image Tone Mapping (TM) approach converting HDR images into Low Dynamic Range (LDR) images so as the tone mapped HDR images ensure good visual quality on LDR display devices in accordance with the observers’ assessment. The proposed algorithm adjusts the logarithm luminance distribution of the HDR image according to a perceptual quantizer with respect to the Human Visual System (HVS) using a piecewise linear function. The parameters’s approximation are induced by L M -norms (l 1 , l 2 , l ∞ ) and the performance of the proposed algorithm is discussed. Simulation results provide good results, both in terms of visual quality and TMQI metric, compared to existing competitive TM approaches.