Realistic camera noise modeling with application to improved HDR synthesis

Artifact remains a long-standing challenge in High Dynamic Range (HDR) reconstruction. Existing methods focus on model designs for artifact mitigation but ignore explicit detection and suppression strategies. Because artifact lacks clear boundaries, distinct shapes, and semantic consistency, and there is no existing dedicated dataset for HDR artifact, progress in direct artifact detection and recovery is impeded. To bridge the gap, we propose a unified HDR reconstruction framework that integrates artifact detection and model optimization. Firstly, we build the first HDR artifact dataset (HADataset), comprising 1,213 diverse multi-exposure Low Dynamic Range (LDR) image sets and 1,765 HDR image pairs with per-pixel artifact annotations. Secondly, we develop an effective HDR artifact detector (HADetector), a robust artifact detection model capable of accurately localizing HDR reconstruction artifact. HADetector plays two pivotal roles: (1) enhancing existing HDR reconstruction models through fine-tuning, and (2) serving as a non-reference image quality assessment (NR-IQA) metric, the Artifact Score (AS), which aligns closely with human visual perception for reliable quality evaluation. Extensive experiments validate the effectiveness and generalizability of our framework, including the HADataset, HADetector, fine-tuning paradigm, and AS metric. The code and datasets are available at: https://github.com/xinyueliii/hdr-artifact-detect-optimize.

Existing single image high dynamic range (HDR) reconstruction methods attempt to expand the range of illuminance. They are not effective in generating plausible textures and colors in the reconstructed results, especially for high-density pixels in ultra-high-definition (UHD) images. To address these problems, we propose a new HDR reconstruction network for UHD images by collaboratively learning color and texture details. First, we propose a dual-path network to extract the content and chromatic features at a reduced resolution of the low dynamic range (LDR) input. These two types of features are used to fit bilateral-space affine models for real-time HDR reconstruction. To extract the main data structure of the LDR input, we propose to use 3D Tucker decomposition and reconstruction to prevent pseudo edges and noise amplification in the learned bilateral grid. As a result, the high-quality content and chromatic features can be reconstructed capitalized on guided bilateral upsampling. Finally, we fuse these two full-resolution feature maps into the HDR reconstructed results. Our proposed method can achieve real-time processing for UHD images (about 160 fps). Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art HDR reconstruction approaches on public benchmarks and real-world UHD images.

Panoramic imaging research on geometry recovery and High Dynamic Range (HDR) reconstruction becomes a trend with the development of Extended Reality (XR). Neural Radiance Fields (NeRF) provide a promising scene representation for both tasks without requiring extensive prior data. How- ever, in the case of inputting sparse Low Dynamic Range (LDR) panoramic images, NeRF often degrades with under-constrained geometry and is unable to reconstruct HDR radiance from LDR inputs. We observe that the radiance from each pixel in panoramic images can be modeled as both a signal to convey scene lighting information and a light source to illuminate other pixels. Hence, we propose the irradiance fields from sparse LDR panoramic images, which increases the observation counts for faithful geometry recovery and leverages the irradiance-radiance attenuation for HDR reconstruction. Extensive experiments demonstrate that the irradiance fields outperform state-of-the-art methods on both geometry recovery and HDR reconstruction and validate their effectiveness. Furthermore, we show a promising byproduct of spatially-varying lighting estimation. The code is available at https://github.com/Lu-Zhan/Pano-NeRF.

Due to limited camera capacities, digital images usually have a narrower dynamic illumination range than real-world scene radiance. To resolve this problem, High Dynamic Range (HDR) reconstruction is proposed to recover the dynamic range to better represent real-world scenes. However, due to different physical imaging parameters, the tone-mapping functions between images and real radiance are highly diverse, which makes HDR reconstruction extremely challenging. Existing solutions can not explicitly clarify a corresponding relationship between the tone-mapping function and the generated HDR image, but this relationship is vital when guiding the reconstruction of HDR images. To address this problem, we propose a method to explicitly estimate the tone mapping function and its corresponding HDR image in one network. Firstly, based on the characteristics of the tone mapping function, we construct a model by a polynomial to describe the trend of the tone curve. To fit this curve, we use a learnable network to estimate the coefficients of the polynomial. This curve will be automatically adjusted according to the tone space of the Low Dynamic Range (LDR) image, and reconstruct the real HDR image. Besides, since all current datasets do not provide the corresponding relationship between the tone mapping function and the LDR image, we construct a new dataset with both synthetic and real images. Extensive experiments show that our method generalizes well under different tone-mapping functions and achieves SOTA performance. The code/dataset is available at https://github.com/jqtangust/EPCE-HDR.git.

This paper addresses the conversion problem of High Dynamic Range (HDR) images into Low Dynamic Range (LDR) images. The aim of this conversion is to ensure a good visual rendering of the displayed Tone Mapped (TM) HDR images in accordance with the observers' assessment. To do so, the proposed algorithm adjusts a piecewise linear function to the logarithm luminance distribution of the HDR image to adapt this transformation to the perceptual quantizer in accordance with the Human Visual System (HVS). The computation of the slope value of the piecewise linear transformation is simplified. Indeed, it is given by the ratio between the probability and the logarithm luminance distance in the sampling bin. This leads to histograms which tend to be flat, thus inducing a contrast enhancement of the tone mapped HDR images in both under-exposed and overexposed areas. Simulation results provide good results, both in terms of visual quality and TMQI metric, compared to existing competitive TM approaches.

International audience

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.

High dynamic range (HDR) image generation had been studied for years. Due to the expense and rareness of HDR cameras, many works try to generate HDR images using several low dynamic range (LDR) images with different exposure conditions. To ensure high-quality HDR image generation, details of the scene should be retained in different LDR images and the exposure parameters of LDR image should be chosen carefully. In this paper, we propose a technique to dynamically determine the suitable exposure parameter for LDR images according to the property of each scene to be captured. Experimental results reveal that better HDR images are always generated with the LDR images captured by the determined exposure, compared to those created by the method with fixed exposures.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

High dynamic range (HDR) images are becoming pervasive due to capturing or rendering of a wider range of luminance, but their special display equipment is difficult to be popularized because of high cost and technological problem. Thus, HDR images must be adapted to the conventional display devices by applying tone mapping (TM) operation, which puts forward higher requirements for intellectual property protection of HDR images. As the robustness presents regional diversity in the low dynamic range (LDR) watermarked image after TM, which is different from the traditional watermarking technologies, a concept of watermarking activity is defined and used to distinguish the essential distinction of watermarking between LDR image and HDR image in this paper. Then, a novel robust HDR image watermarking algorithm is proposed against TM operations. Firstly, based on the hybrid processing of redundant discrete wavelet transform and singular value decomposition, the watermark is embedded by modifying the structure information of the HDR image. Distinguished from LDR image watermarking, the high embedding strength can cause more obvious distortion in the high brightness regions of HDR image than the low brightness regions. Thus, a perceptual brightness mask with low complexity is designed to improve the imperceptibility further. Experimental results show that the proposed algorithm is robust to the existing TM operations, with taking into account the imperceptibility and embedded capacity, which is superior to the current state-of-art HDR image watermarking algorithms.

High Dynamic Range (HDR) images capture the full range of luminance present in real world scenes, and unlike Low Dynamic Range (LDR) images, can simultaneously contain detailed information in the deepest of shadows and the brightest of light sources. In order to render HDR image on LDR displayers, it is often necessary to create LDR depictions of HDR images at the cost of contrast information loss. To reduce the loss, this paper enables to render HDRI (High Dynamic Range Image) with multiple low-bit images periodically. From the viewpoint of a human, the pixel value is fractural. It does not adjust the tones but can reconstruct HDR images.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

Tone-mapping-operators (TMOs) provide a useful means for converting high dynamic range (HDR) images to low dynamic range (LDR) images so that they can be viewed on standard displays, but this may influence the visual quality of experience (QoE) of the end-user. There is a need to understand the impact of TMOs to inform the choice of TMO algorithms for different displays, especially for small-screen-devices (SSDs) such as those used in mobile phones. This is important, as mobile devices are becoming the primary means of consuming multimedia contents. However, few studies have been undertaken to assess the impact of TMOs and viewing devices (especially SSDs) on the visual QoE of the user when using. In this chapter, we evaluate subjectively and objectively, the commonly used TMOs in different displays and resolutions for colour and grey-scale HDR images. Our results show that viewing devices have an influence on the TMOs performance, suggesting the need for a careful choice of TMO to enhance the viewing-QoE of the end-user. As expected, the higher resolution, the better HDR-image quality. Surprisingly, there was no significant difference between the Mean of Opinion Score (MOS) scores for colour and grey-scale images in SSDs. The device and TMOs affect QoE for colour and grey HDR-image equally. We found Shannon entropy (SE) to be a good objective measure of quality for colour and grey HDR images, suggesting that entropy may find use in automated HDR quality control assessment schemes, while; HDR-VDP-2 is a good objective measure for colour HDR image only.

Recently, spatially varying Bidirectional Reflectance Distribution Functions (svBRDF) is widely used as a model to characterize the appearance of materials with varying visual properties over the surface. One of the challenges in image-based svBRDF capture systems rises for surfaces with high specularity and sparkles, which require a dynamic range higher than the dynamic range of cameras. High Dynamic Range Imaging (HDRI) for svBRDF systems with multispectral camera has not been addressed properly in the literature. In HDRI, Camera Response Function (CRF) plays a crucial role in the precision of results specially when measuring metrological data such as spectral svBRDF. In this work, we investigate the effect of CRF assessment on the precision of measurement. Therefore, we have conducted two experiments to measure absolute CRF using reflective chart method as well as estimate relative CRF by Debevec and Malik’s method for a filter wheel multispectral camera to be used in a svBRDF setup. Results are evaluated on two levels: radiance map construction and reflectance calculation, by comparing to telespectroradiometer measurements as ground truth data. Results showed that although the HDRI with measured absolute CRF outputs radiance measurements with the same physical units and in the same scale as ground truth data, HDRI with estimated relative CRF outperformed in terms of the precision of reflectance measurement.

Due to the growing popularity of High Dynamic Range (HDR) images and HDR displays, a large amount of existing Low Dynamic Range (LDR) images are required to be converted to HDR format to benefit HDR advantages, which give rise to some LDR to HDR algorithms. Most of these algorithms especially tackle overexposed areas during expanding, which is the potential to make the image quality worse than that before processing and introduces artifacts. To dispel these problems, we . present a new,LDR to HDR approach, unlike the existing techniques, it focuses on avoiding sophisticated treatment to overexposed areas in dynamic range expansion step. Based on a separating principle, firstly, according to the familiar types of overexposure, the overexposed areas are classified into two categories which are removed and corrected respectively by two kinds of techniques. Secondly, for maintaining color consistency, color recovery is carried out to the preprocessed images. Finally, the LDR image is expanded to HDR. Experiments show that the proposed approach performs well and produced images become more favorable and suitable for applications. The image quality metric also illustrates that we can reveal more details without causing artifacts introduced by other algorithms.