Rendering-Aware HDR Environment Map Prediction from a Single Image

Recently, high dynamic range (HDR) imaging has attracted much attention as a technology to reflect human visual characteristics owing to the development of the display and camera technology. This paper proposes a novel deep neural network model that reconstructs an HDR image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively simple for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range but also has the advantage of restoring the light information of the actual physical world. The proposed method is an end-to-end reconstruction process, and it has the advantage of being able to easily combine a network to extend an additional range. In the experimental results, the proposed method shows quantitative and qualitative improvement in performance, compared with the conventional algorithms.

High dynamic range (HDR) image generation had been studied for years. Due to the expense and rareness of HDR cameras, many works try to generate HDR images using several low dynamic range (LDR) images with different exposure conditions. To ensure high-quality HDR image generation, details of the scene should be retained in different LDR images and the exposure parameters of LDR image should be chosen carefully. In this paper, we propose a technique to dynamically determine the suitable exposure parameter for LDR images according to the property of each scene to be captured. Experimental results reveal that better HDR images are always generated with the LDR images captured by the determined exposure, compared to those created by the method with fixed exposures.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

This study considers a method for reconstructing a high dynamic range (HDR) original image from a single saturated low dynamic range (LDR) image of metallic objects. A deep neural network approach was adopted for the direct mapping of an 8-bit LDR image to HDR. An HDR image database was first constructed using a large number of various metallic objects with different shapes. Each captured HDR image was clipped to create a set of 8-bit LDR images. All pairs of HDR and LDR images were used to train and test the network. Subsequently, a convolutional neural network (CNN) was designed in the form of a deep U-Net-like architecture. The network consisted of an encoder, a decoder, and a skip connection to maintain high image resolution. The CNN algorithm was constructed using the learning functions in MATLAB. The entire network consisted of 32 layers and 85,900 learnable parameters. The performance of the proposed method was examined in experiments using a test image set. The proposed method was also compared with other methods and confirmed to be significantly superior in terms of reconstruction accuracy, histogram fitting, and psychological evaluation.

Deep convolutional neural networks (CNNs) have recently made significant advances in the inverse tone mapping technique, which generates a high dynamic range (HDR) image from a single low dynamic range (LDR) image that has lost information in over- and under-exposed regions. The end-to-end inverse tone mapping approach specifies the dynamic range in advance, thereby limiting dynamic range expansion. In contrast, the method of generating multiple exposure LDR images from a single LDR image and subsequently merging them into an HDR image enables flexible dynamic range expansion. However, existing methods for generating multiple exposure LDR images require an additional network for each exposure value to be changed or a process of recursively inferring images that have different exposure values. Therefore, the number of parameters increases significantly due to the use of additional networks, and an error accumulation problem arises due to recursive inference. To solve this problem, we propose a novel network architecture that can control arbitrary exposure values without adding networks or applying recursive inference. The training method of the auxiliary classifier-generative adversarial network structure is employed to generate the image conditioned on the specified exposure. The proposed network uses a newly designed spatially-adaptive normalization to address the limitation of existing methods that cannot sufficiently restore image detail due to the spatially equivariant nature of the convolution. Spatially-adaptive normalization facilitates restoration of the high frequency component by applying different normalization parameters to each element in the feature map according to the characteristics of the input image. Experimental results show that the proposed method outperforms state-of-the-art methods, yielding a 5.48dB higher average peak signal-to-noise ratio, a 0.05 higher average structure similarity index, a 0.28 higher average multi-scale structure similarity index, and a 7.36 higher average HDR-VDP-2 for various datasets.

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR-HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

A fixed-point tone mapping operation (TMO) instead of a floating-point TMO is proposed in this paper. A TMO generates a low dynamic range (LDR) image from a high dynamic range (HDR) image. Since pixel values of an HDR image are generally expressed in a floating-point data format, e.g. RGBE and OpenEXR, a TMO is also implemented in floating-point arithmetic in conventional approaches. However, it requires a huge computational cost, even though a resulting LDR image is expressed in simple integers. The proposed TMO is implemented with only fixed-point arithmetic. As a result, the proposed method reduces a computational cost. Experimental results shows the PSNR of LDR images in the proposed method are comparable to those of the conventional methods.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

Due to limited camera capacities, digital images usually have a narrower dynamic illumination range than real-world scene radiance. To resolve this problem, High Dynamic Range (HDR) reconstruction is proposed to recover the dynamic range to better represent real-world scenes. However, due to different physical imaging parameters, the tone-mapping functions between images and real radiance are highly diverse, which makes HDR reconstruction extremely challenging. Existing solutions can not explicitly clarify a corresponding relationship between the tone-mapping function and the generated HDR image, but this relationship is vital when guiding the reconstruction of HDR images. To address this problem, we propose a method to explicitly estimate the tone mapping function and its corresponding HDR image in one network. Firstly, based on the characteristics of the tone mapping function, we construct a model by a polynomial to describe the trend of the tone curve. To fit this curve, we use a learnable network to estimate the coefficients of the polynomial. This curve will be automatically adjusted according to the tone space of the Low Dynamic Range (LDR) image, and reconstruct the real HDR image. Besides, since all current datasets do not provide the corresponding relationship between the tone mapping function and the LDR image, we construct a new dataset with both synthetic and real images. Extensive experiments show that our method generalizes well under different tone-mapping functions and achieves SOTA performance. The code/dataset is available at https://github.com/jqtangust/EPCE-HDR.git.

This paper introduces region of interest (ROI) as a functionality to local tone mappings of high dynamic range (HDR) radiance maps. In the proposed scheme, a new quadratic regularization method is provided to decompose the luminance of an HDR radiance map into a base layer with large gradients and a detail layer with small gradients. An ROI based global tone mapping is designed to compress the base layer. Since the dynamic range of a display device is limited, more dynamic ranges are assigned to the base layer in the ROI such that the visual quality of the ROI is preserved better. The detail layer can be amplified to enhance local contrasts such that the output LDR images look sharper. Both automatic and interactive modes are provided. With our automatic mode, the generated low dynamic range (LDR) images look natural and local details are also preserved very well. In addition, a user friendly interactive mode is provided which is also able to produce dynamic LDR images in real time.

We present a high dynamic range (HDR) imaging algorithm that utilizes a modern rank minimization framework. Linear dependency exists among low dynamic range (LDR) images. However, global or local misalignment by camera motion and moving objects breaks down the low-rank structure of LDR images. The proposed algorithm simultaneously estimates global geometric transforms to align LDR images and detects moving objects and under-/over-exposed regions using a rank minimization approach. In the HDR composition step, structural consistency weighting is proposed to generate an artifact-free HDR image from an user-selected reference image. We demonstrate the robustness and effectiveness of the proposed method with real datasets.

Recently, various deep learning-based high dynamic range (HDR) imaging methods have been proposed to overcome the performance limitation of simply merging or fusion of multiple low dynamic range (LDR) images. However, early deep learning-based methods did not fully exploit the inter-image relationship between different exposures. T o consider inter-image features without the dataset dependency, this paper presents a novel HDR method using three-dimensional residual dense network (3D RDN). We first reconstruct a set of differently exposed LDR images in video format. Next, we take the reconstructed video as an input to the 3D RDN, which generates an HDR image as output. Experimental results show that the proposed method provides higher score compared with state-of-the-arts HDR methods. To the best of authors knowledge, the proposed work first adopted the 3D RDN in the field of HDR imaging using more than three LDR images without any additional pre-or post-processing.

High dynamic range(HDR) imaging is the task of re-covering HDR image from one or multiple input Low Dynamic Range (LDR) images. In this paper, we present Gamma-enhanced Spatial Attention Network(GSANet), a novel framework for reconstructing HDR images. This problem comprises two intractable challenges of how to tackle overexposed and underexposed regions and how to overcome the paradox of performance and complexity trade-off. To address the former, after applying gamma correction on the LDR images, we adopt a spatial attention module to adaptively select the most appropriate regions of various exposure low dynamic range images for fusion. For the latter one, we propose an efficient channel attention module, which only involves a handful of parameters while bringing clear performance gain. Experimental results show that the proposed method achieves better visual quality on the HDR dataset. The code will be available at: https://github.com/fancyicookie/GSANet

We report a microlens array camera with variable apertures (MACVA) for high dynamic range (HDR) imaging by using microlens arrays with various sizes of apertures. The MACVA comprises variable apertures, microlens arrays, gap spacers, and a CMOS image sensor. The microlenses with variable apertures capture low dynamic range (LDR) images with different f-stops under single-shot exposure. The reconstructed HDR images clearly exhibit expanded dynamic ranges surpassing LDR images as well as high resolution without motion artifacts, comparable to the maximum MTF50 value observed among the LDR images. This compact camera provides, what we believe to be, a new perspective for various machine vision or mobile devices applications.

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.