Visual saliency guided exposure fusion

High dynamic range (HDR) image generation had been studied for years. Due to the expense and rareness of HDR cameras, many works try to generate HDR images using several low dynamic range (LDR) images with different exposure conditions. To ensure high-quality HDR image generation, details of the scene should be retained in different LDR images and the exposure parameters of LDR image should be chosen carefully. In this paper, we propose a technique to dynamically determine the suitable exposure parameter for LDR images according to the property of each scene to be captured. Experimental results reveal that better HDR images are always generated with the LDR images captured by the determined exposure, compared to those created by the method with fixed exposures.

Recently, high dynamic range (HDR) imaging has attracted much attention as a technology to reflect human visual characteristics owing to the development of the display and camera technology. This paper proposes a novel deep neural network model that reconstructs an HDR image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively simple for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range but also has the advantage of restoring the light information of the actual physical world. The proposed method is an end-to-end reconstruction process, and it has the advantage of being able to easily combine a network to extend an additional range. In the experimental results, the proposed method shows quantitative and qualitative improvement in performance, compared with the conventional algorithms.

A fixed-point tone mapping operation (TMO) instead of a floating-point TMO is proposed in this paper. A TMO generates a low dynamic range (LDR) image from a high dynamic range (HDR) image. Since pixel values of an HDR image are generally expressed in a floating-point data format, e.g. RGBE and OpenEXR, a TMO is also implemented in floating-point arithmetic in conventional approaches. However, it requires a huge computational cost, even though a resulting LDR image is expressed in simple integers. The proposed TMO is implemented with only fixed-point arithmetic. As a result, the proposed method reduces a computational cost. Experimental results shows the PSNR of LDR images in the proposed method are comparable to those of the conventional methods.

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR-HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

Deep convolutional neural networks (CNNs) have recently made significant advances in the inverse tone mapping technique, which generates a high dynamic range (HDR) image from a single low dynamic range (LDR) image that has lost information in over- and under-exposed regions. The end-to-end inverse tone mapping approach specifies the dynamic range in advance, thereby limiting dynamic range expansion. In contrast, the method of generating multiple exposure LDR images from a single LDR image and subsequently merging them into an HDR image enables flexible dynamic range expansion. However, existing methods for generating multiple exposure LDR images require an additional network for each exposure value to be changed or a process of recursively inferring images that have different exposure values. Therefore, the number of parameters increases significantly due to the use of additional networks, and an error accumulation problem arises due to recursive inference. To solve this problem, we propose a novel network architecture that can control arbitrary exposure values without adding networks or applying recursive inference. The training method of the auxiliary classifier-generative adversarial network structure is employed to generate the image conditioned on the specified exposure. The proposed network uses a newly designed spatially-adaptive normalization to address the limitation of existing methods that cannot sufficiently restore image detail due to the spatially equivariant nature of the convolution. Spatially-adaptive normalization facilitates restoration of the high frequency component by applying different normalization parameters to each element in the feature map according to the characteristics of the input image. Experimental results show that the proposed method outperforms state-of-the-art methods, yielding a 5.48dB higher average peak signal-to-noise ratio, a 0.05 higher average structure similarity index, a 0.28 higher average multi-scale structure similarity index, and a 7.36 higher average HDR-VDP-2 for various datasets.

This study considers a method for reconstructing a high dynamic range (HDR) original image from a single saturated low dynamic range (LDR) image of metallic objects. A deep neural network approach was adopted for the direct mapping of an 8-bit LDR image to HDR. An HDR image database was first constructed using a large number of various metallic objects with different shapes. Each captured HDR image was clipped to create a set of 8-bit LDR images. All pairs of HDR and LDR images were used to train and test the network. Subsequently, a convolutional neural network (CNN) was designed in the form of a deep U-Net-like architecture. The network consisted of an encoder, a decoder, and a skip connection to maintain high image resolution. The CNN algorithm was constructed using the learning functions in MATLAB. The entire network consisted of 32 layers and 85,900 learnable parameters. The performance of the proposed method was examined in experiments using a test image set. The proposed method was also compared with other methods and confirmed to be significantly superior in terms of reconstruction accuracy, histogram fitting, and psychological evaluation.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

High dynamic range (HDR) illumination estimation from a single low dynamic range (LDR) image is a significant task in computer vision, graphics, and augmented reality. We present a two-stage deep learning-based method to predict an HDR environment map from a single narrow field-of-view LDR image. We first learn a hybrid parametric representation that sufficiently covers high- and low-frequency illumination components in the environment. Taking the estimated illuminations as guidance, we build a generative adversarial network to synthesize an HDR environment map that enables realistic rendering effects. We specifically consider the rendering effect by supervising the networks using rendering losses in both stages, on the predicted environment map as well as the hybrid illumination representation. Quantitative and qualitative experiments demonstrate that our approach achieves lower relighting errors for virtual object insertion and is preferred by users compared to state-of-the-art methods.

Due to limited camera capacities, digital images usually have a narrower dynamic illumination range than real-world scene radiance. To resolve this problem, High Dynamic Range (HDR) reconstruction is proposed to recover the dynamic range to better represent real-world scenes. However, due to different physical imaging parameters, the tone-mapping functions between images and real radiance are highly diverse, which makes HDR reconstruction extremely challenging. Existing solutions can not explicitly clarify a corresponding relationship between the tone-mapping function and the generated HDR image, but this relationship is vital when guiding the reconstruction of HDR images. To address this problem, we propose a method to explicitly estimate the tone mapping function and its corresponding HDR image in one network. Firstly, based on the characteristics of the tone mapping function, we construct a model by a polynomial to describe the trend of the tone curve. To fit this curve, we use a learnable network to estimate the coefficients of the polynomial. This curve will be automatically adjusted according to the tone space of the Low Dynamic Range (LDR) image, and reconstruct the real HDR image. Besides, since all current datasets do not provide the corresponding relationship between the tone mapping function and the LDR image, we construct a new dataset with both synthetic and real images. Extensive experiments show that our method generalizes well under different tone-mapping functions and achieves SOTA performance. The code/dataset is available at https://github.com/jqtangust/EPCE-HDR.git.

This paper introduces region of interest (ROI) as a functionality to local tone mappings of high dynamic range (HDR) radiance maps. In the proposed scheme, a new quadratic regularization method is provided to decompose the luminance of an HDR radiance map into a base layer with large gradients and a detail layer with small gradients. An ROI based global tone mapping is designed to compress the base layer. Since the dynamic range of a display device is limited, more dynamic ranges are assigned to the base layer in the ROI such that the visual quality of the ROI is preserved better. The detail layer can be amplified to enhance local contrasts such that the output LDR images look sharper. Both automatic and interactive modes are provided. With our automatic mode, the generated low dynamic range (LDR) images look natural and local details are also preserved very well. In addition, a user friendly interactive mode is provided which is also able to produce dynamic LDR images in real time.

New generations of display technologies provide signicantly improved dynamic range over conventional display devices. Current inverse tone mapping schemes require multiple exposure low dynamic range (LDR) images to generate high dynamic range (HDR) images. In this work, we propose an exposure dependent S curve to convert one optimized LDR image to multiple images with different brightness, which are then fused into a virtual real scene HDR image with wide dynamic range. According to our implementation results, the dynamic range can reach about 105. The synthesizer is robust and temporally coherent, and does not require image specific parameter adjustment. This paper also presents the image quality assessment with HDR visual difference predictor (HDR-VDP) and relative entropy contrast, and our work has better performance than other inverse tone mapping operators (iTMOs) for the both image quality assessments.

Nature light has a dynamic range beyond the modern camera can capture. So lots of commercial software applies techniques to reconstruct high dynamic range (HDR) image from low dynamic range (LDR) images. One of the popular methods is to recover the radiance of scene from LDR stacks [2]. However it needs to know the exact exposure time of LDR images and costs much time to solve the camera response function. These defects make it impossible to be built in hardware or real time systems. In this paper, we propose a new technique to synthesize HDR image based on the intensity mapping function (IMF). We first solve the IMF based on cross-histogram map, and then synthesize the pixel values from LDRs as if they have the same exposure time. We test our technique on several demos and get satisfying results with good visual quality in bright and dark scenes. Besides, because our method costs less time than the ‘software’ method, it is more suitable for further hardware application.

High dynamic range(HDR) imaging is the task of re-covering HDR image from one or multiple input Low Dynamic Range (LDR) images. In this paper, we present Gamma-enhanced Spatial Attention Network(GSANet), a novel framework for reconstructing HDR images. This problem comprises two intractable challenges of how to tackle overexposed and underexposed regions and how to overcome the paradox of performance and complexity trade-off. To address the former, after applying gamma correction on the LDR images, we adopt a spatial attention module to adaptively select the most appropriate regions of various exposure low dynamic range images for fusion. For the latter one, we propose an efficient channel attention module, which only involves a handful of parameters while bringing clear performance gain. Experimental results show that the proposed method achieves better visual quality on the HDR dataset. The code will be available at: https://github.com/fancyicookie/GSANet

This paper presents a novel High Dynamic Range (HDR) tone mapping (TM) system based on sub-band architecture. Standard wavelet filters of Daubechies, Symlets, Coiflets and Biorthogonal were used to estimate the proposed system performance in terms of Low Dynamic Range (LDR) image quality and reconstructed HDR image fidelity. During TM stage, the HDR image is firstly decomposed in sub-bands using symmetrical analysis-synthesis filter bank. The transform coefficients are then rescaled using a predefined gain map. The inverse Tone Mapping (iTM) stage is straightforward. Indeed, the LDR image passes through the same sub-band architecture. But, instead of reducing the dynamic range, the LDR content is boosted to an HDR representation. Moreover, in our TM sheme, we included an optimization module to select the gain map components that minimize the reconstruction error, and consequently resulting in high fidelity HDR content. Comparisons with recent state-of-the-art methods have shown that our method provides better results in terms of visual quality and HDR reconstruction fidelity using objective and subjective evaluations.

We present a high dynamic range (HDR) imaging algorithm that utilizes a modern rank minimization framework. Linear dependency exists among low dynamic range (LDR) images. However, global or local misalignment by camera motion and moving objects breaks down the low-rank structure of LDR images. The proposed algorithm simultaneously estimates global geometric transforms to align LDR images and detects moving objects and under-/over-exposed regions using a rank minimization approach. In the HDR composition step, structural consistency weighting is proposed to generate an artifact-free HDR image from an user-selected reference image. We demonstrate the robustness and effectiveness of the proposed method with real datasets.