Deep Controllable Backlight Dimming for HDR Displays

High dynamic range (HDR) image generation had been studied for years. Due to the expense and rareness of HDR cameras, many works try to generate HDR images using several low dynamic range (LDR) images with different exposure conditions. To ensure high-quality HDR image generation, details of the scene should be retained in different LDR images and the exposure parameters of LDR image should be chosen carefully. In this paper, we propose a technique to dynamically determine the suitable exposure parameter for LDR images according to the property of each scene to be captured. Experimental results reveal that better HDR images are always generated with the LDR images captured by the determined exposure, compared to those created by the method with fixed exposures.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

This study considers a method for reconstructing a high dynamic range (HDR) original image from a single saturated low dynamic range (LDR) image of metallic objects. A deep neural network approach was adopted for the direct mapping of an 8-bit LDR image to HDR. An HDR image database was first constructed using a large number of various metallic objects with different shapes. Each captured HDR image was clipped to create a set of 8-bit LDR images. All pairs of HDR and LDR images were used to train and test the network. Subsequently, a convolutional neural network (CNN) was designed in the form of a deep U-Net-like architecture. The network consisted of an encoder, a decoder, and a skip connection to maintain high image resolution. The CNN algorithm was constructed using the learning functions in MATLAB. The entire network consisted of 32 layers and 85,900 learnable parameters. The performance of the proposed method was examined in experiments using a test image set. The proposed method was also compared with other methods and confirmed to be significantly superior in terms of reconstruction accuracy, histogram fitting, and psychological evaluation.

Due to the growing popularity of High Dynamic Range (HDR) images and HDR displays, a large amount of existing Low Dynamic Range (LDR) images are required to be converted to HDR format to benefit HDR advantages, which give rise to some LDR to HDR algorithms. Most of these algorithms especially tackle overexposed areas during expanding, which is the potential to make the image quality worse than that before processing and introduces artifacts. To dispel these problems, we . present a new,LDR to HDR approach, unlike the existing techniques, it focuses on avoiding sophisticated treatment to overexposed areas in dynamic range expansion step. Based on a separating principle, firstly, according to the familiar types of overexposure, the overexposed areas are classified into two categories which are removed and corrected respectively by two kinds of techniques. Secondly, for maintaining color consistency, color recovery is carried out to the preprocessed images. Finally, the LDR image is expanded to HDR. Experiments show that the proposed approach performs well and produced images become more favorable and suitable for applications. The image quality metric also illustrates that we can reveal more details without causing artifacts introduced by other algorithms.

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.

A fixed-point tone mapping operation (TMO) instead of a floating-point TMO is proposed in this paper. A TMO generates a low dynamic range (LDR) image from a high dynamic range (HDR) image. Since pixel values of an HDR image are generally expressed in a floating-point data format, e.g. RGBE and OpenEXR, a TMO is also implemented in floating-point arithmetic in conventional approaches. However, it requires a huge computational cost, even though a resulting LDR image is expressed in simple integers. The proposed TMO is implemented with only fixed-point arithmetic. As a result, the proposed method reduces a computational cost. Experimental results shows the PSNR of LDR images in the proposed method are comparable to those of the conventional methods.

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR-HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

High dynamic range (HDR) images are becoming pervasive due to capturing or rendering of a wider range of luminance, but their special display equipment is difficult to be popularized because of high cost and technological problem. Thus, HDR images must be adapted to the conventional display devices by applying tone mapping (TM) operation, which puts forward higher requirements for intellectual property protection of HDR images. As the robustness presents regional diversity in the low dynamic range (LDR) watermarked image after TM, which is different from the traditional watermarking technologies, a concept of watermarking activity is defined and used to distinguish the essential distinction of watermarking between LDR image and HDR image in this paper. Then, a novel robust HDR image watermarking algorithm is proposed against TM operations. Firstly, based on the hybrid processing of redundant discrete wavelet transform and singular value decomposition, the watermark is embedded by modifying the structure information of the HDR image. Distinguished from LDR image watermarking, the high embedding strength can cause more obvious distortion in the high brightness regions of HDR image than the low brightness regions. Thus, a perceptual brightness mask with low complexity is designed to improve the imperceptibility further. Experimental results show that the proposed algorithm is robust to the existing TM operations, with taking into account the imperceptibility and embedded capacity, which is superior to the current state-of-art HDR image watermarking algorithms.

Recently, high dynamic range (HDR) imaging has attracted much attention as a technology to reflect human visual characteristics owing to the development of the display and camera technology. This paper proposes a novel deep neural network model that reconstructs an HDR image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively simple for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range but also has the advantage of restoring the light information of the actual physical world. The proposed method is an end-to-end reconstruction process, and it has the advantage of being able to easily combine a network to extend an additional range. In the experimental results, the proposed method shows quantitative and qualitative improvement in performance, compared with the conventional algorithms.

Due to limited camera capacities, digital images usually have a narrower dynamic illumination range than real-world scene radiance. To resolve this problem, High Dynamic Range (HDR) reconstruction is proposed to recover the dynamic range to better represent real-world scenes. However, due to different physical imaging parameters, the tone-mapping functions between images and real radiance are highly diverse, which makes HDR reconstruction extremely challenging. Existing solutions can not explicitly clarify a corresponding relationship between the tone-mapping function and the generated HDR image, but this relationship is vital when guiding the reconstruction of HDR images. To address this problem, we propose a method to explicitly estimate the tone mapping function and its corresponding HDR image in one network. Firstly, based on the characteristics of the tone mapping function, we construct a model by a polynomial to describe the trend of the tone curve. To fit this curve, we use a learnable network to estimate the coefficients of the polynomial. This curve will be automatically adjusted according to the tone space of the Low Dynamic Range (LDR) image, and reconstruct the real HDR image. Besides, since all current datasets do not provide the corresponding relationship between the tone mapping function and the LDR image, we construct a new dataset with both synthetic and real images. Extensive experiments show that our method generalizes well under different tone-mapping functions and achieves SOTA performance. The code/dataset is available at https://github.com/jqtangust/EPCE-HDR.git.

This paper addresses the challenging problem of avoiding ghost artifacts in compositing a high dynamic range (HDR) image by multi-exposed images fusion. We propose a convolutional neural network (CNN)-based deghosting algorithm, which treats the estimation of motion maps as a labelling problem. The network input consists of multiple low dynamic range (LDR) images presenting motion-related scene and exposure differences, and the output is a set of binary motion maps where black pixels indicate inconsistency between reference and non-reference images. The network is trained using simulated data, and we propose a novel approach for generating realistic synthetic multiple LDR images and their corresponding ground truth motion maps. The trained CNN model is tested on a variety of dynamic indoor and outdoor scenes. It outperforms the state-of-the-art without requiring camera meta data and is more flexible to handle higher-resolution images under limited GPU memory compared to end-to-end networks for HDR imaging. Our source code, pretrained models, and data will be made publicly available to facilitate future research.

We present a high dynamic range (HDR) imaging algorithm that utilizes a modern rank minimization framework. Linear dependency exists among low dynamic range (LDR) images. However, global or local misalignment by camera motion and moving objects breaks down the low-rank structure of LDR images. The proposed algorithm simultaneously estimates global geometric transforms to align LDR images and detects moving objects and under-/over-exposed regions using a rank minimization approach. In the HDR composition step, structural consistency weighting is proposed to generate an artifact-free HDR image from an user-selected reference image. We demonstrate the robustness and effectiveness of the proposed method with real datasets.

Recently, various deep learning-based high dynamic range (HDR) imaging methods have been proposed to overcome the performance limitation of simply merging or fusion of multiple low dynamic range (LDR) images. However, early deep learning-based methods did not fully exploit the inter-image relationship between different exposures. T o consider inter-image features without the dataset dependency, this paper presents a novel HDR method using three-dimensional residual dense network (3D RDN). We first reconstruct a set of differently exposed LDR images in video format. Next, we take the reconstructed video as an input to the 3D RDN, which generates an HDR image as output. Experimental results show that the proposed method provides higher score compared with state-of-the-arts HDR methods. To the best of authors knowledge, the proposed work first adopted the 3D RDN in the field of HDR imaging using more than three LDR images without any additional pre-or post-processing.

Tone-mapping-operators (TMOs) provide a useful means for converting high dynamic range (HDR) images to low dynamic range (LDR) images so that they can be viewed on standard displays, but this may influence the visual quality of experience (QoE) of the end-user. There is a need to understand the impact of TMOs to inform the choice of TMO algorithms for different displays, especially for small-screen-devices (SSDs) such as those used in mobile phones. This is important, as mobile devices are becoming the primary means of consuming multimedia contents. However, few studies have been undertaken to assess the impact of TMOs and viewing devices (especially SSDs) on the visual QoE of the user when using. In this chapter, we evaluate subjectively and objectively, the commonly used TMOs in different displays and resolutions for colour and grey-scale HDR images. Our results show that viewing devices have an influence on the TMOs performance, suggesting the need for a careful choice of TMO to enhance the viewing-QoE of the end-user. As expected, the higher resolution, the better HDR-image quality. Surprisingly, there was no significant difference between the Mean of Opinion Score (MOS) scores for colour and grey-scale images in SSDs. The device and TMOs affect QoE for colour and grey HDR-image equally. We found Shannon entropy (SE) to be a good objective measure of quality for colour and grey HDR images, suggesting that entropy may find use in automated HDR quality control assessment schemes, while; HDR-VDP-2 is a good objective measure for colour HDR image only.