High dynamic range multispectral imaging using liquid crystal tunable filter

Object detection is critical in autonomous driving, and it is more practical yet challenging to localize objects of unknown categories: an endeavour known as Class-Agnostic Object Detection (CAOD). Existing studies on CAOD predominantly rely on RGB cameras, but these frame-based sensors usually have high latency and limited dynamic range, leading to safety risks under extreme conditions like fast-moving objects, overexposure, and darkness. In this study, we turn to the event-based vision, featured by its sub-millisecond latency and high dynamic range, for robust CAOD. We propose Detecting Every Object in Events (DEOE), an approach aimed at achieving high-speed, class-agnostic object detection in event-based vision. Built upon the fast event-based backbone: recurrent vision transformer, we jointly consider the spatial and temporal consistencies to identify potential objects. The discovered potential objects are assimilated as soft positive samples to avoid being suppressed as backgrounds. Moreover, we introduce a disentangled objectness head to separate the foreground-background classification and novel object discovery tasks, enhancing the model's generalization in localizing novel objects while maintaining a strong ability to filter out the background. Extensive experiments confirm the superiority of our proposed DEOE in both open-set and closed-set settings, outperforming strong baseline methods.

Most recent bidirectional reflectance distribution function (BRDF) measurement systems are the image-based that consist of a light source, a detector, and curved samples. They are useful for measuring the reflectance properties of a material but they have two major drawbacks. They suffer from high cost of BRDF acquisition and also give inaccurate results due to the limited use of spectral bands. In this paper, we propose a novel multispectral HDR imaging system and its efficient characterization method. It combines two promising technologies: high dynamic range (HDR) imaging and multispectral imaging to measure BRDF. We perform a full spectral recovery using camera response curves for each wavelength band and its analysis. For this, we use an HDR camera to capture HDR images and a liquid crystal tunable filter (LCTF) to generate multi-spectral images. Our method can provide an accurate color reproduction of metameric objects as well as a saturated image. Our multi-spectral HDR imaging system provides a very fast data acquisition time and also gives a low system setup cost compared to previous multi-spectral imaging systems and point-based commercial spectroradiometers. We verify the color accuracy of our multi-spectral HDR imaging system in terms of human vision and metamerism using colorimetric and spectral metric.

An accurate characterization method for a multispectral high-dynamic-range (HDR) imaging system is proposed by combining multispectral and HDR imaging technologies. The multispectral HDR imaging system, which can acquire the visible spectrum at many wavelength bands, can provide an accurate color reproduction and physical radiance information of real objects. An HDR camera is used to capture an HDR image without multiple exposures and a liquid crystal tunable filter (LCTF) is used to generate multispectral images. Due to its several limitations in the multispectral HDR imaging system, a carefully designed and an innovative characterization algorithm is presented by considering a logarithmic camera response of the HDR camera and different spectral transmittance of the LCTF. The proposed method efficiently and accurately recovers the full spectrum from the multispectral HDR images using a transformation matrix and provides device-independent color information (e.g., CIEXYZ and CIELAB). The transformation matrix is estimated by training the estimated sensor responses from a multispectral HDR imaging system and the reflectance measurements from a spectroradiometer using Moore–Penrose pseudoinverse matrix.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

Typical digital cameras exhibit a limitation regarding the dynamic range of the scene radiance they can capture. High Dynamic Range (HDR) imaging refers to methods and systems that aim to generate images that exhibit higher dynamic range between the lightest and the darkest parts of the an image. A typical approach for generating HDR images is exposure bracketing where multiple frames, each one with a different exposure setting, are captured and combined to a HDR image of the scene. The large number of images that exposure bracketing requires often leads to motion artefacts that limit the visual quality of the resulting HDR image. In this work, we propose a novel approach in HDR imaging that significantly reduces the necessary number of images. In our proposed system, we employ the notion of random exposure where each pixel of a single frame collects light for a random amount of time. By collecting a small number of such images, the full sequence of low dynamic range images can be reconstructed and subsequently used for HDR generation. The problem is solved by casting the reconstruction of the sequence as a nuclear norm minimization problem following the premises of the recently proposed theory of Matrix Completion. Experimental results suggest that the proposed method is able to reconstruct the sequence from as low as 20% of the images that traditional techniques require with minimal reduction in image quality.

International audience

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

High dynamic range (HDR) images are becoming pervasive due to capturing or rendering of a wider range of luminance, but their special display equipment is difficult to be popularized because of high cost and technological problem. Thus, HDR images must be adapted to the conventional display devices by applying tone mapping (TM) operation, which puts forward higher requirements for intellectual property protection of HDR images. As the robustness presents regional diversity in the low dynamic range (LDR) watermarked image after TM, which is different from the traditional watermarking technologies, a concept of watermarking activity is defined and used to distinguish the essential distinction of watermarking between LDR image and HDR image in this paper. Then, a novel robust HDR image watermarking algorithm is proposed against TM operations. Firstly, based on the hybrid processing of redundant discrete wavelet transform and singular value decomposition, the watermark is embedded by modifying the structure information of the HDR image. Distinguished from LDR image watermarking, the high embedding strength can cause more obvious distortion in the high brightness regions of HDR image than the low brightness regions. Thus, a perceptual brightness mask with low complexity is designed to improve the imperceptibility further. Experimental results show that the proposed algorithm is robust to the existing TM operations, with taking into account the imperceptibility and embedded capacity, which is superior to the current state-of-art HDR image watermarking algorithms.

Tone-mapping-operators (TMOs) provide a useful means for converting high dynamic range (HDR) images to low dynamic range (LDR) images so that they can be viewed on standard displays, but this may influence the visual quality of experience (QoE) of the end-user. There is a need to understand the impact of TMOs to inform the choice of TMO algorithms for different displays, especially for small-screen-devices (SSDs) such as those used in mobile phones. This is important, as mobile devices are becoming the primary means of consuming multimedia contents. However, few studies have been undertaken to assess the impact of TMOs and viewing devices (especially SSDs) on the visual QoE of the user when using. In this chapter, we evaluate subjectively and objectively, the commonly used TMOs in different displays and resolutions for colour and grey-scale HDR images. Our results show that viewing devices have an influence on the TMOs performance, suggesting the need for a careful choice of TMO to enhance the viewing-QoE of the end-user. As expected, the higher resolution, the better HDR-image quality. Surprisingly, there was no significant difference between the Mean of Opinion Score (MOS) scores for colour and grey-scale images in SSDs. The device and TMOs affect QoE for colour and grey HDR-image equally. We found Shannon entropy (SE) to be a good objective measure of quality for colour and grey HDR images, suggesting that entropy may find use in automated HDR quality control assessment schemes, while; HDR-VDP-2 is a good objective measure for colour HDR image only.

Recently, visual representations using high dynamic range (HDR) images become increasingly popular, with advancement of technologies for increasing the dynamic range of image. HDR image is expected to be used in wide-ranging applications such as digital cinema, digital photography and next generation broadcast, because of its high quality and its powerful expression ability. HDR imaging technologies will spread its sphere of influence in imaging industry. In this paper, we review the state-of-the-art studies and the trends of the HDR imaging, in terms of the following three points: (1) HDR imaging sensor and HDR image generation techniques as image acquisition technologies, (2) encode method of HDR images for efficient transmission and storage, (3) human visual system issues associated with reproduction of HDR image.

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.

Current HDR display technology approaches the dynamic range of the fully adapted human vision system. As such, this technology has potential for performing as a surrogate for realworld scenes in the evaluation of the accuracy of high dynamic range (HDR) algorithms that map HDR scenes to the more common, everyday display technology having limited dynamic range. Clearly, use of HDR display technology has the benefit of simplicity in experimental design while maintaining the high dynamic range of the original scene. To evaluate this potential use for HDR displays, seven published versions of well-known tone mapping algorithms were benchmarked for rendering accuracy against each of four real-world scenes constructed in the lab and their corresponding renderings on the Munsell Color Science Laboratory's (MCSL) HDR display. The results between those obtained from the MCSL HDR display and those obtained from the real-world scenes are in good agreement thus validating the HDR display's potential as an evaluation tool in this context.

Some scenes and objects have a wide range of brightness that cannot be captured with a conventional camera. This limitation, which degrades the dynamic range of an imaged scene or object, is addressed by use of high dynamic range (HDR) imaging techniques. With HDR ima- ging techniques, images of a very broad range of intensity can be obtained with conventional cameras. Another limitation of conventional cameras is the range of wavelength that they can capture. Outside the visible wave- lengths, the responsivity of conventional cameras drops; therefore, a con- ventional camera cannot capture images in nonvisible wavelengths. Compressive imaging is a solution for this problem. Compressive imaging reduces the number of pixels of a camera to one, so a camera can be replaced by a detector with one pixel. The range of wavelengths to which such detectors are responsive is much wider than that of a conven- tional camera. A combination of HDR imaging and compressive imaging is introduced and is benefitted from the advantages of both techniques. An algorithm that combines these two techniques is proposed, and results are presented. © 2012 Society of Photo-Optical Instrumentation Engineers

There is a rapid increase in the amount of High Dynamic Range (HDR) display devices that are able to offer significantly better viewing experience with higher contrast and richer colors. However, there is a considerable shortage of content that is able to offer the much-sought-after HDR experience. One approach is to capture new content using specialized HDR-cameras, which are typically fewer and costlier than the conventional cameras. Another approach is to convert existing Standard Dynamic Range (SDR) content into HDR artificially, such that, it can reasonably mimic the viewing experience that a true HDR content would have offered. With the latter approach, we propose a Gaussian Process Regression (GPR) based machine learning method for estimating HDR content from their SDR counterparts. GPR is known as a powerful technique for estimating continuous real-valued functions. Given a set of training SDR-HDR image pairs, our proposed method is able to estimate the reverse tone mapping function that is used to convert SDR signal into its HDR equivalent. Preliminary experimental results indicate that our approach produces visually pleasing HDR images.

Tone mapping is the process of compressing the dynamic range of a scene to make it compatible with displays with limited dynamic range. In this paper, we present a new approach for tone mapping of high dynamic range (HDR) images using edge-preserving filter. We pose it as a variational problem and derive an optimal solution. We compare the results with some of the existing tone mapping algorithms and show that our algorithm compresses the high dynamic range better while preserving the details and avoiding common artefacts. We use an online dynamic range independent metric for analysing the results by the proposed and existing operators and to detect the deviation from the reference HDR image.