Feature point detection under extreme lighting conditions

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

Technique of high dynamic range imaging (HDRI) was introduced into conventional high dynamic range display (HDRD). Sharpness of image was further enhanced by improving local contrast ratio in the HDRI-based high dynamic range display.

Most Convolution Neural Network (CNN) based object detectors, to date, have been optimized for accuracy and/or detection performance on datasets typically comprised of well exposed 8-bits/pixel/channel Standard Dynamic Range (SDR) images. A major existing challenge in this area is to accurately detect objects under extreme/difficult lighting conditions as SDR image trained detectors fail to accurately detect objects under such challenging lighting conditions. In this paper, we address this issue for the first time by introducing High Dynamic Range (HDR) imaging to object detection. HDR imagery can capture and process ≈13 orders of magnitude of scene dynamic range similar to the human eye. HDR trained models are therefore able to extract more salient features from extreme lighting conditions leading to more accurate detections. However, introducing HDR also presents multiple new challenges such as the complete absence of resources and previous literature on such an approach. Here, we introduce a methodology to generate a large scale annotated HDR dataset from any existing SDR dataset and validate the quality of the generated dataset via a robust evaluation technique. We also discuss the challenges of training and validating HDR trained models using existing detectors. Finally, we provide a methodology to create an out of distribution (OOD) HDR dataset to test and compare the performance of HDR and SDR trained detectors under difficult lighting condition. Results suggest that using the proposed methodology, HDR trained models are able to achieve 10 – 12% more accuracy compared to SDR trained models on real-world OOD dataset consisting of high-contrast images under extreme lighting conditions.

Abstract. 3D reconstruction relies on accurate detection, extraction, description and matching of image features. This is even truer for complex architectural scenes that pose needs for 3D models of high quality, without any loss of detail in geometry or color. Illumination conditions influence the radiometric quality of images, as standard sensors cannot depict properly a wide range of intensities in the same scene. Indeed, overexposed or underexposed pixels cause irreplaceable information loss and degrade digital representation. Images taken under extreme lighting environments may be thus prohibitive for feature detection/extraction and consequently for matching and 3D reconstruction. High Dynamic Range (HDR) images could be helpful for these operators because they broaden the limits of illumination range that Standard or Low Dynamic Range (SDR/LDR) images can capture and increase in this way the amount of details contained in the image. Experimental results of this study prove this assumption as they examine state of the art feature detectors applied both on standard dynamic range and HDR images.

Perceptual brightness-based inverse tone mapping for high dynamic range imaging

High Dynamic Range (HDR) imaging captures a wider range of luminance and color than Standard Dynamic Range (SDR), enabling more faithful representations of real-world scenes. Since most displays only support SDR, HDR content must be converted to SDR, often resulting in perceptual distortions like unnatural contrast, loss of detail or color artifacts. This paper evaluates No-Reference Image Quality Assessment (NR-IQA) methods on SDR images derived from HDR using the ESPL-LIVE HDR Image Database, which includes tone-mapped images and images created by multi-exposure fusion, together with associated mean opinion scores (MOS) and standard deviations, providing a reliable ground truth for evaluation. Seven NR-IQA methods, both traditional and Deep Learning (DL)-based, were tested and compared. DL algorithms performed best, with prediction errors below the standard deviation of MOS. Despite moderate correlation values (around 0.5), the results demonstrate the potential of these methods for practical image quality monitoring.

High dynamic range imaging (HDRI) methods in computational photography address situations where the dynamic range of a scene exceeds what can be captured by an image sensor in a single exposure. HDRI techniques have also been used to construct radiance maps in measurement applications; unfortunately, the design and evaluation of HDRI algorithms for use in these applications have received little attention. In this paper, we develop a novel HDRI technique based on pixel-by-pixel Kalman filtering and evaluate its performance using objective metrics that this paper also introduces. In the presented experiments, this new technique achieves as much as 9.4-dB improvement in signal-to-noise ratio and can achieve as much as a 29% improvement in radiometric accuracy over a classic method.

Scenes in real world have dynamic range of radiation that cannot be captured by conventional cameras. High dynamic range imaging is a technique to capture detail images where, in the field of image, intensity variation is extreme. This technique is very useful for biological imaging where the samples have very bright and very dark regions and both parts have useful information. In this article we propose a novel high dynamic range imaging technique based on compressive imaging that uses one single detector instead of camera (array of detectors) to capture an image. Combination of high dynamic range imaging and compressive imaging benefits from imaging with high dynamic range of radiation and advantages of compressive sampling; namely, imaging at regions of optical spectrum where conventional cameras are not readily available and single detectors are available. Additionally, as its name suggests, this technique requires less number of samples (compared to raster scanning). Our experimental results show that high dynamic range compressive imaging system is capable of capturing images with large intensity contrast.

The migration from High Definition (HD) TV to Ultra High Definition (UHD) is already underway. In addition to an increase of picture spatial resolution, UHD will bring more color and higher contrast by introducing Wide Color Gamut (WCG) and High Dynamic Range (HDR) video. As both Standard Dynamic Range (SDR) and HDR devices will coexist in the ecosystem, the transition from Standard Dynamic Range (SDR) to HDR will require distribution solutions supporting some level of backward compatibility. This paper presents a new HDR content distribution scheme, named SL-HDR1, using a single layer codec design and providing SDR compatibility. The solution is based on a pre-encoding HDR-to-SDR conversion, generating a backward compatible SDR video, with side dynamic metadata. The resulting SDR video is then compressed, distributed and decoded using standard-compliant decoders (e.g. HEVC Main 10 compliant). The decoded SDR video can be directly rendered on SDR displays without adaptation. Dynamic metadata of limited size are generated by the pre-processing and used to reconstruct the HDR signal from the decoded SDR video, using a post-processing that is the functional inverse of the pre-processing. Both HDR quality and artistic intent are preserved. Pre- and post-processing are applied independently per picture, do not involve any inter-pixel dependency, and are codec agnostic. Compression performance, and SDR quality are shown to be solidly improved compared to the non-backward and backward-compatible approaches, respectively using the Perceptual Quantization (PQ) and Hybrid Log Gamma (HLG) Opto-Electronic Transfer Functions (OETF).

High dynamic range imaging techniques involve capturing and storing real world radiance values that span many orders of magnitude. However, common display devices can usually reproduce intensity ranges only up to two to three orders of magnitude. Therefore, in order to display a high dynamic range image on a low dynamic range screen, the dynamic range of the image needs to be compressed without losing details or introducing artefacts, and this process is called tone mapping. A good tone mapping operator must be able to produce a low dynamic range image that matches as much as possible the perception of the real world scene. We propose a two stage tone mapping approach, in which the first stage is a global method for range compression based on a gamma curve that equalizes the lightness histogram the best, and the second stage performs local contrast enhancement and color induction using neural activity models for the visual cortex. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

We conducted a large-scale study of human perceptual quality judgments of High Dynamic Range (HDR) and Standard Dynamic Range (SDR) videos subjected to scaling and compression levels and viewed on three different display devices. While conventional expectations are that HDR quality is better than SDR quality, we have found subject preference of HDR versus SDR depends heavily on the display device, as well as on resolution scaling and bitrate. To study this question, we collected more than 23,000 quality ratings from 67 volunteers who watched 356 videos on OLED, QLED, and LCD televisions, and among many other findings, observed that HDR videos were often rated as lower quality than SDR videos at lower bitrates, particularly when viewed on LCD and QLED displays. Since it is of interest to be able to measure the quality of videos under these scenarios, e.g. to inform decisions regarding scaling, compression, and SDR vs HDR, we tested several well-known full-reference and no-reference video quality models on the new database. Towards advancing progress on this problem, we also developed a novel no-reference model called HDRPatchMAX, that uses a contrast-based analysis of classical and bit-depth features to predict quality more accurately than existing metrics.

Physical or virtual models are commonly employed to visualize the conceptual ideas of architects, lighting designers and daylighting researchers. The models are also used to assess the daylighting performance of their buildings, particularly when Complex Fenestration Systems (CFS) are considered. Recent studies have revealed a general tendency of physical models to over- estimate the performance, usually expressed through work plane illuminance and daylight factor profiles, when compared to that of the real building. These discrepancies can be attributed to several experimental errors. To analyze the main sources of error, a set of comparisons between a real building, a virtual model and a physical model was undertaken. The real building in our case consisted of a full scale test module with a south-facing windows designed for experimentation on daylighting systems. A virtual model was a computed model created in Radiance program while the physical model was a scale model (1:10) of the real case. The fenestration systems considered in this study were a simple window (double glazing) and two CFS (Laser-cut panel and Prismatic film). The physical model was placed in outdoor conditions similar to that of the real building as well as under a scanning sky simulator (for both real sky luminance distribution and CIE standard sky); the virtual model simulations were carried out with the program Radiance using the GenSky function (for CIE standard sky) and the Partial Daylight Factor (PDF) method, the later using the real sky luminance distribution acquired by a digital sky scanner at the same time as the real building's daylight performance was assessed. The daylighting performances of the building, daylight factor (DF) for overcast sky and illuminance ratio (IR) for clear sky, were monitored using illuminance meters: a set of sensors for exterior illuminance and another set of equally spaced 7 sensors placed at 1m intervals starting from the window plane for the interior space were used for that purpose. The interior surface luminance of both real building and physical model was measured using a luminance meter and a High Dynamic Range (HDR) imaging technique (within the Photosphere program). The Radiance program was used to determine the interior surface luminance within the virtual model. The measured performance of the real case, physical models and virtual models were compared, the causes of discrepancies between the real building and models were analyzed. The causes of errors that were evaluated were modeling of building details and dimensions, CFS modeling, mocking-up of the photometric properties (surface reflectance and window transmittance), model location as well as photometer features. To study the impact of these error sources on daylighting performance assessment, virtual models created using the Radiance program were used to achieve a sensitivity analysis of modeling errors. The significant factors were considered, leading to a set of modeling guidelines. The experimental study shows that large discrepancies can occur in daylighting performance figures. For example if glazings are omitted from the model's window, a relative divergence of 25% to 40% can be found at different points in the room, suggesting more light entering than actually measured in the real building. Inaccuracy in window transmittance inaccuracy is a major cause of errors commonly found in daylight modeling. In addition, significant discrepancies can be caused by even slight error in surface reflectance values. Only 10% overestimation of surface reflectance modeling leads up to 80% relative errors in work plane illuminance for a simple window and up to 90% for the assessment of CFS. Continuous sky distribution presented more accurate results than 145 sky sectors simulation, particularly when CFS were evaluated. These discrepancies can be reduced by making an effort to mock up the geometric and photometric features including the daylight simulation of the models carefully. A checklist presented in this thesis can be used as a guideline to help the daylight designers to estimate and avoid errors when assessing daylighting performance.

This paper describes a new method to automatically improve scene lighting for video conferencing by learning the photometric mapping between a lower exposure and desired exposure created using High Dynamic Range (HDR) imaging techniques. Once the mapping is learned in a calibration step, it can be used to transform all subsequent images, effectively producing higher dynamic range video without ghosting artifacts. A stereo algorithm is also described that allows multiple exposures to be taken at every frame, useful if the lighting of a scene changes significantly. Results show that this is an effective way to improve face lighting and therefore the overall experience of video conferencing.

Synthetic aperture imaging (SAI) is able to achieve the see through effect by blurring out the off-focus foreground occlusions and reconstructing the in-focus occluded targets from multi-view images. However, very dense occlusions and extreme lighting conditions may bring significant disturbances to the SAI based on conventional frame-based cameras, leading to performance degeneration. To address these problems, we propose a novel SAI system based on the event camera which can produce asynchronous events with extremely low latency and high dynamic range. Thus, it can eliminate the interference of dense occlusions by measuring with almost continuous views, and simultaneously tackle the over/under exposure problems. To reconstruct the occluded targets, we propose a hybrid encoder-decoder network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs). In the hybrid network, the spatio-temporal information of the collected events is first encoded by SNN layers, and then transformed to the visual image of the occluded targets by a style-transfer CNN decoder. Through experiments, the proposed method shows remarkable performance in dealing with very dense occlusions and extreme lighting conditions, and high quality visual images can be reconstructed using pure event data.

In this chapter, depth estimation for stereo pair of High Dynamic Range (HDR) images is proposed. The proposed algorithm consists of two major techniques namely conversion of HDR images to Low Dynamic Range (LDR) images or Standard Dynamic Range (SDR) images and estimating the depth from the converted LDR / SDR stereo images. Local based tone mapping technique is used for the conversion of the HDR images to SDR images. And the depth estimation is done based on the corner features of the stereo pair images and block matching algorithm. Computationally much less expensive cost functions Mean Square Error (MSE) or Mean Absolute Difference (MAD) can be used for block matching algorithms. The proposed algorithm is explained with illustrations and results.