Efficient multi-scale retinex algorithm using multi-rate image processing

As the dynamic range of a digital camera is narrower than that of a real scene, the captured image requires a tone curve or contrast correction to reproduce the information in dark regions. Yet, when using a global correction method, such as histogram-based methods and gamma correction, an unintended contrast enhancement in bright regions can result. Thus, a multiscale retinex algorithm using Gaussian filters was already proposed to enhance the local contrast of a captured image using the ratio between the intensities of an arbitrary pixel in the captured image and its surrounding pixels. The intensity of the surrounding pixels is estimated using Gaussian filters and weights for each filter, and to obtain better results, these Gaussian filters and weights are adjusted in relation to the captured image. Nonetheless, this adjustment is currently a subjective process, as no method has yet been developed for optimizing the Gaussian filters and weights according to the captured image. Therefore, this article proposes local contrast enhancement based on an adaptive multiscale retinex using a Gaussian filter set adapted to the input image. First, the weight of the largest Gaussian filter is determined using the local contrast ratio from the intensity distribution of the input image. The other Gaussian filters and weights for each Gaussian filter in the multiscale retinex are then determined using a visual contrast measure and the maximum color difference of the color patches in the Macbeth color checker. The visual contrast measure is obtained based on the product of the local standard deviation and locally averaged luminance of the image. Meanwhile, to evaluate the halo artifacts generated in large uniform regions that abut to form a high contrast edge, the artifacts are evaluated based on the maximum color difference between each color of the pixels in a patch in the Macbeth color and the averaged color in CIELAB standard color space. When considering the color difference for halo artifacts, the parameters for the Gaussian filters and weights representing a higher visual contrast measure are determined using test images. In addition, to reduce the induced graying-out, the chroma of the resulting image is compensated by preserving the chroma ratio of the input image based on the maximum chroma values of the sRGB color gamut in the lightness–chroma plane. In experiments, the proposed method is shown to improve the local contrast and saturation in a natural way.

This paper proposes a novel approach to reduce the operation time in the Multi-scale Retinex (MSR) while preserving the result image quality. Existing MSR algorithms take much time because they have many arithmetic operations. The proposed algorithm uses MSR weights selection algorithm to determine the important Single-scale Retinex (SSR) among the small, middle, and large SSRs. To reduce the operation time, we reduce the MSR weights selection arithmetic operation but have similar weights. And then, we skip SSR operations to reduce the operation time except dominant SSR when one SSR weight is dominant. Finally, we compare the MSR result image with the original image to reduce graying out. In the experiment, the proposed algorithm has 25%, 13% and 204% performance increment compared with benchmark method1, 2 and 3 and similar result image quality compared with conventional methods.

In this paper, an automatic exposure compensation method is proposed for image enhancement. For the exposure compensation, a novel image segmentation method based on luminance distribution is also proposed. Most single-image-enhancement methods often cause details to be lost in bright areas in images or cannot sufficiently enhance contrasts in dark regions. The image-enhancement method that uses the proposed compensation method enables us to produce high-quality images which well represent both bright and dark areas by fusing pseudo multi-exposure images generated from a single image. Here, pseudo multi-exposure images are automatically generated by the proposed exposure compensation method. To generate effective pseudo multi-exposure images, the proposed segmentation method is utilized for automatic parameter setting in the compensation method. In experiments, image enhancement with the proposed compensation method outperforms state-of-the-art image enhancement methods including Retinex-based methods, in terms of both entropy and statistical naturalness. Moreover, visual comparison results show that the proposed compensation method is effective in producing images that clearly present both bright and dark areas.

With the ability of imaging the temperature distribution of body, infrared imaging is promising in diagnostication and prognostication of diseases. However the poor quality of the raw original infrared images prevented applications and one of the essential problems is the low contrast appearance of the imagined object. In this paper, the image enhancement technique based on the Retinex theory is studied, which is a process that automatically retrieve the visual realism to images. The algorithms, including Frackle-McCann algorithm, McCann99 algorithm, single-scale Retinex algorithm, multi-scale Retinex algorithm and multi-scale Retinex algorithm with color restoration, are experienced to the enhancement of infrared images. The entropy measurements along with the visual inspection were compared and results shown the algorithms based on Retinex theory have the ability in enhancing the infrared image. Out of the algorithms compared, MSRCR demonstrated the best performance.

Image contrast enhancement is an essential operation for various image processing applications, especially for segmentation and feature extraction. However, different contrast enhancement methods contain various performance on the same images. To automatically choose the suitable algorithm for image contrast enhancement, in this paper, we put forward a different Retinex based method according to image feature fusion. First, five features, including gray value, contrast, definition, edge and entropy, are defined to characterize various image. Then, five fusion rules are designed to distinguish images with low contrast, foggy, monotonous color, rich in detail or color. Finally, the single scale Retinex (SSR), the multi-scale Retinex (MSR) and MSR in HSV color space (MSR_HSV) are introduced to improve image contrast. Experiments on various images drawn from different data sets prove the effectiveness of our proposed method.

We address the problem of contrast enhancement for color images. Methods directly derived from gray-level enhancement such as histogram equalization produce significant artifacts, including severe color shifts. Other enhancement techniques that are derived from the Retinex theory may suffer from strong `halo' effects. Our method to enhance images is inspired from the Retinex theory and tries to mimic human color perception. The method helps in achieving color constancy and also results in color contrast enhancement. We express the intensity as a product of illumination and reflectance and estimate these separately. Enhancement is then applied to the illuminant component only. Non-local means filter is used to estimate the illuminant and then the enhancement of the illumination is performed automatically without any manual intervention and multiplied back by the reflectance to obtain enhancement. We compare our results with those from other enhancement techniques and with results from commercial software packages such as PhotoFlair® that uses multi-scale retinex with color restoration (MSRCR) and Picasa¿ and observe that our results are consistently 'visually better'. Finally, we perform a statistical analysis of our results and quantitatively show that our approach produces effective and substantial image enhancement. This is validated by ratings from human observers.

rPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion

The objective of this paper is to develop a simple and efficient image enhancement algorithm for compressing image dynamics and enhancing image contrast in the discrete cosine transform (DCT) domain. The basic idea of this approach is to separate illumination and reflectance components of an image, so that by decreasing the contribution of the illumination the proposed algorithm effectively controls the dynamic range of the image using a contrast measure. The main advantage of the proposed algorithm is to enhance details in the dark and the bright areas with low computations without affecting the compressibility of the original image since it performs on the images in the compressed domain. In order to evaluate the proposed scheme, several base line approaches are described and compared.

The Martian Surface

We have grown blue light-emitting diodes (LEDs) with low threading dislocation densities (TDDs) by using SiN interlayers on Si (111) substrates. Our growth technique using SiN layers makes it possible to decrease twist components (edge-type threading dislocation components). The edge-type TDDs are almost the same values as those of LEDs grown on Al₂O₃ (0001) substrates. EQE of LEDs grown on Si (111) substrates increases with decreasing edge-type dislocation in the low-current-density region, and the EQE of the sample with low TDD is almost as high as that of the LED grown on an Al₂O₃ (0001) substrate at room temperature. It is found that the hot/cold factors (HCFs) of LEDs grown on Si (111) substrates increase with decreasing edge-type dislocations in the low-current-density region, but are less than those of an LED grown on an Al₂O₃ (0001) substrate. Time-resolved photoluminescence (TRPL) shows that the dominant origin of the thermal quenching is edge-type dislocations in our samples, but other defects such as screw-type dislocations also contribute to it. We also found the fluctuated emission patterns consisting of bright and dark areas originated from the difference of Shockley–Read–Hall (SRH) type defect densities in the multi-quantum wells (MQWs) grown on Si (111) substrates. The bright areas spread, and the configurations of the bright areas change into ring-like patterns with reducing edge-type TDDs. We suggest that the internal quantum efficiency (IQE) of dark areas should be promoted to improve the performance of the MQWs grown on Si (111) substrates.

Observations of apparent frost phenomena, occurring preferentially in the Martian bright areas, have in the past led to the conclusion that the bright areas are elevations. The argument hinges on the implicit assumption that, near midday, highlands should be at lower temperatures than lowlands. On the earth, this assumption is valid, because of adiabatic cooling of rising air, a diminished greenhouse effect, and a greater inclination of surfaces to the sun's rays for highlands than for lowlands. It is shown that these factors are unlikely to be important on Mars and that the polar cap recession data, in particular, suggests that the dark areas are highlands. From the radar Doppler spectra, it is found that the Martian dark areas tend to have higher elevations than the adjacent bright areas. Slopes of a few degrees are deduced, and peak elevation differences between 4 and 17 km are inferred. These results may bear on the discrepancies between ground-based spectrometric values and Mariner 4 occultation values of Martian surface pressure and on the discrepancies between the optical and the dynamical oblateness of Mars. They suggest that pressures ∼20 mb are typical of the centers of bright areas, and 6 mb is a typical dark area pressure.

The risks involved in nighttime driving include drowsy drivers and dangerous vehicles. Prominent among the more dangerous vehicles around at night are the larger vehicles which are usually moving faster at night on a highway. In addition, the risk level of driving around larger vehicles rises significantly when the driver’s attention becomes distracted, even for a short period of time. For the purpose of alerting the driver and elevating his or her safety, in this paper we propose two components for any modern vision-based Advanced Drivers Assistance System (ADAS). These two components work separately for the single purpose of alerting the driver in dangerous situations. The purpose of the first component is to ascertain that the driver would be in a sufficiently wakeful state to receive and process warnings; this is the driver drowsiness detection component. The driver drowsiness detection component uses infrared images of the driver to analyze his eyes’ movements using a MSR plus a simple heuristic. This component issues alerts to the driver when the driver’s eyes show distraction and are closed for a longer than usual duration. Experimental results show that this component can detect closed eyes with an accuracy of 94.26% on average, which is comparable to previous results using more sophisticated methods. The purpose of the second component is to alert the driver when the driver’s vehicle is moving around larger vehicles at dusk or night time. The large vehicle detection component accepts images from a regular video driving recorder as input. A bi-level system of classifiers, which included a novel MSR-enhanced KAZE-base Bag-of-Features classifier, is proposed to avoid false negatives. In both components, we propose an improved version of the Multi-Scale Retinex (MSR) algorithm to augment the contrast of the input. Several experiments were performed to test the effects of the MSR and each classifier, and the results are presented in experimental results section of this paper.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

In view of the defects of the classical Retinex image enhancement algorithm, such as poor texture detail retention, halo and over enhancement, and abrupt tone, this paper proposes a low illumination color image enhancement algorithm based on Gabor filter and retinex theory. Firstly, the brightness I component of the image is extracted in the HSI color space, and then the image with the brightness component is enhanced by MSRCR (the color-recovered retinex algorithm). At the same time, the image is enhanced by SSR (the single-scale retinex algorithm) based on Gabor filter in RGB color space, and the image with better texture and edge details is obtained. Finally, the final enhanced image is obtained by weighted fusion. The algorithm is compared with SSR, MSR (the multi-scale retinex algorithm) and MSRCR. The experimental results show that the image information processed by this algorithm is richer in color, hue and the color is closer to the original image, and it effectively reduces the occurrence of halo and over-enhancement. This algorithm can enhance the image of some low illumination color images, and the visual effect of the enhanced image is relatively peaceful.

In practical measurement, we often need to measure the shape of objects with patterns or letters. As far as we know, no paper has ever reported the shape measurement for objects with patterns or letters by Fourier fringe projection profilometry (FPP). In this paper, we propose a method based on the variational decomposition TV-Hilbert-L2 model and multi-scale Retinex (MSR) to measure the shape of objects with patterns and letters by Fourier FPP. In this method, we first use the TV-Hilbert-L2 model to obtain the fringe part, then perform MSR enhancement on the fringe part, and finally decompose the enhanced fringe part with TV-Hilbert-L2 again. We evaluate the performance of this method via application to one computer-simulated noisy fringe projection pattern and two experimental fringe projection patterns with different types of patterns or letters, and comparison with the Fourier transform method, the variational image decomposition TV-Hilbert-L2 model. Furthermore, we apply the proposed method to the dynamic three-dimensional shape measurement of hand posture with pattern. The experimental results show that our method can effectively measure the dynamic shape of objects with patterns or letters from a single-frame fringe projection pattern.