Learning Wasserstein Contrastive Color Histogram Representation for Low-Light Image Enhancement

Low-light image enhancement (LLIE) explores how to refine the illumination and obtain natural normal-light images. Current LLIE methods mainly focus on improving the illumination, but do not consider the color consistency by reasonably incorporating color information into the LLIE process. As a result, color difference usually exists between the enhanced image and ground-truth. To address this issue, we propose a new deep color consistent network termed DCC-Net to retain the color consistency for LLIE. A new "divide and conquer" collaborative strategy is presented, which can jointly preserve color information and enhance the illumination. Specifically, the decoupling strategy of our DCC-Net decouples each color image into two main components, i.e., gray image plus color histogram. Gray image is used to generate reasonable structures and textures, and the color histogram is beneficial for preserving the color consistency. That is, they both are utilized to complete the LLIE task collaboratively. To match the color and content features, and reduce the color consistency gap between enhanced image and ground-truth, we also design a new pyramid color embedding (PCE) module, which can better embed color information into the LLIE process. Extensive experiments on six real datasets show that the enhanced images of our DCC-Net are more natural and colorful, and perform favorably against the state-of-the-art methods.

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.

Self-supervised network for low-light traffic image enhancement based on deep noise and artifacts removal

In this paper, we introduce a hardware accelerator for the Low-Light Image Enhancement (LLIE) network. This accelerator significantly enhances the effectiveness of LLIE while reducing computation time and power consumption. Specifically designed for the LLIE accelerator, we propose an RGB parallel processing algorithm for LLIE. LLIE can estimate individual channel components. The three channels share weights through a pulse array and converge toward the same direction via a self-calibration module. We design hardware acceleration units for different operators in LLIE and conduct testing. LLIE is deployed on Xilinx’s XCU102, with model inference parameters ranging from 0.96[Formula: see text]k to 3456[Formula: see text]k. The latency of LLIE on Field-Programmable Gate Array (FPGA) ranges from 0.19 to 7.69[Formula: see text]ms, with power consumption ranging from 0.4 to 1.68[Formula: see text]W. Compared to GPUs with similar performance, LLIE achieves approximately 20 times acceleration on FPGA, with power consumption only 3[Formula: see text] to 10[Formula: see text] of that of GPUs. Experimental results indicate that LLIE’s performance on FPGA improves with increasing model parameters. This system offers significant advantages in low-light image processing and promises to greatly facilitate real-time processing of low-light images in the future.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

Images captured in low-light conditions usually suffer from very low contrast and underexpose, which cannot be directly utilized in the subsequent computer vision tasks, such as object recognition, detection, identification and tracking. Existing methods include HE based method, Retinex theory based method and deep learning method which may generate undesirable enhanced results including amplified noise, biased colors and extreme boundary. To address this problem, we utilize prior knowledge of Retinex theory and GAN based on data statistic to propose a progressive GAN-based Transfer network to realize the low-light enhancement. In this paper, the image is decomposed by JieP method based on the Retinex model to obtain the reflection and light components, and learn the relationship between the reflection component of the low-light image and normal light image via a reflection decomposition on network (RefDecN), and then generate the reflection component of the low-light image. Then, another illumination transfering network (IllumTransN) is utilized to transfer the light of normal light image to the reflection component to realize low-light enhancement. Experimental results of low-light image enhancement on RAISE, LOL and MEF datasets demonstrate our ProGAN can outperform state-of-the-art methods in terms of objective and subjective quality.KeywordsLow-light enhancementReflection componentIllumination componentRetinex modelGenerative adversarial network

This paper presents an image enhancement model, D2BGAN (Dark to Bright Generative Adversarial Network), to translate low light images to bright images without a paired supervision. We introduce the use of geometric and lighting consistency along with a contextual loss criterion. These when combined with multiscale color, texture and edge discriminators prove to provide competitive results. We performed extensive experiments using benchmark datasets to visually and objectively compare our results. We observed the performance of D2BGAN on real-time driving datasets that are subject to motion blur, noise, and other artifacts. We further demonstrated that our enhanced images can be profitably used in image-understanding tasks. Images processed using our technique obtain the best or second best average scores for three different image quality evaluation methods on the Naturalness Preserved Enhancement (NPE), Low Light Image Enhancement (LIME), Multi-Exposure Image Fusion (MEF) benchmark datasets. Best scores are also obtained on the LOw-Light (LOL) test set and on Berkeley Driving Dataset (BDD) images processed with D2BGAN. Face detection tasks on the DarkFace benchmark dataset show an mAP (mean Average Precision) improvement from 0.209 to 0.301 when images are processed using D2BGAN. mAP further improves to 0.525 when finetuning techniques are adopted.

Low-light image enhancement is widely used in many fields, such as target detection, face recognition, and image segmentation. In recent years, Deep Learning methods have achieved impressive breakthroughs in low-light image enhancement. However, most of them mine high-dimensional features of images by stacking network structures and deepening the depth of the network, which causes more runtime costs for single image enhancement. To reduce inference time while fully extracting local features and global features of low-light images, we propose an Attention-based Broad Self-guided Network for real-world low-light image Enhancement. Compared to U-net structure [1], such a self-guided structure requires a smaller number of parameters and enables us to achieve better effectiveness. To extract local information more efficiently, we designed a Multilevel Guided Densely Connected Attention block, which can be considered as a novel extension of the densely connected blocks in feature space. In addition, we also offer a more efficient module to extract global information from images, which is called the Global Spatial Attention module. The proposed network is validated by lots of mainstream benchmarks. Many experimental results show that the proposed network outperforms most state-of-the-art low-light image Enhancement methods. Our code is available at https://github.com/paullenwyue/ABSGNet.

Despite recent advances in contrast enhancement, it remains difficult for existing methods to simultaneously achieve consistent improvements in image brightness and contrast in both low-light and normal-light images. To address this issue, we revisit 2D histogram equalization methods and extend them to accommodate more diversified and poor lighting conditions. The extension is based on the observation that the contrast needs to be improved by increasing the intensity difference between spatially neighboring pixels, while the degree of increase should be proportional to the difference in their reflectances. On the basis of this observation, we propose to embed the inter-pixel contextual information of image reflectance into the 2D histogram of intensity co-occurrence. An intensity mapping function can thus be derived by solving optimization problems formulated with the 2D histogram, leading to two novel contrast enhancement methods. Qualitative and quantitative evaluations on more than 600 images showed that the proposed methods are superior to state-of-the-art contrast enhancement methods. It was also shown that the reflectance of an image provides important visual information on the significance and objectness of local image areas. Using this reflectance as a clue, the degree of contrast enhancement can be adaptively derived to achieve sufficient brightness improvement in low-light images and to avoid excessive enhancement in normal-light images.

Low-light image enhancement (LLIE) aims to improve the visibility and perceptual quality of images captured under low-light conditions, enhancing as much structure and detail as the captured data allow. Existing methods often rely on deep neural networks to map low-light images to normal-light counterparts within color spaces such as standard red green blue (sRGB) or hue-saturation-value (HSV). However, these approaches are prone to noise interference and lack robustness, resulting in noticeable artifacts in color and brightness. To address these challenges, we propose task-decoupled framework for low-light image enhancement (TDNet), a task-decoupled LLIE framework that regards LLIE as a multi-task learning problem: denoising and color correction, enabling targeted optimization for each sub-task. Specifically, the denoising sub-task further decouples noise suppression across three channels and employs a fine-denoise channel attention module to effectively handle channel-specific noise heterogeneity. Simultaneously, a cross-channel fusion attention module captures global semantic information and inter-channel dependencies, enhancing color consistency and ensuring natural visual aesthetics. In addition, a progressive detail recovery strategy is incorporated to refine structural clarity and edge details. Extensive experiments on multiple benchmark datasets demonstrate that TDNet achieves favorable performance in noise suppression and color correction while maintaining low computational complexity and parameter efficiency (https://github.com/634949684aa/TDNet).

In colonoscopy, the captured images are usually with low-quality appearance, such as non-uniform illumination, low contrast, etc., due to the specialized imaging environment, which may provide poor visual feedback and bring challenges to subsequent disease analysis. Many low-light image enhancement (LIE) algorithms have recently proposed to improve the perceptual quality. However, how to fairly evaluate the quality of enhanced colonoscopy images (ECIs) generated by different LIE algorithms remains a rarely-mentioned and challenging problem. In this study, we carry out a pioneering investigation on perceptual quality assessment of ECIs. Firstly, considering the lack of specific datasets, we collect 300 low-light images with diverse contents during the real-world colonoscopy and conduct rigorous subjective studies to compare the performance of 8 popular LIE methods, resulting in a benchmark dataset (named ECIQAD) for ECIs. Secondly, in view of the distinctive distortion characteristics of ECIs, we propose an effective no-reference Enhanced Colonoscopy Image Quality (ECIQ) method to automatically evaluate the perceptual quality of ECIs via analysis of brightness, contrast, colorfulness, naturalness, and noise. Extensive experiments on ECIQAD demonstrate the superiority of our proposed ECIQ method over 14 mainstream no-reference image quality assessment methods.

Enhancement of low-light images is a challenging task due to the impact of low brightness, low contrast, and high noise. The inability to collect natural labeled data intensifies this problem further. Many researchers have attempted to solve this problem using learning-based approaches; however, most models ignore the impact of noise in low-lit images. In this paper, an encoder-decoder architecture, made up of separable convolution layers that solve the issues encountered in low-light image enhancement, is proposed. The architecture is trained end-to-end on a custom low-light image dataset (LID), comprising both clean and noisy images. We introduce a unique multi-context feature extraction module (MC-FEM) where the input first passes through a feature pyramid of dilated separable convolutions for hierarchical-context feature extraction followed by separable convolutions for feature compression. The model is optimized using a novel three-part loss function that focuses on high-level contextual features, structural similarity, and patch-wise local information. We conducted several ablation studies to determine the optimal model for low-light image enhancement under noisy and noiseless conditions. We have used performance metrics like peak-signal-to-noise ratio, structural similarity index matrix, visual information fidelity, and average brightness to demonstrate the superiority of the proposed work against the state-of-the-art algorithms. Qualitative results presented in this paper prove the strength and suitability of our model for real-time applications.

Low-light images enhancement of real scenes is a challenge task, however, existing algorithms always encounter the problems of over-enhancement, noises amplification and low subjective evaluation. One reason of these problems lies in the missing of high-level vision information. Therefore, this paper present a novel low-light image enhancement framework based on retinex and saliency theories. We first adopts a deep neural network model i.e. Saliency Attentive Model to predict the saliency map of low-light image and detect its salient regions. Then, we utilize one retinex model based method to enhance the whole low-light image. Then we fuse the generated saliency map and the enhanced image together to acquire a well-enhanced image. Experiments verify the significance of our algorithm by comparing with general low-light image enhancement method without saliency.

Low-Light Image Enhancement (LLIE) presents challenges due to texture information loss and uneven illumination, which can distort feature distribution and reduce the quality of the enhanced images. However, current deep learning methods for LLIE only use supervised information from clear images to extract low-light image features, while disregarding the negative information in low-light images (i.e., low illumination and noise). To address these challenges, we propose a novel LLIE method, LACR-VAE, by leveraging the negative information and considering the uneven illumination. In particular, a Light-Aware Contrastive Regularization (LACR) based on contrastive learning is designed to exploit information from both clear and low-light images. The LACR aims to align latent variables of enhanced images with clear images, away from those of low-light images. This allows the method to prioritize essential elements for LLIE and minimize noise and lighting variations. Furthermore, considering the uneven illumination with diverse region sizes and shapes, a Region-CAlibrated Module (RCAM) is present to learn local and global illumination relations among image regions, and an Attention-guided Multi-Scale Module (AMSM) is designed to extract multi-scale features that improve the model’s representation capability. Extensive experiments show that our method achieves superior performance than previous works. Specifically, our method yields a significant enhancement in the National Aeronautics and Space Administration (NASA) testset, achieving an improvement of at least 0.99 in PSNR and 0.0409 in SSIM. Codes and datasets are available at https://github.com/csxuwu/LACR-VAE .

Low-light image enhancement aims to enhance the visibility and contrast of low-light images while eliminating complex degradation issues such as noise, artifacts, and color distortions. Most existing low-light image enhancement methods either focus on quality while neglecting computational efficiency or have limited learning and generalization capabilities. To address these issues, we propose a Bilateral Enhancement Network with signal-to-noise ratio fusion, called BiEnNet, for lightweight and generalizable low-light image enhancement. Specifically, we design a lightweight Bilateral enhancement module with SNR (Signal-to-Noise Ratio) Fusion (BSF), which serves the SNR map of the input low-light image as the interpolation weights to dynamically fuse global brightness features and local detail features extracted from a bilateral network and achieve differentiated enhancement across different regions. To improve the network’s generalization ability, we propose a Luminance Normalization (LNM) module for preprocessing and a Dual-Exposure Processing (DEP) module for post-processing. LNM divides the channels of input features into luminance-related channels and luminance independent channels, and reduces the inconsistency of the degradation distribution of input low-light images by only normalizing the luminance-related channels. DEP learns overexposure and underexposure corrections simultaneously by employing the ReLU activation function, inverting operation, and residual network, which can improve the robustness of enhancement effects under different exposure conditions while reducing network parameters. Experiments on the LOL-V1 dataset shows BiEnNet significantly increased PSNR by 8.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} and SSIM by 3.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to FLW-Net, reduced parameters by 98.78\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and improved computational speed by 52.64\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to the classical KIND.