An efficient three-stage network via Multi-Scale Orthogonal Complementary Transformer for low-light image enhancement

Self-supervised network for low-light traffic image enhancement based on deep noise and artifacts removal

In this paper, we introduce a hardware accelerator for the Low-Light Image Enhancement (LLIE) network. This accelerator significantly enhances the effectiveness of LLIE while reducing computation time and power consumption. Specifically designed for the LLIE accelerator, we propose an RGB parallel processing algorithm for LLIE. LLIE can estimate individual channel components. The three channels share weights through a pulse array and converge toward the same direction via a self-calibration module. We design hardware acceleration units for different operators in LLIE and conduct testing. LLIE is deployed on Xilinx’s XCU102, with model inference parameters ranging from 0.96[Formula: see text]k to 3456[Formula: see text]k. The latency of LLIE on Field-Programmable Gate Array (FPGA) ranges from 0.19 to 7.69[Formula: see text]ms, with power consumption ranging from 0.4 to 1.68[Formula: see text]W. Compared to GPUs with similar performance, LLIE achieves approximately 20 times acceleration on FPGA, with power consumption only 3[Formula: see text] to 10[Formula: see text] of that of GPUs. Experimental results indicate that LLIE’s performance on FPGA improves with increasing model parameters. This system offers significant advantages in low-light image processing and promises to greatly facilitate real-time processing of low-light images in the future.

Low-light image enhancement is widely used in many fields, such as target detection, face recognition, and image segmentation. In recent years, Deep Learning methods have achieved impressive breakthroughs in low-light image enhancement. However, most of them mine high-dimensional features of images by stacking network structures and deepening the depth of the network, which causes more runtime costs for single image enhancement. To reduce inference time while fully extracting local features and global features of low-light images, we propose an Attention-based Broad Self-guided Network for real-world low-light image Enhancement. Compared to U-net structure [1], such a self-guided structure requires a smaller number of parameters and enables us to achieve better effectiveness. To extract local information more efficiently, we designed a Multilevel Guided Densely Connected Attention block, which can be considered as a novel extension of the densely connected blocks in feature space. In addition, we also offer a more efficient module to extract global information from images, which is called the Global Spatial Attention module. The proposed network is validated by lots of mainstream benchmarks. Many experimental results show that the proposed network outperforms most state-of-the-art low-light image Enhancement methods. Our code is available at https://github.com/paullenwyue/ABSGNet.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

Enhancement of low-light images is a challenging task due to the impact of low brightness, low contrast, and high noise. The inability to collect natural labeled data intensifies this problem further. Many researchers have attempted to solve this problem using learning-based approaches; however, most models ignore the impact of noise in low-lit images. In this paper, an encoder-decoder architecture, made up of separable convolution layers that solve the issues encountered in low-light image enhancement, is proposed. The architecture is trained end-to-end on a custom low-light image dataset (LID), comprising both clean and noisy images. We introduce a unique multi-context feature extraction module (MC-FEM) where the input first passes through a feature pyramid of dilated separable convolutions for hierarchical-context feature extraction followed by separable convolutions for feature compression. The model is optimized using a novel three-part loss function that focuses on high-level contextual features, structural similarity, and patch-wise local information. We conducted several ablation studies to determine the optimal model for low-light image enhancement under noisy and noiseless conditions. We have used performance metrics like peak-signal-to-noise ratio, structural similarity index matrix, visual information fidelity, and average brightness to demonstrate the superiority of the proposed work against the state-of-the-art algorithms. Qualitative results presented in this paper prove the strength and suitability of our model for real-time applications.

EENet: An effective and efficient network for single image dehazing

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.

Low-Light Image Enhancement (LLIE) presents challenges due to texture information loss and uneven illumination, which can distort feature distribution and reduce the quality of the enhanced images. However, current deep learning methods for LLIE only use supervised information from clear images to extract low-light image features, while disregarding the negative information in low-light images (i.e., low illumination and noise). To address these challenges, we propose a novel LLIE method, LACR-VAE, by leveraging the negative information and considering the uneven illumination. In particular, a Light-Aware Contrastive Regularization (LACR) based on contrastive learning is designed to exploit information from both clear and low-light images. The LACR aims to align latent variables of enhanced images with clear images, away from those of low-light images. This allows the method to prioritize essential elements for LLIE and minimize noise and lighting variations. Furthermore, considering the uneven illumination with diverse region sizes and shapes, a Region-CAlibrated Module (RCAM) is present to learn local and global illumination relations among image regions, and an Attention-guided Multi-Scale Module (AMSM) is designed to extract multi-scale features that improve the model’s representation capability. Extensive experiments show that our method achieves superior performance than previous works. Specifically, our method yields a significant enhancement in the National Aeronautics and Space Administration (NASA) testset, achieving an improvement of at least 0.99 in PSNR and 0.0409 in SSIM. Codes and datasets are available at https://github.com/csxuwu/LACR-VAE .

Low-light images enhancement of real scenes is a challenge task, however, existing algorithms always encounter the problems of over-enhancement, noises amplification and low subjective evaluation. One reason of these problems lies in the missing of high-level vision information. Therefore, this paper present a novel low-light image enhancement framework based on retinex and saliency theories. We first adopts a deep neural network model i.e. Saliency Attentive Model to predict the saliency map of low-light image and detect its salient regions. Then, we utilize one retinex model based method to enhance the whole low-light image. Then we fuse the generated saliency map and the enhanced image together to acquire a well-enhanced image. Experiments verify the significance of our algorithm by comparing with general low-light image enhancement method without saliency.

Low-light image enhancement aims to enhance the visibility and contrast of low-light images while eliminating complex degradation issues such as noise, artifacts, and color distortions. Most existing low-light image enhancement methods either focus on quality while neglecting computational efficiency or have limited learning and generalization capabilities. To address these issues, we propose a Bilateral Enhancement Network with signal-to-noise ratio fusion, called BiEnNet, for lightweight and generalizable low-light image enhancement. Specifically, we design a lightweight Bilateral enhancement module with SNR (Signal-to-Noise Ratio) Fusion (BSF), which serves the SNR map of the input low-light image as the interpolation weights to dynamically fuse global brightness features and local detail features extracted from a bilateral network and achieve differentiated enhancement across different regions. To improve the network’s generalization ability, we propose a Luminance Normalization (LNM) module for preprocessing and a Dual-Exposure Processing (DEP) module for post-processing. LNM divides the channels of input features into luminance-related channels and luminance independent channels, and reduces the inconsistency of the degradation distribution of input low-light images by only normalizing the luminance-related channels. DEP learns overexposure and underexposure corrections simultaneously by employing the ReLU activation function, inverting operation, and residual network, which can improve the robustness of enhancement effects under different exposure conditions while reducing network parameters. Experiments on the LOL-V1 dataset shows BiEnNet significantly increased PSNR by 8.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} and SSIM by 3.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to FLW-Net, reduced parameters by 98.78\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and improved computational speed by 52.64\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to the classical KIND.

The goal of low-light image enhancement (LLIE) is to enhance perception to restore normal-light images. The primary emphasis of earlier LLIE methods was on enhancing the illumination while paying less attention to the color distortions and noise in the dark. In comparison to the ground truth, the restored images frequently exhibit inconsistent color and residual noise. To this end, this paper introduces a Wasserstein contrastive regularization method (WCR) for LLIE. The WCR regularizes the color histogram (CH) representation of the restored image to keep its color consistency while removing noise. Specifically, the WCR contains two novel designs including a differentiable CH module (DCHM) and a WCR loss. The DCHM serves as a modular component that can be easily integrated into the network to enable end-to-end learning of the image CH. Afterwards, to ensure color consistency, we utilize the Wasserstein distance (WD) to quantify the resemblance of the learnable CHs between the restored image and the normal-light image. Then, the regularized WD is used to construct the WCR loss, which is a triplet loss and takes the normal-light images as positive samples, the low-light images as negative samples, and the restored images as anchor samples. The WCR loss pulls the anchor samples closer to the positive samples and simultaneously pushes them away from the negative samples so as to help the anchors remove the noise in the dark. Notably, the proposed WCR method was only used for training, and was shown to achieve high performance and high speed inference using lightweight networks. Therefore, it is valuable for real-time applications such as night automatic driving and night reversing image enhancement. Extensive evaluations on benchmark datasets such as LOL, FiveK, and UIEB showed that the proposed WCR method achieves superior performance, outperforming existing state-of-the-art methods.

Images taken under low-light conditions often suffer from multiple degradations such as low visibility and unknown noise. Low-light image enhancement is an important task in the field of computer vision. In order to avoid the limited number of samples in paired datasets, several self-supervised enhancement methods have been developed. However, due to the designed illumination gradient prior, most self-supervised enhancement methods based on Retinex cannot effectively constrain the illumination or suppress the amplified real noise. To solve this problem, this paper explores a joint self-supervised enhancement and denoising method for low-light image. Initially, we proposed a new regularization term, named TV-Huber, and developed an adaptive illumination estimation network (AIE-Net) to explore the intrinsic relationship between structure and texture in the illumination map. Next, the camera response model and the learned illumination are then used to enhance the contrast of low-light images and mitigate color shifts. Finally, the learned illumination maps are transformed into illumination masks. Under the assumption of independent and zero-mean noise, selective feature injection is performed on the shallow features extracted by the blind-spot network (BSN) to reduce information loss while removing unknown real noise in the dark area. Extensive experiments show that the proposed method has good generalization ability on five challenging low-light image datasets and outperforms other methods in terms of visual quality and quantitative comparison.

Low-light image enhancement plays very important roles in low-level vision areas. Recent works have built a great deal of deep learning models to address this task. However, these approaches mostly rely on significant architecture engineering and suffer from high computational burden. In this paper, we propose a new method, named Retinex-inspired Unrolling with Architecture Search (RUAS), to construct lightweight yet effective enhancement network for low-light images in real-world scenario. Specifically, building upon Retinex rule, RUAS first establishes models to characterize the intrinsic underexposed structure of low-light images and unroll their optimization processes to construct our holistic propagation structure. Then by designing a cooperative reference-free learning strategy to discover low-light prior architectures from a compact search space, RUAS is able to obtain a top-performing image enhancement network, which is with fast speed and requires few computational resources. Extensive experiments verify the superiority of our RUAS framework against recently proposed state-of-the-art methods. The project page is available at http://dutmedia.org/RUAS/.

This paper presents an image enhancement model, D2BGAN (Dark to Bright Generative Adversarial Network), to translate low light images to bright images without a paired supervision. We introduce the use of geometric and lighting consistency along with a contextual loss criterion. These when combined with multiscale color, texture and edge discriminators prove to provide competitive results. We performed extensive experiments using benchmark datasets to visually and objectively compare our results. We observed the performance of D2BGAN on real-time driving datasets that are subject to motion blur, noise, and other artifacts. We further demonstrated that our enhanced images can be profitably used in image-understanding tasks. Images processed using our technique obtain the best or second best average scores for three different image quality evaluation methods on the Naturalness Preserved Enhancement (NPE), Low Light Image Enhancement (LIME), Multi-Exposure Image Fusion (MEF) benchmark datasets. Best scores are also obtained on the LOw-Light (LOL) test set and on Berkeley Driving Dataset (BDD) images processed with D2BGAN. Face detection tasks on the DarkFace benchmark dataset show an mAP (mean Average Precision) improvement from 0.209 to 0.301 when images are processed using D2BGAN. mAP further improves to 0.525 when finetuning techniques are adopted.

This paper presents a low-light image enhancement method using a convolutional neural network (CNN). Given a lowlight input image, the proposed method converts RGB color space to CIELAB color space. The luminance and chrominance components are separately enhanced. The luminance channel is enhanced using a CNN to enhance the brightness. On the other hand, the chrominance channels are enhanced using a dilated CNN to reduce the color distortion. Experimental results demonstrate that the proposed method can successfully enhance low-light images of a vehicular imaging system without color distortion.