LLIE: Low-Light Image Enhancement Network Based on FPGA

Self-supervised network for low-light traffic image enhancement based on deep noise and artifacts removal

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

Low-light image enhancement is widely used in many fields, such as target detection, face recognition, and image segmentation. In recent years, Deep Learning methods have achieved impressive breakthroughs in low-light image enhancement. However, most of them mine high-dimensional features of images by stacking network structures and deepening the depth of the network, which causes more runtime costs for single image enhancement. To reduce inference time while fully extracting local features and global features of low-light images, we propose an Attention-based Broad Self-guided Network for real-world low-light image Enhancement. Compared to U-net structure [1], such a self-guided structure requires a smaller number of parameters and enables us to achieve better effectiveness. To extract local information more efficiently, we designed a Multilevel Guided Densely Connected Attention block, which can be considered as a novel extension of the densely connected blocks in feature space. In addition, we also offer a more efficient module to extract global information from images, which is called the Global Spatial Attention module. The proposed network is validated by lots of mainstream benchmarks. Many experimental results show that the proposed network outperforms most state-of-the-art low-light image Enhancement methods. Our code is available at https://github.com/paullenwyue/ABSGNet.

Low-light images enhancement of real scenes is a challenge task, however, existing algorithms always encounter the problems of over-enhancement, noises amplification and low subjective evaluation. One reason of these problems lies in the missing of high-level vision information. Therefore, this paper present a novel low-light image enhancement framework based on retinex and saliency theories. We first adopts a deep neural network model i.e. Saliency Attentive Model to predict the saliency map of low-light image and detect its salient regions. Then, we utilize one retinex model based method to enhance the whole low-light image. Then we fuse the generated saliency map and the enhanced image together to acquire a well-enhanced image. Experiments verify the significance of our algorithm by comparing with general low-light image enhancement method without saliency.

Enhancement of low-light images is a challenging task due to the impact of low brightness, low contrast, and high noise. The inability to collect natural labeled data intensifies this problem further. Many researchers have attempted to solve this problem using learning-based approaches; however, most models ignore the impact of noise in low-lit images. In this paper, an encoder-decoder architecture, made up of separable convolution layers that solve the issues encountered in low-light image enhancement, is proposed. The architecture is trained end-to-end on a custom low-light image dataset (LID), comprising both clean and noisy images. We introduce a unique multi-context feature extraction module (MC-FEM) where the input first passes through a feature pyramid of dilated separable convolutions for hierarchical-context feature extraction followed by separable convolutions for feature compression. The model is optimized using a novel three-part loss function that focuses on high-level contextual features, structural similarity, and patch-wise local information. We conducted several ablation studies to determine the optimal model for low-light image enhancement under noisy and noiseless conditions. We have used performance metrics like peak-signal-to-noise ratio, structural similarity index matrix, visual information fidelity, and average brightness to demonstrate the superiority of the proposed work against the state-of-the-art algorithms. Qualitative results presented in this paper prove the strength and suitability of our model for real-time applications.

Low-Light Image Enhancement (LLIE) presents challenges due to texture information loss and uneven illumination, which can distort feature distribution and reduce the quality of the enhanced images. However, current deep learning methods for LLIE only use supervised information from clear images to extract low-light image features, while disregarding the negative information in low-light images (i.e., low illumination and noise). To address these challenges, we propose a novel LLIE method, LACR-VAE, by leveraging the negative information and considering the uneven illumination. In particular, a Light-Aware Contrastive Regularization (LACR) based on contrastive learning is designed to exploit information from both clear and low-light images. The LACR aims to align latent variables of enhanced images with clear images, away from those of low-light images. This allows the method to prioritize essential elements for LLIE and minimize noise and lighting variations. Furthermore, considering the uneven illumination with diverse region sizes and shapes, a Region-CAlibrated Module (RCAM) is present to learn local and global illumination relations among image regions, and an Attention-guided Multi-Scale Module (AMSM) is designed to extract multi-scale features that improve the model’s representation capability. Extensive experiments show that our method achieves superior performance than previous works. Specifically, our method yields a significant enhancement in the National Aeronautics and Space Administration (NASA) testset, achieving an improvement of at least 0.99 in PSNR and 0.0409 in SSIM. Codes and datasets are available at https://github.com/csxuwu/LACR-VAE .

Low-light image enhancement aims to enhance the visibility and contrast of low-light images while eliminating complex degradation issues such as noise, artifacts, and color distortions. Most existing low-light image enhancement methods either focus on quality while neglecting computational efficiency or have limited learning and generalization capabilities. To address these issues, we propose a Bilateral Enhancement Network with signal-to-noise ratio fusion, called BiEnNet, for lightweight and generalizable low-light image enhancement. Specifically, we design a lightweight Bilateral enhancement module with SNR (Signal-to-Noise Ratio) Fusion (BSF), which serves the SNR map of the input low-light image as the interpolation weights to dynamically fuse global brightness features and local detail features extracted from a bilateral network and achieve differentiated enhancement across different regions. To improve the network’s generalization ability, we propose a Luminance Normalization (LNM) module for preprocessing and a Dual-Exposure Processing (DEP) module for post-processing. LNM divides the channels of input features into luminance-related channels and luminance independent channels, and reduces the inconsistency of the degradation distribution of input low-light images by only normalizing the luminance-related channels. DEP learns overexposure and underexposure corrections simultaneously by employing the ReLU activation function, inverting operation, and residual network, which can improve the robustness of enhancement effects under different exposure conditions while reducing network parameters. Experiments on the LOL-V1 dataset shows BiEnNet significantly increased PSNR by 8.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} and SSIM by 3.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to FLW-Net, reduced parameters by 98.78\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and improved computational speed by 52.64\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to the classical KIND.

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.

The direct utilization of low-light images hinders downstream visual tasks. Traditional low-light image enhancement (LLIE) methods, such as Retinex-based networks, require image pairs. A spiking-coding methodology called intensity-to-latency has been used to gradually acquire the structural characteristics of an image. convLSTM has been used to connect the features. This study introduces a simplified DCENet to achieve unsupervised LLIE as well as the spiking coding mode of a spiking neural network. It also applies the comprehensive coding features of convLSTM to improve the subjective and objective effects of LLIE. In the ablation experiment for the proposed structure, the convLSTM structure was replaced by a convolutional neural network, and the classical CBAM attention was introduced for comparison. Five objective evaluation metrics were compared with nine LLIE methods that currently exhibit strong comprehensive performance, with PSNR, SSIM, MSE, UQI, and VIFP exceeding the second place at 4.4% (0.8%), 3.9% (17.2%), 0% (15%), 0.1% (0.2%), and 4.3% (0.9%) on the LOL and SCIE datasets. Further experiments of the user study in five non-reference datasets were conducted to subjectively evaluate the effects depicted in the images. These experiments verified the remarkable performance of the proposed method.

Low-light image enhancement via deep Retinex decomposition and bilateral learning

Low-light image enhancement is a fundamental problem in computer vision. The artifact, noise, insufficient contrast and color distortion are common challenging problems in low-light image enhancement. In this paper, we present a convolutional Long Short-Term Memory (ConvLSTM) network based method to directly restore a normal image from a low-light image, which can be learned in an end-to-end way. Specifically, our base network employs the encoder-decoder structure. Meanwhile, considering that a normal image may correspond to low-light images of different illuminance levels, we adopt a multi-branch structure combined with ConvLSTM to solve this problem. The extensive experiments on two low-light datasets show that our method outperforms the state-of-the-art traditional and deep learning based methods vertified by both quantitative and qualitative evaluation.

This paper presents a low-light image enhancement method using a convolutional neural network (CNN). Given a lowlight input image, the proposed method converts RGB color space to CIELAB color space. The luminance and chrominance components are separately enhanced. The luminance channel is enhanced using a CNN to enhance the brightness. On the other hand, the chrominance channels are enhanced using a dilated CNN to reduce the color distortion. Experimental results demonstrate that the proposed method can successfully enhance low-light images of a vehicular imaging system without color distortion.

Low-light images typically suffer from two problems. First, they have low visibility (i.e., small pixel values). Second, noise becomes significant and disrupts the image content, due to low signal-to-noise ratio. Most existing low-light image enhancement methods, however, learn from noise-negligible datasets. They rely on users having good photographic skills in taking images with low noise. Unfortunately, this is not the case for majority of the low-light images. While concurrently enhancing a low-light image and removing its noise is ill-posed, we observe that noise exhibits different levels of contrast in different frequency layers, and it is much easier to detect noise in the low-frequency layer than in the high one. Inspired by this observation, we propose a frequency-based decomposition- and- enhancement model for low-light image enhancement. Based on this model, we present a novel network that first learns to recover image objects in the low-frequency layer and then enhances high-frequency details based on the recovered image objects. In addition, we have prepared a new low-light image dataset with real noise to facilitate learning. Finally, we have conducted extensive experiments to show that the proposed method outperforms state-of-the-art approaches in enhancing practical noisy low-light images.

A lightweight feature activation guided multi-receptive field attention network for light compensation

With the development of convolutional neural networks, the effectiveness of low-light image enhancement techniques have been greatly advanced. However, the calculate-intensive and memory-intensive characteristics of convolutional neural networks make them difficult to be implemented in mobile platform with low power and limited bandwidth. This paper proposes a complete solution for low-light image enhancement from CNN model to mobile (M2M) FPGA. The proposed solution utilizes a pseudo-symmetry quantization method to compress the low-light image enhancement model, and an accelerator to permit the processing ability of the system significantly. We implemented the whole system on a customized FPGA SOC platform (a low-cost chip, Xilinx Inc. ZYNQ\(^{TM}\)XC7Z035). Extensive experiments show that our method achieved competitive results with the other three platforms, i.e. achieved better speed compare to ARM and CPU; and achieved better power efficiency compared to ARM, CPU, and GPU.