Low-light image enhancement using Gaussian Process for features retrieval

Self-supervised network for low-light traffic image enhancement based on deep noise and artifacts removal

The direct utilization of low-light images hinders downstream visual tasks. Traditional low-light image enhancement (LLIE) methods, such as Retinex-based networks, require image pairs. A spiking-coding methodology called intensity-to-latency has been used to gradually acquire the structural characteristics of an image. convLSTM has been used to connect the features. This study introduces a simplified DCENet to achieve unsupervised LLIE as well as the spiking coding mode of a spiking neural network. It also applies the comprehensive coding features of convLSTM to improve the subjective and objective effects of LLIE. In the ablation experiment for the proposed structure, the convLSTM structure was replaced by a convolutional neural network, and the classical CBAM attention was introduced for comparison. Five objective evaluation metrics were compared with nine LLIE methods that currently exhibit strong comprehensive performance, with PSNR, SSIM, MSE, UQI, and VIFP exceeding the second place at 4.4% (0.8%), 3.9% (17.2%), 0% (15%), 0.1% (0.2%), and 4.3% (0.9%) on the LOL and SCIE datasets. Further experiments of the user study in five non-reference datasets were conducted to subjectively evaluate the effects depicted in the images. These experiments verified the remarkable performance of the proposed method.

Low-light image enhancement has been gradually becoming a hot research topic in recent years due to its wide usage as an important pre-processing step in computer vision tasks. Although numerous methods have achieved promising results, some of them still generate results with detail loss and local distortion. In this paper, we propose an improved generative adversarial network based on contextual information. Specifically, residual dense blocks are adopted in the generator to promote hierarchical feature interaction across multiple layers and enhance features at multiple depths in the network. Then, an attention module integrating multi-scale contextual information is introduced to refine and highlight discriminative features. A hybrid loss function containing perceptual and color component is utilized in the training phase to ensure the overall visual quality. Qualitative and quantitative experimental results on several benchmark datasets demonstrate that our model achieves relatively good results and has good generalization capacity compared to other state-of-the-art low-light enhancement algorithms.

Low-light object detection involves identifying and locating objects in images captured under poor lighting conditions. It plays a significant role in surveillance and security, night pedestrian recognition, and autonomous driving, showcasing broad application prospects. Most existing object detection algorithms and datasets are designed for normal lighting conditions, leading to a significant drop in detection performance when applied to low-light environments. To address this issue, we propose a Low-Light Detection with Low-Light Enhancement (LDWLE) framework. LDWLE is an encoder-decoder architecture where the encoder transforms the raw input data into a compact, abstract representation (encoding), and the decoder gradually generates the target output format from the representation produced by the encoder. Specifically, during training, low-light images are input into the encoder, which produces feature representations that are decoded by two separate decoders: an object detection decoder and a low-light image enhancement decoder. Both decoders share the same encoder and are trained jointly. Throughout the training process, the two decoders optimize each other, guiding the low-light image enhancement towards improvements that benefit object detection. If the input image is normally lit, it first passes through a low-light image conversion module to be transformed into a low-light image before being fed into the encoder. If the input image is already a low-light image, it is directly input into the encoder. During the testing phase, the model can be evaluated in the same way as a standard object detection algorithm. Compared to existing object detection algorithms, LDWLE can train a low-light robust object detection model using standard, normally lit object detection datasets. Additionally, LDWLE is a versatile training framework that can be implemented on most one-stage object detection algorithms. These algorithms typically consist of three components: the backbone, neck, and head. In this framework, the backbone functions as the encoder, while the neck and head form the object detection decoder. Extensive experiments on the COCO, VOC, and ExDark datasets have demonstrated the effectiveness of LDWLE in low-light object detection. In quantitative measurements, it achieves an AP of 25.5 and 38.4 on the synthetic datasets COCO-d and VOC-d, respectively, and achieves the best AP of 30.5 on the real-world dataset ExDark. In qualitative measurements, LDWLE can accurately detect most objects on both public real-world low-light datasets and self-collected ones, demonstrating strong adaptability to varying lighting conditions and multi-scale objects.

This paper presents a low-light image enhancement method using a convolutional neural network (CNN). Given a lowlight input image, the proposed method converts RGB color space to CIELAB color space. The luminance and chrominance components are separately enhanced. The luminance channel is enhanced using a CNN to enhance the brightness. On the other hand, the chrominance channels are enhanced using a dilated CNN to reduce the color distortion. Experimental results demonstrate that the proposed method can successfully enhance low-light images of a vehicular imaging system without color distortion.

In this paper, we introduce a hardware accelerator for the Low-Light Image Enhancement (LLIE) network. This accelerator significantly enhances the effectiveness of LLIE while reducing computation time and power consumption. Specifically designed for the LLIE accelerator, we propose an RGB parallel processing algorithm for LLIE. LLIE can estimate individual channel components. The three channels share weights through a pulse array and converge toward the same direction via a self-calibration module. We design hardware acceleration units for different operators in LLIE and conduct testing. LLIE is deployed on Xilinx’s XCU102, with model inference parameters ranging from 0.96[Formula: see text]k to 3456[Formula: see text]k. The latency of LLIE on Field-Programmable Gate Array (FPGA) ranges from 0.19 to 7.69[Formula: see text]ms, with power consumption ranging from 0.4 to 1.68[Formula: see text]W. Compared to GPUs with similar performance, LLIE achieves approximately 20 times acceleration on FPGA, with power consumption only 3[Formula: see text] to 10[Formula: see text] of that of GPUs. Experimental results indicate that LLIE’s performance on FPGA improves with increasing model parameters. This system offers significant advantages in low-light image processing and promises to greatly facilitate real-time processing of low-light images in the future.

A wavelet-based local and global feature fusion network (LAGN) is proposed for low-light image enhancement, aiming to enhance image details and restore colors in dark areas. This study focuses on addressing three key issues in low-light image enhancement: Enhancing low-light images using LAGN to preserve image details and colors; extracting image edge information via wavelet transform to enhance image details; and extracting local and global features of images through convolutional neural networks and Transformer to improve image contrast. Comparisons with state-of-the-art methods on two datasets verify that LAGN achieves the best performance in terms of details, brightness, and contrast.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

Low-light is a challenging environment for both human and computer vision to perform tasks such as object classification and detection. Recent works have shown potential in employing enhancements algorithms to support and improve such tasks in low-light, however there has not been any focused analysis to understand the direct effects that low-light enhancement have on an object detector. This work aims to quantify and visualize such effects on the multi-level abstractions involved in network-based object detection. First, low-light image enhancement algorithms are employed to enhance real low-light images, and then followed by deploying an object detection network on the low-light as well as the enhanced counterparts. A comparison of the activations in different layers, representing the detection features, are used to generate statistics in order to quantify the enhancements' contribution to detection. Finally, this framework was used to analyze several low-light image enhancement algorithms and identify their impact on the detection model and task. This framework can also be easily generalized to any convolutional neural network-based models for the analysis of different enhancements algorithms and tasks.

Progressive feature fusion for SNR-aware low-light image enhancement

UHD Low-light image enhancement via interpretable bilateral learning

This paper proposes a deep convolutional neural network-based low-light image enhancement method. In order to adaptively enhance the image brightness, a convolutional neural network with convolutional modules is designed. Lowlight image is firstly down-sampled into sub-images. Then an illumination map is obtained from the input image to provide additional information to the network. The network works on a tensor that consists of sub-images and illumination map, achieving a good performance in brightness increasing and structure preservation. The enhanced result is reconstructed from the output sub-images. Experimental results demonstrate the effectiveness of the proposed method in low-light image enhancement.

Remote sensing image (RSI) scene classification is the foundation and important technology of ground object detection, land use management and geographic analysis. During recent years, convolutional neural networks (CNNs) have achieved significant success and are widely applied in RSI scene classification. However, crafted images that serve as adversarial examples can potentially fool CNNs with high confidence and are hard for human eyes to interpret. For the increasing security and robust requirements of RSI scene classification, the adversarial example problem poses a serious problem for the classification results derived from systems using CNN models, which has not been fully recognized by previous research. In this study, to explore the properties of adversarial examples of RSI scene classification, we create different scenarios by testing two major attack algorithms (i.e., the fast gradient sign method (FGSM) and basic iterative method (BIM)) trained on different RSI benchmark datasets to fool CNNs (i.e., InceptionV1, ResNet and a simple CNN). In the experiment, our results show that CNNs of RSI scene classification are also vulnerable to adversarial examples, and some of them have a fooling rate of over 80%. These adversarial examples are affected by the architecture of CNNs and the type of RSI dataset. InceptionV1 has a fooling rate of less than 5%, which is lower than the others. Adversarial examples generated on the UCM dataset are easier than other datasets. Importantly, we also find that the classes of adversarial examples have an attack selectivity property. Misclassifications of adversarial examples of RSIs are related to the similarity of the original classes in the CNN feature space. Attack selectivity reveals potential classes of adversarial examples and provides insights into the design of defensive algorithms in future research.

Feature detectors and descriptors have been successfully used for various computer vision tasks, such as video object tracking and content-based image retrieval. Many methods use image gradients in different stages of the detection-description pipeline to describe local image structures. Recently, some, or all, of these stages have been replaced by convolutional neural networks (CNNs), in order to increase their performance. A detector is defined as a selection problem, which makes it more challenging to implement as a CNN. They are therefore generally defined as regressors, converting input images to score maps and keypoints can be selected with non-maximum suppression. This paper discusses and compares several recent methods that use CNNs for keypoint detection. Experiments are performed both on the CNN based approaches, as well as a selection of conventional methods. In addition to qualitative measures defined on keypoints and descriptors, the bag-of-words (BoW) model is used to implement an image retrieval application, in order to determine how the methods perform in practice. The results show that each type of features are best in different contexts.

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.