Exploring the Contributions of Low-light Image Enhancement to Network-based Object Detection

Low-light object detection involves identifying and locating objects in images captured under poor lighting conditions. It plays a significant role in surveillance and security, night pedestrian recognition, and autonomous driving, showcasing broad application prospects. Most existing object detection algorithms and datasets are designed for normal lighting conditions, leading to a significant drop in detection performance when applied to low-light environments. To address this issue, we propose a Low-Light Detection with Low-Light Enhancement (LDWLE) framework. LDWLE is an encoder-decoder architecture where the encoder transforms the raw input data into a compact, abstract representation (encoding), and the decoder gradually generates the target output format from the representation produced by the encoder. Specifically, during training, low-light images are input into the encoder, which produces feature representations that are decoded by two separate decoders: an object detection decoder and a low-light image enhancement decoder. Both decoders share the same encoder and are trained jointly. Throughout the training process, the two decoders optimize each other, guiding the low-light image enhancement towards improvements that benefit object detection. If the input image is normally lit, it first passes through a low-light image conversion module to be transformed into a low-light image before being fed into the encoder. If the input image is already a low-light image, it is directly input into the encoder. During the testing phase, the model can be evaluated in the same way as a standard object detection algorithm. Compared to existing object detection algorithms, LDWLE can train a low-light robust object detection model using standard, normally lit object detection datasets. Additionally, LDWLE is a versatile training framework that can be implemented on most one-stage object detection algorithms. These algorithms typically consist of three components: the backbone, neck, and head. In this framework, the backbone functions as the encoder, while the neck and head form the object detection decoder. Extensive experiments on the COCO, VOC, and ExDark datasets have demonstrated the effectiveness of LDWLE in low-light object detection. In quantitative measurements, it achieves an AP of 25.5 and 38.4 on the synthetic datasets COCO-d and VOC-d, respectively, and achieves the best AP of 30.5 on the real-world dataset ExDark. In qualitative measurements, LDWLE can accurately detect most objects on both public real-world low-light datasets and self-collected ones, demonstrating strong adaptability to varying lighting conditions and multi-scale objects.

Low light images often have low visibility, which not only affects the visual effect, but also reduces the performance of algorithms that require high-quality input. Aiming at the problem of low light image enhancement, this paper proposes a composite enhancement algorithm. Firstly, the dark channel prior model and retinex model are combined by two adjustable parameters to obtain a new enhancement model DeRetinex. Then, according to the duality of the dehazing model and retinex theory, the image of the previous step is inverted, and the DeRetinex model is used for the second enhancement, which can eliminate the haze caused by enhancement. Compared with the existing mainstream algorithms, the proposed algorithm has the advantages of avoiding over exposure, rich texture details, low noise and high color recovery.

The ability of the device to capture high-resolution images poses a new challenge to image processing, and most of the existing low-light image enhancement algorithms are designed for low-resolution images, and there are problems such as unclear details and colour distortion when dealing with high-resolution images. In this paper, an edge and colour information guided high resolution low light image enhancement algorithm is proposed using the texture information and colour information contained in the image itself. In order to improve the limitation of local feature learning of convolutional neural network, an edge decoder is introduced, which helps to capture the key information at a long distance in the image and improves the encoding of boundary semantic information. In addition, in order to deal with high-resolution images, a sparse attention mechanism is introduced in the contextual attention block, which focuses on the important information in the image and effectively reduces noise interference. On the other hand, the colour decoder effectively uses the chromaticity cues of the low-light image itself to improve the accuracy of colour information recovery

Low light image enhancement algorithm based on improved multi-objective grey wolf optimization with detail feature enhancement

With the widespread application of image enhancement technology in the digital protection of intangible cultural heritage, research on improving low light image enhancement algorithms is receiving increasing attention. Embroidery, as an important component of intangible cultural heritage, has always been highly valued for its image enhancement work in both industry and academia. In recent years, deep learning (DL) has demonstrated outstanding performance in areas such as image classification, object detection, and image enhancement, providing new ideas for enhancing embroidery textures of different types. However, how to improve image illumination while preserving detail clarity and avoiding overfitting remains an urgent challenge that needs to be addressed. This article proposes an embroidery texture image enhancement model based on Generative Adversarial Network (GAN) to meet the demand for digital storage of intangible cultural heritage. This model significantly improves processing efficiency and effectively avoids overfitting by combining attention mechanisms while reducing the number of network parameters. The results indicate that our model not only significantly improves the brightness performance of the image, but also makes the enhanced embroidery texture smoother and more natural, with clearer details.

Low-light image enhancement using Gaussian Process for features retrieval

Self-supervised network for low-light traffic image enhancement based on deep noise and artifacts removal

In this paper, we introduce a hardware accelerator for the Low-Light Image Enhancement (LLIE) network. This accelerator significantly enhances the effectiveness of LLIE while reducing computation time and power consumption. Specifically designed for the LLIE accelerator, we propose an RGB parallel processing algorithm for LLIE. LLIE can estimate individual channel components. The three channels share weights through a pulse array and converge toward the same direction via a self-calibration module. We design hardware acceleration units for different operators in LLIE and conduct testing. LLIE is deployed on Xilinx’s XCU102, with model inference parameters ranging from 0.96[Formula: see text]k to 3456[Formula: see text]k. The latency of LLIE on Field-Programmable Gate Array (FPGA) ranges from 0.19 to 7.69[Formula: see text]ms, with power consumption ranging from 0.4 to 1.68[Formula: see text]W. Compared to GPUs with similar performance, LLIE achieves approximately 20 times acceleration on FPGA, with power consumption only 3[Formula: see text] to 10[Formula: see text] of that of GPUs. Experimental results indicate that LLIE’s performance on FPGA improves with increasing model parameters. This system offers significant advantages in low-light image processing and promises to greatly facilitate real-time processing of low-light images in the future.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

Low-light image enhancement via deep Retinex decomposition and bilateral learning

Images captured in low-light conditions usually suffer from very low contrast and underexpose, which cannot be directly utilized in the subsequent computer vision tasks, such as object recognition, detection, identification and tracking. Existing methods include HE based method, Retinex theory based method and deep learning method which may generate undesirable enhanced results including amplified noise, biased colors and extreme boundary. To address this problem, we utilize prior knowledge of Retinex theory and GAN based on data statistic to propose a progressive GAN-based Transfer network to realize the low-light enhancement. In this paper, the image is decomposed by JieP method based on the Retinex model to obtain the reflection and light components, and learn the relationship between the reflection component of the low-light image and normal light image via a reflection decomposition on network (RefDecN), and then generate the reflection component of the low-light image. Then, another illumination transfering network (IllumTransN) is utilized to transfer the light of normal light image to the reflection component to realize low-light enhancement. Experimental results of low-light image enhancement on RAISE, LOL and MEF datasets demonstrate our ProGAN can outperform state-of-the-art methods in terms of objective and subjective quality.KeywordsLow-light enhancementReflection componentIllumination componentRetinex modelGenerative adversarial network

Channel Self-Attention Based Low-Light Image Enhancement Network

In colonoscopy, the captured images are usually with low-quality appearance, such as non-uniform illumination, low contrast, etc., due to the specialized imaging environment, which may provide poor visual feedback and bring challenges to subsequent disease analysis. Many low-light image enhancement (LIE) algorithms have recently proposed to improve the perceptual quality. However, how to fairly evaluate the quality of enhanced colonoscopy images (ECIs) generated by different LIE algorithms remains a rarely-mentioned and challenging problem. In this study, we carry out a pioneering investigation on perceptual quality assessment of ECIs. Firstly, considering the lack of specific datasets, we collect 300 low-light images with diverse contents during the real-world colonoscopy and conduct rigorous subjective studies to compare the performance of 8 popular LIE methods, resulting in a benchmark dataset (named ECIQAD) for ECIs. Secondly, in view of the distinctive distortion characteristics of ECIs, we propose an effective no-reference Enhanced Colonoscopy Image Quality (ECIQ) method to automatically evaluate the perceptual quality of ECIs via analysis of brightness, contrast, colorfulness, naturalness, and noise. Extensive experiments on ECIQAD demonstrate the superiority of our proposed ECIQ method over 14 mainstream no-reference image quality assessment methods.

In recent years, deep learning-based algorithms for low-light images have achieved SOTA performance. Traditional unsupervised learning methods typically use image-to-image transformation models to solve low-light image enhancement problems. However they do not perform well on real images and scenes because of the over-reliance on paired real and synthetic image datasets that do not suppress the noise in existing scenes. This is mainly due to domain migration. To address the effects of domain migration, we propose an unsupervised domain adaptation paradigm. This includes a different image translation module, an image illumination enhancement module and a noise suppression module. In order to reduce the impact of domain migration, we align two domains through a two-way translation network. We use the images before and after translation and join the network for training at the same time to maintain a consistency loss between the same group of images. At this stage, we incorporate real low-light images into the enhancement network training using the properties of well-illuminated images (Retinex theory) to further improve domain adaptation. Through ablation and comparative experiments, our proposed method outperforms recent approaches under various metrics about visual quality and subjective feelings.

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.