Contextual Information Aided Generative Adversarial Network for Low-Light Image Enhancement

Low-light image enhancement using Gaussian Process for features retrieval

Image restoration is a critical task in computer vision that involves recovering an original image from corrupted or damaged versions of it. Traditional image restoration relies on interpolation and completion techniques, such as Navier–Stokes equations and the fast multipole boundary element method. However, these methods often fail to capture the advanced semantic information of an image, leading to inaccurate restoration results. In recent years, generative adversarial networks (GANs) have emerged as a practical approach in computer vision for image restoration tasks, as they can address image restoration issues related to damaged or missing image information. GANs are a deep‐learning network structure with a generator and discriminator. In image restoration, the generator is employed to restore damaged images, and the discriminator is used to assess the authenticity of the repair results. Competition between the generator and discriminator improves the quality of the repair results. This study proposes a novel generative image restoration method that utilizes contextual and perceptual semantic information mechanisms to strengthen GANs. Our approach demonstrates the effectiveness of GANs in restoring images through learning how to fuse the missing or damaged parts of an image with the undamaged parts surrounding them, resulting in visually appealing restoration results.

High-resolution construction images are essential for quality monitoring and progress management. Addressing the challenge that existing super-resolution models often struggle to balance reconstruction quality with real-time inference speed in complex construction environments, this paper proposes a super-resolution reconstruction model based on attention mechanisms and Generative Adversarial Networks (GANs). The generator captures fine details through a Multi-scale Shallow Feature Extraction (MSFE) module. The deep feature mapping module introduces the SORRDB (SFT-Octave-Residual in Residual Dense Block) as its fundamental unit. This module integrates SFT layers for dynamic spatial feature modulation and incorporates Octave Convolution to decouple high- and low-frequency information, thereby reducing computational redundancy, while scSE attention mechanisms further enhance high-level structural features. The discriminator integrates Swin Transformer to capture both local and global features, enhancing the quality and realism of generated images. Experimental results on the SODA dataset demonstrate that the model achieves superior performance with PSNR, SSIM, and LPIPS scores of 28.063 dB, 0.825, and 0.197, respectively. Crucially, the model achieves an inference speed of 32.0 FPS on an NVIDIA GeForce RTX 4070, meeting the requirements for real-time monitoring. Therefore, the proposed model not only significantly enhances the perceptual quality and structural integrity of construction images but also achieves an optimal balance between performance and efficiency, offering substantial benefits for intelligent construction monitoring.

The purpose of image fusion is to combine the source images of the same scene into a single composite image with more useful information and better visual effects. Fusion GAN has made a breakthrough in this field by proposing to use the generative adversarial network to fuse images. In some cases, considering retain infrared radiation information and gradient information at the same time, the existing fusion methods ignore the image contrast and other elements. To this end, we propose a new end-to-end network structure based on generative adversarial networks (GANs), termed as FLGC-Fusion GAN. In the generator, using the learnable grouping convolution can improve the efficiency of the model and save computing resources. Therefore, we can have a better trade-off between the accuracy and speed of the model. Besides, we take the residual dense block as the basic network building unit and use the perception characteristics of the inactive as content loss characteristics of input, achieving the effect of deep network supervision. Experimental results on two public datasets show that the proposed method performs well in subjective visual performance and objective criteria and has obvious advantages over other current typical methods.

Low-light object detection involves identifying and locating objects in images captured under poor lighting conditions. It plays a significant role in surveillance and security, night pedestrian recognition, and autonomous driving, showcasing broad application prospects. Most existing object detection algorithms and datasets are designed for normal lighting conditions, leading to a significant drop in detection performance when applied to low-light environments. To address this issue, we propose a Low-Light Detection with Low-Light Enhancement (LDWLE) framework. LDWLE is an encoder-decoder architecture where the encoder transforms the raw input data into a compact, abstract representation (encoding), and the decoder gradually generates the target output format from the representation produced by the encoder. Specifically, during training, low-light images are input into the encoder, which produces feature representations that are decoded by two separate decoders: an object detection decoder and a low-light image enhancement decoder. Both decoders share the same encoder and are trained jointly. Throughout the training process, the two decoders optimize each other, guiding the low-light image enhancement towards improvements that benefit object detection. If the input image is normally lit, it first passes through a low-light image conversion module to be transformed into a low-light image before being fed into the encoder. If the input image is already a low-light image, it is directly input into the encoder. During the testing phase, the model can be evaluated in the same way as a standard object detection algorithm. Compared to existing object detection algorithms, LDWLE can train a low-light robust object detection model using standard, normally lit object detection datasets. Additionally, LDWLE is a versatile training framework that can be implemented on most one-stage object detection algorithms. These algorithms typically consist of three components: the backbone, neck, and head. In this framework, the backbone functions as the encoder, while the neck and head form the object detection decoder. Extensive experiments on the COCO, VOC, and ExDark datasets have demonstrated the effectiveness of LDWLE in low-light object detection. In quantitative measurements, it achieves an AP of 25.5 and 38.4 on the synthetic datasets COCO-d and VOC-d, respectively, and achieves the best AP of 30.5 on the real-world dataset ExDark. In qualitative measurements, LDWLE can accurately detect most objects on both public real-world low-light datasets and self-collected ones, demonstrating strong adaptability to varying lighting conditions and multi-scale objects.

Aiming at the problems of over-enhancement of local regions and lack of detail processing of key regions in the overall enhancement of images captured under low-light conditions, this paper proposes a low-light image enhancement method based on local region exposure fusion, which is based on the design of an unsupervised deep learning network model for the overexposed and underexposed regions of the target image, and the prediction of the source image and the hierarchical image by the filter to fuse the local information in order to achieve higher-quality of low-light image enhancement. By introducing a multi-scale residual network for layer decomposition of the image, the key information of the source image at multiple levels is obtained; a deeper hybrid attention mechanism is used to obtain a sequence of filters composed of different convolutional kernel sizes, which adaptively focuses on the local information of the image at each layer, and completes the prediction of the fusion region of the adaptive filters; the whole fusion uses an unsupervised gradient optimization loss to obtain the gradient information and achieve the optimization of the model. Experiments on the SICE dataset show that our method performs well compared with the most representative current methods in the field of exposure fusion and low-light enhancement, can effectively balance the visual disparity of the image, reconstruct high-quality images, and in terms of target detection, exhibits good performance and results, providing technical support for the practical application of low-light image enhancement.

Low-light is a challenging environment for both human and computer vision to perform tasks such as object classification and detection. Recent works have shown potential in employing enhancements algorithms to support and improve such tasks in low-light, however there has not been any focused analysis to understand the direct effects that low-light enhancement have on an object detector. This work aims to quantify and visualize such effects on the multi-level abstractions involved in network-based object detection. First, low-light image enhancement algorithms are employed to enhance real low-light images, and then followed by deploying an object detection network on the low-light as well as the enhanced counterparts. A comparison of the activations in different layers, representing the detection features, are used to generate statistics in order to quantify the enhancements' contribution to detection. Finally, this framework was used to analyze several low-light image enhancement algorithms and identify their impact on the detection model and task. This framework can also be easily generalized to any convolutional neural network-based models for the analysis of different enhancements algorithms and tasks.

In recent years, enhancement of underwater images based on generative adversarial network (GAN) has been widely used. Aiming at the shortcoming that the texture details of the images generated by GAN are not clear enough, an underwater image pre-process unit and a generator including a perceptual subnet and a refine subnet are designed to improve visual quality of underwater images. The role of the perceptual subnet is to maintain the structure and semantic information of the input image. Refine subnet is to make the generated image clearer in the details of the texture. Specially, considering that underwater images may prevent the network from learning effectively, an underwater image pre-process unit is proposed to improve the quality of underwater images. To maintain the structure and semantic information of underwater images and preserve texture details of underwater images, the perceptual subnet and the refine subnet are proposed, respectively. Then, color-structure perception loss is proposed to obtain good performance in color and structure, and content loss and detail loss are proposed to keep content consistent and sharper texture. Ablation study verify the rationality and effectiveness of each loss function. Finally, subjective and objective experiments prove that the proposed method can obtain underwater images with higher visual quality and clearer texture details.

DivGAN: A diversity enforcing generative adversarial network for mode collapse reduction

Low-light Image Enhancement (LLIE) aims to restore images captured under insufficient illumination to a normally lit appearance with adequate brightness, clear structures, and natural colors. However, existing methods still perform poorly in extremely dark conditions, where structural distortion, loss of details, amplified noise, and color imbalance frequently occur, making it difficult to maintain both structural fidelity and perceptual quality. To address these challenges, this paper proposes CA-YUVNet, a cross-attention-based low-light enhancement model. The model converts input images from RGB to YUV space to decouple luminance and chrominance, thereby suppressing noise propagation and color distortion commonly observed in low-light scenes. Furthermore, a cross-branch crossattention mechanism is introduced in the U and V branches to mitigate color shifts and chromatic artifacts. To further improve detail restoration and color consistency, a Bipolar Channel Attention Fusion (BCAF) module is designed, which integrates multi-scale convolution and deep feature fusion to enhance noise suppression and structural refinement. Experimental results on the LOL-v1, LOL-v2-real, and LOL-v2-synthetic datasets demonstrate that CA-YUVNet outperforms existing methods, validating its effectiveness and robustness.

Progressive feature fusion for SNR-aware low-light image enhancement

Captured images often suffer from issues like color distortion, detail loss, and significant noise. Therefore, it is necessary to improve image quality for reliable threat detection. Balancing brightness enhancement with the preservation of natural colors and details is particularly challenging in low-light image enhancement. To address these issues, this paper proposes an unsupervised low-light image enhancement approach using a U-net neural network with Retinex theory and a Convolutional Block Attention Module (CBAM). This method leverages Retinex-based decomposition to separate and enhance the reflectance map, ensuring visibility and contrast without introducing artifacts. A local adaptive enhancement function improves the brightness of the reflection map, while the designed loss function addresses illumination smoothness, brightness enhancement, color restoration, and denoising. Experiments validate the effectiveness of our method, revealing improved image brightness, reduced color deviation, and superior color restoration compared to leading approaches.

Low-light image enhancement is crucial for improving the quality of applications such as nighttime photography and security surveillance. The core challenges lie in significant noise amplification due to weak signals, severe detail loss from underexposure, and motion blur caused by handheld shooting, all of which compromise image clarity and realism. Existing multi-frame low-light enhancement methods typically focus on pixel-level fusion, lacking a holistic grasp of the final image’s perceptual quality. Inspired by the GPP-LLIE framework, we are the first to integrate generative perceptual priors from Vision-Language Models (VLMs) with Transformer-based temporal fusion. By employing an adaptive alignment strategy and a novel perceptual prior-guided cross-frame attention mechanism, our method achieves effective information alignment and complementary enhancement across burst frames. The proposed method demonstrates superior performance on the SID dataset, excelling in both quantitative metrics and visual quality.

Low-light image enhancement in zero-shot area has gained attention as it relies on image priors for brightness adjustment avoiding the need for paired low/normal-light images. However, complex lighting and noise affect prior extraction, leading to noise and artifacts in reconstructed images. To address this issue, we propose a Self-Attention-guided Generative Adversarial Network (SA-GAN) for low-light image enhancement, which will improve the visual quality and details of images. Furthermore, a multi-objective adaptive balancing mechanism is introduced to dynamically optimize the weights of multiple loss functions, achieving optimal enhancement. Extensive experiments on multiple benchmarks demonstrate that the proposed method achieves performance comparable to state-of-the-art approaches.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.