Self-Attention Guided Low-Light Image Enhancement GAN with adaptive Multi-Objective Loss Optimization

Image dehazing has always been a challenging topic in image processing. The development of deep learning methods, especially the generative adversarial networks (GAN), provides a new way for image dehazing. In recent years, many deep learning methods based on GAN have been applied to image dehazing. However, GAN has two problems in image dehazing. Firstly, For haze image, haze not only reduces the quality of the image but also blurs the details of the image. For GAN network, it is difficult for the generator to restore the details of the whole image while removing the haze. Secondly, GAN model is defined as a minimax problem, which weakens the loss function. It is difficult to distinguish whether GAN is making progress in the training process. Therefore, we propose a guided generative adversarial dehazing network (GGADN). Different from other generation adversarial networks, GGADN adds a guided module on the generator. The guided module verifies the network of each layer of the generator. At the same time, the details of the map generated by each layer are strengthened. Network training is based on the pre-trained VGG feature model and L1-regularized gradient prior which is developed by new loss function parameters. From the dehazing results of synthetic images and real images, the proposed method is better than the state-of-the-art dehazing methods.

In recent years, enhancement of underwater images based on generative adversarial network (GAN) has been widely used. Aiming at the shortcoming that the texture details of the images generated by GAN are not clear enough, an underwater image pre-process unit and a generator including a perceptual subnet and a refine subnet are designed to improve visual quality of underwater images. The role of the perceptual subnet is to maintain the structure and semantic information of the input image. Refine subnet is to make the generated image clearer in the details of the texture. Specially, considering that underwater images may prevent the network from learning effectively, an underwater image pre-process unit is proposed to improve the quality of underwater images. To maintain the structure and semantic information of underwater images and preserve texture details of underwater images, the perceptual subnet and the refine subnet are proposed, respectively. Then, color-structure perception loss is proposed to obtain good performance in color and structure, and content loss and detail loss are proposed to keep content consistent and sharper texture. Ablation study verify the rationality and effectiveness of each loss function. Finally, subjective and objective experiments prove that the proposed method can obtain underwater images with higher visual quality and clearer texture details.

In this paper, we introduce a hardware accelerator for the Low-Light Image Enhancement (LLIE) network. This accelerator significantly enhances the effectiveness of LLIE while reducing computation time and power consumption. Specifically designed for the LLIE accelerator, we propose an RGB parallel processing algorithm for LLIE. LLIE can estimate individual channel components. The three channels share weights through a pulse array and converge toward the same direction via a self-calibration module. We design hardware acceleration units for different operators in LLIE and conduct testing. LLIE is deployed on Xilinx’s XCU102, with model inference parameters ranging from 0.96[Formula: see text]k to 3456[Formula: see text]k. The latency of LLIE on Field-Programmable Gate Array (FPGA) ranges from 0.19 to 7.69[Formula: see text]ms, with power consumption ranging from 0.4 to 1.68[Formula: see text]W. Compared to GPUs with similar performance, LLIE achieves approximately 20 times acceleration on FPGA, with power consumption only 3[Formula: see text] to 10[Formula: see text] of that of GPUs. Experimental results indicate that LLIE’s performance on FPGA improves with increasing model parameters. This system offers significant advantages in low-light image processing and promises to greatly facilitate real-time processing of low-light images in the future.

With the widespread adoption of Generative Adversarial Networks (GANs) in image generation and processing, enhancing their generation quality and visualization capabilities has become a prominent research focus. This study introduces a deep learning framework that integrates multi-scale information chart generation with visualization enhancement to improve the performance of GAN-based image generation models across various domains. Based on the Self-Attention Generative Adversarial Network (SAGAN), which leverages self-attention mechanisms to capture long-range dependencies in images, the proposed approach significantly enhances image quality and detail representation. The framework incorporates a multi-scale feature extraction method to optimize the feature maps at each layer of the generative network. Experimental results demonstrate that SAGAN outperforms traditional GAN models in terms of image clarity, detail preservation, and visual effects. The proposed model achieves notable improvements in diversity and generalization, with a mutual information content of 0.91, clustering uniformity of 0.89, and inter-cluster dissimilarity of 0.92 on the CelebA dataset. Furthermore, in terms of image quality, SAGAN attains a Structural Similarity Index Measure (SSIM) of 0.94 and a Peak Signal-to-Noise Ratio (PSNR) of 30.1, surpassing traditional GANs by a significant margin.

Images captured in a dark environment may suffer from low visibility, which may degrade the visual aesthetics of images and the performance of vision-based systems. Extensive studies have focused on the low-light image enhancement (LIE) problem. However, we observe that even though the state-of-the-art LIE methods may oversharpen the low-light image and introduce visual artifacts. To reduce the overshoot effects of LIE, this paper proposes an illumination-aware image quality assessment, called LIE-IQA, for the enhanced low-light images. Since directly using the IQA of degraded image may fail to perform well, the proposed LIE-IQA is an illumination-aware and learnable metric. At first, the reflectance and shading components of both the enhanced low-light image and reference image are extracted by intrinsic image decomposition. Then, we use the weighted similarity between the VGG-based feature of enhanced low-light image and reference image to obtain LIE-IQA, where the weight of the measurement can be learned from pairs of data on benchmark dataset. Qualitative and quantitative experiments illustrate the superiority of the LIE-IQA to measure the image quality of LIE on different datasets, including a new IQA dataset built for LIE. We also use the LIE-IQA as a regularization of a loss function to optimize an end-to-end LIE method, and the results indicate the potential of the optimization framework with LIE-IQA to reduce the overshoot effects of low-light enhancement.

This study introduces an innovative unsupervised low-light image enhancement method, termed GDLLE. The key innovation lies in decoupling the luminance and content representations of an image, thereby establishing a bidirectional mapping between low-light and normal-light images, which significantly enhances the visual quality of low-light images. In contrast to traditional supervised learning approaches that rely on paired datasets, the GDLLE method does not require paired low-light and normal-light image pairs. Instead, it enhances low-light images by extracting luminance masks and content information from unpaired images. Furthermore, the study incorporates an adaptive normalization strategy and a multi-scale enhancement framework to optimize both the global structure and local details of the image. To further improve the enhancement performance, a self-supervised mechanism based on cyclic consistency is introduced, which leverages cyclic consistency loss and self-feature preservation loss to ensure the quality of the generated images. Experimental results demonstrate that the GDLLE method outperforms existing approaches on several benchmark datasets, compared with the baseline models, GDLLE has achieved an improvement of 0.46 dB in the PSNR metric, validating its effectiveness and superiority in the unsupervised low-light image enhancement task. This research offers a novel solution in the field of low-light image enhancement, with promising potential for superior image quality improvement in practical applications.

In response to the challenges of blur distortion, low contrast and color fading in underwater images, caused by complex environmental factors and light attenuation, this study presents a novel underwater image enhancement method that leverages the U-Net architecture and channel attention mechanism fusion generative adversarial network (GAN), named UAEGAN. UAEGAN is built on the framework of GAN, combining the U-Net structure with a channel attention mechanism to construct a generator network, reducing the loss of low-level information during feature extraction and enhancing image details. Additionally, the algorithm employs a PatchGAN discriminator, which improves image resolution and detail representation by performing fine-grained true/false judgments on local image patches. Finally, the visual quality of the enhanced image is further optimized through the weighted fusion of multiple loss functions. Experimental results on the UIEB dataset indicate that UAEGAN outperforms the latest methods in terms of both visual quality and numerical metrics. The algorithm effectively enhances the clarity and visual quality of underwater images, providing strong support for subsequent underwater image processing tasks and applications.

A wavelet-based local and global feature fusion network (LAGN) is proposed for low-light image enhancement, aiming to enhance image details and restore colors in dark areas. This study focuses on addressing three key issues in low-light image enhancement: Enhancing low-light images using LAGN to preserve image details and colors; extracting image edge information via wavelet transform to enhance image details; and extracting local and global features of images through convolutional neural networks and Transformer to improve image contrast. Comparisons with state-of-the-art methods on two datasets verify that LAGN achieves the best performance in terms of details, brightness, and contrast.

Low-light image enhancement (LLIE) aims to improve the visibility and perceptual quality of images captured under low-light conditions, enhancing as much structure and detail as the captured data allow. Existing methods often rely on deep neural networks to map low-light images to normal-light counterparts within color spaces such as standard red green blue (sRGB) or hue-saturation-value (HSV). However, these approaches are prone to noise interference and lack robustness, resulting in noticeable artifacts in color and brightness. To address these challenges, we propose task-decoupled framework for low-light image enhancement (TDNet), a task-decoupled LLIE framework that regards LLIE as a multi-task learning problem: denoising and color correction, enabling targeted optimization for each sub-task. Specifically, the denoising sub-task further decouples noise suppression across three channels and employs a fine-denoise channel attention module to effectively handle channel-specific noise heterogeneity. Simultaneously, a cross-channel fusion attention module captures global semantic information and inter-channel dependencies, enhancing color consistency and ensuring natural visual aesthetics. In addition, a progressive detail recovery strategy is incorporated to refine structural clarity and edge details. Extensive experiments on multiple benchmark datasets demonstrate that TDNet achieves favorable performance in noise suppression and color correction while maintaining low computational complexity and parameter efficiency (https://github.com/634949684aa/TDNet).

Low-light image enhancement aims to enhance the visibility and contrast of low-light images while eliminating complex degradation issues such as noise, artifacts, and color distortions. Most existing low-light image enhancement methods either focus on quality while neglecting computational efficiency or have limited learning and generalization capabilities. To address these issues, we propose a Bilateral Enhancement Network with signal-to-noise ratio fusion, called BiEnNet, for lightweight and generalizable low-light image enhancement. Specifically, we design a lightweight Bilateral enhancement module with SNR (Signal-to-Noise Ratio) Fusion (BSF), which serves the SNR map of the input low-light image as the interpolation weights to dynamically fuse global brightness features and local detail features extracted from a bilateral network and achieve differentiated enhancement across different regions. To improve the network’s generalization ability, we propose a Luminance Normalization (LNM) module for preprocessing and a Dual-Exposure Processing (DEP) module for post-processing. LNM divides the channels of input features into luminance-related channels and luminance independent channels, and reduces the inconsistency of the degradation distribution of input low-light images by only normalizing the luminance-related channels. DEP learns overexposure and underexposure corrections simultaneously by employing the ReLU activation function, inverting operation, and residual network, which can improve the robustness of enhancement effects under different exposure conditions while reducing network parameters. Experiments on the LOL-V1 dataset shows BiEnNet significantly increased PSNR by 8.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} and SSIM by 3.6\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to FLW-Net, reduced parameters by 98.78\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and improved computational speed by 52.64\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to the classical KIND.

PurposeMagnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance.Materials and methods3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett’s S score and Fleiss’ kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively.ResultsIn the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63 mm and 0.92/5.91 mm for the bladder and 0.84/10.61 mm and 0.83/9.71 mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs’ boundaries compared with the original MRI.ConclusionThe bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs’ auto-segmentation outputs generated by the GAN.

Low-light image enhancement via deep Retinex decomposition and bilateral learning

The super-resolution reconstruction in diffusion-weighted imaging of preoperative rectal MR using generative adversarial network (GAN): Image quality and T-stage assessment

The low-light image enhancement task aims to improve the visibility of information in the dark to obtain more data and utilize it, while also improving the visual quality of the image. In this paper, we propose a dual cross-attention multi-stage embedding network (DCMENet) for fast and accurate enhancement of low-light images into high-quality images with high visibility. The problem that enhanced images tend to have more noise in them, which affects the image quality, is improved by introducing an attention mechanism in the encoder–decoder structure. In addition, the encoder–decoder can focus most of its attention on the dark areas of the image and better attend to the detailed features in the image that are obscured by the dark areas. In particular, the poor performance of the Transformer when the dataset size is small is solved by fusing the CNN-Attention and Transformer in the encoder. Considering the purpose of the low-light image enhancement task, we raise the importance of recovering image detail information to the same level as reconstructing the lighting. For features such as texture details in images, cascade extraction using spatial attention and pixel attention can reduce the model complexity while the performance is also improved. Finally, the global features obtained by the encoder–decoder are fused into the shallow feature extraction structure to reconstruct the illumination while guiding the network for the focused extraction of information in the dark. The proposed DCMENet achieves the best results in both objective quality assessment and subjective evaluation, while for the computer vision tasks working in low-light environments as well, the enhanced images using the DCMENet proposed in this paper show the best performance.

InfraGAN: A GAN architecture to transfer visible images to infrared domain