GAMSNet: Deblurring via Generative Adversarial and Multi-Scale Networks

In the automatic sorting process of express delivery, a three-segment code is used to represent a specific area assigned by a specific delivery person. In the process of obtaining the courier order information, the camera is affected by factors such as light, noise, and subject shake, which will cause the information on the courier order to be blurred, and some information will be lost. Therefore, this paper proposes an image text deblurring method based on a generative adversarial network. The model of the algorithm consists of two generative adversarial networks, combined with Wasserstein distance, using a combination of adversarial loss and perceptual loss on unpaired datasets to train the network model to restore the captured blurred images into clear and natural image. Compared with the traditional method, the advantage of this method is that the loss function between the input and output images can be calculated indirectly through the positive and negative generative adversarial networks. The Wasserstein distance can achieve a more stable training process and a more realistic generation effect. The constraints of adversarial loss and perceptual loss make the model capable of training on unpaired datasets. The experimental results on the GOPRO test dataset and the self-built unpaired dataset showed that the two indicators, peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), increased by 13.3% and 3%, respectively. The human perception test results demonstrated that the algorithm proposed in this paper was better than the traditional blur algorithm as the deblurring effect was better.

Objective. Computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in medical imaging modalities, and provide valuable information for clinical diagnosis and treatment. However, due to hardware limitations and radiation safety concerns, the acquired images are often limited in resolution. Super-resolution reconstruction (SR) techniques have been developed to enhance the resolution of CT and MRI slices, which can potentially improve diagnostic accuracy. To capture more useful feature information and reconstruct higher quality super-resolution images, we proposed a novel hybrid framework SR model based on generative adversarial networks. Approach. The proposed SR model combines frequency domain and perceptual loss functions, which can work in both frequency domain and image domain (spatial domain). The proposed SR model consists of 4 parts: (i) the discrete Fourier transform (DFT) operation transforms the image from the image domain to frequency domain; (ii) a complex residual U-net performs SR in the frequency domain; (iii) the inverse discrete Fourier transform (iDFT) operation based on data fusion transforms the image from the frequency domain to image domain; (iv) an enhanced residual U-net network is used for SR of image domain. Main results. Experimental results on bladder MRI slices, abdomen CT slices, and brain MRI slices show that the proposed SR model outperforms state-of-the-art SR methods in terms of visual quality and objective quality metric such as the structural similarity (SSIM) and the peak signal-to-noise ratio (PSNR), which proves that the proposed model has better generalization and robustness. (Bladder dataset: upscaling factor of 2: SSIM = 0.913, PSNR = 31.203; upscaling factor of 4: SSIM = 0.821, PSNR = 28.604. Abdomen dataset: upscaling factor of 2: SSIM = 0.929, PSNR = 32.594; upscaling factor of 4: SSIM = 0.834, PSNR = 27.050. Brain dataset: SSIM = 0.861, PSNR = 26.945). Significance. Our proposed SR model is capable of SR for CT and MRI slices. The SR results provide a reliable and effective foundation for clinical diagnosis and treatment.

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

Satellite cloud image extrapolation technology plays a crucial role in enhancing the timeliness and accuracy of weather forecasting. Traditional extrapolation methods, such as optical flow and Generative Adversarial Networks, often exhibit limitations when handling the nonlinear motion and complex evolution of cloud clusters. The Dynamic Multi-Scale Voxel Flow Network (DMVFN) model effectively captures multi-scale motion features through its pyramid structure and adaptive voxelization strategy, enabling high-precision prediction of cloud cluster evolution. This study trained three models using FY-4B satellite data from the 10.8 μm infrared channel covering East Asia between June and August 2023. Three representative cases from 2024 were selected for comparative experiments between DMVFN, GAN, and optical flow methods. The results demonstrate that DMVFN outperforms the other methods across key metrics, including Structural Similarity (SSIM), Peak Signal-to-Noise Ratio (PSNR), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). It excels in preserving cloud map structural integrity while minimizing prediction errors. The model exhibits excellent stability and temporal consistency, providing effective technical support for enhancing short-term cloud system forecasting capabilities and supporting meteorological disaster prevention and emergency response.

Recent studies have achieved great progress on accuracy and speed of single image super-resolution (SISR) based on neural networks. Most current SISR methods use mean squared error (MSE) loss as objective function. As a result, they can get high peak signal-to-noise ratios (PSNR) which are however not in full agreement with the visual qualities by experiments, and thus the output from these methods could be prone to blurry and over-smoothed. Especially at large upscaling factors, the output images are perceptually unsatisfactory in general. In this paper, we firstly propose a novel residual network architecture based on generative adversarial network (GAN) for image super-resolution (SR), which is capable of inferring photo-realistic images for 4\(\times \) upscaling factors. Perceptual loss is applied as the objective function to make output image sharper and more real. In addition, we adopt some tricks to preprocess the input dataset and use improved techniques to train the generator and discriminator separately, which are proved to be effective for the result. We validate our GAN-based approach on CelebA dataset with mean opinion score (MOS) as performance measure. The results demonstrate that the proposed approach performs better than previous methods.

In this study, we propose a CNN-GAN-based real-time processing technique for filtering images of underwater cables used in power systems. This addresses the excessive interference impurities that are frequently observed in images captured by remotely operated vehicles (ROVs). The process begins with the input of the original image into the convolutional neural network (CNN). Subsequently, the training outcomes, which serve as input parameters for the generative adversarial network (GAN), facilitate the filtering process. The system also calculates both the structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR), performing model updates via backward propagation. This technique utilizes deep learning technologies to achieve rapid, real-time filtering of underwater cable images. The experimental results reveal that the loss function of the CNN reaches 0.16 with an accuracy of 97.5%, while the loss function of the adversarial GAN network approaches 0.05. Compared with traditional methods such as DDN, JORDOR, RESCAN, and PRENet, the proposed CNN-GAN algorithm exhibits superior performance, as evidenced by the higher PSNR and SSIM values. Specifically, for clear water images, the PSNR reaches 29.86 dB and the SSIM is 0.9045. For severely polluted images, the PSNR is 28.67 dB and the SSIM is 0.8965, while for unevenly illuminated images, the PSNR and SSIM values are 24.37 dB and 0.88, respectively. These enhancements significantly benefit the monitoring and maintenance of power systems.

Image dehazing has always been a challenging topic in image processing. The development of deep learning methods, especially the generative adversarial networks (GAN), provides a new way for image dehazing. In recent years, many deep learning methods based on GAN have been applied to image dehazing. However, GAN has two problems in image dehazing. Firstly, For haze image, haze not only reduces the quality of the image but also blurs the details of the image. For GAN network, it is difficult for the generator to restore the details of the whole image while removing the haze. Secondly, GAN model is defined as a minimax problem, which weakens the loss function. It is difficult to distinguish whether GAN is making progress in the training process. Therefore, we propose a guided generative adversarial dehazing network (GGADN). Different from other generation adversarial networks, GGADN adds a guided module on the generator. The guided module verifies the network of each layer of the generator. At the same time, the details of the map generated by each layer are strengthened. Network training is based on the pre-trained VGG feature model and L1-regularized gradient prior which is developed by new loss function parameters. From the dehazing results of synthetic images and real images, the proposed method is better than the state-of-the-art dehazing methods.

Generative Adversarial Networks (GANs) have been used as a deep learning approach to solving physics and engineering problems. Using deep learning for these problems is attractive in that reasonably accurate models can be inferred from only raw data, eliminating the need to define the exact physical equations governing a problem. We expand on previous work [Farimani et al. 2017] using GANs to generate steady-state solutions to the two-dimensional heat equation. Using a basic conditional GAN (cGAN), we generate accurate solutions for rectangular domains conditioned on four edge boundary conditions (MAE%3C0.5%). For finding steady-state solutions over arbitrary two-dimensional domains (not constrained to rectangles), we use a cGAN designed for image-to-image translation. We train this GAN on various types of geometric domains (circles, squares, triangles, shapes with one circular or rectangular hole), achieving accurate results on test data made up of geometries similar to those in training (MAE%3C1%). For both of these GANs, we experiment with different loss function terms, showing that a term using the gradients of solution images significantly improves the basic cGAN but not the image-to-image GAN. Lastly, we show that the image-to-image GAN performs poorly when applied to two-dimensional geometries that vary in structure from training data (MAE%3C8% for shapes with multiple holes or different shaped holes). This demonstrates the cGAN's lack of generalizability. While the cGAN is an accurate and computationally efficient method when trained and tested on similarly structured data, it is a much less reliable method when applied to data that is slightly different in structure from the training data.

It is difficult for laser scanning confocal microscopy to obtain high- or ultra-high-resolution laser confocal images directly, which affects the deep mining and use of the embedded information in laser confocal images and forms a technical bottleneck in the in-depth exploration of the microscopic physiological and biochemical processes of plants. The super-resolution reconstruction model (SRGAN), which is based on a generative adversarial network and super-resolution reconstruction model (SRResNet), which is based on a residual network, was used to obtain single and secondary super-resolution reconstruction images of laser confocal images of the root cells of the hyperaccumulator Solanum nigrum. Using the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and mean opinion score (MOS), the models were evaluated by the image effects after reconstruction and were applied to the recognition of endocytic vesicles in Solanum nigrum root cells. The results showed that the single reconstruction and the secondary reconstruction of SRGAN and SRResNet improved the resolution of laser confocal images. PSNR, SSIM, and MOS were clearly improved, with a maximum PSNR of 47.690. The maximum increment of PSNR and SSIM of the secondary reconstruction images reached 21.7% and 2.8%, respectively, and the objective evaluation of the image quality was good. However, overall MOS was less than that of the single reconstruction, the perceptual quality was weakened, and the time cost was more than 130 times greater. The reconstruction effect of SRResNet was better than that of SRGAN. When SRGAN and SRResNet were used for the recognition of endocytic vesicles in Solanum nigrum root cells, the clarity of the reconstructed images was obviously improved, the boundary of the endocytic vesicles was clearer, and the number of identified endocytic vesicles increased from 6 to 9 and 10, respectively, and the mean fluorescence intensity was enhanced by 14.4% and 7.8%, respectively. Relevant research and achievements are of great significance for promoting the application of deep learning methods and image super-resolution reconstruction technology in laser confocal image studies.

Pseudo CT Synthesis Using Cone-Beam CT of Cervical Cancer with GAN-Based Neural Network Model

Advanced Chinese ink painting also includes work-brush flower and bird paintings with brilliant colors, in contrast to traditional ink paintings that often only use water, ink, and black and white. This serves as the foundation for our investigation into a generalized transfer problem involving ink and wash, or an ink painting coloring problem. Our goal is to automatically colorize black and white ink paintings using deep neural networks. This study can serve as a guide for coloring ink paintings and broaden the range of applications for ink painting style transfer. The high-level semantic information and low-level local features of ink paintings cannot be successfully extracted using the current generalized style transfer approach (colorization algorithm). The resulting images have muddy borders and low color saturation. In order to improve the accuracy and coherence of the coloring of ink paintings, we build training by combining the global and local features of ink paintings with the achievements of generative adversarial networks already made in the field of colorization. Comparative and objective evaluations of the experimental portion are made using metrics like peak signal-to-noise ratio (PSNR), structural similarity (SSIM), colorfulness, and user studies. Additionally, our approach beats the previous comparison approaches in terms of creative expression, color richness, and color overflow management.

Underwater image enhancement is a necessary process to improve the quality of images captured in underwater environments. The underwater environment presents unique challenges that can significantly impact the quality of images, including reduced light and color, distortion and blurring, limited visibility and reflections, and the effects of particles and debris. Enhancing underwater images can help to overcome these challenges and improve the visibility of the captured scene. This can be especially important in applications such as underwater exploration, marine biology, and underwater archaeology, where clear and detailed images are crucial for analysis and decision-making. This study compares Contrast Limited Adaptive Histogram Equalization (CLAHE) and Generative Adversarial Networks (GAN). Images generated from both algorithms are compared on the basis of Peak Signal Noise Ratio (PSNR) values. PSNR value of image generated by GAN is greater than PSNR value obtained by CLAHE, so in order to obtain a highly enhanced underwater image GAN will be suitable.

Future frame prediction is a challenging task in computer vision with practical applications in areas such as video generation, autonomous driving, and robotics. Traditional recurrent neural networks have limited effectiveness in capturing long-range dependencies between frames, and combining convolutional neural networks (CNNs) with recurrent networks has limitations in modeling complex dependencies. Generative adversarial networks have shown promising results, but they are computationally expensive and suffer from instability during training. In this article, we propose a novel approach for future frame prediction that combines the encoding capabilities of 3-D CNNs with the sequence modeling capabilities of Transformers. We also propose a spatial self-attention mechanism and a novel neighborhood pixel intensity loss to preserve structural information and local intensity, respectively. Our approach outperforms existing methods in terms of structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and learned perceptual image patch similarity (LPIPS) scores on five public datasets. More precisely, our model exhibited an average improvement of 4.64%, 18.5%, and 42% concerning SSIM, PSNR, and LPIPS for the second most proficient method correspondingly, across all datasets. The results demonstrate the effectiveness of our proposed method in generating high-quality predictions of future frames.

In this study, we evaluated the improvement of image quality in digital breast tomosynthesis under low-radiation dose conditions of pre-reconstruction processing using conditional generative adversarial networks [cGAN (pix2pix)]. Pix2pix pre-reconstruction processing with filtered back projection (FBP) was compared with and without multiscale bilateral filtering (MSBF) during pre-reconstruction processing. Noise reduction and preserve contrast rates were compared using full width at half-maximum (FWHM), contrast-to-noise ratio (CNR), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) in the in-focus plane using a BR3D phantom at various radiation doses [reference-dose (automatic exposure control reference dose: AECrd), 50% and 75% reduction of AECrd] and phantom thicknesses (40 mm, 50 mm, and 60 mm). The overall performance of pix2pix pre-reconstruction processing was effective in terms of FWHM, PSNR, and SSIM. At ~50% radiation-dose reduction, FWHM yielded good results independently of the microcalcification size used in the BR3D phantom, and good noise reduction and preserved contrast. PSNR results showed that pix2pix pre-reconstruction processing represented the minimum in the error with reference FBP images at an approximately 50% reduction in radiation-dose. SSIM analysis indicated that pix2pix pre-reconstruction processing yielded superior similarity when compared with and without MSBF pre-reconstruction processing at ~50% radiation-dose reduction, with features most similar to the reference FBP images. Thus, pix2pix pre-reconstruction processing is promising for reducing noise with preserve contrast and radiation-dose reduction in clinical practice.

In current times, reconstruction of remote sensing images using super-resolution is a prominent topic of study. Remote sensing data have a complex spatial distribution. Compared with natural pictures, remote sensing pictures often contain subtler and more complicated information. Most super-resolution reconstruction algorithms cannot restore all the information contained in remote sensing images when reconstructing them. The content of some areas in the reconstructed images may be too smooth, and some areas may even have color changes, resulting in lower quality reconstructed images. In response to the problems presenting in current reconstruction algorithms about super-resolution, this article proposes the SRGAN-MSAM-DRC model (SRGAN model with multi-scale attention mechanism and dense residual connection). This model roots in generative adversarial networks and incorporates multi-scale attention mechanisms and dense residual connections into the generator. Furthermore, residual blocks are incorporated into the discriminator. We use some remote sensing image datasets of real-world data to evaluate this model, and the results indicate the SRGAN-MSAM-DRC model has shown enhancements in three evaluation metrics for reconstructed images about super-resolution. Compared to the basic SRGAN model, the SSIM (structural similarity), PSNR (peak signal-to-noise ratio), and IE (image entropy) increase by 5.0%, 4.0%, and 4.1%, respectively. From the results, we know the quality of the reconstructed images of remote sensing using the SRGAN-MSAM-DRC model is better than basic SRGAN model, and verifies that the model has good applicability and performance in reconstruction of remote sensing images using super-resolution.