(Retracted) Super-resolution generative adversarial networks using autoencoder reduce dimension

In recent years, the study of a single image super-resolution (SISR) is crucial to improving image resolution and using hardware technology to improve image resolution. SISR is widely used in satellite remote sensing, video surveillance, and medical image processing because it mainly relies on deep learning algorithms to realize the conversion from low-resolution (LR) images to high-resolution images. It has the advantages of low cost, simple operation, and high efficiency. This paper proposes an image super-resolution method based on a generative adversarial network named text localization generative adversarial nets (TLGAN) model. The method is improved based on super-resolution generative adversarial networks (SRGAN), and the batch normalization layer is removed, which significantly reduces the computational burden of the model. In TLGAN model, we used the transfer learning method to pre-trained the model on the large dataset ImageNet, and then apply the pre-trained model to the cartoon image data set animes to achieve image super-resolution. Experimental results report that the proposed method has the advantages of fast running speed and excellent visual perception of super-resolution images compared with bicubic interpolation and SRGAN method.

Super-resolution reconstruction of pneumocystis carinii pneumonia images based on generative confrontation network

Since the neural network was introduced into the super-resolution (SR) field, many SR deep models have been proposed and have achieved excellent results. However, there are two main drawbacks: one is that the methods based on the best peak-signal-to-noise ratio (PSNR) do not have enough comfortable visual quality; the other is that although the SR models based on generative adversarial network (GAN) have satisfactory visual quality, the structure of the reconstructed image has apparent defects. Therefore, according to the characteristics that human eyes are sensitive to high-frequency components in images, this article proposes an improved image SRGAN model based on high-frequency information fusion (HIFGAN). It builds a feature extraction network for high-frequency information fusion by designing a lightweight spatial attention module and improving the network architecture of enhanced super-resolution GAN (ESRGAN). It makes the generator in the GAN network have better feature recovery ability, reduces the dependence of the later training on the decider and loss function, and makes the generated image structure more consistent with the real situation. In addition, we build a high-frequency loss function to optimize the training of the generator network. Detailed experimental results show that HIFGAN performs excellently in both objective criterion evaluation and subjective visual effect. Compared with the state-of-the-art GAN-based SR networks, the reconstructed image by our model is more precise and complete in texture details.

Generating High-Resolution Climate Prediction through Generative Adversarial Network

Reconstruction of sub‐mm 3D pavement images using recursive generative adversarial network for faster texture measurement

The goal of speech super-resolution (SSR) or speech bandwidth expansion is to generate the missing high-frequency components for a given low-resolution speech signal. It has the potential to improve the quality of telecommunications. We propose a new method for SSR that leverages the generative adversarial networks (GANs) and a regularization method for stabilizing the GAN training. The generator network is a convolutional autoencoder with 1D convolution kernels, operating along time-axis and generating the high-frequency log-power spectra from the low-frequency log-power spectra input. We employ two recent deep neural network (DNN) based approaches to compare them with our proposed method, including both objective speech quality metrics and subjective perceptual tests. We show that our proposed method outperforms the baseline methods in terms of both objective and subjective evaluations.

Images created using the Super Resolution method can generate more information compared to their low resolution counterparts. A super-resolved image, which is created using an original image captured by an imaging source is not only more meaningful to human perception but also has advantages on downstream tasks such as object detection and pattern recognition. In this work, we aim to apply the Super Resolution method to the Aerial Images captured for surveillance to enable more information about the original scenes. To achieve this Super Resolution Generative Adversarial Network (SRGAN), which is based on the Generative Adversarial Networks architecture is used. We also applied transfer learning methodology to achieve better image quality. Public xView and DOTA datasets which contain images mostly captured by satellites around the world are used to train a generative model via SRGAN architecture. Furthermore, DIV2K dataset is used to pre-train a generative model, and then the transfer learning technique is used to train separate models on xView and DOTA validation datasets. Perceptual Index (PI) and Root Mean Squared Error (RMSE) which are used on European Conference on Computer Vision -Perceptual Image Restoration and Manipulation Workshop 2018 are computed as the performance metrics. We have seen that the model which gives the best PI results, i.e. better perceptual quality, on xView and DOTA validation datasets is the one trained using the DIV2K dataset and the model which gives the best RMSE results, i.e. better reconstruction quality, is the one trained using the transfer learning technique.

Acquiring High Resolution (HR) Magnetic Resonance (MR) images requires the patient to remain still for long periods of time, which causes patient discomfort and increases the probability of motion induced image artifacts. A possible solution is to acquire low resolution (LR) images and to process them with the Super Resolution Generative Adversarial Network (SRGAN) to create a super-resolved version. This work applies SRGAN to MR images of the prostate and performs three experiments. The first experiment explores improving the in-plane MR image resolution by factors of 4 and 8, and shows that, while the PSNR and SSIM (Structural SIMilarity) metrics are lower than the isotropic bicubic interpolation baseline, the SRGAN is able to create images that have high edge fidelity. The second experiment explores anisotropic super-resolution via synthetic images, in that the input images to the network are anisotropically downsampled versions of HR images. This experiment demonstrates the ability of the modified SRGAN to perform anisotropic super-resolution, with quantitative image metrics that are comparable to those of the anisotropic bicubic interpolation baseline. Finally, the third experiment applies a modified version of the SRGAN to super-resolve anisotropic images obtained from the through-plane slices of the volumetric MR data. The output super-resolved images contain a significant amount of high frequency information that make them visually close to their HR counterparts. Overall, the promising results from each experiment show that super-resolution for MR images is a successful technique and that producing isotropic MR image volumes from anisotropic slices is an achievable goal.

In urethra-sparing radiation therapy, prostatic urinary tract visualization is important in decreasing the urinary side effect. A methodology has been developed to visualize the prostatic urinary tract using post-urination magnetic resonance imaging (PU-MRI) without a urethral catheter. This study investigated whether the combination of PU-MRI and super-resolution (SR) deep learning models improves the visibility of the prostatic urinary tract. We enrolled 30 patients who had previously undergone real-time-image-gated spot scanning proton therapy by insertion of fiducial markers. PU-MRI was performed using a non-contrast high-resolution two-dimensional T2-weighted turbo spin-echo imaging sequence. Four different SR deep learning models were used: the enhanced deep SR network (EDSR), widely activated SR network (WDSR), SR generative adversarial network (SRGAN), and residual dense network (RDN). The complex wavelet structural similarity index measure (CW-SSIM) was used to quantitatively assess the performance of the proposed SR images compared to PU-MRI. Two radiation oncologists used a 1-to-5 scale to subjectively evaluate the visibility of the prostatic urinary tract. Cohen's weighted kappa (k) was used as a measure of agreement of inter-operator reliability. The mean CW-SSIM in EDSR, WDSR, SRGAN, and RDN was 99.86%, 99.89%, 99.30%, and 99.67%, respectively. The mean prostatic urinary tract visibility scores of the radiation oncologists were 3.70 and 3.53 for PU-MRI (k = 0.93), 3.67 and 2.70 for EDSR (k = 0.89), 3.70 and 2.73 for WDSR (k = 0.88), 3.67 and 2.73 for SRGAN (k = 0.88), and 4.37 and 3.73 for RDN (k = 0.93), respectively. The results suggest that SR images using RDN are similar to the original images, and the SR deep learning models subjectively improve the visibility of the prostatic urinary tract.

Restoration of damaged images is a fundamental problem that has been attempted before the advent of digital image processing technology. In this paper, one of the deep neural network technologies (GAN), we propose an image restoration network using Generative Adversarial Network. The proposed system is the image generation network, the generation result plateIt consists of a star network. Old documents not only contain information, but also we can learn about historical people's thought and consciousness from the past. Old document restoration is referred to as the restoration of documents that are usually made of parchment which are damaged either naturally or artificially. Missing regions in old documents are filled based on the current visual data which is a hard task in image inpainting. In this paper, we present Super Resolution Generative Adversarial Network (SRGAN) and semantic image inpainting for restoring the old documents so that they can be reused.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Deep learning superresolution (SR) is a promising approach to reduce MRI scan time without requiring custom sequences or iterative reconstruction. Previous deep learning SR approaches have generated low-resolution training images by simple k-space truncation, but this does not properly model in-plane turbo spin echo (TSE) MRI resolution degradation, which has variable T2 relaxation effects in different k-space regions. To fill this gap, we developed a T2 -deblurred deep learning SR method for the SR of 3D-TSE images. A SR generative adversarial network was trained using physically realistic resolution degradation (asymmetric T2 weighting of raw high-resolution k-space data). For comparison, we trained the same network structure on previous degradation models without TSE physics modeling. We tested all models for both retrospective and prospective SR with 3 × 3 acceleration factor (in the two phase-encoding directions) of genetically engineered mouse embryo model TSE-MR images. The proposed method can produce high-quality 3 × 3 SR images for a typical 500-slice volume with 6-7 mouse embryos. Because 3 × 3 SR was performed, the image acquisition time can be reduced from 15 h to 1.7 h. Compared to previous SR methods without TSE modeling, the proposed method achieved the best quantitative imaging metrics for both retrospective and prospective evaluations and achieved the best imaging-quality expert scores for prospective evaluation. The proposed T2 -deblurring method improved accuracy and image quality of deep learning-based SR of TSE MRI. This method has the potential to accelerate TSE image acquisition by a factor of up to 9.

Recently, there have been significant advances in image super-resolution based on generative adversarial networks (GANs) to achieve breakthroughs in generating more images with high subjective quality. However, there are remaining challenges needs to be met, such as simultaneously recovering the finer texture details for large upscaling factors and mitigating the geometric transformation effects. In this paper, we propose a novel robust super-resolution GAN (i.e. namely RSR-GAN) which can simultaneously perform both the geometric transformation and recovering the finer texture details. Specifically, since the performance of the generator depends on the discreminator, we propose a novel discriminator design by incorporating the spatial transformer module with residual learning to improve the discrimination of fake and true images through removing the geometric noise, in order to enhance the super-resolution of geometric corrected images. Finally, to further improve the perceptual quality, we introduce an additional DCT loss term into the existing loss function. Extensive experiments, measured by both PSNR and SSIM measurements, show that our proposed method achieves a high level of robustness against a number of geometric transformations, including rotation, translation, a combination of rotation and scaling effects, and a cobmination of rotaion, transalation and scaling effects. Benchmarked by the existing state-of-the-arts SR methods, our proposed delivers superior performances on a wide range of datasets which are publicly available and widely adopted across research communities.

Super-resolution (SR) techniques present a suitable solution to increase the image resolution acquired using an ultrasound device characterized by a low image resolution. This can be particularly beneficial in low-resource imaging settings. This work surveys advanced SR techniques applied to enhance the resolution and quality of fetal ultrasound images, focusing Dual back-projection based internal learning (DBPISR) technique, which utilizes internal learning for blind super-resolution, as opposed to blind super-resolution generative adversarial network (BSRGAN), real-world enhanced super-resolution generative adversarial network (Real-ESRGAN), swin transformer for image restoration (SwinIR) and SwinIR-Large. The dual back-projection approach enhances SR by iteratively refining downscaling and super-resolution processes through a dual network training method, achieving high accuracy in kernel estimation and image reconstruction. Real-ESRGAN uses synthetic data to simulate complex real-world degradations, incorporating a U-shaped network (U-Net) discriminator to improve training stability and visual performance. BSRGAN addresses the limitations of traditional degradation models by introducing a realistic and comprehensive degradation process involving blur, downsampling, and noise, leading to superior real-world SR performance. Swin models (SwinIR and SwinIR_large) employ a Swin Transformer architecture for image restoration, excelling in capturing long-range dependencies and complex structures, resulting in an outstanding performance in PSNR, SSIM, NIQE, and BRISQUE metrics. The tested images, sourced from five developing countries and often of lower quality, enabled us to show that these approaches can help enhance the quality of the images. Evaluations on fetal ultrasound images reveal that these methods significantly enhance image quality, with DBPISR, Real-ESRGAN, BSRGAN, SwinIR, and SwinIR-Large showing notable improvements in PSNR and SSIM, thereby highlighting their potential for improving the resolution and diagnostic utility of fetal ultrasound images. We evaluated the five aforementioned Super-Resolution models, analyzing their impact on both image quality and classification tasks. Our findings indicate that these techniques hold great potential for enhancing the evaluation of medical images, particularly in development countries. Among the models tested, Real-ESRGAN consistently enhanced both image quality and diagnostic accuracy, even when challenged by limited and variable datasets. This finding was further supported by deploying the ConvNext-base classifier, which demonstrated improved performance when applied to the super-resolved images. Real-ESRGAN’s capacity to enhance image quality, and in turn, classification accuracy, highlights its potential to address the resource constraints often encountered in these settings.

Optimizing the discriminator in Generative Adversarial Networks (GANs) to completion in the inner training loop is computationally prohibitive, and on finite datasets would result in overfitting. To address this, a common update strategy is to alternate between k optimization steps for the discriminator D and one optimization step for the generator G. This strategy is repeated in various GAN algorithms where k is selected empirically. In this paper, we show that this update strategy is not optimal in terms of accuracy and convergence speed, and propose a new update strategy for networks with Wasserstein GAN (WGAN) group related loss functions (e.g. WGAN, WGAN-GP, Deblur GAN, and Super resolution GAN). The proposed update strategy is based on a loss change ratio comparison of G and D. We demonstrate that the proposed strategy improves both convergence speed and accuracy.