Joint-attention Discriminator for Accurate Super-resolution via Adversarial Training

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Super-resolution (SR) techniques present a suitable solution to increase the image resolution acquired using an ultrasound device characterized by a low image resolution. This can be particularly beneficial in low-resource imaging settings. This work surveys advanced SR techniques applied to enhance the resolution and quality of fetal ultrasound images, focusing Dual back-projection based internal learning (DBPISR) technique, which utilizes internal learning for blind super-resolution, as opposed to blind super-resolution generative adversarial network (BSRGAN), real-world enhanced super-resolution generative adversarial network (Real-ESRGAN), swin transformer for image restoration (SwinIR) and SwinIR-Large. The dual back-projection approach enhances SR by iteratively refining downscaling and super-resolution processes through a dual network training method, achieving high accuracy in kernel estimation and image reconstruction. Real-ESRGAN uses synthetic data to simulate complex real-world degradations, incorporating a U-shaped network (U-Net) discriminator to improve training stability and visual performance. BSRGAN addresses the limitations of traditional degradation models by introducing a realistic and comprehensive degradation process involving blur, downsampling, and noise, leading to superior real-world SR performance. Swin models (SwinIR and SwinIR_large) employ a Swin Transformer architecture for image restoration, excelling in capturing long-range dependencies and complex structures, resulting in an outstanding performance in PSNR, SSIM, NIQE, and BRISQUE metrics. The tested images, sourced from five developing countries and often of lower quality, enabled us to show that these approaches can help enhance the quality of the images. Evaluations on fetal ultrasound images reveal that these methods significantly enhance image quality, with DBPISR, Real-ESRGAN, BSRGAN, SwinIR, and SwinIR-Large showing notable improvements in PSNR and SSIM, thereby highlighting their potential for improving the resolution and diagnostic utility of fetal ultrasound images. We evaluated the five aforementioned Super-Resolution models, analyzing their impact on both image quality and classification tasks. Our findings indicate that these techniques hold great potential for enhancing the evaluation of medical images, particularly in development countries. Among the models tested, Real-ESRGAN consistently enhanced both image quality and diagnostic accuracy, even when challenged by limited and variable datasets. This finding was further supported by deploying the ConvNext-base classifier, which demonstrated improved performance when applied to the super-resolved images. Real-ESRGAN’s capacity to enhance image quality, and in turn, classification accuracy, highlights its potential to address the resource constraints often encountered in these settings.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

Single Image Super-Resolution (SISR) stands at the forefront of image processing, offering a transformative solution to elevate low-resolution images to a higher fidelity.Leveraging deep neural networks, particularly Generative Adversarial Networks (GANs), this study aims to enrich the quality of images, especially in domains like medical imaging, public surveillance, and historical image restoration. Traditional methods often fail to preserve crucial details during up scaling, prompting the adoption of deep learning techniques which have demonstrated remarkable breakthroughs in accuracy and efficiency. By harnessing the adversarial training process inherent in GANs, we seek to propel the boundaries of image enhancement, striving to faithfully reconstruct high-resolution renditions while retaining essential nuances from their low-resolution counterparts. Through comprehensive performance evaluations utilizing publicly available datasets, we conduct rigorous quantitative and qualitative analyses to gauge the efficacy of various GAN architectures in SISR. This meticulous examination allows us to assess not only the fidelity of the reconstructed images but also the computational efficiency and scalability of each approach, essential considerations for real-world applications. However, alongside the promises of GAN-based SISR come inherent challenges that warrant attention. Issues such as training instability, mode collapse, and imageartifacts present hurdles that must be addressed to fully harness the potential of GANs in image enhancement.Additionally, the integration of domain-specific expertise, particularly in fields like medical imaging, can further enhance the utility and efficacy of GAN-based super-resolution techniques.In conclusion, while the journey towards achieving high-quality, high-resolution images may be fraught with challenges, the integration of GANs into SISR represents a significant leap forward, promising to revolutionize various industries and deepen our understanding of visual data.

In recent years, the study of a single image super-resolution (SISR) is crucial to improving image resolution and using hardware technology to improve image resolution. SISR is widely used in satellite remote sensing, video surveillance, and medical image processing because it mainly relies on deep learning algorithms to realize the conversion from low-resolution (LR) images to high-resolution images. It has the advantages of low cost, simple operation, and high efficiency. This paper proposes an image super-resolution method based on a generative adversarial network named text localization generative adversarial nets (TLGAN) model. The method is improved based on super-resolution generative adversarial networks (SRGAN), and the batch normalization layer is removed, which significantly reduces the computational burden of the model. In TLGAN model, we used the transfer learning method to pre-trained the model on the large dataset ImageNet, and then apply the pre-trained model to the cartoon image data set animes to achieve image super-resolution. Experimental results report that the proposed method has the advantages of fast running speed and excellent visual perception of super-resolution images compared with bicubic interpolation and SRGAN method.

In order to enhance the visual effects of the reconstructed images in the single image super-resolution, and solve the problem of instability of the training phase of super-resolution with generative adversary network (SRGAN), a new super-resolution model is built with more depth and width, and the corresponding super-resolution algorithm is proposed. The network structure of the original SRGAN is modified, so that deeper and wider convolutional networks can be used with high efficiency. Furthermore, the loss function is employed to balance the training of the generative network and the discriminator network, and solve the instability problem in the training phase. The training images for the network are obtained from the database of ImageNet on the web as other researchers. Experimental results show that images reconstructed by the proposed algorithm has better visual effects than those by the original SRGAN. In addition, the objective measures of PSNR and SSIM of the reconstructed images have also been improved.

Recent studies have shown that Deep Learning (DL) algorithms can significantly improve Super Resolution (SR) performance. Single image SR is useful in producing High Resolution (HR) images from their Low Resolution (LR) counterparts. The motivation for SR is the potential to assist algorithms such as object detection, localization, and classification. Insufficient work has been conducted using Generative Adversarial Networks (GANs) for SR on infrared (IR) images despite its promising ability to increase object detection accuracy by extracting more precise features from a given image. This work adopts the idea of a relativistic GAN that utilizes Residual in Residual Dense blocks (RRDBs) for feature ex- traction, a novel residual image addition, and a Pixel Transposed Convolutional Layer (PixelTCL) for up-sampling. Recent work has validated the use of GANs for Visible Light (VL) images, making them a strong candidate. The inclusion of these components produce more realistic and natural features while also receiving superior metric values.

Since the neural network was introduced into the super-resolution (SR) field, many SR deep models have been proposed and have achieved excellent results. However, there are two main drawbacks: one is that the methods based on the best peak-signal-to-noise ratio (PSNR) do not have enough comfortable visual quality; the other is that although the SR models based on generative adversarial network (GAN) have satisfactory visual quality, the structure of the reconstructed image has apparent defects. Therefore, according to the characteristics that human eyes are sensitive to high-frequency components in images, this article proposes an improved image SRGAN model based on high-frequency information fusion (HIFGAN). It builds a feature extraction network for high-frequency information fusion by designing a lightweight spatial attention module and improving the network architecture of enhanced super-resolution GAN (ESRGAN). It makes the generator in the GAN network have better feature recovery ability, reduces the dependence of the later training on the decider and loss function, and makes the generated image structure more consistent with the real situation. In addition, we build a high-frequency loss function to optimize the training of the generator network. Detailed experimental results show that HIFGAN performs excellently in both objective criterion evaluation and subjective visual effect. Compared with the state-of-the-art GAN-based SR networks, the reconstructed image by our model is more precise and complete in texture details.

With the availability of commercial Light Field (LF) cameras, LF imaging has emerged as an up-and-coming technology in computational photography. However, the spatial resolution is significantly constrained in commercial micro-lens-based LF cameras because of the inherent multiplexing of spatial and angular information. Therefore, it becomes the main bottleneck for other applications of light field cameras. This paper proposes an adaptation module in a pre-trained Single Image Super-Resolution (SISR) network to leverage the powerful SISR model instead of using highly engineered light field imaging domain-specific Super Resolution models. The adaption module consists of a Sub-aperture Shift block and a fusion block. It is an adaptation in the SISR network to further exploit the spatial and angular information in LF images to improve the super-resolution performance. Experimental validation shows that the proposed method outperforms existing light field super-resolution algorithms. It also achieves PSNR gains of more than 1 dB across all the datasets as compared to the same pre-trained SISR models for scale factor 2, and PSNR gains 0.6 − 1 dB for scale factor 4.

Single-image super-resolution based on an improved asymmetric Laplacian pyramid structure

This paper considers a deep Generative Adversarial Networks (GAN) based method referred to as the Perception-Enhanced Super-Resolution (PESR) for Single Image Super Resolution (SISR) that enhances the perceptual quality of the reconstructed images by considering the following three issues: (1) ease GAN training by replacing an absolute with a relativistic discriminator, (2) include in the loss function a mechanism to emphasize difficult training samples which are generally rich in texture and (3) provide a flexible quality control scheme at test time to trade-off between perception and fidelity. Based on extensive experiments on six benchmark datasets, PESR outperforms recent state-of-the-art SISR methods in terms of perceptual quality. The code is available at https://github.com/thangvubk/PESR.

Single image super resolution is a very important computer vision task, with a wide range of applications. In recent years, the depth of the super-resolution model has been constantly increasing, but with a small increase in performance, it has brought a huge amount of computation and memory consumption. In this work, in order to make the super resolution models more effective, we proposed a novel single image super resolution method via recursive squeeze and excitation networks (SESR). By introducing the squeeze and excitation module, our SESR can model the interdependencies and relationships between channels and that makes our model more efficiency. In addition, the recursive structure and progressive reconstruction method in our model minimized the layers and parameters and enabled SESR to simultaneously train multi-scale super resolution in a single model. After evaluating on four benchmark test sets, our model is proved to be above the state-of-the-art methods in terms of speed and accuracy.

Superresolution (SR) has provided an effective solution to the increasing need for high-resolution images in remote sensing applications. Among various SR methods, deep learning-based SR (DLSR) has made a significant breakthrough. However, supervised DLSR methods require a considerable amount of training data, which is hardly available in the remote sensing field. To address this issue, some research works have recently proposed and revealed the capability of deep learning in unsupervised SR. This article presents an efficient unsupervised SR (EUSR) deep learning model using dense skip connections, which boosts the reconstruction performance in parallel with the reduction of computational burden. To do this, several blocks containing densely connected convolutional layers are implemented to increase the depth of the model. Some skip connections also concatenate feature maps of different blocks to enable better SR performance. Moreover, a bottle-neck block abstracts the feature maps in fewer feature maps to remarkably reduce the computational burden. According to our experiments, the proposed EUSR leads to better results than the state-of-the-art DLSR method in terms of reconstruction quality with less computational burden. Furthermore, results indicate that the EUSR is more robust than its rival in dealing with images of different classes and larger sizes.

To reduce external disturbances and achieve high vertical resolution, the scanning time for white-light interference microscopy is very short. Because capturing high-resolution (HR) images is time consuming, low-resolution (LR) images are acquired instead. However, HR images are more desirable because they contain more details. To ensure high vertical resolution and high image resolution, one feasible solution is to process the scanned LR images to HR images by single image super-resolution (SISR). In this paper, an interference image super-resolution (IISR) model based on a generative adversarial network (GAN) is proposed. The generator is based on the enhanced super-resolution generative adversarial network (ESRGAN) architecture. With the aim of acquiring more realistic images, the discriminator network is designed using a modified DenseNet architecture, in which the pooling layers are replaced with dilated convolutional layers. The perceptual loss is optimized, and the content loss is upgraded to a continuously differentiable piecewise function. Various microscopy images are tested, including images with and without interference fringes. The IISR model has been proven to restore LR images to HR images. The comparative experiments prove that the proposed model achieves better visual quality than other models, preserving more realistic details.

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.