Multi image super resolution of MRI images using generative adversarial network

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

This article explores the application of Generative Adversarial Networks (GANs) in improving subsurface rock analysis. GANs generate realistic images of rock features, enhancing the accuracy and efficiency of estimating rock properties. The objective is to advance the field of subsurface rock analysis, offering insights into the potential of GAN techniques. The article reviews recent developments in data-driven rock reconstruction technology, specifically focusing on GANs. It highlights GANs’ ability to generate and estimate rock features like porosity and permeability. These GAN methods are complemented by discriminators to select or reject features, making them integral to the reconstruction model. The article provides a framework for engineers and researchers to effectively utilize these techniques for rock reconstruction, offering an assessment of their strengths and limitations. Digital rock reconstruction, aided by GANs, has promising implications for site selection and recovery. GANs enhance image resolution, improving the field of view and dynamism of parameters in rock analysis. However, GANs heavily rely on high-quality data, which poses a limitation. To address this, conditional GANs have been proposed. The article comprehensively reviews the latest developments in GAN methods for digital rock reconstruction, offering valuable insights for engineers and researchers using GANs for accurate subsurface rock analysis. It also proposes ways to enhance these methods and advance the field. The novelty of this article lies in its exploration of Generative Adversarial Networks (GANs) in the context of subsurface rock analysis. It emphasizes the potential of GAN techniques to generate realistic rock images and estimate properties like porosity and permeability. Additionally, the article discusses the integration of GANs into the digital rock concept, highlighting their role in improving predictive models and supporting decision-making in the oil and gas industry's resource extraction and production strategies.

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.

In the past few years, medium and high-resolution data became freely available for downloading. It provides great opportunity for researchers not to select between solving the task with high-resolution data on small territory or on global scale, but with low-resolution satellite images. Due to high spectral and spatial resolution of the data, Sentinel-1 and Sentinel-2 are very popular sources of information. Nevertheless, in practice if we would like to receive final product in 10 m resolution we should use bands with 10 m resolution. Sentinel-2 has four such bands, but also has other bands, especially red-edge 20 m resolution bands that are useful for vegetation analysis and often are omitted due to lower resolution. Thus, in this study we propose methodology for enhancing resolution (super-resolution) of the existing low-resolution images to higher resolution images. The main idea is to use advanced methods of deep learning - Generative Adversarial Networks (GAN) and train it to increase the resolution for the satellite images. Experimental results for the Sentinel-2 data showed that this approach is efficient and could be used for creating high resolution products.

As a new unsupervised learning algorithm framework, generative adversarial networks(GAN) have been favored by more and more researchers, and it has become a research hotspot now. GAN is inspired by the two-person zero-sum game theory in game theory. Its unique adversarial training idea can generate high-quality samples and has more powerful feature learning and feature expression capabilities than traditional machine learning algorithms. At present, GAN has achieved remarkable success in the field of computer vision, especially in the field of sample generation. Every year, a large number of GAN-related research papers are produced, reflecting the fiery degree of research on GAN model. Aiming at the hot model of GAN, first introduce the research status of GAN; then introduce the theory and framework of GAN, which analyzes in detail why the gradient disappears and the mode collapses during the training of GAN; then discussed some typical GAN improvement models, and summarized their theoretical improvements, advantages, limitations, application scenarios and implementation costs; Finally, the application results of GAN in data generation, image super-resolution, and image style conversion are shown, and the current challenges and future research directions of GAN are discussed.

Face Recognition is a challenging task present in different applications and systems. An existing challenge is to recognize faces when imaging conditions are adverse, for example when images come from low-quality cameras or when the subject and the camera are far apart, thus impacting the accuracy of these recognizing systems. Super-Resolution techniques can be used to improve both image resolution and quality, hopefully improving the accuracy of the face recognition task. Among these techniques, the actual state-of-the-art uses Generative Adversarial Networks. One promising option is to train Super-Resolution and Face Recognition as one single network, conducting the network to learn super resolution features that will improve its capability when recognizing faces. In the present work, we trained a super resolution face recognition model using a jointly-learn approach, combining a generative network for super resolution and a ResNet50 for Face Recognition. The model was trained with a discriminator network, following the generative adversarial training. The images generated by the network were convincing, but we could not converge the face recognition model. We hope that our contributions could help future works on this topic. Code is publicly available at https://github.com/OliRafa/SRFR-GAN.KeywordsSuper-resolutionFace RecognitionGenerative Adversarial NetworksMachine learning

High-resolution digital elevation models (DEMs) are widely used in many fields such as mapping, hydrology, meteorology and geology, where they can improve the accuracy and reliability of many geographic analysis applications as an input. However, due to the high cost and difficulty of acquiring high-resolution DEMs, as well as the problems of edge smoothing, data distortion and fractures in reconstructed ground surfaces with traditional super-resolution DEM reconstruction techniques. Inspired by the excellence of generative adversarial neural networks in super-resolution image analysis, this paper investigates an approach for DEM super-resolution reconstruction with deep residual generative adversarial network. An advanced DEM Super-resolution Generative Adversarial Network (D-SRCAGAN) is proposed in this paper, which can reconstruct a quadruple higher resolution DEM by using low-resolution DEM. Compared with the bicubic and SRGAN methods, the D-SRCAGAN method reconstruction results can retain more topographic features and obtain higher RMSE values.

Aiming at the object recognition through UAV, an intelligent object recognition model based on YOLO and Generative adversarial network is proposed in this paper. Firstly, the solution is given, and an object recognition model that can realize intelligent recognition is established. Then, in order to improve the resolution of the identified images, an image resolution enhancement model based on generative adversarial networks is built. After that, the structure and parameters of the recognition model and image resolution enhancement model are adjusted through the simulation experiments to improve the accuracy and robustness of the object recognition. Finally, the object recognition model based on YOLO and generative adversarial network in this paper is verified through UAV.

Using a Generative Adversarial Network (GAN) has proven its ability to successfully implement realistic images in image translation fields. It has its successful implementation in the sketch-to-image translation, too. Generative adversarial networks are widely used for the purpose of image translation. Most discriminators in generative adversarial networks use encoder or decoder blocks for image segmentation and classification tasks. U-net-based architecture is mostly used in the generator but rarely in the discriminator. If used in the discriminator, it is used for image resolution increment and segmentation tasks. In this research, a U-net-based discriminator is used for image translation tasks. U-net-based discriminator uses local and global differences between the real and fake images, which helps maintain global and local data representation. Resnet-9, used in the generator, uses skip connections, shortcuts, and concatenations, enabling information to flow from earlier to later layers. This helps preserve the original image features and solves the vanishing gradient problems in normal generators. The use of a strong discriminator and effective generator helps in the improvement system's performance. The available dataset was unpaired at the same time. Datasets from various sources were combined and formed a sketch-image pair. The input is a 512x256 human sketch and a corresponding real image pair. The image pair is split into sketch and image with dimensions 256x256. The system's output is the human face image of the corresponding sketch.

Abstract: High-resolution satellite imagery is vital for applications such as environmental monitoring, urban planning, and defense. However, the availability of such data is limited by sensor constraints and acquisition costs. This paper investigates the use of Generative Adversarial Networks (GANs) to enhance the resolution of satellite images, using low-resolution input and learning to generate high-quality details. We explore and evaluate three GAN variants—SRGAN, ESRGAN, and WDSR-GAN—on two publicly available datasets: SpaceNet and UC Merced. Quantitative results indicate that ESRGAN achieves the highest performance, with a Peak Signal-to-Noise Ratio (PSNR) of 28.7 dB and Structural Similarity Index (SSIM) of 0.91. Regression and visual fidelity analysis confirm its robustness across diverse landscapes, showing its potential for improving satellite image usability. Keywords: satellite imagery, super-resolution, GANs, SRGAN, ESRGAN, WDSR, PSNR, SSIM, SpaceNet, UC Merced, deep learning

Although the majority of existing License Plate (LP) recognition techniques have significant improvements in accuracy, they are still limited to ideal situations in which training data is correctly annotated with restricted scenarios. Moreover, images or videos are frequently used in monitoring systems that have Low Resolution (LR) quality. In this work, the problem of LP detection in digital images is addressed in the images of a naturalistic environment. Single-stage character segmentation and recognition are combined with adversarial Super-Resolution (SR) approaches to improve the quality of the LP by processing the LR images into High-Resolution (HR) images. This work proposes effective changes to the SRGAN network regarding the number of layers, an activation function, and the appropriate loss regularization using Total Variation (TV) loss. The main paper contribution can be summarized into presenting an end-to-end deep learning framework based on generative adversarial networks (GAN), which is able to generate realistic super-resolution images. Also, proposed adding a TV regularization to the loss function to help the model enhance the resolution of images. The proposed SRGAN can handle tiny <inline-formula> <tex-math notation="LaTeX">$72\times 72$ </tex-math></inline-formula> images of LPs. The paper explores how SRGAN performed over different datasets from many aspects, such as visual analysis, PSNR, SSIM, and Optical Character Recognition (OCR). The experiments demonstrate that the suggested SRGAN can generate high-resolution images that improve the accuracy of the license plate recognition stage compared to other systems.

The thermal camera systems can be used in all kinds of applications that require the detection of heat change, but thermal imaging systems are highly costly systems. In recent years, developments in the field of deep learning have increased the success by obtaining quality results compared to traditional methods. In this paper, thermal images of neonates (healthy - unhealthy) obtained from a high-resolution thermal camera were used and these images were evaluated as high resolution (ground truth) images. Later, these thermal images were downscaled at 1/2, 1/4, 1/8 ratios, and three different datasets consisting of low-resolution images in different sizes were obtained. In this way, super-resolution applications have been carried out on the deep network model developed based on generative adversarial networks (GAN) by using three different datasets. The successful performance of the results was evaluated with PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). In addition, healthy - unhealthy classification application was carried out by means of a classifier network developed based on convolutional neural networks (CNN) to evaluate the super-resolution images obtained using different datasets. The obtained results show the importance of combining medical thermal imaging with super-resolution methods.

In recent years, the study of a single image super-resolution (SISR) is crucial to improving image resolution and using hardware technology to improve image resolution. SISR is widely used in satellite remote sensing, video surveillance, and medical image processing because it mainly relies on deep learning algorithms to realize the conversion from low-resolution (LR) images to high-resolution images. It has the advantages of low cost, simple operation, and high efficiency. This paper proposes an image super-resolution method based on a generative adversarial network named text localization generative adversarial nets (TLGAN) model. The method is improved based on super-resolution generative adversarial networks (SRGAN), and the batch normalization layer is removed, which significantly reduces the computational burden of the model. In TLGAN model, we used the transfer learning method to pre-trained the model on the large dataset ImageNet, and then apply the pre-trained model to the cartoon image data set animes to achieve image super-resolution. Experimental results report that the proposed method has the advantages of fast running speed and excellent visual perception of super-resolution images compared with bicubic interpolation and SRGAN method.

Single Image Super-Resolution (SISR) stands at the forefront of image processing, offering a transformative solution to elevate low-resolution images to a higher fidelity.Leveraging deep neural networks, particularly Generative Adversarial Networks (GANs), this study aims to enrich the quality of images, especially in domains like medical imaging, public surveillance, and historical image restoration. Traditional methods often fail to preserve crucial details during up scaling, prompting the adoption of deep learning techniques which have demonstrated remarkable breakthroughs in accuracy and efficiency. By harnessing the adversarial training process inherent in GANs, we seek to propel the boundaries of image enhancement, striving to faithfully reconstruct high-resolution renditions while retaining essential nuances from their low-resolution counterparts. Through comprehensive performance evaluations utilizing publicly available datasets, we conduct rigorous quantitative and qualitative analyses to gauge the efficacy of various GAN architectures in SISR. This meticulous examination allows us to assess not only the fidelity of the reconstructed images but also the computational efficiency and scalability of each approach, essential considerations for real-world applications. However, alongside the promises of GAN-based SISR come inherent challenges that warrant attention. Issues such as training instability, mode collapse, and imageartifacts present hurdles that must be addressed to fully harness the potential of GANs in image enhancement.Additionally, the integration of domain-specific expertise, particularly in fields like medical imaging, can further enhance the utility and efficacy of GAN-based super-resolution techniques.In conclusion, while the journey towards achieving high-quality, high-resolution images may be fraught with challenges, the integration of GANs into SISR represents a significant leap forward, promising to revolutionize various industries and deepen our understanding of visual data.

Images created using the Super Resolution method can generate more information compared to their low resolution counterparts. A super-resolved image, which is created using an original image captured by an imaging source is not only more meaningful to human perception but also has advantages on downstream tasks such as object detection and pattern recognition. In this work, we aim to apply the Super Resolution method to the Aerial Images captured for surveillance to enable more information about the original scenes. To achieve this Super Resolution Generative Adversarial Network (SRGAN), which is based on the Generative Adversarial Networks architecture is used. We also applied transfer learning methodology to achieve better image quality. Public xView and DOTA datasets which contain images mostly captured by satellites around the world are used to train a generative model via SRGAN architecture. Furthermore, DIV2K dataset is used to pre-train a generative model, and then the transfer learning technique is used to train separate models on xView and DOTA validation datasets. Perceptual Index (PI) and Root Mean Squared Error (RMSE) which are used on European Conference on Computer Vision -Perceptual Image Restoration and Manipulation Workshop 2018 are computed as the performance metrics. We have seen that the model which gives the best PI results, i.e. better perceptual quality, on xView and DOTA validation datasets is the one trained using the DIV2K dataset and the model which gives the best RMSE results, i.e. better reconstruction quality, is the one trained using the transfer learning technique.