Edge-Enhanced GAN for Remote Sensing Image Superresolution

Generative adversarial network (GAN) has been widely applied to produce super-resolution (SR) image with real perception and texture details. In most existing SR approaches, the training objective typically measures a pixel-wise average distance between the SR and high-resolution (HR) images. However, as the degradation function of different images from HR to low resolution (LR) is generally different, optimizing such metrics often leads to certain unpleasant artificial traces. Unlike the prevalent GAN inversion methods that require expensive image-specific optimization at runtime, we present an alternative formulation by directly leveraging latent representation produced by a pretrained AutoEncoder. We call this improved method reduce dimension super-resolved GAN (RD-SRGAN). RD-SRGAN first obtains the latent feature representation of LR image by a pretrained AutoEncoder as input to the generator network. This process not only reduces noise effects but also decreases the overall computational complexity. On the other hand, the residual between the ground truth and the produced images replaces the produced images as input to the discriminator network, and a 2D zero mean Gaussian noise with controllable low variance replaces the real images as another input to the discriminator network. By leveraging the feature representation and properties of the 2D zero mean Gaussian noise, we restrict the optimization space to produce an SR image. Therefore, the residual of the generated SR images tend to approximates to a Gaussian noise, which introduces useless deviation information as little as possible. Experimental results show that RD-SRGAN can benefit from these strategies and achieve improved fidelity and naturalness comparison to existing methods. Switching the pretrained AutoEncoder allows the method to deal with images from diverse categories, e.g., remote sensing satellite imaging, medical imaging, and astronomy.

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

The resolution of remote-sensing images is directly related to the application value of data products. Due to differences in imaging characteristics between digital cameras and remote-sensing cameras, the existing network models cannot get the best super resolution (SR) results of satellite images. In response to the requirements of remote-sensing image production, we propose a single image super-resolution (SISR) reconstruction method for specific type of remote-sensing satellite. First, we measure and model the imaging degradation phenomenon of remote-sensing satellite, including the image blur and noise model. Then, high-quality aerial images are down-sampled and degraded to construct paired training image datasets. We chose the most popular Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) as the basic structure and optimized the number of Residual-in-Residual Dense Block (RRDB) modules to further improve the processing efficiency. Finally, we perform a series of quantitative measurements of the SR image results, including image interpretation capability, reconstruction accuracy, ground resolution distance, and data processing efficiency, using higher resolution remote-sensing images as the benchmark. The experimental results demonstrate that our proposed method has higher interpretation capability and reconstruction accuracy for the SR processing of specific type remote-sensing satellite. Our proposed method is evaluated within a real satellite image product, that demonstrated it has the capability of pipeline super-resolution processing of remote sensing data products.

Abstract. As deep learning models advance, their use in Super Resolution (SR) tasks has become pivotal for enhancing remote sensing low-resolution (LR) satellite images. However, the decision making processes within these models remain opaque, especially in remote sensing applications where transparency is critical. This paper focuses on applying Explainable Artificial Intelligence (XAI) techniques, particularly Local Attribution Maps (LAMs), to analyze and interpret the internal behavior of both general purpose and remote sensing specific SR neural networks. General purpose models like Generative Adversarial Network for Super Resolution (SRGAN), Enhanced Deep Super-Resolution Network (EDSR), Efficient Super-Resolution Transformer (ESRT), and Hybrid-Attention Transformer (HAT), although highly effective in SR tasks, were originally designed for broader image enhancement challenges; in contrast, Hybrid-Scale Self-Similarity Exploitation Network (HSENet) and Multi-scale Enhanced Network (MEN) are tailored for the unique complexities of remote sensing, such as varied textures and intricate scene features. By leveraging LAMs, we highlight how different networks prioritize and process features such as edges, textures, and high frequency details to generate super resolved outputs. The comparative study between general purpose and remote sensing specific networks outlines each model’s strengths and weaknesses in managing the data present in remote sensing imagery. This approach addresses the previously unexplored application of LAMs in remote sensing for SR, contributing to the research in this field and providing deeper insights into the interpretability and transparency of widely used SR models. Furthermore, by drawing parallels between feature attribution in SR and classification tasks, we suggest new pathways for integrating semantic information to refine model transparency and performance in remote sensing applications.

In this paper, we propose a simple but powerful idea to improve super-resolution (SR) methods based on convolutional neural networks (CNNs). We consider a linear manifold, which is the set of all SR images whose downsampling results are the same as the input image, and apply the orthogonal projection onto this linear manifold in the output layers of the CNNs. The proposed method can guarantee the consistency between the SR image and the input image and reduce the mean squared error. The proposed method is especially effective for SR methods based on generative adversarial networks (GANs), composed of one generator and one discriminator, since the generator can learn high-frequency components while maintaining low-frequency ones. Experiments show the effectiveness of the proposed technique for a GAN-based SR method. Finally we introduce an idea of extension to noisy images.

Sensor resolution, data acquisition conditions and multiple compression degradation of images during transmission often lead to quality degradation of remotely sensed images, which affects the fine analysis and identification of features. In order to improve the image resolution, this study adopts a blind super-resolution reconstruction method based on deep learning, proposes a super-resolution framework based on generative adversarial network (GAN), designs ARDB as the generator and U-Net as the discriminator, and explores the optimization strategy of the loss function. By constructing high-resolution and low-resolution paired remote sensing image datasets and implementing step-by-step training on the BasicSR platform, the model performance is finally evaluated by a variety of metrics, and the results show that the present model has a significant advantage over other super-resolution methods in terms of visual effect and quantitative evaluation.

As an important display mode of underwater environments, the sonar image has limitations on the resolution, which often leads to problems with low resolution of underwater objects. Therefore, the image super-resolution algorithm is needed to transform the images from low-resolution to high-resolution. It can improve the visual effect and contribute to subsequent processing such as 3D reconstruction and object recognition. This paper proposes a method for sonar image super-resolution based on generative adversarial networks (GAN). By comparing the super-resolution effects of various interpolation and convolutional neural network algorithms on sonar images, a Residual-in-Residual Dense Block network is employed as the generator of GAN since it has the low distortion and high perceptual quality. Because the sonar image training set does not have enough data, the generator utilizes the transfer learning on the sonar images to produce an optimized network model which is more suitable for super-resolution of sonar image. The VGG19 network is employed as the discriminator. In addition, the perceptual loss is introduced into the loss function of $$\hbox {S}^2\hbox {RGAN}$$ to further improve the perceptual quality of super-resolution images. The experimental results indicate that the proposed $$\hbox {S}^2\hbox {RGAN}$$ shows excellent performance. The generated super-resolution images of $$\hbox {S}^2\hbox {RGAN}$$ have the remarkable advantages of both lower distortion and higher perceptual quality comparing with other methods. Because $$\hbox {S}^2\hbox {RGAN}$$ focuses more on the reality and overall visual effect of super-resolution sonar images, it is suitable for various underwater situations.

Chapter 16 - Remote Sensing in Lineament Identification: Examples from Western India

The thermal camera systems can be used in all kinds of applications that require the detection of heat change, but thermal imaging systems are highly costly systems. In recent years, developments in the field of deep learning have increased the success by obtaining quality results compared to traditional methods. In this paper, thermal images of neonates (healthy - unhealthy) obtained from a high-resolution thermal camera were used and these images were evaluated as high resolution (ground truth) images. Later, these thermal images were downscaled at 1/2, 1/4, 1/8 ratios, and three different datasets consisting of low-resolution images in different sizes were obtained. In this way, super-resolution applications have been carried out on the deep network model developed based on generative adversarial networks (GAN) by using three different datasets. The successful performance of the results was evaluated with PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). In addition, healthy - unhealthy classification application was carried out by means of a classifier network developed based on convolutional neural networks (CNN) to evaluate the super-resolution images obtained using different datasets. The obtained results show the importance of combining medical thermal imaging with super-resolution methods.

Superresolution (SR) has provided an effective solution to the increasing need for high-resolution images in remote sensing applications. Among various SR methods, deep learning-based SR (DLSR) has made a significant breakthrough. However, supervised DLSR methods require a considerable amount of training data, which is hardly available in the remote sensing field. To address this issue, some research works have recently proposed and revealed the capability of deep learning in unsupervised SR. This article presents an efficient unsupervised SR (EUSR) deep learning model using dense skip connections, which boosts the reconstruction performance in parallel with the reduction of computational burden. To do this, several blocks containing densely connected convolutional layers are implemented to increase the depth of the model. Some skip connections also concatenate feature maps of different blocks to enable better SR performance. Moreover, a bottle-neck block abstracts the feature maps in fewer feature maps to remarkably reduce the computational burden. According to our experiments, the proposed EUSR leads to better results than the state-of-the-art DLSR method in terms of reconstruction quality with less computational burden. Furthermore, results indicate that the EUSR is more robust than its rival in dealing with images of different classes and larger sizes.

Generating High-Resolution Climate Prediction through Generative Adversarial Network

Deep learning superresolution (SR) is a promising approach to reduce MRI scan time without requiring custom sequences or iterative reconstruction. Previous deep learning SR approaches have generated low-resolution training images by simple k-space truncation, but this does not properly model in-plane turbo spin echo (TSE) MRI resolution degradation, which has variable T2 relaxation effects in different k-space regions. To fill this gap, we developed a T2 -deblurred deep learning SR method for the SR of 3D-TSE images. A SR generative adversarial network was trained using physically realistic resolution degradation (asymmetric T2 weighting of raw high-resolution k-space data). For comparison, we trained the same network structure on previous degradation models without TSE physics modeling. We tested all models for both retrospective and prospective SR with 3 × 3 acceleration factor (in the two phase-encoding directions) of genetically engineered mouse embryo model TSE-MR images. The proposed method can produce high-quality 3 × 3 SR images for a typical 500-slice volume with 6-7 mouse embryos. Because 3 × 3 SR was performed, the image acquisition time can be reduced from 15 h to 1.7 h. Compared to previous SR methods without TSE modeling, the proposed method achieved the best quantitative imaging metrics for both retrospective and prospective evaluations and achieved the best imaging-quality expert scores for prospective evaluation. The proposed T2 -deblurring method improved accuracy and image quality of deep learning-based SR of TSE MRI. This method has the potential to accelerate TSE image acquisition by a factor of up to 9.

Abstract: High-resolution satellite imagery is vital for applications such as environmental monitoring, urban planning, and defense. However, the availability of such data is limited by sensor constraints and acquisition costs. This paper investigates the use of Generative Adversarial Networks (GANs) to enhance the resolution of satellite images, using low-resolution input and learning to generate high-quality details. We explore and evaluate three GAN variants—SRGAN, ESRGAN, and WDSR-GAN—on two publicly available datasets: SpaceNet and UC Merced. Quantitative results indicate that ESRGAN achieves the highest performance, with a Peak Signal-to-Noise Ratio (PSNR) of 28.7 dB and Structural Similarity Index (SSIM) of 0.91. Regression and visual fidelity analysis confirm its robustness across diverse landscapes, showing its potential for improving satellite image usability. Keywords: satellite imagery, super-resolution, GANs, SRGAN, ESRGAN, WDSR, PSNR, SSIM, SpaceNet, UC Merced, deep learning

Super resolution (SR) techniques play a critical role in enhancing the quality of satellite imagery by reconstructing high resolution (HR) images from low resolution (LR) inputs. This study investigates the application of the Enhanced Super Resolution Generative Adversarial Network (ESRGAN) to improve the spatial resolution of satellite images. The ESRGAN framework integrates a Residual-in-Residual Dense Block (RRDB) generator, a Relativistic average GAN (RaGAN) discriminator, and an improved perceptual loss function to achieve sharper textures and more realistic structures. The 4× Satellite Image Super Resolution dataset, consisting of paired LR (2 m/pixel) and HR (0.5 m/pixel) images, was employed. Quantitative evaluation was conducted using the Peak Signal-to-Noise Ratio (PSNR), with Lanczos5 interpolation used as a baseline for comparison. Experimental results on ten test images demonstrate that ESRGAN consistently outperforms Lanczos5. Qualitative assessments further confirm that ESRGAN effectively restores high frequency details, enhances fine textures, and preserves object boundaries more accurately than the interpolation method. These findings highlight ESRGAN’s potential as an advanced solution for improving visual quality and detail preservation in high resolution satellite imagery applications.

As the largest alpine peat swamp wetland distribution area in the world, the Zoige has important ecological functions, including water conservation and biodiversity maintenance. In the past 20 years, the regional ecological protection and restoration measures continuously strengthened under the leadership of the local government have led to gradual improvements in the ecological environment of the region. In this study, multisource satellite remote-sensing image data were used to carry out quantitative monitoring and assessment of the main ecological elements (vegetation and water), as well as the regional leading ecosystem service function in the Zoige. Combined with local ecological protection management policies and measures, we analyzed the characteristics and effectiveness of ecological protection. We compared the ecosystem change trends of the Zoige reserve and the county, from 2001 to 2020, and found that the fractional vegetation cover (FVC) of Zoige county has increased at a rate of 0.25%/year. The growth rate was highest between 2015 and 2020, and the growth rate of FVC in the Zoige Wetland National Nature Reserve is approximately 1.89-fold that of the whole county. The water area also shows similar variation characteristics. On the whole, the water conservation capacity of the Zoige showed a significant increase from 2001 to 2020. We used high-resolution satellite remote-sensing images to capture the details of land use changes brought about by local ecological protection policies and measures, and together with macroecological indicators, we reflected on the effectiveness of regional ecological protection measures. We observed that the ecological effects of nature reserves are more direct and rapid, and the amount of water conservation within the nature reserve is about 1 × 104 m3/km2 higher than that of the surrounding grasslands. Satellite remote-sensing images can not only capture the multiscale change information of ecological indicators, such as vegetation and water, in a timely manner, but can also help us to identify the effectiveness of conservation measures by distinguishing and analyzing the causes of these changes.