Enhanced Zero-Shot Learning Algorithm for Super-Resolution Reconstruction of Remote Sensing Images

Super-resolution (SR) reconstruction is one of the effective ways to improve the spatial resolution of remote sensing images, but the blur kernel estimation without any additional prior information is the key to the quality of SR reconstruction. Facing the above problem, a blind super-resolution method via sparse representation and transformed self-similarity is proposed in this paper. In this method, the blur kernel as well as the reconstructed high-resolution image are estimated at the same time using the structural self-similarity from a single image, a down-sampled version of the observed image is used as the training samples for the dictionary learning, and the internal patch search space is expanded using the transformed self-similarity to improve the quality of estimation. Experiment results show that, our method performs well both in quality of kernel estimation and SR reconstruction.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

The topic of super-resolution image reconstruction has recently received considerable attention within the research area. Super-resolution image reconstruction methods attempt to create a single high-resolution image from a number of low-resolution images (or a video sequence). The Bayesian maximum a posteriori (MAP) estimation method is widely used for super-resolution image reconstruction. So far, existing super-resolution reconstruction methods are all confronted with the problem of slow convergence and expensive computation. To satisfy the requirement of real-time application, we propose a high-efficiency super-resolution reconstruction algorithm that solves two key bottlenecks in the multi-frame MAP framework. The first breakthrough is to select the Armijo rule to identify the step length instead of the exact line search. The second one is to approximately compute the gradient of the MAP objective function using analytic representation instead of numerical calculation. Experimental results show that the proposed algorithm is applicable to real-time super-resolution reconstruction of low-resolution image sequences.

In this paper, we propose a new super resolution (SR) reconstruction method to handle license plate numbers of vehicles in real traffic videos. Recently, SR reconstruction shemes based on regularization have been demonstrated to be effective because SR reconstrction is an ill-posed problem. Working within this promising framework, the residual data (RD) term can be weighted according to the differences among the observed LR images in the SR reconstruction model. Moreover, L1 norm is used to measure the RD term in order to improve the robustness of our method. Experiments show the proposed method improves the subjective visual quality of the high resolution images.

Image explicit prior has made breakthrough progress in the super-resolution (SR) due to the additional supervisory information provided. However, existing explicit prior-guided SR methods directly use the Gaussian gradient or Laplacian gradient prior, which cannot fit the gradient distribution of remote sensing images. Through the statistics of gradient probability density distribution of the remote sensing image dataset, we found that the hyper-Laplacian prior can fit the heavy-tailed distribution better, which aroused us to use the hyper-Laplacian before facilitating the SR reconstruction. We propose a novel hyper-Laplacian prior SR method for remote sensing images in this manuscript. Specifically, our model consists of three components: rough reconstruction subnetwork (RRS), hyper-Laplacian prior subnetwork (HPS), and image refinement enhancement subnetwork (RES). In the RRS, we reconstruct low-resolution (LR) images into rough SR images by a set of resblocks. In the HPS, we first introduce the hyper-Laplacian prior for LR images to provide an additional texture. Hereafter, we set up a prior loss which imposes a second-order supervision on the SR image. Like the previous image space loss function, it helps the model to gather the geometric structure of the image. Finally, the outputs of the RRS and HPS are fused and then fed to the RES for high-quality image reconstruction. Numerous studies of SR reconstruction and segmentation on UCMerced, PatternNet, and OpenBayes datasets confirm that our method is superior compared to state-of-the-art methods.

Infrared images hold significant application value in fields such as military reconnaissance, security surveillance, and medical diagnosis. However, issues like low resolution, high noise, and complex degradation characteristics severely hinder their practical application effectiveness. This paper introduces an infrared super-resolution reconstruction algorithm based on a random degradation model and Generative Adversarial Networks (GANs), addressing the diversity of infrared image degradation processes. The primary contribution lies in explicitly modeling key degradation parameters of infrared images (such as blur kernel and noise distribution) using a random degradation model, generating diverse low-resolution to high-resolution image pairs, and significantly enhancing the model’s generalization ability for complex degradations. Experiments on the Airo infrared dataset and a self-built infrared dataset demonstrate that when the resolution is increased by a factor of 2, 4, and 8, the 4× resolution reconstructed images exhibit notable advantages in terms of texture detail and noise suppression. Especially in 4× super-resolution reconstruction, compared to three typical deep learning algorithms, our algorithm improves the Peak Signal-to-Noise Ratio (PSNR) by 3.046 dB, 1.8489 dB, and 0.2108 dB, respectively, and the Structural Similarity Index (SSIM) by 0.0387 (4.76%), 0.0287 (3.48%), and 0.0131 (1.56%), respectively, with perceptual similarity decreasing by 0.2465, 0.13344, and 0.0514 (lower values indicate better perceptual quality). Subjective visual assessments further validate the algorithm’s significant advantages in noise reduction and weak texture restoration. This study proposes an infrared image super-resolution reconstruction method based on random degradation modeling, which holds significant theoretical and practical value in complex infrared degradation scenarios.

Color images have a wider range of applications than gray images. There are two ways to extend the traditional super-resolution reconstruction method to color images: Super resolution reconstructs each channel of the color image individually; Change the RGB color bands into YCrCb color bands, then super-resolution reconstructs the luminance component and interpolates the chrominance components.These algorithms cannot effectively utilize the property that the edges and textures are similar in the RGB channels, and the results of those methods may lead to color artifacts. Aiming to solve these problems, we propose a new super-resolution method based on cross channel prior. First, a cross channel prior is proposed to describe the similarity of gradient in RGB channels. Then, a new super-resolution method is proposed for color images via combination of the cross channel prior and the traditional super-resolution methods. Finally, the proposed method reconstructs the color channels alternately. The experimental results show that the proposed method could effectively suppress the generation of color artifacts and improve the quality of the reconstructed images.

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

Raman spectroscopy visualization is a challenging task due to the interference of complex background noise and the number of selected measurement points. In this paper, a super-resolution image reconstruction algorithm for Raman spectroscopy is studied to convert raw Raman data into pseudo-color super-resolution imaging. Firstly, the Raman spectrum data of a single measurement point is measured multiple times to calculate the mean value to remove the random background noise, and innovatively introduce the Retinex algorithm and the median filtering algorithm which improve the signal-to-noise ratio. The novel method of using deep neural network performs a super-resolution reconstruction operation on the gray image. An adaptive guided filter that automatically adjusts the filter radius and penalty factor is proposed to highlight the contour of the cell, and the super-resolution reconstruction of the pseudo-color image of the Raman spectrum is realized. The average signal-to-noise ratio of the reconstructed pseudo-color image sub-band reaches 14.29 db, and the average value of information entropy reaches 4.30 db. The results show that the Raman-based cell pseudo-color image super-resolution reconstruction algorithm is an effective tool to effectively remove noise and high-resolution visualization. The contrast experiments show that the pseudo-color image Kullback–Leiber (KL) entropy of the color image obtained by the method is small, the boundary is obvious, and the noise is small, which provide technical support for the development of sophisticated single-cell imaging Raman spectroscopy instruments.

The high-resolution image is the prerequisite of information acquisition and precise analysis. Multi-frame super-resolution images reconstruction technologies are able to address many image degraded issues (caused by external shooting environment), such as detail information lost, blurred edges, and so forth. According to the nanoscale memristor, a Multi-channel Memristive Pulse Coupled Neural Network (MMPCNN) model is proposed. This model is able to simulate the adaptive-variable linking coefficient in pulse coupled neural network. Meanwhile, the proposed network is applied to the multi-frame super resolution reconstruction for fusing the registered low resolution images. Furthermore, the sparse coding based super resolution method is performed to improve the original high-resolution image. Finally, a series of computer experiments and the relevant subjective/objective analysis jointly illustrate the validity and effectiveness of the entire scheme.

Weeds are one of the primary factors that reduce crop productivity by competing for nutrients and water, causing the plant to lose weight and resulting in reduced grain yield. Traditional agricultural practices often rely on uniform herbicide application, which can contaminate soil and raise costs. On agricultural land, selective weed treatment are an efficient and cost-effective way to control weeds that require a deep learning-based crop and weed segmentation system. Many existing crop and weed segmentation research works focus on achieving precise crop and weed segmentation results, rather than building lightweight models to deploy on edge devices. To attain this, we develop an effective and efficient convolutional neural network, namely the Concatenated Attention U-Net with Convolutional Block Attention Module (CAUC). By integrating Linear Concatenated Blocks (LCB), Attention Gate (AG) connections, and Convolutional Block Attention Module (CBAM), the proposed model efficiently utilizes feature maps among its architectural components to achieve superior performance. Depth-wise convolution layers and 1 × 1 convolution layers in LCBs reduce computational complexity. To enable the proposed model to identify the weed portions in multiple crop fields, we integrated three datasets in this research work, namely the Crop/Weed Field Image Dataset (CWFID), Sugar Beet, and Sunflower datasets. Experimental results on carrot, sugar beet, and sunflower crop datasets demonstrate high Accuracy (99.09%), MIoU (81.02%), and F1-score (99.06%), with a modest model size (5.6 MB) and computational parameters (0.377 million). We developed a lightweight computer vision application (13.7 MB) to demonstrate the model’s efficacy on low-computational devices.

Acquiring the growth information of tobacco plants quickly and accurately is an important prerequisite for their daily management. In view of the problem that the convolutional neural network is easily affected by background information, this study introduces the Convolutional Block Attention Module into the U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net. This method is mainly composed of an encoder and a decoder. Before the feature map is decoded, it will go through the Convolutional Block Attention Module to filter more important features and inhibit the invalid features. In the decoder, in order to fuse the features of objects in different scales, the feature maps in the decoder are up-sampled and then concatenated. This information aggregation solves the problem of difficult object detection at different scales to some extent. The results of this method are compared with four current mainstream deep learning models(U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net, DeeplabV3+, PSP-Net, U-Net). The result shows: 1) The U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net network based on the Convolutional Block Attention Module has the highest extraction accuracy for the UAV visible light remote senseing of tobacco(accuracy was 98.53%, recall was 96.00%,F1-Score was 96.38%, kappa coefficient was 0.95, IOU was 93.02%); 2)In the extraction of the three test areas, the extraction value of this method is highly consistent with the label area value(the area ratios of a total of 13 plants in the three regions were kept between 90% and 100%),the effect is much higher than that of DeeplabV3+ and PSP-Net, and slightly higher than the extraction effect of U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net and U-Net. 3)In the segmentation of three typical canopy closure regions of A, B, and C, most of the accuracy factors in different regions can reach 94% and above. It can be seen that the method in this study has certain advantages in feature extraction and has certain value in practical application.

Super-resolution (SR) reconstruction, particularly on face images, can be widely used in forensic analysis and video surveillance. In this paper, we investigate the statistical characteristics of face images and incorporate them into SR reconstruction in terms of deterministic sets. Based on the set theoretic formulation, the projection onto convex sets (POCS) algorithm is applied to find the solution to face image reconstruction. Compared with the traditional POCS based SR methods, the proposed approach imposes additional constraints to the solution. The experimental results on frontal face images show that the proposed approach gains a better performance both on noise suppression and reconstruction quality.

Timely acquiring the earthquake-induced damage of buildings is crucial for emergency assessment and post-disaster rescue. Optical remote sensing is a typical method for obtaining seismic data due to its wide coverage and fast response speed. Convolutional neural networks (CNNs) are widely applied for remote sensing image recognition. However, insufficient extraction and expression ability of global correlations between local image patches limit the performance of dense building segmentation. This paper proposes an improved Swin Transformer to segment dense urban buildings from remote sensing images with complex backgrounds. The original Swin Transformer is used as a backbone of the encoder, and a convolutional block attention module is employed in the linear embedding and patch merging stages to focus on significant features. Hierarchical feature maps are then fused to strengthen the feature extraction process and fed into the UPerNet (as the decoder) to obtain the final segmentation map. Collapsed and non-collapsed buildings are labeled from remote sensing images of the Yushu and Beichuan earthquakes. Data augmentations of horizontal and vertical flipping, brightness adjustment, uniform fogging, and non-uniform fogging are performed to simulate actual situations. The effectiveness and superiority of the proposed method over the original Swin Transformer and several mature CNN-based segmentation models are validated by ablation experiments and comparative studies. The results show that the mean intersection-over-union of the improved Swin Transformer reaches 88.53%, achieving an improvement of 1.3% compared to the original model. The stability, robustness, and generalization ability of dense building recognition under complex weather disturbances are also validated.

Superresolution reconstruction (SRR) is an effective approach for improving spatial resolution of image, which need not change the original imaging system hardware. Among the SRR methods, jointly estimate image registration and superresolution can overcome the serve aliased between low resolution images. However, further studies are still necessary for choose appropriate prior mode for better preserving edge of image. Also it is important for the choice of adaptive regularization parameters in the SRR based on regularization/maximum a posteriori (MAP) method. In this paper, we propose a method for joint image registration and SRR method based on MAP framework. In the reconstruction, the initial high resolution image is firstly obtained by the shift-and-add method. Then, the edge-preserved MRF model is used to improve the reconstructed image quality. Moreover, adaptive regularization method is used in the SRR. Experimental results demonstrate our improved SRR method effective.