Directionally adaptive single frame image super resolution

We explore in this paper an efficient algorithmic solution to single image super-resolution (SR). We propose the gCLSR, namely graph-Constrained Least Squares Regression, to super-resolve a high-resolution (HR) image from a single low-resolution (LR) observation. The basic idea of gCLSR is to learn a projection matrix mapping the LR image patch to the HR image patch space while preserving the intrinsic geometric structure of the original HR image patch manifold. Even if gCLSR resembles other manifold learning-based SR methods in preserving the local geometric structure of HR and LR image patch manifolds, the innovation of gCLSR lies in that it preserves the intrinsic geometric structure of the original HR image patch manifold rather than the LR image patch manifold, which may be contaminated by image degeneration (e.g., blurring, down-sampling and noise). Upon acquiring the projection matrix, the target HR image can be simply super-resolved from a single LR image without the need of HR-LR training pairs, which favors resource-limited applications. Experiments on images from the public database show that gCLSR method can achieve competitive quality as state-of-the-art methods, while gCLSR is much more efficient in computation than some state-of-the-art methods.

Abstract. Single image super resolution (SISR) technology has been attracted much attention from remote sensing community due to its proven potentials in remote sensing applications. Existing SISR techniques varying from conventional interpolation methods to different network architectures. Generative adversarial networks (GANs) are one of the latest network architectures proven a greater potential as a SISR method whereas least attention has been given by the remote sensing community. Several studies have already been carried out on this context. However, yet there is no generalized GAN based approach to super resolve remote sensing imageries. Therefore, this study investigated the potentials of enhanced super resolution generative adversarial (ESRGAN) model to super resolve very high to medium resolution images from high to coarse resolution images for remote sensing applications. Two models were trained and Worldview-3 (WV3) images used as for very high resolution images. Whereas, down sampled WV3 and Sentinel-2(S2) were used as low resolution counterparts. Model performances were qualitatively and quantitatively analysed using standard metrics such as PSNR, SSIM, UIQI, CC, SAM, SID. Evaluation results emphasised super resolved images were preserved the original quality of the satellite images to a greater extent while improving its ground resolution.

Special Section on Enabling Technologies for Network-Based Applications

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Iterative back-projection (IBP) technique is commonly used to minimize the super resolution (SR) reconstruction error and acquire a high spatial resolution image with appealing visual effect. However, the IBP based SR method suffers from the chessboard and ringing effect around edges. In this paper, we employ total variation (TV) as a part of penalty function to minimize the reconstruction error, which helps to avoid across-edge smoothing and suppress the above-mentioned artifacts simultaneously. The experimental results validate that the proposed algorithm can promote the SR reconstruction precision considerably.

Single image super resolution is a very important computer vision task, with a wide range of applications. In recent years, the depth of the super-resolution model has been constantly increasing, but with a small increase in performance, it has brought a huge amount of computation and memory consumption. In this work, in order to make the super resolution models more effective, we proposed a novel single image super resolution method via recursive squeeze and excitation networks (SESR). By introducing the squeeze and excitation module, our SESR can model the interdependencies and relationships between channels and that makes our model more efficiency. In addition, the recursive structure and progressive reconstruction method in our model minimized the layers and parameters and enabled SESR to simultaneously train multi-scale super resolution in a single model. After evaluating on four benchmark test sets, our model is proved to be above the state-of-the-art methods in terms of speed and accuracy.

Traditional works have shown that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Make full use of these multi-scale information can improve the image restoration performance. However, the current proposed deep learning based restoration methods do not take the multi-scale information into account. In this paper, we propose a dilated convolution based inception module to learn multi-scale information and design a deep network for single image super-resolution. Different dilated convolution learns different scale feature, then the inception module concatenates all these features to fuse multi-scale information. In order to increase the reception field of our network to catch more contextual information, we cascade multiple inception modules to constitute a deep network to conduct single image super-resolution. With the novel dilated convolution based inception module, the proposed end-to-end single image super-resolution network can take advantage of multi-scale information to improve image super-resolution performance. Experimental results show that our proposed method outperforms many state-of-the-art single image super-resolution methods.

The external example-driven single image super-resolution (SISR) method that uses a deep convolutional neural network (CNN) has exhibited superior performance as compared to previously developed SISR methods. However, the advantages of jointly using external and internal examples on a deep CNN framework have not been sufficiently investigated. In this letter, we present a novel method for single image super-resolution by exploiting a complementary relation between external and internal example-based SISR methods. The proposed deep CNN model consists of two subnetworks, a global residual network and a self-residual network, to utilize the advantages of both external and internal examples. In contrast with conventional joint SISR methods, the proposed method is the first deep CNN-based SISR method that does not require a retraining process, which tends to be inefficient. The proposed method outperformed existing methods in both quantitative and qualitative evaluations.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

Lightweight single image super-resolution with attentive residual refinement network

Recently deep convolutional neural network (CNN) has made significant achievement in Single Image Super-Resolution (SISR). Most CNN-based SISR methods used the default L2 norm of the error. However, for Hyperspectral Image (HSI), this loss function may bring spectral inconsistencies. The main reason is that most methods did not pay much attention to spectral loss. To HSI, the loss function should capture not only spatial information but also spectral consistency. In this paper, a Multi-Losses Function Network (MLFN) simultaneously considering spatial and spectral information is proposed, and is composed of two parts: one is Concatenate Dense Residual Network (CDRN), and the other is Loss Network (LN). CDRN is an image reconstruction network which can utilize the hierarchical features extracted from the low-resolution image. LN includes pixel-wise spatial loss and spectral loss which drive the learning of the entire reconstruction model. The experimental results prove that the proposed MLFN can enhance spatial resolution with the consistency of the spectrum of HSI preserved.

Self-supervised cycle-consistent learning for scale-arbitrary real-world single image super-resolution

We propose an intuitive formulation for single image super resolution (SISR), and an algorithm based on it that outperforms the state of the art. We model the SISR problem around an aspect of natural images that is often overlooked - sharp edges - which are important for perceptual quality, and troublesome for interpolation. We justify the use of low resolution (LR) Markovian neighborhoods to estimate high resolution (HR) pixels corresponding to only the central pixel of the LR neighborhood based on our formulation. The formulation also lends itself to learning LR to HR mapping based on their pairs as training examples. We propose a learning algorithm based on polynomial neural networks to learn this mapping. Our formulation and algorithm provide further insight into performance of various single image super resolution methods.

Single image super-resolution based on compressive sensing and improved TV minimization sparse recovery