An Iterative Back-Projection Technique for Single Image Super Resolution with Natural Texture Preservation

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

In this paper, we propose a new super resolution technique based on iterative interpolation followed by registering them using back projection (BP). Firstly the low resolution image is interpolated and then decimated to four low resolution images. The four low resolution images are interpolated and registered by using BP in order to generate a sharper high resolution image then high resolution image is down sampled and back to the first step. The proposed method has been tested on some bench mark images. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) results as well as the visual results shows the superiority of the proposed technique over the conventional and state-of-art image super resolution techniques. In Average, the PSNR is 2.72 dB higher than the bicubic interpolation.

Single image super-resolution using a deep encoder–decoder symmetrical network with iterative back projection

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.

Deep Feature Blend Attention: A New Frontier in Super Resolution Image Generation

In recent years, the study of a single image super-resolution (SISR) is crucial to improving image resolution and using hardware technology to improve image resolution. SISR is widely used in satellite remote sensing, video surveillance, and medical image processing because it mainly relies on deep learning algorithms to realize the conversion from low-resolution (LR) images to high-resolution images. It has the advantages of low cost, simple operation, and high efficiency. This paper proposes an image super-resolution method based on a generative adversarial network named text localization generative adversarial nets (TLGAN) model. The method is improved based on super-resolution generative adversarial networks (SRGAN), and the batch normalization layer is removed, which significantly reduces the computational burden of the model. In TLGAN model, we used the transfer learning method to pre-trained the model on the large dataset ImageNet, and then apply the pre-trained model to the cartoon image data set animes to achieve image super-resolution. Experimental results report that the proposed method has the advantages of fast running speed and excellent visual perception of super-resolution images compared with bicubic interpolation and SRGAN method.

This paper presents a technique that used to enhance the image resolution. The input low resolution (LR) image is decomposed into low frequency (LF) and high frequency (HF) sub-bands by using lifting wavelet transform (LWT) and discrete wavelet transform (DWT). The resolution enhancement is done by interpolating HF sub-bands that are generated using LWT from the LR image. The HF sub-bands of DWT are used to preserve the edges of the images. The operation is done on HF sub-bands of LWT and DWT. The HF sub-bands that are generated from the DWT are interpolated and perform addition to the respective interpolated HF sub-bands of LWT. The HF sub-bands that are modified and the LR image will again interpolate and undergoes the inverse lifting wavelet transform (ILWT) to reconstruct the high resolution (HR) image. This implemented method results efficient over conventional methods.

Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">supervised</i> manner has resulted in state-of-the-art results. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR’s distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network’s generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UDSR</i> ) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics.

Single image super-resolution via internal gradient similarity

This paper proposes an unsupervised single-image Super-Resolution(SR) model using cycleGAN and domain discriminator to solve the problem of SR with unknown degradation using unpaired dataset. In previous approaches, paired dataset is required for training with assumed levels of image degradation. In real world SR applications, however, training sets are typically not of low and high resolution image pairs, but only low resolution images with unknown degradation are provided as inputs. To address the problem, we introduce a cycle-in-cycle GAN based unsupervised learning model using an unpaired dataset. In addition, we combine several losses attributed to image contents, such as pixel-wise loss, VGG feature loss and SSIM loss, for stable learning and performance improvement. We also propose a domain discriminator, which consists of noise discriminator, texture discriminator and color discriminator, to guide generated images to follow target domain distribution rather than source domain. We validate effectiveness of our model in quantitative and qualitative experiments using NTIRE2020 real-world SR challenge dataset.

Example learning-based image super-resolution (SR) is recognized as an effective way to produce a high-resolution (HR) image with the help of an external training set. The effectiveness of learning-based SR methods, however, depends highly upon the consistency between the supporting training set and low-resolution (LR) images to be handled. To reduce the adverse effect brought by incompatible high-frequency details in the training set, we propose a single image SR approach by learning multiscale self-similarities from an LR image itself. The proposed SR approach is based upon an observation that small patches in natural images tend to redundantly repeat themselves many times both within the same scale and across different scales. To synthesize the missing details, we establish the HR-LR patch pairs using the initial LR input and its down-sampled version to capture the similarities across different scales and utilize the neighbor embedding algorithm to estimate the relationship between the LR and HR image pairs. To fully exploit the similarities across various scales inside the input LR image, we accumulate the previous resultant images as training examples for the subsequent reconstruction processes and adopt a gradual magnification scheme to upscale the LR input to the desired size step by step. In addition, to preserve sharper edges and suppress aliasing artifacts, we further apply the nonlocal means method to learn the similarity within the same scale and formulate a nonlocal prior regularization term to well pose SR estimation under a reconstruction-based SR framework. Experimental results demonstrate that the proposed method can produce compelling SR recovery both quantitatively and perceptually in comparison with other state-of-the-art baselines.

Due to the factors like processing power limitations and channel capabilities images are often down sampled and transmitted at low bit rates resulting in a low resolution compressed image. High resolution images can be reconstructed from several blurred, noisy and down sampled low resolution images using a computational process know as super resolution reconstruction. Super-resolution is the process of combining multiple aliased low-quality images to produce a high resolution, high-quality image. The problem of recovering a high resolution image progressively from a sequence of low resolution compressed images is considered. In this paper we propose a novel DCT based progressive image display algorithm by stressing on the encoding and decoding process. At the encoder we consider a set of low resolution images which are corrupted by additive white Gaussian noise and motion blur. The low resolution images are compressed using 8 by 8 blocks DCT and noise is filtered using our proposed novel zonal filter. Multiframe fusion is performed in order to obtain a single noise free image. At the decoder the image is reconstructed progressively by transmitting the coarser image first followed by the detail image. And finally a super resolution image is reconstructed by applying our proposed novel adaptive interpolation technique. We have performed both objective and subjective analysis of the reconstructed image, and the resultant image has better super resolution factor, and a higher ISNR and PSNR. A comparative study done with Iterative Back Projection (IBP) and Projection on to Convex Sets (POCS),Papoulis Grechberg, FFT based Super resolution Reconstruction shows that our method has out performed the previous contributions.

Recently deep convolutional neural network (CNN) has made significant achievement in Single Image Super-Resolution (SISR). Most CNN-based SISR methods used the default L2 norm of the error. However, for Hyperspectral Image (HSI), this loss function may bring spectral inconsistencies. The main reason is that most methods did not pay much attention to spectral loss. To HSI, the loss function should capture not only spatial information but also spectral consistency. In this paper, a Multi-Losses Function Network (MLFN) simultaneously considering spatial and spectral information is proposed, and is composed of two parts: one is Concatenate Dense Residual Network (CDRN), and the other is Loss Network (LN). CDRN is an image reconstruction network which can utilize the hierarchical features extracted from the low-resolution image. LN includes pixel-wise spatial loss and spectral loss which drive the learning of the entire reconstruction model. The experimental results prove that the proposed MLFN can enhance spatial resolution with the consistency of the spectrum of HSI preserved.

The single image super resolution recovers missing high resolution details so as to reconstruct a high resolution image from a single low resolution image. This paper proposes a novel directionally adaptive, learning-based, single image super resolution method using multiple direction wavelet transform, called directionlets. Here, critically sampled directionlets are used to capture directional features effectively and to extract edge information along different directions from a set of available high resolution images. This information is used as the training set for super resolving a low resolution input image. The directionlet coefficients at finer scales of its high resolution image are learned locally from this training set and the inverse directionlet transform recovers the super resolved high resolution image. The simulation results showed that the proposed directionlet approach outperforms standard interpolation techniques like cubic spline interpolation as well as standard wavelet-based learning, both visually and in terms of the mean squared error (MSE) values. The SNR scores for cubic spline interpolation, wavelet and directionlet method are 13.6998 dB, 23.8324 dB and 30.8654 dB respectively for Barbara.

Recent deep learning approaches in single image super-resolution (SISR) can generate high-definition textures for super-resolved (SR) images. However, they tend to hallucinate fake textures and even produce artifacts. An alternative to SISR, reference-based SR (RefSR) approaches use high-resolution (HR) reference (Ref) images to provide HR details that are missing in the low-resolution (LR) input image. We propose a novel framework that leverages existing SISR approaches and enhances them with RefSR. Specifically, we refine the output of SISR methods using neural texture transfer, where HR features are queried from the Ref images. The query is conducted by computing the similarity of textural and semantic features between the input image and the Ref images. The most similar HR features, patch-wise, to the LR image is used to augment the SR image through an augmentation network. In the case of dissimilar Ref images from the LR input image, we prevent performance degradation by including the similarity scores in the input features of the network. Furthermore, we use random texture patches during the training to condition our augmentation network to not always trust the queried texture features. Different from past RefSR approaches, our method can use arbitrary Ref images and its lower-bound performance is based on the SR image. We showcase that our method drastically improves the performance of the base SISR approach.