SRDiff: Single image super-resolution with diffusion probabilistic models

The single image super resolution recovers missing high resolution details so as to reconstruct a high resolution image from a single low resolution image. This paper proposes a novel directionally adaptive, learning-based, single image super resolution method using multiple direction wavelet transform, called directionlets. Here, critically sampled directionlets are used to capture directional features effectively and to extract edge information along different directions from a set of available high resolution images. This information is used as the training set for super resolving a low resolution input image. The directionlet coefficients at finer scales of its high resolution image are learned locally from this training set and the inverse directionlet transform recovers the super resolved high resolution image. The simulation results showed that the proposed directionlet approach outperforms standard interpolation techniques like cubic spline interpolation as well as standard wavelet-based learning, both visually and in terms of the mean squared error (MSE) values. The SNR scores for cubic spline interpolation, wavelet and directionlet method are 13.6998 dB, 23.8324 dB and 30.8654 dB respectively for Barbara.

Single image super resolution based on sparse domain selection

Example learning-based image super-resolution (SR) is recognized as an effective way to produce a high-resolution (HR) image with the help of an external training set. The effectiveness of learning-based SR methods, however, depends highly upon the consistency between the supporting training set and low-resolution (LR) images to be handled. To reduce the adverse effect brought by incompatible high-frequency details in the training set, we propose a single image SR approach by learning multiscale self-similarities from an LR image itself. The proposed SR approach is based upon an observation that small patches in natural images tend to redundantly repeat themselves many times both within the same scale and across different scales. To synthesize the missing details, we establish the HR-LR patch pairs using the initial LR input and its down-sampled version to capture the similarities across different scales and utilize the neighbor embedding algorithm to estimate the relationship between the LR and HR image pairs. To fully exploit the similarities across various scales inside the input LR image, we accumulate the previous resultant images as training examples for the subsequent reconstruction processes and adopt a gradual magnification scheme to upscale the LR input to the desired size step by step. In addition, to preserve sharper edges and suppress aliasing artifacts, we further apply the nonlocal means method to learn the similarity within the same scale and formulate a nonlocal prior regularization term to well pose SR estimation under a reconstruction-based SR framework. Experimental results demonstrate that the proposed method can produce compelling SR recovery both quantitatively and perceptually in comparison with other state-of-the-art baselines.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

In this study, a novel single image super-resolution (SR) method, which uses a generated dictionary from pairs of high-resolution (HR) images and their corresponding low-resolution (LR) representations, is proposed. First, HR and LR dictionaries are created by dividing HR and LR images into patches Afterwards, when performing SR, the distance between every patch of the input LR image and those of available LR patches in the LR dictionary are calculated. The minimum distance between the input LR patch and those in the LR dictionary is taken, and its counterpart from the HR dictionary will be passed through an illumination enhancement process resulting in consistency of illumination between neighbour patches. This process is applied to all patches of the LR image. Finally, in order to remove the blocking effect caused by merging the patches, an average of the obtained HR image and the interpolated image is calculated. Furthermore, it is shown that the stabe of dictionaries is reducible to a great degree. The speed of the system is improved by 62.5%. The quantitative and qualitative analyses of the experimental results show the superiority of the proposed technique over the conventional and state-of-the-art methods.

Recent deep learning approaches in single image super-resolution (SISR) can generate high-definition textures for super-resolved (SR) images. However, they tend to hallucinate fake textures and even produce artifacts. An alternative to SISR, reference-based SR (RefSR) approaches use high-resolution (HR) reference (Ref) images to provide HR details that are missing in the low-resolution (LR) input image. We propose a novel framework that leverages existing SISR approaches and enhances them with RefSR. Specifically, we refine the output of SISR methods using neural texture transfer, where HR features are queried from the Ref images. The query is conducted by computing the similarity of textural and semantic features between the input image and the Ref images. The most similar HR features, patch-wise, to the LR image is used to augment the SR image through an augmentation network. In the case of dissimilar Ref images from the LR input image, we prevent performance degradation by including the similarity scores in the input features of the network. Furthermore, we use random texture patches during the training to condition our augmentation network to not always trust the queried texture features. Different from past RefSR approaches, our method can use arbitrary Ref images and its lower-bound performance is based on the SR image. We showcase that our method drastically improves the performance of the base SISR approach.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

We present a self-learning approach for single image super-resolution (SR), with the ability to preserve high frequency components such as edges in resulting high resolution (HR) images. Given a low-resolution (LR) input image, we construct its image pyramid and produce a super pixel dataset. By extracting context information from the super-pixels, we propose to deploy context-specific contour let transform on them in order to model the relationship (via support vector regression) between the input patches and their associated directional high-frequency responses. These learned models are applied to predict the SR output with satisfactory quality. Unlike prior learning-based SR methods, our approach advances a self-learning technique and does not require the self similarity of image patches within or across image scales. More importantly, we do not need to collect training LR/HR image data in advance and only require a single LR input image. Empirical results verify the effectiveness of our approach, which quantitatively and qualitatively outperforms existing interpolation or learning-based SR methods.

Super-resolution (SR) has become more vital, because of its capability to generate high-quality ultra-high definition (UHD) high-resolution (HR) images from low-resolution (LR) input images. Conventional SR methods entail high computational complexity, which makes them difficult to be implemented for up-scaling of full-high-definition input images into UHD-resolution images. Nevertheless, our previous super-interpolation (SI) method showed a good compromise between Peak-Signal-to-Noise Ratio (PSNR) performances and computational complexity. However, since SI only utilizes simple linear mappings, it may fail to precisely reconstruct HR patches with complex texture. In this paper, we present a novel SR method, which inherits the large-to-small patch conversion scheme from SI but uses global regression based on local linear mappings (GLM). Thus, our new SR method is called GLM-SI. In GLM-SI, each LR input patch is divided into 25 overlapped subpatches. Next, based on the local properties of these subpatches, 25 different local linear mappings are applied to the current LR input patch to generate 25 HR patch candidates, which are then regressed into one final HR patch using a global regressor. The local linear mappings are learned cluster-wise in our off-line training phase. The main contribution of this paper is as follows: Previously, linear-mapping-based conventional SR methods, including SI only used one simple yet coarse linear mapping to each patch to reconstruct its HR version. On the contrary, for each LR input patch, our GLM-SI is the first to apply a combination of multiple local linear mappings, where each local linear mapping is found according to local properties of the current LR patch. Therefore, it can better approximate nonlinear LR-to-HR mappings for HR patches with complex texture. Experiment results show that the proposed GLM-SI method outperforms most of the state-of-the-art methods, and shows comparable PSNR performance with much lower computational complexity when compared with a super-resolution method based on convolutional neural nets (SRCNN15). Compared with the previous SI method that is limited with a scale factor of 2, GLM-SI shows superior performance with average 0.79 dB higher in PSNR, and can be used for scale factors of 3 or higher.

Single Image Super-Resolution (SISR) stands at the forefront of image processing, offering a transformative solution to elevate low-resolution images to a higher fidelity.Leveraging deep neural networks, particularly Generative Adversarial Networks (GANs), this study aims to enrich the quality of images, especially in domains like medical imaging, public surveillance, and historical image restoration. Traditional methods often fail to preserve crucial details during up scaling, prompting the adoption of deep learning techniques which have demonstrated remarkable breakthroughs in accuracy and efficiency. By harnessing the adversarial training process inherent in GANs, we seek to propel the boundaries of image enhancement, striving to faithfully reconstruct high-resolution renditions while retaining essential nuances from their low-resolution counterparts. Through comprehensive performance evaluations utilizing publicly available datasets, we conduct rigorous quantitative and qualitative analyses to gauge the efficacy of various GAN architectures in SISR. This meticulous examination allows us to assess not only the fidelity of the reconstructed images but also the computational efficiency and scalability of each approach, essential considerations for real-world applications. However, alongside the promises of GAN-based SISR come inherent challenges that warrant attention. Issues such as training instability, mode collapse, and imageartifacts present hurdles that must be addressed to fully harness the potential of GANs in image enhancement.Additionally, the integration of domain-specific expertise, particularly in fields like medical imaging, can further enhance the utility and efficacy of GAN-based super-resolution techniques.In conclusion, while the journey towards achieving high-quality, high-resolution images may be fraught with challenges, the integration of GANs into SISR represents a significant leap forward, promising to revolutionize various industries and deepen our understanding of visual data.

Self-supervised cycle-consistent learning for scale-arbitrary real-world single image super-resolution

Streaming low bitrate while preserving high-quality video content is a crucial topic in multimedia and video surveillance. In this work, we explore the problem of spatially and temporally reconstructing high-resolution (HR) frames from a high frame-rate low-resolution (LR) sequence and a few temporally subsampled HR frames. The targeted problem is essentially different from the problems handled by typical super-resolution (SR) methods such as single-image SR and video SR, which attempt to reconstruct HR images using only LR images. To tackle the targeted problem, we propose a deep quality transfer network, based on the convolutional neural network (CNN), which consists of modules including generation and selection of HR pixel candidates, fusion with LR input, residual learning and bidirectional architecture. The proposed CNN model has real-time performance at inference stage. The empirical studies have verified the generality of the proposed CNN model showing significant quality gains for video enhancement.

Single image super-resolution (SR) aims at generating a high-resolution (HR) image from one low-resolution (LR) input. In this paper, we focus on single image SR by using a confidence growing model based on an example-based super resolution approach. Compared to previous works that reconstruct high-resolution image in a raster scan order, the new proposed method reconstructs the patches using a new confidence measure. More confident reconstructions are propagated to neighboring areas by enforcing a smoothness constraint in selecting patches. We also adopt hierarchical clustering to construct a training set to speed up processing. Experimental results demonstrate that this simple method outperforms existing state-of-the-art algorithms on a the given benchmark SR test images.

The aim of signal image super resolution is to generate a high-resolution image from a given low-resolution (LR) input. Learning regression for low-high resolution patch pairs show promise in image super resolution. Gaussian process regression (GPR) is a typically probability model, has strong compatibility, but it is time consuming for building large numbers of samples modeling. In this paper, we extend a GPR based method and propose a nearest neighbor local Gaussian process regression (NNLGPR) that learns the function relationship from the low-resolution patch to the corresponding high frequency patch efficiently. At first, we search the nearest neighbor patches in the sample set for each input low resolution patch. Secondly, we establish local Gaussian process regression model from the searched nearest neighbor low-resolution patches to their corresponding high frequency patches for each low resolution patch. And then, the initial high resolution patch is restored by combining the up-scaled low-resolution patch with the high frequency patch. Finally, the non-local similarity and IBP are utilized to improve the quality of reconstruction image. Experimental results show that our approach acquires better results in the peak signal to noise ratio and the visual effects against several state of the arts methods.

Positron Emission Tomography (PET) is a nuclear, in-vivo medical imaging technology which can make 3D images of tissue metabolic act, in which high dose of tracer is needed to obtain a high quality PET image, which affects patients' health. Due to the limitations of using high dose tracer and limitations of physical imaging systems, it is not easy to get an image in desired resolution. Simplest approach to generate a high resolution image is by post processing. Single Image Super Resolution (SISR) is a post processing procedure to retrieve a high resolution image from a low resolution input. We propose a convolutional neural network trained on PET images which can estimate a high resolution PET image from its input low resolution image.