Autocorrelation-based interlaced to progressive format conversion

Nowadays Super Resolution (SR) is a trending term in many fields of image processing area where resolution describes the degree of details an image holds like, pixel count, sharpness and clarity. In many applications captured images are of low quality known as low resolution (LR) image, these images hold very less details. Therefore there is a need to convert such LR images to High resolution (HR) images which can be obtained by various SR methods like, interpolation methods, frequency domain based methods, reconstruction based methods and learning based methods. HR images obtained from interpolation methods contain reconstruction artifacts like edge blurs, edge halos and ringing effects, whereas learning based methods produce good results but predict unknown HR pixel values based only on input LR/HR image pairs. Thus, to overcome above mentioned disadvantages we proposed modified interpolation method for obtaining HR image, which predicts unknown HR pixel values from interpolated LR patch and corresponding HR patch using multilayer perceptron (MLP) and discrete wavelet transform (DWT). Experimental results subjectively and objectively show that for considered test images corresponding HR images obtained by our proposed method are of better quality than classical bicubic, Chopade et al. and Man et al. method.

This paper presents an edge-directed image interpolation algorithm. In the proposed algorithm, the edge directions are implicitly estimated with a statistical-based approach. In opposite to explicit edge directions, the local edge directions are indicated by length-16 weighting vectors. Implicitly, the weighting vectors are used to formulate geometric regularity (GR) constraint (smoothness along edges and sharpness across edges) and the GR constraint is imposed on the interpolated image through the Markov random field (MRF) model. Furthermore, under the maximum a posteriori-MRF framework, the desired interpolated image corresponds to the minimal energy state of a 2-D random field given the low-resolution image. Simulated annealing methods are used to search for the minimal energy state from the state space. To lower the computational complexity of MRF, a single-pass implementation is designed, which performs nearly as well as the iterative optimization. Simulation results show that the proposed MRF model-based edge-directed interpolation method produces edges with strong geometric regularity. Compared to traditional methods and other edge-directed interpolation methods, the proposed method improves the subjective quality of the interpolated edges while maintaining a high PSNR level.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

Image super-resolution is of great interest in medical imaging field. However, different from natural images studied in computer vision field, the low-resolution (LR) medical imaging data is often a stack of high-resolution (HR) 2D slices with large slice thickness. Consequently, the goal of super-resolution for medical imaging data is to reconstruct the missing slice(s) between any two consecutive slices. Since some modalities (e.g., T1-weighted MR image) are often acquired with high-resolution (HR) image, it is intuitive to harness the prior self-similarity information in the HR image for guiding the super-resolution of LR image (e.g., T2-weighted MR image). The conventional way is to find the profile of patchwise self-similarity in the HR image and then use it to reconstruct the missing information at the same location of LR image. However, the local morphological patterns could vary significantly across the LR and HR images, due to the use of different imaging protocols. Therefore, such direct (un-supervised) adaption of self-similarity profile from HR image is often not effective in revealing the actual information in the LR image. To this end, we propose to employ the existing image information in the LR image to supervise the estimation of self-similarity profile by requiring it not only being optimal in representing patches in the HR image, but also producing less reconstruction errors for the existing image information in the LR image. Moreover, to make the anatomical structures spatially consistent in the reconstructed image, we simultaneously estimate the self-similarity profiles for a stack of patches across consecutive slices by solving a group sparse patch representation problem. We have evaluated our proposed super-resolution method on both simulated brain MR images and real patient images with multiple sclerosis lesion, achieving promising results with more anatomical details and sharpness.

Many deep learning (DL) frameworks have demonstrated state-of-the-art performance in the super-resolution (SR) task of magnetic resonance imaging, but most performances have been achieved with simulated low-resolution (LR) images rather than LR images from real acquisition. Due to the limited generalizability of the SR network, enhancement is not guaranteed for real LR images because of the unreality of the training LR images. In this study, we proposed a DL-based SR framework with an emphasis on data construction to achieve better performance on real LR MR images. The framework comprised two steps: (a) downsampling training using a generative adversarial network (GAN) to construct more realistic and perfectly matched LR/high-resolution (HR) pairs. The downsampling GAN input was real LR and HR images. The generator translated the HR images to LR images and the discriminator distinguished the patch-level difference between the synthetic and real LR images. (b) SR training was performed using an enhance4d deep super-resolution network (EDSR). In the controlled experiments, three EDSRs were trained using our proposed method, Gaussian blur, and k-space zero-filling. As for the data, liver MR images were obtained from 24 patients using breath-hold serial LR and HR scans (only HR images were used in the conventional methods). The k-space zero-filling group delivered almost zero enhancement on the real LR images and the Gaussian group produced a considerable number of artifacts. The proposed method exhibited significantly better resolution enhancement and fewer artifacts compared with the other two networks. Our method outperformed the Gaussian method by an improvement of 0.111 ± 0.016 in the structural similarity index and 2.76 ± 0.98 dB in the peak signal-to-noise ratio. The blind/reference-less image spatial quality evaluator metric of the conventional Gaussian method and proposed method were 46.6 ± 4.2 and 34.1 ± 2.4, respectively.

Cross-Resolution Person Re-Identification (re-ID) aims to match images with disparate resolutions arising from variations in camera hardware and shooting distances. Most conventional works utilize Super-Resolution (SR) models to recover Low Resolution (LR) images to High Resolution (HR) images. However, because the SR models cannot completely compensate for the missing information in the LR images, there is still a large gap between the HR image recovered from the LR images and the real HR images. To tackle this challenge, we propose a novel Multi-Scale Image- and Feature-Level Alignment (MSIFLA) framework to align the images on multiple resolution scales at both the image and feature level. Specifically, (i) we design a Cascaded Multi-Scale Resolution Reconstruction (CMSR2) module, which is composed of three cascaded Image Reconstruction (IR) networks, and can continuously reconstruct multiple variables of different resolution scales from low to high for each image, regardless of image resolution. The reconstructed images with specific resolution scales are of similar distribution; therefore, the images are aligned on multiple resolution scales at the image level. (ii) We propose a Multi-Resolution Representation Learning (MR2L) module which consists of three-person re-ID networks to encourage the IR models to preserve the ID-discriminative information during training separately. Each re-ID network focuses on mining discriminative information from a specific scale without the disturbance from various resolutions. By matching the extracted features on three resolution scales, the images with different resolutions are also aligned at the feature-level. We conduct extensive experiments on multiple public cross-resolution person re-ID datasets to demonstrate the superiority of the proposed method. In addition, the generalization of MSIFLA in handling cross-resolution retrieval tasks is verified on the UAV vehicle dataset.

Multimodal magnetic resonance images (e.g., T1-weighted image (TIWI) and T2-weighted image (T2WI)) are used for accurate medical imaging analysis. Different modal images have different resolution depending on pulse sequence parameters under limited data acquisition time. Therefore, interpolation methods are used to match the low-resolution (LR) image with the high-resolution (HR) image. However, the interpolation causes blurring that affects analysis accuracy. Although some recent works such as non-local-means (NLM) filter have manifested impressive super-resolution (SR) performance with available HR modal images, the filter has high computational cost. Therefore, we propose a fast SR framework with iterative-guided back projection, which incorporates iterative back projection with a guided filter (GF) method for resolution enhancement of LR images (e.g., T2WI) by referring HR images in another modality image (e.g., T1WI). The proposed method not only achieves both high accuracy than conventional interpolation methods and original GF and computational efficiency by applying an integral 3D image technique. In addition, although the proposed method is slightly inferior in accuracy visually than the state-of-the-art NLM filter, it can run 22 times faster than the state-of-theart method in expanding three times in the slice-select direction from 180 × 216 × 60 voxels to 180 × 216 × 180 voxels. The computational time of our method is about 1 min only. Therefore, the proposed method will be applied to various applications in practice, including not only multimodal MR images but also multimodal image analysis such as computed tomography (CT) and positron emission tomography (PET).

Image super-resolution (SR) can be seen as finding a nonlinear mapping between low-resolution (LR) image and high-resolution (HR) image. Convolutional neural networks (CNN) based image SR methods have achieved great success based on features extracted from pretrained networks. However, the previous works less consider multiscale features. In this paper, we propose a SR cascaded multi-column convolutional neural network (SRCMCN) to recover HR image by learning multi-scale features from the corresponding LR image. Compared to existing CNN based image SR methods, the proposed method directly takes the original LR image as input to learn the residual between interpolated LR image and corresponding HR image, which will reduce the computational cost of convolution due to the small input size. Besides, in order to achieve high-quality reconstruction, we optimize our networks using mean absolute error (MAE) loss function. Experimental results demonstrate that our proposed method outperforms several state-of-the-art image SR methods in terms of both reconstructed image quality and computational efficiency.

Single image super-resolution based on compressive sensing and improved TV minimization sparse recovery

This paper presents a technique that used to enhance the image resolution. The input low resolution (LR) image is decomposed into low frequency (LF) and high frequency (HF) sub-bands by using lifting wavelet transform (LWT) and discrete wavelet transform (DWT). The resolution enhancement is done by interpolating HF sub-bands that are generated using LWT from the LR image. The HF sub-bands of DWT are used to preserve the edges of the images. The operation is done on HF sub-bands of LWT and DWT. The HF sub-bands that are generated from the DWT are interpolated and perform addition to the respective interpolated HF sub-bands of LWT. The HF sub-bands that are modified and the LR image will again interpolate and undergoes the inverse lifting wavelet transform (ILWT) to reconstruct the high resolution (HR) image. This implemented method results efficient over conventional methods.

In this paper, we demonstrate simple algorithms that project low resolution (LR) images differing in subpixel shifts on a high resolution (HR) also called super resolution (SR) grid. The algorithms are very effective in accuracy as well as time efficiency. A number of spatial interpolation techniques using nearest neighbor, inverse-distance weighted averages, Radial Basis Functions (RBF) etc. are used in projection. For best accuracy of reconstructing SR image by a factor of two requires four LR images differing in four independent subpixel shifts. The algorithm has two steps: i) registration of low resolution images and (ii) shifting the low resolution images to align with reference image and projecting them on high resolution grid based on the shifts of each low resolution image using different interpolation techniques. Experiments are conducted by simulating low resolution images by subpixel shifts and subsampling of original high resolution image and the reconstructing the high resolution images from the simulated low resolution images. The results of accuracy of reconstruction are compared by using mean squared error measure between original high resolution image and reconstructed image. The algorithm was tested on remote sensing images and found to outperform previously proposed techniques such as Iterative Back Projection algorithm (IBP), Maximum Likelihood (ML) algorithms. The algorithms are robust and are not overly sensitive to the registration inaccuracies.

We propose a new high-quality up-scaling technique that extends the existing example based super-resolution (SR) framework. Our approach is based on the fundamental idea that a low-resolution (LR) image could be generated from any of the multiple possible high-resolution (HR) images. Therefore it would be more natural to use multiple predictors of HR patch from LR patch instead of single one. In this work we build a generic framework to estimate an HR image from LR one using an adaptive prior (select the predictor locally) based on the local statistics of LR images. We use natural image patch prior as the HR image statistics. We partition the natural images into documents and group them to discover the inherent topics using probabilistic Latent Semantic Analysis (pLSA) and also learn the dual dictionaries of HR and LR image patch pairs for each of the topics using sparse dictionary learning technique. Then for test image we infer locally which topic it corresponds to and then we use the corresponding learned dual dictionary to generate HR image. Experimental results show the effectiveness of our method over existing state-of-art methods.KeywordsSparse RepresentationSparse CodeSuper ResolutionProbabilistic Latent Semantic AnalysisBicubic InterpolationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

High accuracy image registration is critical for the success of multi-frame super-resolution. Conventionally, the shift between images are estimated directly based on the under-sampled low-resolution (LR) image data. However, the high-frequency of LR data is unreliable due to the aliasing effect of sub-sampling, which will deteriorate the accuracy of registration. This paper proposes to resolve the aliasing by converting the LR images to high-resolution (HR) domain and then perform registration on the restored HR spectrum. To recover the HR spectrum for each corresponding LR image, we fuse the LR images into one HR image and project the estimation difference back to the reconstructed HR spectrum iteratively. To address the unequal reliabilities of different restored frequencies, weighted least square is employed to improve the precision of registration. Experimental results show that the proposed method can outperform other existing methods and improve the quality of super-resolution image.

Acquiring degraded images with paired high-resolution (HR) images is often challenging, impeding the advance of image super-resolution in real-world applications. By generating realistic low-resolution (LR) images with degradation similar to that in real-world scenarios, simulated paired LR-HR data can be constructed for supervised training. However, most of the existing work ignores the degradation uncertainty of the generated realistic LR images, since only one LR image has been generated given an HR image. To address this weakness, we propose learning the degradation uncertainty of generated LR images and sampling multiple LR images from the learned LR image (mean) and degradation uncertainty (variance) and construct LR-HR pairs to train the super-resolution (SR) networks. Specifically, uncertainty can be learned by minimizing the proposed loss based on Kullback-Leibler (KL) divergence. Furthermore, the uncertainty in the feature domain is exploited by a novel perceptual loss; and we propose to calculate the adversarial loss from the gradient information in the SR stage for stable training performance and better visual quality. Experimental results on popular real-world datasets show that our proposed method has performed better than other unsupervised approaches.

Super-resolution guided knowledge distillation for low-resolution image classification