Initial Image Selection and its Influence on Super-Resolution Reconstruction

Due to the factors like processing power limitations and channel capabilities images are often down sampled and transmitted at low bit rates resulting in a low resolution compressed image. High resolution images can be reconstructed from several blurred, noisy and down sampled low resolution images using a computational process know as super resolution reconstruction. The problem of recovering a high resolution image from a sequence of low resolution compressed images is considered. In this paper, we propose lifting schemes for intentionally introducing down sampling of the high resolution image sequence before compression and then utilize super resolution techniques for generating a high resolution image at the decoder. Lifting wavelet transform has its advantages over the ordinary wavelet transform by way of reduction in memory required for its implementation. This is possible because lifting transform uses in-place computation. The lifting coefficients replace the image samples present in the respective memory locations. In our proposed approach the forward lifting is applied to the high resolution images which are compressed using Set Portioning in Hierarchical Trees (SPHIT), the compressed images are transmitted which results in low resolution images at the encoder, at the decoder super resolution techniques are applied; lifting scheme based fusion is performed, then decoded using DSPIHT and Inverse lifting is applied. In order to remove noise from the reconstructed image soft thresholding is performed and the blur is removed using blind deconvolution, and finally interpolated using our novel interpolation technique. We have performed both objective and subjective analysis of the reconstructed image, and the resultant image has better super resolution factor, and a higher ISNR and PSNR.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

Due to the factors like processing power limitations and channel capabilities images are often down sampled and transmitted at low bit rates resulting in a low resolution compressed image. High resolution images can be reconstructed from several blurred, noisy and down sampled low resolution images using a computational process know as super resolution reconstruction. Super-resolution is the process of combining multiple aliased low-quality images to produce a high resolution, high-quality image. The problem of recovering a high resolution image progressively from a sequence of low resolution compressed images is considered. In this paper we propose a novel DCT based progressive image display algorithm by stressing on the encoding and decoding process. At the encoder we consider a set of low resolution images which are corrupted by additive white Gaussian noise and motion blur. The low resolution images are compressed using 8 by 8 blocks DCT and noise is filtered using our proposed novel zonal filter. Multiframe fusion is performed in order to obtain a single noise free image. At the decoder the image is reconstructed progressively by transmitting the coarser image first followed by the detail image. And finally a super resolution image is reconstructed by applying our proposed novel adaptive interpolation technique. We have performed both objective and subjective analysis of the reconstructed image, and the resultant image has better super resolution factor, and a higher ISNR and PSNR. A comparative study done with Iterative Back Projection (IBP) and Projection on to Convex Sets (POCS),Papoulis Grechberg, FFT based Super resolution Reconstruction shows that our method has out performed the previous contributions.

In this paper we propose a fast approach for edge preserving super-resolution (SR) based on learning of contourlet coefficients. Given a low resolution test image, we first obtain an initial HR estimate i.e., a close approximation to SR image by learning the contourlet coefficients from a training database consisting of low resolution (LR) and high resolution (HR) images. The final SR image is obtained by using a regularization framework in which both the SR and the LR images are modeled as separate homogeneous Markov Random Fields (MRFs). The LR image formation process is modeled as a decimated and noisy version of the SR image and the final cost function is minimized by using a gradient descent method. Novelty of our approach lies in preserving the edges in the final SR image while using a non edge preserving MRF prior. This is definitely advantageous since it avoids the use of discontinuity preserving prior and hence the computationally taxing optimization methods. The edges in the final SR correspond to those learned from the initial HR estimate. The use of MRF on the low resolution image imposes an additional constraint on the final solution and hence we expect a better solution. In addition, we use the initial HR image for estimating the decimation matrix entries as well as for learning the corresponding MRF parameter. We show the effectiveness of the proposed approach by conducting the experiments on images captured using a real camera.

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.

Image super-resolution is of great interest in medical imaging field. However, different from natural images studied in computer vision field, the low-resolution (LR) medical imaging data is often a stack of high-resolution (HR) 2D slices with large slice thickness. Consequently, the goal of super-resolution for medical imaging data is to reconstruct the missing slice(s) between any two consecutive slices. Since some modalities (e.g., T1-weighted MR image) are often acquired with high-resolution (HR) image, it is intuitive to harness the prior self-similarity information in the HR image for guiding the super-resolution of LR image (e.g., T2-weighted MR image). The conventional way is to find the profile of patchwise self-similarity in the HR image and then use it to reconstruct the missing information at the same location of LR image. However, the local morphological patterns could vary significantly across the LR and HR images, due to the use of different imaging protocols. Therefore, such direct (un-supervised) adaption of self-similarity profile from HR image is often not effective in revealing the actual information in the LR image. To this end, we propose to employ the existing image information in the LR image to supervise the estimation of self-similarity profile by requiring it not only being optimal in representing patches in the HR image, but also producing less reconstruction errors for the existing image information in the LR image. Moreover, to make the anatomical structures spatially consistent in the reconstructed image, we simultaneously estimate the self-similarity profiles for a stack of patches across consecutive slices by solving a group sparse patch representation problem. We have evaluated our proposed super-resolution method on both simulated brain MR images and real patient images with multiple sclerosis lesion, achieving promising results with more anatomical details and sharpness.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

Spatial resolution of image is a critical index of image quality, and also is an important parameter in the area of image application. However, there are many factors which can lower and degenerate the quality of images in the process of capturing images. An effective solution to this problem is the super-resolution image reconstruction, that means reconstructing a high resolution (HR) image from some low resolution (LR) images, and eliminating the additive noise and blur which is caused by the size of the detector and optical at the same time. In this paper,one of project of convex set method for reconstructing a super-resolution image from multiframe undersampling images is introduced.First,a very important factor-the model of the process of imaging in spatial domain iterative method for super-resolution image reconstruction is mentioned. Next the reconstruction model and its steps are introduced through introducing the theory of project of convex set method. At last, the technique is available by experiments. The result indicates the project of convex set method for super-resolution image reconstruction has preferable effect on multiframe undersampling remote sensing images, and the technique of super-resolution image reconstruction can used in lots of fields such as remote sensing, computer vision, public security, physic imaging and so on.

A new multi-frame image super resolution (SR) algorithm via Bayesian modeling with natural image prior modeled by fields of experts (FoE) is proposed. Multi-frame SR can be used to obtain a high resolution (HR) image from a set of degraded low resolution (LR) images without changing any hardware device. However, SR is well known to be an ill-posed problem. So state-of-the-art solutions usually formulate the problem with Bayesian modeling techniques, which infer the HR image based on not only the LR input images but also on prior information about the HR image. Current Bayesian SR approaches typically use simple prior models such as L1 norm, TV prior and Laplacian prior, which cannot exploit the statistics of natural scenes well. In this paper, a Bayesian multi-frame image SR approach using a FOE model as the prior for natural images is presented. The Maximum a Posteriori (MAP) framework is used for estimating the HR image. The proposed method cannot only capture the statistics of natural images well, but also require less computational power than the other Bayesian modelling methods such as Sampling methods and Approximate inference. The proposed method shows superior or comparable results to the state-of-art multi-frame SR methods.

Super-resolution (SR) can be defined as a method of reconstructing high resolution(HR) image by increasing spatial resolution of one or more low resolution(LR) image counterparts. SR is widely applied in areas like satellite imaging, drones, surveillance and medical imaging etc. SR techniques can be broadly classified as external database driven or internal database driven approaches. The first approach is time consuming during training phase, in learning LR-HR patch relationship but computes fast during testing phase. The second approach directly constructs the super resolved image from existing LR image without the need of any training procedures but found to be computationally intensive during test time. This paper focuses on reducing the computation time of transformed self exemplar based internal database driven technique [1] that uses 3D scene geometry and internal image characteristics for super resolution by exploiting its highly parallelizable nature. On parallelizing the original algorithm we achieved a speedup of 6 on multi-core, upto 10 on GPU, around 14 on hybrid platform of multi-core and GPU. Using our hybrid implementation we can achieve 32x super resolution factor. We compare our results with SRCNN[2] (Super Resolution Convolution Neural Network) an external database SR technique which has a very good performance among the current start-of-the-art SR methods.

Deep Feature Blend Attention: A New Frontier in Super Resolution Image Generation

In this paper, we demonstrate simple algorithms that project low resolution (LR) images differing in subpixel shifts on a high resolution (HR) also called super resolution (SR) grid. The algorithms are very effective in accuracy as well as time efficiency. A number of spatial interpolation techniques using nearest neighbor, inverse-distance weighted averages, Radial Basis Functions (RBF) etc. are used in projection. For best accuracy of reconstructing SR image by a factor of two requires four LR images differing in four independent subpixel shifts. The algorithm has two steps: i) registration of low resolution images and (ii) shifting the low resolution images to align with reference image and projecting them on high resolution grid based on the shifts of each low resolution image using different interpolation techniques. Experiments are conducted by simulating low resolution images by subpixel shifts and subsampling of original high resolution image and the reconstructing the high resolution images from the simulated low resolution images. The results of accuracy of reconstruction are compared by using mean squared error measure between original high resolution image and reconstructed image. The algorithm was tested on remote sensing images and found to outperform previously proposed techniques such as Iterative Back Projection algorithm (IBP), Maximum Likelihood (ML) algorithms. The algorithms are robust and are not overly sensitive to the registration inaccuracies.

Multi-modal imaging requires image fusion to combine advantages of different types of sensors and requires super-resolution (SR) because of limited spatial resolution of source images. In this study, a novel framework is proposed for unification of image SR and the fusion process to obtain a high-resolution (HR)-fused image from a set of low-resolution (LR) multi-modal images. The jointly trained dictionaries of LR patches and corresponding HR patches are used for sparse representation of LR source image patches and HR-fused image patches, respectively. The sparse coefficients vectors for corresponding patches of source LR images are determined by using orthogonal matching pursuit and a local information content-based metric is employed to fuse these sparse coefficients. The corresponding HR-fused image patch is obtained by combining elements of the HR dictionary as per the fused coefficients of the LR image patches. The experimental results on sets of multi-modal images exhibited that the proposed method outperformed the existing fusion and SR techniques in terms of visual quality and image fusion quality metrics.

Nowadays Super Resolution (SR) is a trending term in many fields of image processing area where resolution describes the degree of details an image holds like, pixel count, sharpness and clarity. In many applications captured images are of low quality known as low resolution (LR) image, these images hold very less details. Therefore there is a need to convert such LR images to High resolution (HR) images which can be obtained by various SR methods like, interpolation methods, frequency domain based methods, reconstruction based methods and learning based methods. HR images obtained from interpolation methods contain reconstruction artifacts like edge blurs, edge halos and ringing effects, whereas learning based methods produce good results but predict unknown HR pixel values based only on input LR/HR image pairs. Thus, to overcome above mentioned disadvantages we proposed modified interpolation method for obtaining HR image, which predicts unknown HR pixel values from interpolated LR patch and corresponding HR patch using multilayer perceptron (MLP) and discrete wavelet transform (DWT). Experimental results subjectively and objectively show that for considered test images corresponding HR images obtained by our proposed method are of better quality than classical bicubic, Chopade et al. and Man et al. method.

Super-resolution (SR) techniques produce a high resolution (HR) image from a set of low-resolution (LR) undersampled images. Usually, SR problems are posed as estimation problems where the LR images are contaminated by stationary noise. However, in real SR problems is very common to have non-stationary noise due to problems in the registration of the images or outliers. SR methods that address this type of problems are called robust. In this paper we propose a novel robust SR method that employs a median filter directly in the data from the LR images, before proceeding to the interpolation and deblurring steps that are common in SR. We compare this new method with other robust SR methods with synthetic and real data, proving that it outperforms the other methods in both cases.KeywordsProjective MethodMedian FilterRegistration ErrorPepper NoiseIrregular SampleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.