Iterative Multi-Frame Super-Resolution Image Reconstruction via Variance-Based Fidelity to the Data

Coping with the limited spatial resolution of images, which is caused by the physical limitations of image sensors, is the objective of this thesis. Initially, an effort to model the scanner function when generating a document copy by means of simple models is made. In a task of scanner function simulation the proposed model should be preferred over the Gaussian and Cauchy models met in bibliography as it is equivalent in performance, simpler in implementation and does not present any dependence on certain scanner characteristics. Afterwards, new methods for improving images spatial resolution are formulated. A nonuniform interpolation method for Super-Resolution (SR) image reconstruction is proposed. Experimentation proves that the proposed method employing Kriging interpolation predominates over the method which creates the high-resolution grid by means of the weighted nearest neighbor interpolation that is a conventional interpolation technique. Also, three new methods for stochastic regularized SR image reconstruction are presented. The Tukey error norm in combination with the Bilateral Total Variation (BTV) regularization, the Lorentzian error norm in combination with the BTV regularization and the Huber error norm combined with the BTV regularization are the three proposed methods. An additional novelty is the direct comparison of the three estimators Tukey, Lorentzian and Huber in the task of super-resolution image reconstruction, thus in rejecting outliers. The performance of the proposed methods proves superior to that of a regularized SR technique met in bibliography. Experimental results verify the robust statistics theory. Moreover, a novel study which considers the effect of each one of the data-fidelity and regularization terms on the SR image reconstruction result is carried out. The conclusions reached help to select an effective SR image reconstruction method, among several potential ones, for a given low-resolution sequence of frames. Finally, an image interpolation method employing a neural network is proposed. The presented training procedure results in the network learning the scanner point spread function. Experimental results prove that the proposed technique predominates over the classical algorithms of bicubic and spline interpolation. The proposed method is novel as it treats, for the first time, the issue of the training data presentation order to the neural network input.

A new multi-frame image super resolution (SR) algorithm via Bayesian modeling with natural image prior modeled by fields of experts (FoE) is proposed. Multi-frame SR can be used to obtain a high resolution (HR) image from a set of degraded low resolution (LR) images without changing any hardware device. However, SR is well known to be an ill-posed problem. So state-of-the-art solutions usually formulate the problem with Bayesian modeling techniques, which infer the HR image based on not only the LR input images but also on prior information about the HR image. Current Bayesian SR approaches typically use simple prior models such as L1 norm, TV prior and Laplacian prior, which cannot exploit the statistics of natural scenes well. In this paper, a Bayesian multi-frame image SR approach using a FOE model as the prior for natural images is presented. The Maximum a Posteriori (MAP) framework is used for estimating the HR image. The proposed method cannot only capture the statistics of natural images well, but also require less computational power than the other Bayesian modelling methods such as Sampling methods and Approximate inference. The proposed method shows superior or comparable results to the state-of-art multi-frame SR methods.

In this paper a context based super-resolution (SR) image reconstruction method is proposed. The proposed maximum a-posteriori (MAP) based estimator identifies local gradients and textures for selecting the optimal SR method for the region of interest. Texture segmentation and gradient map estimation are done prior to the reconstruction stage. Gradient direction is used for optimal noise reduction along the edges for non-textured regions. On the other hand, regularization term is cancelled for textured regions so that the resultant method reduces to maximum likelihood (ML) solution. It is demonstrated on Brodatz Texture Database that ML solution gives the best PSNR values on textures compared to the regularized SR methods in the literature. Experimental results show that the proposed hybrid method has superior performance in terms of Peak Signal-to-Noise-Ratio (PSNR), Structural Similarity Index Measure (SSIM) compared the SR methods in the literature.

In this paper, trade-off between effect of registration error and number of images used in the process of super resolution image reconstruction is studied. Super Resolution image reconstruction is three phase process, of which registration is of at most importance. Super resolution image reconstruction uses set of low resolution images to reconstruct high resolution image during registration. The study demonstrates the effects of registration error and benefit of more number of low resolution images on the quality of reconstructed image. Study reveals that the registration error degrades the reconstructed image and without better registration methodology, a better super resolution method is still not of any use. It is noticed that without further improvement in the registration technique, not much improvement can be achieved by increasing number of input low resolution images.

Deep Feature Blend Attention: A New Frontier in Super Resolution Image Generation

In this paper, a novel method of blind Super-Resolution (SR) image restoration is presented. First, a learning based blur identification method is proposed to identify the blur parameter in which Sobel operator and Vector Quantization (VQ) are used for extracting feature vectors. Then a super-resolution image is reconstructed by a new hybrid MAP/POCS method where the data fidelity term is minimized by l 1 norm and regularization term is defined on the high frequency sub-bands offered by Stationary Wavelet Transform (SWT) to incorporate the smoothness of the discontinuity field. Simulation results demonstrate the effectiveness and robustness of our method.

Super-resolution (SR) reconstruction produces one or a series of high-resolution images from a series of low-resolution images. In this paper, we apply the regularization-based SR image reconstruction method on the basis of multi-frame image SR. Fisrstly, a linear observation model is utilized to associate the recorded LR images with the unknown reconstructed HR image estimates, and we apply the bilateral total variation operator as a regularization term. Moreover, the basic principal of this algorithm is presented, and we thoroughly analyze the selection of the cost-function and the regularization term by comparing of experiments. According to some connective experiments, the algorithm is proved to be effective and robust, and it can better preserve the details of the image.

In the Bayesian image super resolution (SR), a regularisation term is minimised along with a data‐fidelity term to generate a high‐resolution (HR) image from input low‐resolution (LR) image. The regularisation term is incorporated into the SR to fulfil a prior knowledge over the HR image. For instance, smoothness in the background and foreground regions is a prior knowledge about document images. The bilateral total variation (BTV), as a known regularisation term, uniformly smooths the image in all directions while preserving the edges. In this study, the authors present a document image SR method by introducing a new regularisation term called the stroke width‐based directional total variation (SWDTV). It is a modified version of the BTV, and adaptively performs smoothing based on the local width and local direction of the character strokes. By analysing the input LR image and its interpolation, the local stroke width and local stroke direction in the output image are approximated. This information is encapsulated into the SWDTV regularisation term. By minimising the linear combination of the regularisation and data‐fidelity terms, the HR image is reconstructed. The authors’ method achieves significant improvements in visual image quality and optical character recognition accuracy compared to related non‐example‐based document image SR methods.

The reconstruction of a super-resolution (SR) image from multiple low-resolution (LR) frames has drawn a lot of research attention over the past years. The SR reconstruction is an inherent ill-posed minimization problem. Though regularization constraint may be accomplished to derive a single solution, to implement a traditional approach which the data fusing is processed at high-resolution (HR) grids may still be a great challenge due to complexity as well as high computing load. In this paper we introduce a new approach which both the blur and noise removal would be simultaneously carried out at the LR grids by taking pixelwise time average upon the available LR frames after registration. This approach is characterized by simplicity and real-time applicability, and the simulation results demonstrate that the compatible performance can be achieved.

In this paper a high-resolution (MR) image is reconstructed from a sequence of subpixel shifted, aliased low-resolution (LR) frames by means of a novel nonuniform interpolation super-resolution (SR) method. A gradient-based algorithm estimates the horizontal and vertical shifts for each frame. Then, the uniformly spaced sampling points of the UK image are produced by means of Kriging interpolation. Wiener filtering is employed to deal with the restoration problem. The novelty of the proposed nonuniform interpolation approach to SR image reconstruction lies in the employment of Kriging interpolation technique. Comparisons with the original image demonstrate the superiority of our method to a conventional nonuniform interpolation one of SR image reconstruction.

Super-resolution of cardiac magnetic resonance images using Laplacian Pyramid based on Generative Adversarial Networks

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

Color images have a wider range of applications than gray images. There are two ways to extend the traditional super-resolution reconstruction method to color images: Super resolution reconstructs each channel of the color image individually; Change the RGB color bands into YCrCb color bands, then super-resolution reconstructs the luminance component and interpolates the chrominance components.These algorithms cannot effectively utilize the property that the edges and textures are similar in the RGB channels, and the results of those methods may lead to color artifacts. Aiming to solve these problems, we propose a new super-resolution method based on cross channel prior. First, a cross channel prior is proposed to describe the similarity of gradient in RGB channels. Then, a new super-resolution method is proposed for color images via combination of the cross channel prior and the traditional super-resolution methods. Finally, the proposed method reconstructs the color channels alternately. The experimental results show that the proposed method could effectively suppress the generation of color artifacts and improve the quality of the reconstructed images.

Super Resolution (SR) is an interesting topic in video and image research. Super resolution image reconstruction (SRR) is one of the commonest SR technologies and has been proposed in many forms. SRR reconstructs a highresolution image from multiple low-resolution images. However, the low-resolution images require an aliasing process prior to reconstruction. The present paper analyzes SRR in the frequency domain, and demonstrates that SRR is unsuitable for general applications.

Feedback attention network for cardiac magnetic resonance imaging super-resolution