Graph-Based Blind Image Deblurring From a Single Photograph.

Blind image deconvolution, i.e., estimating both the latent image and the blur kernel from the only observed blurry image, is a severely ill-posed inverse problem. In this paper, we propose a blur kernel estimation method for blind motion deblurring using sparse representation and cross-scale self-similarity of image patches as priors to recover the latent sharp image from a single blurry image. Sparse representation indicates that image patches can always be represented well as a sparse linear combination of atoms in an appropriate dictionary. Cross-scale self-similarity results in that any image patch can in some way be well approximated by a number of other similar patches across different image scales. Our method is based on the observations that almost any image patch in a natural image has multiple similar patches in down-sampled versions of the image, and down-sampling produces image patches that are sharper than those in the blurry image itself. In our method, the dictionary for sparse representation is trained adaptively from sharper patches sampled from the down-sampled latent image estimate to make the similar patches of the latent sharp image well represented sparsely, and meanwhile, all patches from the latent image estimate are optimized to be as close to the sharper similar patches searched from the down-sampled version to enforce the sharp recovery of the latent image by constructing a non-local regularization. Experimental results on both simulated and real blurry images demonstrate that our method outperforms state-of-the-art blind deblurring methods.

A deep variational Bayesian framework for blind image deblurring

Blind image deblurring problem, estimating the blur kernel and latent image from a single blurry image, is the a severely ill-posed problem. Since the blur kernel plays an important role in image deblurring problem, we focus on how to effectively apply Elastic-Net prior(ENP) to estimate blur kernel. During the estimation of kernel blur, the ENP is used to maintain the slight edge and removing insignificant texture of intermediate image. Thus, a kernel estimation method is proposed, based on Maximum a Posteriori(MAP) method with ENP. And then, the half-quadratic splitting method is used to solve the non-convex problem of this model. Finally, the latent image can be estimated by using the hyper-Laplacian priors and the estimated blur kernel. Experimental results show that the proposed method can obtain better blur kernel and improve the quality of the latent image estimation, against with the state-of-the art methods.

Blind image deblurring is a long-standing challenge in remote sensing image restoration tasks. It aims to recover a latent sharp image from a blurry image while the blur kernel is unknown. To solve this problem, many image priors-based algorithms and learning-based algorithms have been proposed. However, most of these methods are based on a single blurry image. Due to the lack of high frequency information, the images restored by these algorithms still have some deficiencies in edge and texture details. In this work, we propose a novel deep learning model named Reference-Based Multi-Level Features Fusion Deblurring Network (Ref-MFFDN), which registers the reference image and the blurry image in the multi-level feature space and transfers the high-quality textures from registered reference features to assist image deblurring. Comparative experiments on the testing set prove that our Ref-MFFDN outperforms many state-of-the-art single image deblurring approaches in both quantitative evaluation and visual results, which indicates the effectiveness of using reference images in remote sensing image deblurring tasks. More ablation experiments demonstrates the robustness of Ref-MFFDN to the input image size, the effectiveness of multi-level features fusion network (MFFN) and the effect of different feature levels in multi-feature extractor (MFE) on algorithm performance.

Blind image deblurring is a conundrum because there are infinitely many pairs of latent image and blur kernel. To get a stable and reasonable deblurred image, proper prior knowledge of the latent image and the blur kernel is urgently required. Different from the recent works on the statistical observations of the difference between the blurred image and the clean one, our method is built on the surface-aware strategy arising from the intrinsic geometrical consideration. This approach facilitates the blur kernel estimation due to the preserved sharp edges in the intermediate latent image. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on deblurring the text and natural images. Moreover, our method can achieve attractive results in some challenging cases, such as low-illumination images with large saturated regions and impulse noise. A direct extension of our method to the non-uniform deblurring problem also validates the effectiveness of the surface-aware prior.

Objectives: To develop image deblurring algorithm to effectively recover the original image from the captured blurry and noisy image. Methods: A novel image deblurring algorithm using adaptive priors is proposed. The adaptive priors vary the sparsity induced based on factors such as the type of blur affecting the image, image region, statistical parameters regarding the blur kernel and/or statistical predictions on the blur kernel. The proposed algorithm using adaptive priors improves the quality of the deblurred result, when compared with various recent image deblurring algorithms. Finding: the work contains a case study with regard to certain standard parameters. It is observed that proposed method is better in terms of frequency response, Peak signal-tonoise ratio (PSNR) and Structural Similarity Index (SSIM) values in comparison with other priors. Novelty: the proposed priors lead to the most effective results for image deblurring using the Bayesian framework. The proposed method improves the performance by 30% in PSNR and 45% in SSIM values in dB with uniform kernel size 12 * 12 and improves performance by 30 % in PSNR and 32% SSIM with a standard deviation of 3.5. The proposed method enhances the frequency response of the real-time image restoration process. Applications: Some important applications include the restoration of medical images such as MRI images, CT images where the intensity of these radiations is maintained to avoid damage to human organs. Remote sensing images captured through drones at a specified time cannot be retaken and hence restoring such images is very important. Similarly restoring images from CCTV footage, astronomical images. Keywords: Blind Image Deblurring; Maximum a Posteriori Estimation; Image priors; Regularization; Structural Similarity Index (SSIM) and Peak signal-to-noise ratio (PSNR)

Single-image nonparametric blind super-resolution is a fundamental image restoration problem yet largely ignored in the past decades among the computational photography and computer vision communities. An interesting phenomenon is observed that learning-based single-image super-resolution (SR) has been experiencing a rapid development since the boom of the sparse representation in 2005s and especially the representation learning in 2010s, wherein the high-res image is generally blurred by a supposed bicubic or Gaussian blur kernel. However, the parametric assumption on the form of blur kernels does not hold in most practical applications because in real low-res imaging a high-res image can undergo complex blur processes, e.g., Gaussian-shaped kernels of varying sizes, ellipse-shaped kernels of varying orientations, curvilinear kernels of varying trajectories. The paper is mainly motivated by one of our previous works: Shao and Elad (in: Zhang (ed) ICIG 2015, Part III, Lecture notes in computer science, Springer, Cham, 2015). Specifically, we take one step further in this paper and present a type of adaptive heavy-tailed image priors, which result in a new regularized formulation for nonparametric blind super-resolution. The new image priors can be expressed and understood as a generalized integration of the normalized sparsity measure and relative total variation. Although it seems that the proposed priors are simple, the core merit of the priors is their practical capability for the challenging task of nonparametric blur kernel estimation for both super-resolution and deblurring. Harnessing the priors, a higher-quality intermediate high-res image becomes possible and therefore more accurate blur kernel estimation can be accomplished. A great many experiments are performed on both synthetic and real-world blurred low-res images, demonstrating the comparative or even superior performance of the proposed algorithm convincingly. Meanwhile, the proposed priors are demonstrated quite applicable to blind image deblurring which is a degenerated problem of nonparametric blind SR.

Existing video super-resolution (SR) algorithms usually assume that the blur kernels in the degradation process are known and do not model the blur kernels in the restoration. However, this assumption does not hold for video SR and usually leads to over-smoothed super-resolved images. In this paper, we propose a deep convolutional neural network (CNN) model to solve video SR by a blur kernel modeling approach. The proposed deep CNN model consists of motion blur estimation, motion estimation, and latent image restoration modules. The motion blur estimation module is used to provide reliable blur kernels. With the estimated blur kernel, we develop an image deconvolution method based on the image formation model of video SR to generate intermediate latent images so that some sharp image contents can be restored well. However, the generated intermediate latent images may contain artifacts. To generate high-quality images, we use the motion estimation module to explore the information from adjacent frames, where the motion estimation can constrain the deep CNN model for better image restoration. We show that the proposed algorithm is able to generate clearer images with finer structural details. Extensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.

Learning-based methods have been becoming the mainstream of single image super resolution (SR) technologies. It is effective to generate a high-resolution image from a single low resolution input. However, the quality of training data and the computational demand are two main problems. We propose a novel process framework for single image SR tasks aiming at these two problems, which consists of blur kernel estimation (BKE) and dictionary learning. BKE is utilized for improving the quality of training samples and compatibility between training samples and test samples, which is realized by minimizing the dissimilarity between cross-scale patches iteratively (intuitively be equivalent to maximizing the similarity of cross-scale patches). A selective patch processing (SPP) strategy is adopted in BKE and sparse recovery to reduce the number of patches needed to be processed. The fact that nature images usually contain continuity and discontinuity simultaneously ensures the feasibility of SPP. The experimental results show that our method produces more precise estimation for blurring kernel and better SR effect than several state-of-the-art SR algorithms on equal conditions, but needs much less computation time.

Infrared images of power equipment play an important role in power equipment status monitoring and fault identification. Aiming to resolve the problems of low resolution and insufficient clarity in the application of infrared images, we propose a blind super-resolution algorithm based on the theory of compressed sensing. It includes an improved blur kernel estimation method combined with compressed sensing theory and an improved infrared image super-resolution reconstruction algorithm based on block compressed sensing theory. In the blur kernel estimation method, we propose a blur kernel estimation algorithm under the compressed sensing framework to realize the estimation of the blur kernel from low-resolution images. In the estimation process, we define a new norm to constrain the gradient image in the iterative process by analyzing the significant edge intensity changes before and after the image is blurred. With the norm, the salient edges can be selected and enhanced, the intermediate latent image generated by the iteration can move closer to the clear image, and the accuracy of the blur kernel estimation can be improved. For the super-resolution reconstruction algorithm, we introduce a blur matrix and a regular total variation term into the traditional compressed sensing model and design a two-step total variation sparse iteration (TwTVSI) algorithm. Therefore, while ensuring the computational efficiency, the boundary effect caused by the block processing inside the image is removed. In addition, the design of the TwTVSI algorithm can effectively process the super-resolution model of compressed sensing with a sparse dictionary, thereby breaking through the reconstruction performance limitation of the traditional regularized super-resolution method of compressed sensing due to the lack of sparseness in the signal transform domain. The final experimental results also verify the effectiveness of our blind super-resolution algorithm.

Solving blind image deblurring usually requires defining a data fitting function and image priors. While existing algorithms mainly focus on developing image priors for blur kernel estimation and non-blind deconvolution, only a few methods consider the effect of data fitting functions. In contrast to the state-of-the-art methods that use a single or a fixed data fitting term, we propose a data-driven approach to learn effective data fitting functions from a large set of motion blurred images with the associated ground truth blur kernels. The learned data fitting function facilitates estimating accurate blur kernels for generic scenes and domain-specific problems with corresponding image priors. In addition, we extend the learning approach for data fitting function to latent image restoration and nonuniform deblurring. Extensive experiments on challenging motion blurred images demonstrate the proposed algorithm performs favorably against the state-of-the-art methods.

This work addresses image recovery problem in the presence of salt-and-pepper noise and image blur. The salt-and-pepper noise reviewed as the impulsive noise, in this paper, is modeled as a sparse signal because of its impulsiveness. To accurately reconstruct the clean image and the blur kernel, the framelet domains are exploited to sparsely represent the image and the blur kernel. From the reformulations conducted, a joint estimation is devised to simultaneously perform the image recovery, the salt-and-pepper noise suppression and the blur kernel estimation under a optimization framework. To solve the optimization problem, an efficient solver based on accelerated proximal gradient (APG) is developed to obtain the joint estimation solution. Numerical studies demonstrate the superior performance of the joint estimation algorithm compared with the state-of-the-art approaches in terms of both objective and subjective evaluation standards.

In this paper, we investigate super-resolution image restoration from multiple images, which are possibly degraded with large motion blur. The blur kernel for each input image is separately estimated. This is unlike many existing super-resolution algorithms, which assume identical blur kernel for all input images. We also do not make any restrictions on the motion fields among images; that is, we estimate dense motion field without simplifications such as parametric motion. We present a two-step algorithm: In the first step, each input image is deblurred using the estimated blur kernel. In the second step, super-resolution restoration is applied to the deblurred images. Because the estimated blur kernels may not be accurate, we propose a weighted cost function for the super-resolution restoration step, where a weight associated with an input image reflects the reliability of the corresponding kernel estimate and the deblurred image. We provide experimental results from real video data captured with a hand-held camera, and show that the proposed weighting scheme is robust to motion deblurring errors.

Estimating blur kernels from real world images is a challenging problem as the linear image formation assumption does not hold when significant outliers, such as saturated pixels and non-Gaussian noise, are present. While some existing non-blind deblurring algorithms can deal with outliers to a certain extent, few blind deblurring methods are developed to well estimate the blur kernels from the blurred images with outliers. In this paper, we present an algorithm to address this problem by exploiting reliable edges and removing outliers in the intermediate latent images, thereby estimating blur kernels robustly. We analyze the effects of outliers on kernel estimation and show that most state-of-the-art blind deblurring methods may recover delta kernels when blurred images contain significant outliers. We propose a robust energy function which describes the properties of outliers for the final latent image restoration. Furthermore, we show that the proposed algorithm can be applied to improve existing methods to deblur images with outliers. Extensive experiments on different kinds of challenging blurry images with significant amount of outliers demonstrate the proposed algorithm performs favorably against the state-of-the-art methods.

A self-adaptive learning method for motion blur kernel estimation of the single image