Triply Complementary Priors for Image Restoration.

As a long-standing task, image restoration aims to recover the latent sharp image from its degraded counterpart. In recent years, owing to the strong ability of self-attention in capturing long-range dependencies, Transformer based methods have achieved promising performance on multifarious image restoration tasks. However, the canonical self-attention leads to quadratic complexity with respect to input size, hindering its further applications in image restoration. In this paper, we propose a Strip Attention Network (SANet) for image restoration to integrate information in a more efficient and effective manner. Specifically, a strip attention unit is proposed to harvest the contextual information for each pixel from its adjacent pixels in the same row or column. By employing this operation in different directions, each location can perceive information from an expanded region. Furthermore, we apply various receptive fields in different feature groups to enhance representation learning. Incorporating these designs into a U-shaped backbone, our SANet performs favorably against state-of-the-art algorithms on several image restoration tasks. The code is available at https://github.com/c-yn/SANet.

Digital Image Restoration is a technique used to recover original image information from degraded images. Digital images lose information because of factors like noise, blurring and camera misfocus. Image restoration plays a significant role in improving the quality of pictures obtained from unmanned aircrafts, poor quality video tape of a crime scene and blurred satellite images. In this paper we review how the field of image restoration has evolved over the years, and how the techniques of image restoration are making inroads into various applications, thus improving users perception. The article also portrays recent advances and patents in the field of image restoration. Keywords: Image deblurring, image restoration, classical approaches to image restoration, adaptive approaches, median filter, image inpainting, inverse filtering, patents on image restoration applications

Image restoration (IR) methods based on neural network algorithms have shown great success. However, the hardware circuits that can perform real-time IR task with high-effective analog computation are few in the literature. To address such problem, we propose a memristor-based continuous Hopfield neural network (HNN) circuit for processing the IR task in this work. In our circuit, a single memristor crossbar array is used to represent synaptic weights and perform matrix operations. Current feedback operation amplifiers are utilized to achieve integral operation and output function. Given these designs, the proposed circuit can perform continuous recursive operations in parallel and process different optimization problems with the programmability of the memristor array. On the basis of the proposed circuit, binary and greyscale image restorations are conducted through self-organizing network operations, providing a hardware implementation platform for IR tasks. Comparative simulations show the designed HNN circuit provides effective improvements in terms of speed and accuracy compared with software simulation. Moreover, the hardware circuit shows good robustness to memristive variation and input noise.

In image restoration tasks, like denoising and super resolution, continual modulation of restoration levels is of great importance for real-world applications, but has failed most of existing deep learning based image restoration methods. Learning from discrete and fixed restoration levels, deep models cannot be easily generalized to data of continuous and unseen levels. This topic is rarely touched in literature, due to the difficulty of modulating well-trained models with certain hyper-parameters. We make a step forward by proposing a unified CNN framework that consists of few additional parameters than a single-level model yet could handle arbitrary restoration levels between a start and an end level. The additional module, namely AdaFM layer, performs channel-wise feature modification, and can adapt a model to another restoration level with high accuracy. By simply tweaking an interpolation coefficient, the intermediate model - AdaFM-Net could generate smooth and continuous restoration effects without artifacts. Extensive experiments on three image restoration tasks demonstrate the effectiveness of both model training and modulation testing. Besides, we carefully investigate the properties of AdaFM layers, providing a detailed guidance on the usage of the proposed method.

Background: Recently, sparse representation has been significantly used in various image inverse problems, such as image deblurring, super resolution and compressive sensing, and has shown promising results. The key issue of sparse representation is how to find a reasonable dictionary, by which the image can present more sparsity, as described in various patents. Method: In this paper, we address the image restoration and propose a novel cost function. Considering the significant difference of underlying structure within different patches, we independently train the dictionary using a set of self-similarity patches to present each patch more sparsely. Result: To solve the proposed cost function, an approach based on alternating optimization is presented to obtain the approximate solution. Conclusion: Experimental results demonstrate that the proposed method is superior to many existing excellent algorithms. Keywords: Image restoration, sparse representation, dictionary learning, optimization, image inversion.

Focusing on features with higher importance degree: A significance-wise mechanism for image restoration

Satellites have many high-, medium-, and low-frequency micro vibration sources that lead to the optical axis jitter of the optical load and subsequently degrade the remote sensing image quality. To address this problem, this paper developed an image motion detection and restoration method based on an inertial reference laser, and describe edits principle and key components. To verify the feasibility and performance of this method, this paper also built an image motion measurement and restoration system based on an inertial reference laser, which comprised a camera (including the inertial reference laser unit and a Hartmann wavefront sensor), an integrating sphere, a simulated image target, a parallel light pope, a vibration isolation platform, a vibration generator, and a 6 degrees of freedom platform. The image restoration principle was also described. The background noise in the experiment environment was measured, and an image motion measurement accuracy experiment was performed. Verification experiments of image restoration were also conducted under various working conditions. The experiment results showed that the error of image motion detection based on the inertial reference laser was less than 0.12 pixels (root mean square). By using image motion data to improve image quality, the modulation transfer function (MTF) of the restored image was increased to 1.61–1.88 times that of the original image MTF. The image motion data could be used as feedback to the fast steering mirror to compensate for the satellite jitter in real time and to directly obtain high-quality images.

Due to proper use of the image sparsity, CS (compressed sensing) image compression has made great achievements in the field of image compression without the constraint of Nyquist sampling law. A great deal of researches indicate that there exist obvious statistical and structural priors regularity for image information distribution, while traditional CS image compression algorithms only use the sparse characteristic of the image information. In this paper, we propose a CS image reconstruction algorithm based on the joint statistical and structural priors which can achieve efficient image reconstruction via a small amount of measurements. With the full use of inter-scale and intra-scale relations of the sparse coefficients, we optimize the iterative hard thresholding CS algorithm specifically by building a GSM model to constrain the local coefficients distribution statistically, and a tree model to constrain the global coefficients distribution structurally. Extensive simulations have been conducted and the results show that the proposed method has achieved a considerable promotion both on the speed and the PSNR gain of image reconstruction, compared with the traditional recovery algorithms under the same compression ratio.

A practical generative adversarial network architecture for restoring damaged character photographs

Image degradation characteristics and restoration based on regularization for diffractive imaging

Score-based diffusion methods provide a powerful strategy to solve image restoration tasks by flexibly combining a pre-trained foundational prior model with a likelihood function specified during test time . Such methods are predominantly derived from two stochastic processes: reversing Ornstein–Uhlenbeck, which underpins the celebrated denoising diffusion probabilistic models (DDPM) and denoising diffusion implicit models (DDIM), and the Langevin diffusion process. The solutions delivered by DDPM and DDIM are often remarkably realistic, but they are not always consistent with measurements because of likelihood intractability issues and the associated required approximations. Alternatively, using a Langevin process circumvents the intractable likelihood issue, but usually leads to restoration results of inferior quality. This paper presents a novel and highly computationally efficient image restoration method that carefully embeds a foundational DDPM denoiser within an empirical Bayesian Langevin algorithm, which jointly calibrates key model hyper-parameters as it estimates the model’s posterior mean. Extensive experimental results on three canonical tasks (image deblurring, super-resolution, and inpainting) demonstrate that the proposed approach improves on state-of-the-art strategies both in image estimation accuracy and computing time.

Local to non-local: Multi-scale progressive attention network for image restoration

To demonstrate the feasibility of compressed sensing (CS) to accelerate the acquisition of hyperpolarized (HP) 129 Xe multi-b diffusion MRI for quantitative assessments of lung microstructural morphometry. Six healthy subjects and six chronic obstructive pulmonary disease (COPD) subjects underwent HP 129 Xe multi-b diffusion MRI (b=0, 10, 20, 30, and 40s/cm2 ). First, a fully sampled (FS) acquisition of HP 129 Xe multi-b diffusion MRI was conducted in one healthy subject. The acquired FS dataset was retrospectively undersampled in the phase encoding direction, and an optimal twofold undersampled pattern was then obtained by minimizing mean absolute error (MAE) between retrospective CS (rCS) and FS MR images. Next, the FS and CS acquisitions during separate breath holds were performed on five healthy subjects (including the above one). Additionally, the FS and CS synchronous acquisitions during a single breath hold were performed on the sixth healthy subject and one COPD subject. However, only CS acquisitions were conducted in the rest of the five COPD subjects. Finally, all the acquired FS, rCS and CS MR images were used to obtain morphometric parameters, including acinar duct radius (R), acinar lumen radius (r), alveolar sleeve depth (h), mean linear intercept (Lm ), and surface-to-volume ratio (SVR). The Wilcoxon signed-rank test and the Bland-Altman plot were employed to assess the fidelity of the CS reconstruction. Moreover, the t-test was used to demonstrate the effectiveness of the multi-b diffusion MRI with CS in clinical applications. The retrospective results demonstrated that there was no statistically significant difference between rCS and FS measurements using the Wilcoxon signed-rank test (P>0.05). Good agreement between measurements obtained with the CS and FS acquisitions during separate breath holds was demonstrated in Bland-Altman plots of slice differences. Specifically, the mean biases of the R, r, h, Lm , and SVR between the CS and FS acquisitions were 1.0%, 2.6%, -0.03%, 1.5%, and -5.5%, respectively. Good agreement between measurements with the CS and FS acquisitions was also observed during the single breath-hold experiments. Furthermore, there were significant differences between the morphometric parameters for the healthy and COPD subjects (P<0.05). Our study has shown that HP 129 Xe multi-b diffusion MRI with CS could be beneficial in lung microstructural assessments by acquiring less data while maintaining the consistent results with the FS acquisitions.

Image restoration remains a challenging task in image processing. Numerous methods tackle this problem, often solved by minimizing a non-smooth penalized co-log-likelihood function. Although the solution is easily interpretable with theoretic guarantees, its estimation relies on an optimization process that can take time. Considering the research effort in deep learning for image classification and segmentation, this class of methods offers a serious alternative to perform image restoration but stays challenging to solve inverse problems. In this work, we design a deep network, named DeepPDNet, built from primal-dual proximal iterations associated with the minimization of a standard penalized likelihood with an analysis prior, allowing us to take advantage of both worlds. We reformulate a specific instance of the Condat-Vu primal-dual hybrid gradient (PDHG) algorithm as a deep network with fixed layers. The learned parameters are both the PDHG algorithm step-sizes and the analysis linear operator involved in the penalization (including the regularization parameter). These parameters are allowed to vary from a layer to another one. Two different learning strategies: "Full learning" and "Partial learning" are proposed, the first one is the most efficient numerically while the second one relies on standard constraints ensuring convergence in the standard PDHG iterations. Moreover, global and local sparse analysis prior are studied to seek a better feature representation. We apply the proposed methods to image restoration on the MNIST and BSD68 datasets and to single image super-resolution on the BSD100 and SET14 datasets. Extensive results show that the proposed DeepPDNet demonstrates excellent performance on the MNIST and the more complex BSD68, BSD100, and SET14 datasets for image restoration and single image super-resolution task.

Compressed image restoration via deep deblocker driven unified framework