Neural‐network‐based regularization methods for inverse problems in imaging

Traditionally, analytical methods have been used to solve imaging problems such as image restoration, inpainting, and superresolution (SR). In recent years, the fields of machine and deep learning have gained a lot of momentum in solving such imaging problems, often surpassing the performance provided by analytical approaches. Unlike analytical methods for which the problem is explicitly defined and domain-knowledge carefully engineered into the solution, deep neural networks (DNNs) do not benefit from such prior knowledge and instead make use of large data sets to learn the unknown solution to the inverse problem. In this article, we review deep-learning techniques for solving such inverse problems in imaging. More specifically, we review the popular neural network architectures used for imaging tasks, offering some insight as to how these deep-learning tools can solve the inverse problem. Furthermore, we address some fundamental questions, such as how deeplearning and analytical methods can be combined to provide better solutions to the inverse problem in addition to providing a discussion on the current limitations and future directions of the use of deep learning for solving inverse problem in imaging.

Ultrasound computed tomography (USCT) receives increasing attention because of its capability to reconstruct quantitative information about the material property distribution as images with superior resolution. However, one roadblock for the wide adoption of relevant techniques is the high demand for computational resources and the long processing time for solving a large inverse problem in imaging. To alleviate the associated challenges, a two-stage inversion scheme is proposed: 1) the ultrasound scanning signals are first processed using a full waveform inversion (FWI) technique with a single iteration to rapidly create a model (image) with embedded wave speed distribution; 2) the corresponding image will be further improved by feeding into a pre-trained deep neural network. The deep learning models presented in this paper are built upon two architectures to instantaneously solve the associated inverse problems and to produce a high-resolution image in real-time. The first is based on 1D convolutional neural network (1D-CNN) layers with an autoencoder structure. The second implements additional layers and skip connections inspired by a U-Net architecture. The resultant superior reconstructions from both CNNs demonstrate that the proposed framework produces a high-resolution image from a rapidly-generated, low-resolution image in real-time, with dramatically improved results.

Effective representation methods and proper signal priors are crucial in most signal processing applications. In this thesis we focus on different structured models and we design appropriate schemes that allow the discovery of low dimensional latent structures that characterise and identify the signals. Motivated by the highly non-linear structure of most datasets, we firstly investigate the geometry of manifolds. Manifolds are low dimensional, non-linear structures that are naturally employed to describe sets of strongly related signals such as the images of a 3-D object captured from different viewpoints. However, the use of manifolds in applications is not straightforward due to their usually non-analytic and non-linear form. We propose here a way to `disassemble' a manifold into simpler components by approximating it with affine subspaces. Our objective is to discover a set of low dimensional affine subspaces that can represent manifold data accurately while preserving the manifold's structure. To this end, we employ a greedy technique that iteratively merges manifold samples into groups based on the difference of local tangents. We use our algorithm to approximate synthetic and real manifolds and to demonstrate that it is competitive to state-of-the-art techniques. Then, we consider structured sparse representations of signals and we propose a new sparsity model, where signals are essentially composed of a small number of structured {\it molecules }. We define the molecules to be linear combinations of a small number of atoms in a redundant dictionary. Our multi-level model takes into account the energy distribution of the significant signal components in addition to their support. It permits to define typical visual patterns and recognise them in prototypical or deformed form. We define a new structural difference measure between molecules and their deformed versions, which is based on their sparse codes and we create an algorithm for decomposing signals into molecules that can account for different deviations in the internal molecule structure. Our experiments verify the benefits of the new image model in various restoration tasks and they confirm that the development of proper models that extend the mere notion of sparsity can be very useful for various inverse problems in imaging. Finally, we investigate the problem of learning molecule representations directly in the sparse code domain. We constrain sparse codes to be linear combinations of a few, possibly deformed, molecules and we design an algorithm that can learn the structure from the codes without transforming them back into the signal domain. To this end, we take advantage of our structural difference which is based on the sparse codes and we devise a scheme for representing the codes with molecules and learn the molecules at the same time. To illustrate the effectiveness of our proposed algorithm we apply it to various synthetic and real datasets and we compare the results with traditional sparse coding and dictionary learning techniques. From the experiments, we verify the superior performance of our scheme in interpreting and recognising correctly the underlying structure. In short, in this thesis we are interested in low-dimensional, structured models. Among the various choices, we focus on manifolds and sparse representations and we propose schemes that enhance their structural properties and highlight their effectiveness in signal representations.

A two-dimensional inverse problem in imaging the thermal conductivity of a non-homogeneous medium

The Newsletter is a key element in further enhancing the value of the journal to the inverse problems community. So why not be a part of this exciting forum by sending to our Bristol office material suitable for inclusion under any of the categories mentioned above. Your contributions will be very welcome. Book review Introduction to Inverse Problems in Imaging M Bertero and P Boccacci 1998 Bristol: Institute of Physics 362 pp ISBN 0-7503-0435-9 (pbk) £25.00, $49.00 This book shows that several problems in imaging are in fact linear inverse problems. In general inverse problems are ill-posed in the sense that small perturbations in the (measured) data have a significant influence on the output. Consequently, numerical methods developed in a general setting of inverse problems which cure the ill-posedness (such methods are called regularization methods) can be applied to solve problems in imaging. This is an excellent textbook on the principle of linear inverse problems, methods of their numerical solution and practical applications in imaging. It introduces basic ideas and methods for the solution of inverse and ill-posed problems while avoiding the mathematics of functional treatment of operator equations. This goal is achieved by a bottom-up presentation: the authors focus on the two problems of image deconvolution and tomography (bottom) and develop regularization theory specifically for these two problems. Most other books which are concerned with the numerical solution of inverse problems present a top-down approach. That is, a general regularization theory (top) is developed which is then applied to particular problems. The distinguished presentation makes the book very useful to students in applied mathematics, practitioners in engineering science and image processing. This book has the best prerequisites to establish a close link between the mathematical fields of imaging and inverse problems. Mainly the book focuses on two topics: image deconvolution and linear imaging systems. 1. In image deconvolution the goal is to restore a space invariant blurred image. This part of the book contains mathematical tools in image deconvolution, examples of space invariant imaging systems, a comparison of image deconvolution techniques and low-pass filtering techniques, the analysis of the ill-posedness of the deconvolution problem, regularization methods for deconvolution such as constrained least squares regularization, Tikhonov regularization, iterative regularization and statistical methods. Since the authors focus on particular problems they are able to analyse various methods for deconvolution on a very concrete level. As an example I would like to mention that the authors analyse Fourier methods for deconvolution. Thus they provide an optimal starting point to the theory of inverse problems for people working in the area of signal and image processing. 2. The second part of the book considers linear imaging systems that are not of a convolution type such as tomography problems, diffraction problems and inverse scattering problems. Efficient numerical methods for the solution of this class of problems such as singular value decomposition, Tikhonov regularization methods and Fourier-based methods are presented. O Scherzer Universität Linz

The concept of region of sensitivity is central to the field of diffuse optics and is closely related to the Jacobian matrix used to solve the inverse problem in imaging. It is well-known that, in diffuse reflectance, the region of sensitivity associated with a given source-detector pair is shaped as a banana, and features maximal sensitivity to the portions of the sample that are closest to the source and the detector. We have recently introduced a dual-slope method based on a special arrangement of two sources and two detectors, which results in deeper and more localized regions of sensitivity, resembling the shapes of different kinds of nuts. Here, we report the regions of sensitivity associated with a variety of source-detector arrangements for dual-slope measurements of intensity and phase with frequency-domain spectroscopy (modulation frequency: 140 MHz) in a medium with absorption and reduced scattering coefficients of 0.1 cm-1 and 12 cm-1, respectively. The main result is that the depth of maximum sensitivity, considering only cases that use source-detector separations of 25 and 35 mm, progressively increases as we consider single-distance intensity (2.0 mm), dual-slope intensity (4.6 mm), single-distance phase (7.5 mm), and dual-slope phase (10.9 mm). These results indicate the importance of dual-slope measurements, and even more so of phase measurements, when it is desirable to selectively probe deeper portions of a sample with diffuse optics. This is certainly the case in non-invasive optical studies of brain, muscle, and breast tissue, which are located underneath superficial tissue at variable depths.

Wavelet decompositions of integral operators have proven their efficiency in reducing computing times for many problems, ranging from the simulation of waves or fluids to the resolution of inverse problems in imaging. Unfortunately, computing the decomposition is itself a hard problem which is oftentimes out of reach for large-scale problems. The objective of this work is to design fast decomposition algorithms based on another representation called product-convolution expansion. This decomposition can be evaluated efficiently, assuming that a few impulse responses of the operator are available, but it is usually less efficient than the wavelet decomposition when incorporated in iterative methods. The proposed decomposition algorithms, run in quasi-linear time and we provide some numerical experiments to assess its performance for an imaging problem involving space-varying blurs.

This book provides researchers and engineers in the imaging field with the skills they need to effectively deal with nonlinear inverse problems associated with different imaging modalities, including impedance imaging, optical tomography, elastography, and electrical source imaging. Focusing on numerically implementable methods, the book bridges the gap between theory and applications, helping readers tackle problems in applied mathematics and engineering. Complete, self-contained coverage includes basic concepts, models, computational methods, numerical simulations, examples, and case studies. Provides a step-by-step progressive treatment of topics for ease of understanding. Discusses the underlying physical phenomena as well as implementation details of image reconstruction algorithms as prerequisites for finding solutions to non linear inverse problems with practical significance and value. Includes end of chapter problems, case studies and examples with solutions throughout the book. Companion website will provide further examples and solutions, experimental data sets, open problems, teaching material such as PowerPoint slides and software including MATLAB m files. Essential reading for Graduate students and researchers in imaging science working across the areas of applied mathematics, biomedical engineering, and electrical engineering and specifically those involved in nonlinear imaging techniques, impedance imaging, optical tomography, elastography, and electrical source imaging

In this work, a method for obtaining pixel-wise error bounds in Bayesian regularization of inverse imaging problems is introduced. The proposed method employs estimates of the posterior variance together with techniques from conformal prediction in order to obtain coverage guarantees for the error bounds, without making any assumption on the underlying data distribution. It is generally applicable to Bayesian regularization approaches, independent, e.g., of the concrete choice of the prior. Furthermore, the coverage guarantees can also be obtained in case only approximate sampling from the posterior is possible. With this in particular, the proposed framework is able to incorporate any learned prior in a black-box manner. Guaranteed coverage without assumptions on the underlying distributions is only achievable since the magnitude of the error bounds is, in general, unknown in advance. Nevertheless, experiments with multiple regularization approaches presented in the paper confirm that in practice, the obtained error bounds are rather tight. For realizing the numerical experiments, also a novel primal-dual Langevin algorithm for sampling from non-smooth distributions is introduced in this work.

An alternative differential framework for the existing integral-based electromagnetic inverse problems is proposed. This differential form, supplemented by the corresponding boundary conditions, overcomes the existing limitations of the integral-based inverse problems, i.e. considering the imaging antennas as point sources, the essential homogeneity along one axis, and the necessity of using a background matching medium. These limitations result in reconstructing two-dimensional images with certain level of error, and the practical problems associated with selecting suitable dielectric properties of the background medium. Thus, the ability of the proposed technique in reconstructing accurate three-dimensional images, including the dimensions and real structure of the imaging antennas and avoiding the essential use of matching medium, are unique advancements in solving inverse electromagnetic problems. To validate this proposed novel method, a biomedical inverse electromagnetic problem in head imaging is successfully solved.

X-ray Computed Tomography (XCT) is a widely used non-destructive imaging technique. With advancements in hardware technology, including X-ray source and detector improvements, there is increasing demand for higher spatial resolution, faster acquisition times, and enhanced imaging quality. Moreover, customers seek better performance by addressing challenges such as metal artifacts and imaging low-attenuation objects. These evolving requirements have intensified the need to solve many ill-conditioned inversion problems inherent in X-ray imaging, driving innovations in algorithm development.This thesis introduces model-based approaches to address two ill-conditioned inverse problems in X-ray imaging. In the first portion of the thesis, we address the inverse problem of estimating the effective spectrum for different X-ray systems using transmission CT measurement of homogeneous metal rods with known material and dimensions. We proposed two approaches to estimate the effective spectrum. First, we propose a dictionary-based spectral estimation (DictSE) algorithm that represents the unknown spectral response using an over-complete dictionary and finds the optimal sparse representation of the spectrum. Second, we propose a model-based spectral calibration (MBSC) algorithm that models the effective spectrum as a function of some physically meaningful parameters and estimates the spectral response by estimating a limited number of parameters with multi-voltage or multi-filtration dataset. Using simulated and measured data, we demonstrate that MBSC outperforms other methods' accuracy and robustness. In the second portion of the thesis, we address the inverse problem of reconstructing images from sparse-view CT scans, a critical task in many applications to achieve faster acquisition times. We propose a direct reconstruction method called Recurrent Stacked Back Projection (RSBP), which leverages a deep recurrent neural network on the Stacked Back Projections (SBP) to improve reconstruction quality. Using simulated and experimental data, we demonstrate that our method produces more high-quality reconstructions from sparse measurements than the reconstructions from filter backprojection (FBP) and model-based iterative reconstruction (MBIR).

Continuous-time image reconstruction using differential equations for computed tomography

Image reconstruction in computed tomography can be treated as an inverse problem, namely, obtaining pixel values of a tomographic image from measured projections. However, a seriously degraded image with artifacts is produced when a certain part of the projections is inaccurate or missing. A novel method for simultaneously obtaining a reconstructed image and an estimated projection by solving an initial-value problem of differential equations is proposed. A system of differential equations is constructed on the basis of optimizing a cost function of unknown variables for an image and a projection. Three systems described by nonlinear differential equations are constructed, and the stability of a set of equilibria corresponding to an optimized solution for each system is proved by using the Lyapunov stability theorem. To validate the theoretical result given by the proposed method, metal artifact reduction was numerically performed.

We consider the use of conjugate-gradient-like iterative methods for the solution of integral equations arising from an inverse problem in acoustics in a bounded three dimensional region. The inverse problem is the computation of the normal velocities on the boundary of a region from pressure measurements on an interior surface. The pressure satisfies the Helmholtz equation in the region. Two formulations are considered: one based on the representation of pressures by a single layer potential and the other based on the Helmholtz-Kirchhoff integral equation. Both formulations can be used to approximate the Neumann Green’s function as an alternative. The integral equations are all ill-posed and are discretized by a boundary element method. The resulting liner systems are ill-conditoned and a (smooth) regularized solutions must be sought. Two regularization rules, including a new one, for conjugate-gradient-like methods are applied and found to have advantages over a standard method based on the truncated singular value decomposition using generalized cross validation. Due to the occurence of multiple singular values for our integral operators, conjugate gradient methods compute the optimal solution in the first few iterations and prove to be particularly fast for these large scale acoustics problems.

Cerenkov luminescence tomography (CLT) is a promising non-invasive optical imaging method with three-dimensional semiquantitative in vivo imaging capability. However, CLT itself relies on Cerenkov radiation, a low-intensity radiation, making CLT reconstruction more challenging than other imaging modalities. In order to solve the ill-posed inverse problem of CLT imaging, some numerical optimization or regularization methods need to be applied. However, in commonly used methods for solving inverse problems, parameter selection significantly influences the results. Therefore, this paper proposed a probabilistic energy distribution density region scaling (P-EDDRS) framework. In this framework, multiple reconstruction iterations are performed, and the Cerenkov source distribution of each reconstruction is treated as random variables. According to the spatial energy distribution density, the new region of interest (ROI) is solved. The size of the region required for the next operation was determined dynamically by combining the intensity characteristics. In addition, each reconstruction source distribution is given a probability weight value, and the prior probability in the subsequent reconstruction is refreshed. Last, all the reconstruction source distributions are weighted with the corresponding probability weights to get the final Cerenkov source distribution. To evaluate the performance of the P-EDDRS framework in CLT, this article performed numerical simulation, in vivo pseudotumor model mouse experiment, and breast cancer mouse experiment. Experimental results show that this reconstruction framework has better positioning accuracy and shape recovery ability and can optimize the reconstruction effect of multiple algorithms on CLT.