On a heuristic stopping rule for the regularization of inverse problems by the augmented Lagrangian method

<p style='text-indent:20px;'>Total generalization variation (TGV) is a very powerful and important regularization for various inverse problems and computer vision tasks. In this paper, we propose a semismooth Newton based augmented Lagrangian method for solving this problem. The augmented Lagrangian method (also called as method of multipliers) is widely used for lots of smooth or nonsmooth variational problems. However, its efficiency heavily depends on solving the corresponding coupled and nonlinear system together and simultaneously. With efficient primal-dual semismooth Newton methods for the challenging and highly coupled nonlinear subproblems involving total generalized variation, we develop a highly efficient and competitive augmented Lagrangian method compared with some fast first-order method. With the analysis of the metric subregularities of the corresponding functions, we give both the global convergence and local linear convergence rate for the proposed augmented Lagrangian methods.</p>

Geophysical Inverse Theory and Regularization Problems

—GeoFEM (Iizuka et al., 1999) is a parallel finite element analysis system intended for multi-physics, multi-scale problems of solid earth field phenomena. Very large linear elastic problems have already been resolved by parallel computation with GeoFEM. The next stage is to examine large-scale nonlinear problems using GeoFEM. The analysis of large-scale contact problems for fault zones is particularly important in the development of models that simulate the occurrence and cycle of earthquakes.¶This paper proposes a parallel FEM using an iterative solver with the augmented Lagrange method for solving large-scale contact problems. Direct solvers are presently applied to contact problem analysis because the matrix for such problems is ill-conditioned. However, direct solvers are not suitable for large-scale matrices. The augmented Lagrange method can improve the matrix conditions. The present study evaluates a parallel FEM using an iterative solver with the augmented Lagrange method. Analysis of a contact problem with the augmented Lagrange method revealed that an optimal penalty parameter exists and that large-scale parallel contact analysis using the iterative solver with localized preconditioning is promising.

The term, "Waveform inversion" (WFI), refers to a collection of techniques that use the information from seismic data to derive high-fidelity earth models for seismic imaging. The attractiveness of WFI lies mainly in its lack of approximations, at least in a theoretical sense, in contrast to other model determination techniques such as semblance or tomography. However, a whole raft of approximations must be made to make the technique viable with today'scomputing technology and restrictions of seismic acquisition. These are collectively referred to as "waveform inversion strategies" and in this paper we mainly discuss regularization and preconditioning strategies. Because the wavefields need to be accurately modeled to represent the kinematics of all the waves during WFI iterations, the effects of anisotropy often help to improve WFI results. In this paper, forward modeling and its adjoint computation are based on acoustic wave equations in vertical transversely isotropic (VTI)media. We introduce a multi-parameter inversion for P-wave velocity and anisotropy parameters. WFI is a highly nonlinear, ill-posed problem. We introduce additional information and turn the unconstrained optimization problem into a constrained optimization problem in order to reduce the ill-posedness. The geophysics of the problem leads to appropriate constraints, such as restriction of model parameters, or information from well logs. In this paper, we use well logs as constraints and solve the problem using the augmented Lagrangian method (ALM), a mathematical method that replaces a constrained optimization problem by a series of unconstrained problems. The ALM with well constraints aims at preserving velocity characteristics from welllogs and providing us with more reliable velocity updates. This paper presents the acoustic anisotropic WFI implementation using ALM with well constraints. It also discusses practical strategies for regularization and preconditioning and their influences on the models that are obtained from WFI. We illustrate these approaches on a 2D synthetic Marmousi example and another application to 3D VSOOBC data from the Green Canyon area of the Gulf of Mexico. From the results, we show that multi-parameter VTI WFI with ALM provides us with more useful and reliable model updates. To further evaluate our WFI results, we also compare offset gathers and RTM images and illustrate their significant improvement susing updated models generated from WFI.

The Newsletter is a key element in further enhancing the value of the journal to the inverse problems community. So why not be a part of this exciting forum by sending to our Bristol office material suitable for inclusion under any of the categories mentioned above. Your contributions will be very welcome. Book review Introduction to Inverse Problems in Imaging M Bertero and P Boccacci 1998 Bristol: Institute of Physics 362 pp ISBN 0-7503-0435-9 (pbk) £25.00, $49.00 This book shows that several problems in imaging are in fact linear inverse problems. In general inverse problems are ill-posed in the sense that small perturbations in the (measured) data have a significant influence on the output. Consequently, numerical methods developed in a general setting of inverse problems which cure the ill-posedness (such methods are called regularization methods) can be applied to solve problems in imaging. This is an excellent textbook on the principle of linear inverse problems, methods of their numerical solution and practical applications in imaging. It introduces basic ideas and methods for the solution of inverse and ill-posed problems while avoiding the mathematics of functional treatment of operator equations. This goal is achieved by a bottom-up presentation: the authors focus on the two problems of image deconvolution and tomography (bottom) and develop regularization theory specifically for these two problems. Most other books which are concerned with the numerical solution of inverse problems present a top-down approach. That is, a general regularization theory (top) is developed which is then applied to particular problems. The distinguished presentation makes the book very useful to students in applied mathematics, practitioners in engineering science and image processing. This book has the best prerequisites to establish a close link between the mathematical fields of imaging and inverse problems. Mainly the book focuses on two topics: image deconvolution and linear imaging systems. 1. In image deconvolution the goal is to restore a space invariant blurred image. This part of the book contains mathematical tools in image deconvolution, examples of space invariant imaging systems, a comparison of image deconvolution techniques and low-pass filtering techniques, the analysis of the ill-posedness of the deconvolution problem, regularization methods for deconvolution such as constrained least squares regularization, Tikhonov regularization, iterative regularization and statistical methods. Since the authors focus on particular problems they are able to analyse various methods for deconvolution on a very concrete level. As an example I would like to mention that the authors analyse Fourier methods for deconvolution. Thus they provide an optimal starting point to the theory of inverse problems for people working in the area of signal and image processing. 2. The second part of the book considers linear imaging systems that are not of a convolution type such as tomography problems, diffraction problems and inverse scattering problems. Efficient numerical methods for the solution of this class of problems such as singular value decomposition, Tikhonov regularization methods and Fourier-based methods are presented. O Scherzer Universität Linz

It is known that the steepest-descent method converges normally at the first few iterations, and then it slows down. We modify the original steplength and descent direction by an optimization argument with the new steplength as being a merit function to be maximized. An optimal iterative algorithm with <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M1" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path id="x1D45A" d="M766 88q-40 -45 -83.5 -72.5t-63.5 -27.5q-39 0 -16 103l49 224q15 68 -10 68q-36 0 -112.5 -78.5t-105.5 -133.5q-27 -101 -38 -165q-35 -4 -77 -18l-6 6q42 153 70 288q12 55 10 78t-19 23q-38 0 -114 -80.5t-101 -131.5q-24 -68 -41 -165q-36 -3 -76 -18l-8 6&#xA;q56 202 87 347q7 33 -3 33q-8 0 -31.5 -17.5t-41.5 -35.5l-12 28q42 45 84 72t63 27q44 0 10 -124l-20 -75h2q92 121 193 178q36 21 64 21q62 0 28 -159l-8 -37h2q50 67 103.5 112t94.5 65q39 19 62 19q60 0 23 -156l-45 -189q-9 -39 2 -39q17 0 71 49z" /></g> </svg>-vector descent direction in a Krylov subspace is constructed, of which the <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M2" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><use xlink:href="#x1D45A"/></g> </svg> optimal weighting parameters are solved in closed-form to accelerate the convergence speed in solving ill-posed linear problems. The optimally generalized steepest-descent algorithm (OGSDA) is proven to be convergent with very fast convergence speed, accurate and robust against noisy disturbance, which is confirmed by numerical tests of some well-known ill-posed linear problems and linear inverse problems.

We propose an adaptive total generalized variation (TGV) based model, aiming at achieving a balance between edge preservation and region smoothness for image denoising. The variable splitting (VS) and the classical augmented Lagrangian method (ALM) are used to solve the proposed model. With the proposed adaptive model and ALM, the regularization parameter, which balances the data fidelity and the regularizer, is refreshed with a closed form in each iterate, and the image denoising can be accomplished without manual interference. Numerical results indicate that our method is effective in staircasing effect suppression and holds superiority over some other state‐of‐the‐art methods both in quantitative and in qualitative assessment.

This paper presents a new point-to-face contact algorithm for contacts between two polyhedrons with planar boundaries. A new discrete numerical method called three-dimensional discontinuous deformation analysis (3-D DDA) is used and formulations of normal contact submatrices based on the proposed algorithm are derived. The presented algorithm is a simple and efficient method and it can be easily coded into a computer program. This approach does not need to use an iterative algorithm in each time step to obtain the contact plane, unlike the ‘Common-Plane’ method applied in the existing 3-D DDA. In the present 3-D DDA method, block contact constraints are enforced using the penalty method. This approach is quite simple, but may lead to inaccuracies that may be large for small values of the penalty number. The penalty method also creates block contact overlap, which violates the physical constraints of the problem. These limitations are overcome by using the augmented Lagrangian method that is used for normal contacts in this research. This point-to-face contact model has been programmed and some illustrative examples are provided to demonstrate the new contact rule between two blocks. A comparison between results obtained by using the augmented Lagrangian method and the penalty method is presented as well.

We present an Augmented Lagrangian Method (ALM) for solving image reconstruction problems with a cost function consisting of multiple regularization functions with a data fidelity constraint. The presented technique is used to solve inverse problems related to image reconstruction, including compressed sensing formulations. Our contributions include an improvement for reducing the number of computations required by an existing ALM method, an approach for obtaining the proximal mapping associated with p-norm based regularizers, and lastly a particular ALM for the constrained image reconstruction problem with a hybrid cost function including a weighted sum of the p-norm and the total variation of the image. We present examples from Synthetic Aperture Radar imaging and Computed Tomography.

Splitting-based conjugate gradient method for a multi-dimensional linear inverse heat conduction problem

Many works have shown strong connections between learning and regularization techniques for ill-posed inverse problems. A careful analysis shows that a rigorous connection between learning and regularization for inverse problem is not straightforward. In this study, pattern recognition will be viewed as an ill-posed inverse problem and applications of methods from the theory of inverse problems to pattern recognition are studied. A new learning algorithm derived from a well-known regularization model is generated and applied to the task of reconstruction of an inhomogeneous object as pattern recognition. Particularly, it is demonstrated that pattern recognition can be reformulated in terms of inverse problems defined by a Riesz-type kernel. This reformulation can be employed to design a learning algorithm based on a numerical solution of a system of linear equations. Finally, numerical experiments have been carried out with synthetic experimental data considering a reasonable level of noise. Good recoveries have been achieved with this methodology, and the results of these simulations are compatible with the existing methods. The comparison results show that the Regularization-based learning algorithm (RBA) obtains a promising performance on the majority of the test problems. In prospects, this method can be used for the creation of automated systems for diagnostics, testing, and control in various fields of scientific and applied research, as well as in industry.

The Bayesian inference is widely used in many scientific and engineering problems, especially in the linear inverse problems in infinite-dimensional setting where the unknowns are functions. In such problems, choosing an appropriate prior distribution is an important task. Especially when the function to infer has much detail information, such as many sharp jumps, corners, and the discontinuous and nonsmooth oscillation, the so-called total variation-Gaussian (TG) prior is proposed in function space to address it. However, the TG prior is easy to lead the blocky (staircase) effect in numerical results. In this work, we present a fractional order-TG (FTG) hybrid prior to deal with such problems, where the fractional order total variation (FTV) term is used to capture the detail information of the unknowns and simultaneously uses the Gaussian measure to ensure that it results in a well-defined posterior measure. For the numerical implementations of linear inverse problems in function spaces, we also propose an efficient independence sampler based on a transport map, which uses a proposal distribution derived from a diagonal map, and the acceptance probability associated to the proposal is independent of discretization dimensionality. And in order to take full advantage of the transport map, the hierarchical Bayesian framework is applied to flexibly determine the regularization parameter. Finally we provide some numerical examples to demonstrate the performance of the FTG prior and the efficiency and robustness of the proposed independence sampler method.

Ill-posed linear inverse problems appear frequently in various signal processing applications. It can be very useful to have theoretical characterizations that quantify the level of ill-posedness for a given inverse problem and the degree of ambiguity that may exist about its solution. Traditional measures of ill-posedness, such as the condition number of a matrix, provide characterizations that are global in nature. While such characterizations can be powerful, they can also fail to provide full insight into situations where certain entries of the solution vector are more or less ambiguous than others. In this work, we derive novel theoretical lower- and upper-bounds that apply to individual entries of the solution vector, and are valid for all potential solution vectors that are nearly data-consistent. These bounds are agnostic to the noise statistics and the specific method used to solve the inverse problem, and are also shown to be tight. In addition, our results also lead us to introduce an entrywise version of the traditional condition number, which provides a substantially more nuanced characterization of scenarios where certain elements of the solution vector are less sensitive to perturbations than others. Our results are illustrated in an application to magnetic resonance imaging reconstruction, and we include discussions of practical computation methods for large-scale inverse problems, connections between our new theory and the traditional Cramér-Rao bound under statistical modeling assumptions, and potential extensions to cases involving constraints beyond just data-consistency.

Because balanced constraints can overcome the problems of trivial solutions of data classification via minimum cut method, many techniques with different balanced strategies have been proposed to improve data classification accuracy. However, their performances have not been compared comprehensively so far. In this paper, we investigate seven balanced classification methods under the discrete non-local total variational framework and compare their accuracy performances on graph. The two-class classification problem with equality constraints, inequality constraints and Ratio Cut, Normalized Cut, Cheeger Cut models are investigated. For cases of equality constraint, we firstly compare the Penalty Function Method (PFM) and the Augmented Lagrangian Method (ALM), which can transform the constrained problems into unconstrained ones, to show the advantages of ALM. The other cases are all solved using the ALM also. In order to make the comparison fairly, we solve all models using ALM method and using the same proportion of fidelity points and the same neighborhood size on graph. Experimental results demonstrate ALM with the equality balanced constraint has the best classification accuracy compared with other six constraints. 200 words.

In this paper, we propose an image segmentation model where an $L^1$ variant of the Euler's elastica energy is used as boundary regularization. An interesting feature of this model lies in its preference for convex segmentation contours. However, due to the high order and non-differentiability of Euler's elastica energy, it is nontrivial to minimize the associated functional. As in recent work on the ordinary $L^2$-Euler's elastica model in imaging, we propose using an augmented Lagrangian method to tackle the minimization problem. Specifically, we design a novel augmented Lagrangian functional that deals with the mean curvature term differently than in previous works. The new treatment reduces the number of Lagrange multipliers employed, and more importantly, it helps represent the curvature more effectively and faithfully. Numerical experiments validate the efficiency of the proposed augmented Lagrangian method and also demonstrate new features of this particular segmentation model, such as shape driven and data driven properties.