Noise-reducing cascadic multilevel methods for linear discrete ill-posed problems

An extrapolated TSVD method for linear discrete ill-posed problems with Kronecker structure

Invertible smoothing preconditioners for linear discrete ill-posed problems

Combining approximate solutions for linear discrete ill-posed problems

The purpose of this paper is to bring a series of rather surprising results which have appeared in the technical literature to the attention of a wider audience. We describe the results informally here; precise definitions and results will be given below. Our subject is the solution of an ill-posed problem on a digital computer. Such a problem can at best only be solved approximately. We say a problem is solvable if we can compute an "-approximation for any positive " and is unsolvable if we cannot compute an "-approximation, even for arbitrarily large ". It was shown in [17] that a linear problem is unsolvable iff it is ill-posed. Are there circumstances under which one can avoid this negative conclusion? It is common to associate an operator S with an ill-posed problem. If S is linear, then a problem is ill-posed iff S is unbounded. Note that unboundedness is a worst case concept; the supremum of S operating on elements in the unit ball of its domain is infinite. This suggests introducing the concept of a problem’s being ill-posed on the average if the expectation of S with respect to a measure is infinite and being well-posed on the average if this expectation is finite. These concepts were introduced in [17], where it was shown that if the measure is Gaussian and the linear operator S is measurable, then a linear problem is unsolvable on the average iff it is ill-posed on the average. A natural next step is to find a linear ill-posed problem which is also ill-posed on the average. Several attempts to do this for Gaussian measures were unsuccessful. This suggested the question: Is every linear problem well-posed on the average for any Gaussian measure? This question was answered in the affirmative in [5] and [15]. The conclusion in the title follows immediately from this and the preceding result. What if the measure is non-Gaussian? Now the situation is more complicated. In particular, there are problems that are even ill-posed on the average. In this paper, we round out the picture by presenting examples of such problems for non-Gaussian measures.

This chapter discusses iterative methods for the solution of very large severely ill-conditioned linear systems of equations that arise from the discretization of linear ill-posed problems. The right-hand side vector represents the given data and is assumed to be contaminated by errors. Solution methods proposed in the literature employ some form of filtering to reduce the influence of the error in the right-hand side on the computed approximate solution. The amount of filtering is determined by a parameter often referred to as the regularization parameter. We discuss how the filtering affects the computed approximate solution and consider the selection of regularization parameter. Methods in which a suitable value of the regularization parameter is determined during the computation without user intervention are emphasized. New iterative solution methods based on expanding explicitly chosen filter functions in terms of Chebyshev polynomials are presented. The properties of these methods are illustrated with applications to image restoration.

A stopping criterion for iterative regularization methods

Tikhonov regularization is a popular method to approximate solutions of linear discrete ill-posed problems when the observed or measured data is contaminated by noise. Multiparameter Tikhonov regularization may improve the quality of the computed approximate solutions. We propose a new iterative method for large-scale multiparameter Tikhonov regularization with general regularization operators based on a multidirectional subspace expansion. The multidirectional subspace expansion may be combined with subspace truncation to avoid excessive growth of the search space. Furthermore, we introduce a simple and effective parameter selection strategy based on the discrepancy principle and related to perturbation results.

The Landweber iteration method is one of the most popular methods for the solution of linear discrete ill-posed problems. The diversity of physical problems and the diversity of operators that result from them leads us to think about updating the main methods and algorithms to achieve the best results. We considered in this work the linear operator equation and the use of a new version of the Landweber iterative method as an iterative solver. The main goal of updating the Landweber iteration method is to make the iteration process fast and more accurate. We used a polar decomposition to achieve a symmetric positive definite operator instead of an identity operator in the classical Landweber method. We carried out the convergence and other necessary analyses to prove the usability of the new iteration method. The residual method was used as an analysis method to rate the convergence of the iteration. The modified iterative method was compared to the classical Landweber method. A numerical experiment illustrates the effectiveness of this method by applying it to solve the inverse boundary value problem of the heat equation (IBVP).

An ill-posed problem which involves heterogonous data can yield good results if the weight of observations is properly introduced into the adjustment model. Variance component estimation can be used in this respect to update and improve the weights based on the results of the adjustment. The variance component estimation will not be as simple as that is in an ordinary adjustment problem, because the result of the solution of an ill-posed problem contains a bias due to stabilizing the adjustment model. This paper investigates the variance component estimation in those ill-posed problems solved by the truncation singular value decomposition. The biases of the variance components are analyzed and the biased-corrected and the biased-corrected non-negative estimators of the variance components are developed. The derivations show that in order to estimate unbiased variance components, it suffices to estimate and remove the bias from the estimated residuals.

A Highly Accurate Technique for Interpolations Using Very High-Order Polynomials, and Its Applications to Some Ill-Posed Linear Problems

Tikhonov regularization is one of the most popular approaches to solving linear discrete ill-posed problems. The choice of the regularization matrix may significantly affect the quality of the computed solution. When the regularization matrix is the identity, iterated Tikhonov regularization can yield computed approximate solutions of higher quality than (standard) Tikhonov regularization. This paper provides an analysis of iterated Tikhonov regularization with a regularization matrix different from the identity. Computed examples illustrate the performance of this method.

Iterative Tikhonov regularization of tensor equations based on the Arnoldi process and some of its generalizations

Geophysical Inverse Theory and Regularization Problems

It is known that the steepest-descent method converges normally at the first few iterations, and then it slows down. We modify the original steplength and descent direction by an optimization argument with the new steplength as being a merit function to be maximized. An optimal iterative algorithm with <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M1" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path id="x1D45A" d="M766 88q-40 -45 -83.5 -72.5t-63.5 -27.5q-39 0 -16 103l49 224q15 68 -10 68q-36 0 -112.5 -78.5t-105.5 -133.5q-27 -101 -38 -165q-35 -4 -77 -18l-6 6q42 153 70 288q12 55 10 78t-19 23q-38 0 -114 -80.5t-101 -131.5q-24 -68 -41 -165q-36 -3 -76 -18l-8 6&#xA;q56 202 87 347q7 33 -3 33q-8 0 -31.5 -17.5t-41.5 -35.5l-12 28q42 45 84 72t63 27q44 0 10 -124l-20 -75h2q92 121 193 178q36 21 64 21q62 0 28 -159l-8 -37h2q50 67 103.5 112t94.5 65q39 19 62 19q60 0 23 -156l-45 -189q-9 -39 2 -39q17 0 71 49z" /></g> </svg>-vector descent direction in a Krylov subspace is constructed, of which the <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M2" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><use xlink:href="#x1D45A"/></g> </svg> optimal weighting parameters are solved in closed-form to accelerate the convergence speed in solving ill-posed linear problems. The optimally generalized steepest-descent algorithm (OGSDA) is proven to be convergent with very fast convergence speed, accurate and robust against noisy disturbance, which is confirmed by numerical tests of some well-known ill-posed linear problems and linear inverse problems.

In the present paper, we consider the application of overcomplete dictionaries to the solution of general ill-posed linear inverse problems. In the context of regression problems, there has been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso and its variants, is based on maximizing the likelihood, and relies on stringent assumptions on the dictionary, the so-called compatibility conditions, for a proof of its convergence rates. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction properties imposed by the linear operator. In what follows, we bypass this difficulty by a novel approach, which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations, and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide sharp oracle inequalities for the risk in a non-asymptotic setting.