Sparsity reconstruction for bioluminescence tomography based on an augmented Lagrangian method

As an important optical molecular imaging technique, bioluminescence tomography (BLT) offers an inexpensive and sensitive means for non-invasively imaging a variety of physiological and pathological activities at cellular and molecular levels in living small animals. The key problem of BLT is to recover the distribution of the internal bioluminescence sources from limited measurements on the surface. Considering the sparsity of the light source distribution, we directly formulate the inverse problem of BLT into an l 0 -norm minimization model and present a smoothed l 0 -norm (SL0) based reconstruction algorithm. By approximating the discontinuous l0 norm with a suitable continuous function, the SL0 norm method solves the problem of intractable computational load of the minimal l 0 search as well as high sensitivity of l 0 -norm to noise. Numerical experiments on a mouse atlas demonstrate that the proposed SL0 norm based reconstruction method can obtain whole domain reconstruction without any a priori knowledge of the source permissible region, yielding almost the same reconstruction results to those of l 1 norm methods.

A new general mathematical framework for bioluminescence tomography

The term, "Waveform inversion" (WFI), refers to a collection of techniques that use the information from seismic data to derive high-fidelity earth models for seismic imaging. The attractiveness of WFI lies mainly in its lack of approximations, at least in a theoretical sense, in contrast to other model determination techniques such as semblance or tomography. However, a whole raft of approximations must be made to make the technique viable with today'scomputing technology and restrictions of seismic acquisition. These are collectively referred to as "waveform inversion strategies" and in this paper we mainly discuss regularization and preconditioning strategies. Because the wavefields need to be accurately modeled to represent the kinematics of all the waves during WFI iterations, the effects of anisotropy often help to improve WFI results. In this paper, forward modeling and its adjoint computation are based on acoustic wave equations in vertical transversely isotropic (VTI)media. We introduce a multi-parameter inversion for P-wave velocity and anisotropy parameters. WFI is a highly nonlinear, ill-posed problem. We introduce additional information and turn the unconstrained optimization problem into a constrained optimization problem in order to reduce the ill-posedness. The geophysics of the problem leads to appropriate constraints, such as restriction of model parameters, or information from well logs. In this paper, we use well logs as constraints and solve the problem using the augmented Lagrangian method (ALM), a mathematical method that replaces a constrained optimization problem by a series of unconstrained problems. The ALM with well constraints aims at preserving velocity characteristics from welllogs and providing us with more reliable velocity updates. This paper presents the acoustic anisotropic WFI implementation using ALM with well constraints. It also discusses practical strategies for regularization and preconditioning and their influences on the models that are obtained from WFI. We illustrate these approaches on a 2D synthetic Marmousi example and another application to 3D VSOOBC data from the Green Canyon area of the Gulf of Mexico. From the results, we show that multi-parameter VTI WFI with ALM provides us with more useful and reliable model updates. To further evaluate our WFI results, we also compare offset gathers and RTM images and illustrate their significant improvement susing updated models generated from WFI.

Bioluminescence tomography (BLT) is a powerful non-invasive molecular imaging tool for in vivo studies of glioma in mice. However, because of the light scattering and resulted ill-posed problems, it is challenging to develop a sufficient reconstruction method, which can accurately locate the tumor and define the tumor morphology in three-dimension. In this paper, we proposed a novel Gaussian weighted Laplace prior (GWLP) regularization method. It considered the variance of the bioluminescence energy between any two voxels inside an organ had a non-linear inverse relationship with their Gaussian distance to solve the over-smoothed tumor morphology in BLT reconstruction. We compared the GWLP with conventional Tikhonov and Laplace regularization methods through various numerical simulations and in vivo orthotopic glioma mouse model experiments. The in vivo magnetic resonance imaging and ex vivo green fluorescent protein images and hematoxylin-eosin stained images of whole head cryoslicing specimens were utilized as gold standards. The results demonstrated that GWLP achieved the highest accuracy in tumor localization and tumor morphology preservation. To the best of our knowledge, this is the first study that achieved such accurate BLT morphological reconstruction of orthotopic glioma without using any segmented tumor structure from any other structural imaging modalities as the prior for reconstruction guidance. This enabled BLT more suitable and practical for in vivo imaging of orthotopic glioma mouse models.

The Newsletter is a key element in further enhancing the value of the journal to the inverse problems community. So why not be a part of this exciting forum by sending to our Bristol office material suitable for inclusion under any of the categories mentioned above. Your contributions will be very welcome. Book review Introduction to Inverse Problems in Imaging M Bertero and P Boccacci 1998 Bristol: Institute of Physics 362 pp ISBN 0-7503-0435-9 (pbk) £25.00, $49.00 This book shows that several problems in imaging are in fact linear inverse problems. In general inverse problems are ill-posed in the sense that small perturbations in the (measured) data have a significant influence on the output. Consequently, numerical methods developed in a general setting of inverse problems which cure the ill-posedness (such methods are called regularization methods) can be applied to solve problems in imaging. This is an excellent textbook on the principle of linear inverse problems, methods of their numerical solution and practical applications in imaging. It introduces basic ideas and methods for the solution of inverse and ill-posed problems while avoiding the mathematics of functional treatment of operator equations. This goal is achieved by a bottom-up presentation: the authors focus on the two problems of image deconvolution and tomography (bottom) and develop regularization theory specifically for these two problems. Most other books which are concerned with the numerical solution of inverse problems present a top-down approach. That is, a general regularization theory (top) is developed which is then applied to particular problems. The distinguished presentation makes the book very useful to students in applied mathematics, practitioners in engineering science and image processing. This book has the best prerequisites to establish a close link between the mathematical fields of imaging and inverse problems. Mainly the book focuses on two topics: image deconvolution and linear imaging systems. 1. In image deconvolution the goal is to restore a space invariant blurred image. This part of the book contains mathematical tools in image deconvolution, examples of space invariant imaging systems, a comparison of image deconvolution techniques and low-pass filtering techniques, the analysis of the ill-posedness of the deconvolution problem, regularization methods for deconvolution such as constrained least squares regularization, Tikhonov regularization, iterative regularization and statistical methods. Since the authors focus on particular problems they are able to analyse various methods for deconvolution on a very concrete level. As an example I would like to mention that the authors analyse Fourier methods for deconvolution. Thus they provide an optimal starting point to the theory of inverse problems for people working in the area of signal and image processing. 2. The second part of the book considers linear imaging systems that are not of a convolution type such as tomography problems, diffraction problems and inverse scattering problems. Efficient numerical methods for the solution of this class of problems such as singular value decomposition, Tikhonov regularization methods and Fourier-based methods are presented. O Scherzer Universität Linz

For interior tomography based on truncated Hilbert transform (THT), the recently proposed truncated singular value decomposition (TSVD) reconstruction method uses a regularizati on parameter given directly. In this paper, a method of choosing the regularization parameter is presented based on L-curve to get an optimal regularization parameter in theoretical sense. Furthermore, we develop a Tikhonov regula rization method in comparison to TSVD. Simulation results indicate that both of the two regularization methods with the optimal regularization parameters have good performances on the image quality for both cases of noise-free and noisy projections. Keywords: Interior Tomography, Truncated Hilbert Transform, L-curve, Truncated Singular Value Decomposition, Tikhonov Regularization 1. INTRODUCTION It had been believed that the solution to the interior tomography is not unique fo r a long time. Recently, inspired by the concept of the differential backprojection (DBP) proposed by Noo et al [1] and Pan et al [2], a novel solution for the interior problem was proposed with numerical results demonstrat ed that the interior problem can be uniquely and stably solved if a sub-region inside a region of interest (ROI) is known [3]-[5]. Similar results were also independently proposed by other researchers [6]-[8]. In these papers, image reconstruction was reduced to the inversion of THT via the projection onto convex set (POCS) method, which is iterativ e and computationally expensive. To over above limitations, TSVD as a regularization method has been proposed for inversing the THT, which runs two orders of magnitude faster than the iterative method [9]. However the regularization parame ter in this method is given directly without theoretical analysis. The selection of the regularization parameter that controls the trade-off between fidelity to the data and constraint on the reconstruction result is a primary problem in the regularization method. In this paper, we present a method of selecting the optimal regularization based on the L-curve for TSVD method for solving the interior tomography based on truncated Hilbert transform. In this method, the optimal regularization parameter is computed theoretically using the maximum curvature of the L-curve. Moreover, we develop a Tikhonov regularization method in comparison to TSVD. Root Mean Sq uare Error (RMSE) of reconstr ucted image with respect to different regularization parameters validates our selecting regularization parameter method. The paper is organized as follows: section 2 gives Mathematical formulation, TSVD, Tikhonov regularization, the L-curve. Section 3 exhibits the simulation results. Section 4 is conclusion.

An augmented Lagrangian approach with a variable transformation in nonlinear programming

This paper proposes an algorithm for the data processing of multispectral thermometry suitable for high temperatures ranging from 1500 to 3000K. The algorithm is based on Planck's law, transforming the temperature inversion problem into an equality-constrained optimization problem and employing the augmented Lagrangian method (ALM) as the solution approach. To address the issue of ALM being greatly influenced by the initial iteration point, this paper proposes the modified equilibrium optimizer algorithm to provide the emissivity initial point for ALM, forming the modified equilibrium optimizer-augmented Lagrangian method. Simulation experiments for hypothetical targets, tungsten, and solid rocket engine plumes have demonstrated the effectiveness and superiority of the proposed algorithm, offering a feasible and efficient solution for the field of multispectral high-temperature online real-time temperature measurement. In addition, the experiment on 2219 aluminum alloy has demonstrated the effectiveness of the algorithm in practical applications.

In this paper, we explore the application of total variation regularization method for bioluminescence tomography (BLT) with an adaptive regularization parameter choice approach. Since BLT is a seriously ill-posed problem, therefore, l(2) regularized methods are frequently adopted to recover the bi-oluminescent sources. However, l(2) regularized methods typically lead to smooth reconstructions. In this paper, we investigated the use of total variation (TV) regularization to improve the quality of BLT reconstruction. Furthermore, the regularization parameter in TV method was chosen adaptively to make the proposed algorithm more stable. Results on simulation data provide evidence that the reconstructed source can be localized accurately compared with l(2) method. Meanwhile, the effectiveness of utility of the parameter choice were illustrated. Finally, different levels of noisy data were added to validate the performance of the proposed algorithm.

L1 regularization is effective for feature selection, but the resulting optimization is challenging due to the non-differentiability of the 1-norm. In this paper we compare state-of-the-art optimization techniques to solve this problem across several loss functions. Furthermore, we propose two new techniques. The first is based on a smooth (differentiable) convex approximation for the L1 regularizer that does not depend on any assumptions about the loss function used. The other technique is a new strategy that addresses the non-differentiability of the L1-regularizer by casting the problem as a constrained optimization problem that is then solved using a specialized gradient projection method. Extensive comparisons show that our newly proposed approaches consistently rank among the best in terms of convergence speed and efficiency by measuring the number of function evaluations required.KeywordsFeature SelectionLoss FunctionStep LengthLine SearchConstrain Optimization ProblemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

A deterministic global optimization algorithm based on augmented Lagrangian is proposed for solving the nonlinear programming with cone constraints, which is often encountered in technical design and operational research. At each outer iteration, the method requires the -global minimization of the augmented Lagrangian of problems with cone constraints, where. The convergence to an-global minimizer of the original problem is proved. Key words: Global optimization, augmented Lagrangians, cone optimization, algorithms.

In this paper, we introduce a novel upper bound formulation of Limit Analysis applied to plane strain problems. This formulation can considered to be a variant of an existing two-field mixed formulation based on the augmented Lagrangian method also developed by the authors. In contrast to the previous formulation, two independent field approximations are now used to define the velocity field, globally and at element level. It is shown that this feature allows a governing system of uncoupled linear equations to be obtained. Our main concern is to achieve a formulation with appropriate characteristics for implementation in an efficient parallel algorithm. Some numerical examples are presented in order to illustrate the current model performance.

This chapter introduces two very important concepts in constrained nonlinear optimization. These are penalty and augmented Lagrangian concepts. The idea is that both these concepts replace the original problem by a sequence of sub-problems in which the constraints are expressed by terms added to the objective function. The penalty concept is implemented in two different methods. The quadratic penalty method adds to the objective function a multiple of the square of the violation of each constraint and solves a sequence of unconstrained optimization sub-problems. Simple and enough intuitive, this approach has some important deficiencies. The nonsmooth exact penalty method, on the other hand, solves a single unconstrained optimization problem. In this approach, a popular function is the l1 penalty function. The problem with this method is that the nonsmoothness may create complications in numerical implementations. Finally, the second concept is the multiplier method or the augmented Lagrangian method, which explicitly uses Lagrange multiplier estimates in order to avoid the ill-conditioning of the quadratic penalty method.

In this paper, we propose an image segmentation model where an $L^1$ variant of the Euler's elastica energy is used as boundary regularization. An interesting feature of this model lies in its preference for convex segmentation contours. However, due to the high order and non-differentiability of Euler's elastica energy, it is nontrivial to minimize the associated functional. As in recent work on the ordinary $L^2$-Euler's elastica model in imaging, we propose using an augmented Lagrangian method to tackle the minimization problem. Specifically, we design a novel augmented Lagrangian functional that deals with the mean curvature term differently than in previous works. The new treatment reduces the number of Lagrange multipliers employed, and more importantly, it helps represent the curvature more effectively and faithfully. Numerical experiments validate the efficiency of the proposed augmented Lagrangian method and also demonstrate new features of this particular segmentation model, such as shape driven and data driven properties.