A novel augmented Lagrangian approach for Limit Analysis computations

Bioluminescence imaging (BLI) is an optical molecular imaging modality for monitoring physiological and pathological activities at the molecular level. The information of bioluminescent probe distribution in small animals can be threedimensionally and quantitatively obtained by bioluminescence tomography (BLT). Due to ill-posed nature, BLT may bear multiple solutions and aberrant reconstruction in the presence of measurement noise and optical parameter mismatches. Among different regularization methods, L2-type regularization strategy is the most popular and commonly-applied method, which minimizes the output-least-square formulation incorporated with the l2-norm regularization term to stabilize the problem. However, it often imposes over-smoothing on the reconstruction results. In contrast, for many practical applications, such as early detection of tumors, the volumes of the bioluminescent sources are very small compared with the whole body. In this paper, L1 regularization is used to fully take advantage of the sparsity prior knowledge and improve both efficiency and stability. And then a reconstruction method based on the augmented Lagrangian approach is proposed, which considers the BLT problem as the constrained optimization problem and employs the Bregman iterative method to deal with it. By using "divide and conquer" approach, the optimization problem can be exactly and fast solved by iteratively solving a sequence of unconstrained subproblems. To evaluate the performance of the proposed method in turbid mouse geometry, stimulate experiments with a heterogeneous 3D mouse atlas are conducted. In addition, physical experiments further demonstrate the potential of the proposed algorithm in practical applications.

In this paper we consider a nonmonotone (mixed) variational inequality (VI) model with (nonlinear) convex conic constraints. Through developing an equivalent Lagrangian function-like primal-dual saddle point system for the VI model in question, we introduce an augmented Lagrangian primal-dual method, called ALAVI (Augmented Lagrangian Approach to Variational Inequality) in the paper, for solving a general constrained VI model. Under an assumption, called the primal-dual variational coherence condition in the paper, we prove the convergence of ALAVI. Next, we show that many existing generalized monotonicity properties are sufficient—though by no means necessary—to imply the abovementioned coherence condition and thus are sufficient to ensure convergence of ALAVI. Under that assumption, we further show that ALAVI has in fact an [Formula: see text] global rate of convergence where k is the iteration count. By introducing a new gap function, this rate further improves to be [Formula: see text] if the mapping is monotone. Finally, we show that under a metric subregularity condition, even if the VI model may be nonmonotone, the local convergence rate of ALAVI improves to be linear. Numerical experiments on some randomly generated highly nonlinear and nonmonotone VI problems show the practical efficacy of the newly proposed method. Funding: L. Zhao and D. Zhu were partially supported by the Major Project of the National Natural Science Foundation of China [Grant 72293582], the National Key R&D Program of China [Grant 2023YFA0915202], and the Fundamental Research Funds for the Central Universities (the Interdisciplinary Program of Shanghai Jiao Tong University) [Grant YG2024QNA36]. L. Zhao was partially supported by the Startup Fund for Young Faculty at SJTU (SFYF at SJTU) [Grant 22X010503839].

In this paper, we propose an image segmentation model where an $L^1$ variant of the Euler's elastica energy is used as boundary regularization. An interesting feature of this model lies in its preference for convex segmentation contours. However, due to the high order and non-differentiability of Euler's elastica energy, it is nontrivial to minimize the associated functional. As in recent work on the ordinary $L^2$-Euler's elastica model in imaging, we propose using an augmented Lagrangian method to tackle the minimization problem. Specifically, we design a novel augmented Lagrangian functional that deals with the mean curvature term differently than in previous works. The new treatment reduces the number of Lagrange multipliers employed, and more importantly, it helps represent the curvature more effectively and faithfully. Numerical experiments validate the efficiency of the proposed augmented Lagrangian method and also demonstrate new features of this particular segmentation model, such as shape driven and data driven properties.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

Due to the possible lack of primal-dual-type error bounds, it was not clear whether the Karush–Kuhn–Tucker (KKT) residuals of the sequence generated by the augmented Lagrangian method (ALM) for solving convex composite conic programming (CCCP) problems converge superlinearly. In this paper, we resolve this issue by establishing the R-superlinear convergence of the KKT residuals generated by the ALM under only a mild quadratic growth condition on the dual of CCCP, with easy-to-implement stopping criteria for the augmented Lagrangian subproblems. This discovery may help to explain the good numerical performance of several recently developed semismooth Newton-CG based ALM solvers for linear and convex quadratic semidefinite programming.

A deterministic global optimization algorithm based on augmented Lagrangian is proposed for solving the nonlinear programming with cone constraints, which is often encountered in technical design and operational research. At each outer iteration, the method requires the -global minimization of the augmented Lagrangian of problems with cone constraints, where. The convergence to an-global minimizer of the original problem is proved. Key words: Global optimization, augmented Lagrangians, cone optimization, algorithms.

<p style='text-indent:20px;'>Total generalization variation (TGV) is a very powerful and important regularization for various inverse problems and computer vision tasks. In this paper, we propose a semismooth Newton based augmented Lagrangian method for solving this problem. The augmented Lagrangian method (also called as method of multipliers) is widely used for lots of smooth or nonsmooth variational problems. However, its efficiency heavily depends on solving the corresponding coupled and nonlinear system together and simultaneously. With efficient primal-dual semismooth Newton methods for the challenging and highly coupled nonlinear subproblems involving total generalized variation, we develop a highly efficient and competitive augmented Lagrangian method compared with some fast first-order method. With the analysis of the metric subregularities of the corresponding functions, we give both the global convergence and local linear convergence rate for the proposed augmented Lagrangian methods.</p>

This chapter introduces two very important concepts in constrained nonlinear optimization. These are penalty and augmented Lagrangian concepts. The idea is that both these concepts replace the original problem by a sequence of sub-problems in which the constraints are expressed by terms added to the objective function. The penalty concept is implemented in two different methods. The quadratic penalty method adds to the objective function a multiple of the square of the violation of each constraint and solves a sequence of unconstrained optimization sub-problems. Simple and enough intuitive, this approach has some important deficiencies. The nonsmooth exact penalty method, on the other hand, solves a single unconstrained optimization problem. In this approach, a popular function is the l1 penalty function. The problem with this method is that the nonsmoothness may create complications in numerical implementations. Finally, the second concept is the multiplier method or the augmented Lagrangian method, which explicitly uses Lagrange multiplier estimates in order to avoid the ill-conditioning of the quadratic penalty method.

The performance of a new implementation of the augmented Lagrangian method is evaluated on a range of explicit and structural sizing optimization problems. The results are compared with those obtained using other mathematical programming methods. The implementation uses a first-order Lagrange multiplier update and the Hessian of the augmented Lagrangian function is approximated using partitioned secant updating. A number of different secant updates are evaluated. The results show the formulation to be superior to other implementations of augmented Lagrangian methods reported in the literature and that, under certain conditions, the method approaches the performance of the state-of-the-art SQP and SAM methods. Of the secant updates, the symmetric-rank-one update, is superior to the other updates including the BFGS scheme. It is suggested that the individual function, secant updating employed may be usefully applied in contexts where structural analysis and optimization are performed simultaneously, as in the simultaneous analysis and design method. In such cases the functions are partially separable and the associated Hessians are of low rank.

Computational modeling of the heart is a subject of substantial medical and scientific interest, which may contribute to increase the understanding of several phenomena associated with cardiac physiological and pathological states. Modeling the mechanics of the heart have led to considerable insights, but it still represents a complex and a demanding computational problem, especially in a strongly coupled electromechanical setting. Passive cardiac tissue is commonly modeled as hyperelastic and is characterized by quasi-incompressible, orthotropic, and nonlinear material behavior. These factors are known to be very challenging for the numerical solution of the model. The near-incompressibility is known to cause numerical issues such as the well-known locking phenomenon and ill-conditioning of the stiffness matrix. In this work, the augmented Lagrangian method is used to handle the nearly incompressible condition. This approach can potentially improve computational performance by reducing the condition number of the stiffness matrix and thereby improving the convergence of iterative solvers. We also improve the performance of iterative solvers by the use of an algebraic multigrid preconditioner. Numerical results of the augmented Lagrangian method combined with a preconditioned iterative solver for a cardiac mechanics benchmark suite are presented to show its improved performance.

This paper presents a new point-to-face contact algorithm for contacts between two polyhedrons with planar boundaries. A new discrete numerical method called three-dimensional discontinuous deformation analysis (3-D DDA) is used and formulations of normal contact submatrices based on the proposed algorithm are derived. The presented algorithm is a simple and efficient method and it can be easily coded into a computer program. This approach does not need to use an iterative algorithm in each time step to obtain the contact plane, unlike the ‘Common-Plane’ method applied in the existing 3-D DDA. In the present 3-D DDA method, block contact constraints are enforced using the penalty method. This approach is quite simple, but may lead to inaccuracies that may be large for small values of the penalty number. The penalty method also creates block contact overlap, which violates the physical constraints of the problem. These limitations are overcome by using the augmented Lagrangian method that is used for normal contacts in this research. This point-to-face contact model has been programmed and some illustrative examples are provided to demonstrate the new contact rule between two blocks. A comparison between results obtained by using the augmented Lagrangian method and the penalty method is presented as well.

PARALLEL AND LARGE SCALE MATRIX COMPUTATIONS IN CONTROL: SOME IDEAS

We present an Augmented Lagrangian Method (ALM) for solving image reconstruction problems with a cost function consisting of multiple regularization functions with a data fidelity constraint. The presented technique is used to solve inverse problems related to image reconstruction, including compressed sensing formulations. Our contributions include an improvement for reducing the number of computations required by an existing ALM method, an approach for obtaining the proximal mapping associated with p-norm based regularizers, and lastly a particular ALM for the constrained image reconstruction problem with a hybrid cost function including a weighted sum of the p-norm and the total variation of the image. We present examples from Synthetic Aperture Radar imaging and Computed Tomography.

ABSTRACTThe proximal method of multipliers was proposed by Rockafellar [Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math Oper Res. 1976;1:97–116] for solving convex programming and it is a kind of proximal point method applied to convex programming. In this paper, we apply this method for solving nonlinear semidefinite programming problems, in which subproblems have better properties than those from the augmented Lagrange method. We prove that, under the linear independence constraint qualification and the strong second-order sufficiency optimality condition, the rate of convergence of the proximal method of multipliers, for a nonlinear semidefinite programming problem, is linear and the ratio constant is proportional to 1/c, where c is the penalty parameter that exceeds a threshold . Moreover, the rate of convergence of the proximal method of multipliers is superlinear when the parameter c increases to .

The term, "Waveform inversion" (WFI), refers to a collection of techniques that use the information from seismic data to derive high-fidelity earth models for seismic imaging. The attractiveness of WFI lies mainly in its lack of approximations, at least in a theoretical sense, in contrast to other model determination techniques such as semblance or tomography. However, a whole raft of approximations must be made to make the technique viable with today'scomputing technology and restrictions of seismic acquisition. These are collectively referred to as "waveform inversion strategies" and in this paper we mainly discuss regularization and preconditioning strategies. Because the wavefields need to be accurately modeled to represent the kinematics of all the waves during WFI iterations, the effects of anisotropy often help to improve WFI results. In this paper, forward modeling and its adjoint computation are based on acoustic wave equations in vertical transversely isotropic (VTI)media. We introduce a multi-parameter inversion for P-wave velocity and anisotropy parameters. WFI is a highly nonlinear, ill-posed problem. We introduce additional information and turn the unconstrained optimization problem into a constrained optimization problem in order to reduce the ill-posedness. The geophysics of the problem leads to appropriate constraints, such as restriction of model parameters, or information from well logs. In this paper, we use well logs as constraints and solve the problem using the augmented Lagrangian method (ALM), a mathematical method that replaces a constrained optimization problem by a series of unconstrained problems. The ALM with well constraints aims at preserving velocity characteristics from welllogs and providing us with more reliable velocity updates. This paper presents the acoustic anisotropic WFI implementation using ALM with well constraints. It also discusses practical strategies for regularization and preconditioning and their influences on the models that are obtained from WFI. We illustrate these approaches on a 2D synthetic Marmousi example and another application to 3D VSOOBC data from the Green Canyon area of the Gulf of Mexico. From the results, we show that multi-parameter VTI WFI with ALM provides us with more useful and reliable model updates. To further evaluate our WFI results, we also compare offset gathers and RTM images and illustrate their significant improvement susing updated models generated from WFI.