Design optimization in electrostatic field analysis using the BEM and the augmented Lagrangian method

<p style='text-indent:20px;'>Total generalization variation (TGV) is a very powerful and important regularization for various inverse problems and computer vision tasks. In this paper, we propose a semismooth Newton based augmented Lagrangian method for solving this problem. The augmented Lagrangian method (also called as method of multipliers) is widely used for lots of smooth or nonsmooth variational problems. However, its efficiency heavily depends on solving the corresponding coupled and nonlinear system together and simultaneously. With efficient primal-dual semismooth Newton methods for the challenging and highly coupled nonlinear subproblems involving total generalized variation, we develop a highly efficient and competitive augmented Lagrangian method compared with some fast first-order method. With the analysis of the metric subregularities of the corresponding functions, we give both the global convergence and local linear convergence rate for the proposed augmented Lagrangian methods.</p>

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

The accurate and fast calculation of surface electric field of DC converter valve shield system have guiding significance in the designing process. However, the electrostatic field analysis is a large‐scale problem with low efficiency using the conventional numerical algorithm. In order to solve this problem, fast multipole curved boundary element method (FMCBEM) was proposed. The FMCBEM is based on the indirect boundary integral equation of electrostatic field. Galerkin curved boundary element method based on coordinate transformation is used to improve the accuracy, then fast multipole method is used to improve the calculation speed and reduce the memory usage. Through simple examples, the accuracy and efficiency of the algorithm were verified. Compared with finite element method, it was found that the accuracy can meet the requirements of engineering. The efficiency was obviously better than the conventional boundary element method. Applying this algorithm, the surface electric fields of different shield systems of ±160 kV converter valve, which the degree of freedoms is more than 220 thousand, were analysed using desktop computer. This algorithm can calculate the large‐scale electric field problem fast and accurately, which is an effective tool for the optimisation design.

—GeoFEM (Iizuka et al., 1999) is a parallel finite element analysis system intended for multi-physics, multi-scale problems of solid earth field phenomena. Very large linear elastic problems have already been resolved by parallel computation with GeoFEM. The next stage is to examine large-scale nonlinear problems using GeoFEM. The analysis of large-scale contact problems for fault zones is particularly important in the development of models that simulate the occurrence and cycle of earthquakes.¶This paper proposes a parallel FEM using an iterative solver with the augmented Lagrange method for solving large-scale contact problems. Direct solvers are presently applied to contact problem analysis because the matrix for such problems is ill-conditioned. However, direct solvers are not suitable for large-scale matrices. The augmented Lagrange method can improve the matrix conditions. The present study evaluates a parallel FEM using an iterative solver with the augmented Lagrange method. Analysis of a contact problem with the augmented Lagrange method revealed that an optimal penalty parameter exists and that large-scale parallel contact analysis using the iterative solver with localized preconditioning is promising.

This paper presents a new point-to-face contact algorithm for contacts between two polyhedrons with planar boundaries. A new discrete numerical method called three-dimensional discontinuous deformation analysis (3-D DDA) is used and formulations of normal contact submatrices based on the proposed algorithm are derived. The presented algorithm is a simple and efficient method and it can be easily coded into a computer program. This approach does not need to use an iterative algorithm in each time step to obtain the contact plane, unlike the ‘Common-Plane’ method applied in the existing 3-D DDA. In the present 3-D DDA method, block contact constraints are enforced using the penalty method. This approach is quite simple, but may lead to inaccuracies that may be large for small values of the penalty number. The penalty method also creates block contact overlap, which violates the physical constraints of the problem. These limitations are overcome by using the augmented Lagrangian method that is used for normal contacts in this research. This point-to-face contact model has been programmed and some illustrative examples are provided to demonstrate the new contact rule between two blocks. A comparison between results obtained by using the augmented Lagrangian method and the penalty method is presented as well.

As the dedication suggests, many scientists in several countries have contributed significantly during these last fifty years to make conjugate gradient the efficient and versatile method it is today. However, this article is not limited to conjugate gradient and from that point of view, it owes a lot to M. Hestenes for his major contributions, not only to conjugate gradient, but also to calculus of variations and augmented Lagrangian methods. Indeed, in this article after investigating the solution of obstacle problems for elliptic and parabolic linear operators by a methodology combining penalty, Newton’s method and conjugate gradient algorithms, we shall apply the above methodology to the search for non-negative solutions to an Eikonal system with Dirichlet boundary conditions, after transforming it into a (non-convex problem from Calculus of Variations; in order to solve this non-convex minimization problem we shall apply an operator-splitting m ethod, and as shown in, e.g., [1], [2] operator splitting methods have close links with augmented Lagrangian methods and therefore with M. Hestenes contributions to Applied and Computational Mathematics. Since, in this article, we make a systematic use of conjugate gradient algorithms, and of cyclic reduction methods for the fast solution of discrete linear elliptic problems, a special credit has to be given to G. H. Golub for his outstanding contributions to these topics (among many other contributions, which include, in fact, complementarity methods for the solution of discrete obstacle problems). We deeply regret he could not attend the Jyvaskyla Conference on “50 Years of Conjugate Gradients”.

In this paper, we propose an image segmentation model where an $L^1$ variant of the Euler's elastica energy is used as boundary regularization. An interesting feature of this model lies in its preference for convex segmentation contours. However, due to the high order and non-differentiability of Euler's elastica energy, it is nontrivial to minimize the associated functional. As in recent work on the ordinary $L^2$-Euler's elastica model in imaging, we propose using an augmented Lagrangian method to tackle the minimization problem. Specifically, we design a novel augmented Lagrangian functional that deals with the mean curvature term differently than in previous works. The new treatment reduces the number of Lagrange multipliers employed, and more importantly, it helps represent the curvature more effectively and faithfully. Numerical experiments validate the efficiency of the proposed augmented Lagrangian method and also demonstrate new features of this particular segmentation model, such as shape driven and data driven properties.

The boundary element method (BEM) is an efficient numerical method that approximates solutions of various boundary value problems. Despite its success little research has been performed on the conditioning of the linear systems that appear in the BEM. For a Laplace equation with Dirichlet boundary conditions a remarkable phenomenon is observed; the corresponding boundary integral equation (BIE) is singular for a certain critical size of the 2D domain. As a consequence the discrete counterpart of the BIE, the linear system, is singular too, or at least ill-conditioned. This is reflected by the condition number of the system matrix, which is infinitely large, or at least very large. When the condition number of the BEM-matrix is large, the linear system is difficult to solve and the solution of the system is very sensible to perturbations in the boundary data. For a Laplace equation with mixed boundary conditions a similar phenomenon is observed. The corresponding BEM-matrix consists of two blocks; one block originates from the BEM-matrix belonging to the Dirichlet problem, the other block originates from the BEM-matrix belonging to the Neumann problem. The composite matrix inherits the solvability problems from the Dirichlet block. In other words, for the Laplace equation with mixed boundary conditions there exists also a critical size of the 2D domain for which the BEM-matrix has an infinitely large condition number. Hence the size and shape of the domain affects the solvability of the BEM problem. The critical size of the domain for which the BIE becomes singular is related to the logarithmic capacity of the domain. The logarithmic capacity is a positive real number that is a function of the size and shape of the domain. If this logarithmic capacity is equal to one, the domain is a critical domain, and for this domain the BIE becomes singular. Thus by computing the logarithmic capacity we can a-priori determine whether the BIE will be singular or not. The logarithmic capacity depends linearly on the scale of the domain, and thus a domain with logarithmic capacity equal to one can always be found by rescaling the domain. Unfortunately the logarithmic capacity can only be computed analytically for a few simple domains; for more involved domains the logarithmic capacity can be estimated though. There are several possibilities to avoid large condition numbers, i.e. singular BIEs that appear at critical domains. The first option is to rescale the domain such that the logarithmic capacity is unequal to one. One can also add a supplementary condition to the BIE and the linear system. A drawback of this option is that the linear system has more equations than unknowns and different techniques are required to solve the system. A third option is to slightly modify the fundamental solution of the Laplace operator. This fundamental solution directly appears in the BIE and it can be shown that a suitable modification yields BIEs that do not become singular. The critical domains for which the BIEs become singular do not restrict to Laplace equations only. Also for BIEs applied to the biharmonic equation or the elastostatic equations and the Stokes equations such critical domains exist. As the last two equations are vectorial equations, also the corresponding BIE consists of two equations. As a consequence two critical domains can be found for which these BIEs become singular. To obtain nonsingular BIEs techniques similar to the Laplace case can be used. Unfortunately we cannot a-priori determine the sizes for which the BIEs becomes singular, and thus do not know to what size we should rescale the domain to obtain nonsingular BIEs. The existence of critical domains is in essence caused by the logarithmic term in the fundamental solutions for the elliptic boundary value problems in 2D. This logarithmic term does not depend linearly on the size of the domain. When a domain is scaled, i.e. multiplied by a scale factor, the argument of the logarithm is also multiplied by this scale factor, but the logarithm turns this into an additive term. Thus the logarithm transforms multiplication into addition. This affects the BIEs in such a way that critical domains can appear. The fundamental solutions of boundary value problems in 3D do not contain a logarithmic term. Hence scaling of the domain does not affect the fundamental solution, and consequently also the BIE is not affected. Hence we may safely rescale 3D domains without the risk to encounter a critical domain. An example in which a domain takes many different sizes and shapes is the blowing problem. In this problem a viscous fluid is blown to a desired shape. Typically the time is discretised into a set of discrete time steps, and at each step the shape of the fluid is computed by solving the Stokes equations. When attempting to simulate this problem in 2D, we meet a large number of 2D domains, and we risk that one of these domains is equal to or approaches a critical domain. In such a case the BEM will have difficulties with solving the Stokes equations for that particular domain. When simulating the blowing problem in 3D, no critical domains are encountered. It turns out that the BEM is a very efficient numerical method for this particular 3D problem with a moving boundary. As we are merely interested in the shape of the fluid, we only need to know the flow of its boundary. The BEM does exactly that; it does not compute the flow at the interior of the fluid. Furthermore it is rather easy to include other effects from the blowing problem in the model, such as gravity, surface tension and friction from the contact of the fluid with a wall. As only the boundary of the fluid is discretised, the system matrices that appear in the BEM are smaller than the system matrices that appear when solving the problem with a finite element method, for example. Though the BEM-matrices are dense, while the finite element matrices are sparse, the computational effort for the BEM is relatively low. In short, the BEMis a very appropriate numerical method when solving blowing problems.

<p>The primary objective of this study is to develop new computational methods for robust design optimization (RDO) and reliability-based design optimization (RBDO) of high-dimensional, complex engineering systems. Four major research directions, all anchored in polynomial dimensional decomposition (PDD), have been defined to meet the objective. They involve: (1) development of new sensitivity analysis methods for RDO and RBDO; (2) development of novel optimization methods for solving RDO problems; (3) development of novel optimization methods for solving RBDO problems; and (4) development of a novel scheme and formulation to solve stochastic design optimization problems with both distributional and structural design parameters.</p> <p>The major achievements are as follows. Firstly, three new computational methods were developed for calculating design sensitivities of statistical moments and reliability of high-dimensional complex systems subject to random inputs. The first method represents a novel integration of PDD of a multivariate stochastic response function and score functions, leading to analytical expressions of design sensitivities of the first two moments. The second and third methods, relevant to probability distribution or reliability analysis, exploit two distinct combinations built on PDD: the PDD-SPA method, entailing the saddlepoint approximation (SPA) and score functions; and the PDD-MCS method, utilizing the embedded Monte Carlo simulation (MCS) of the PDD approximation and score functions. For all three methods developed, both the statistical moments or failure probabilities and their design sensitivities are both determined concurrently from a single stochastic analysis or simulation. Secondly, four new methods were developed for RDO of complex engineering systems. The methods involve PDD of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms. The methods, depending on how statistical moment and sensitivity analyses are dovetailed with an optimization algorithm, encompass direct, single-step, sequential, and multi-point single-step design processes. Thirdly, two new methods were developed for RBDO of complex engineering systems. The methods involve an adaptive-sparse polynomial dimensional decomposition (AS-PDD) of a high-dimensional stochastic response for reliability analysis, a novel integration of AS-PDD and score functions for calculating the sensitivities of the failure probability with respect to design variables, and standard gradient-based optimization algorithms, resulting in a multi-point, single-step design process. The two methods, depending on how the failure probability and its design sensitivities are evaluated, exploit two distinct combinations built on AS-PDD: the AS-PDD-SPA method, entailing SPA and score functions; and the AS-PDD-MCS method, utilizing the embedded MCS of the AS-PDD approximation and score functions. In addition, a new method, named as the augmented PDD method, was developed for RDO and RBDO subject to mixed design variables, comprising both distributional and structural design variables. The method comprises a new augmented PDD of a high-dimensional stochastic response for statistical moment and reliability analyses; an integration of the augmented PDD, score functions, and finite-difference approximation for calculating the sensitivities of the first two moments and the failure probability with respect to distributional and structural design variables; and standard gradient-based optimization algorithms, leading to a multi-point, single-step design process.</p> <p>The innovative formulations of statistical moment and reliability analysis, design sensitivity analysis, and optimization algorithms have achieved not only highly accurate but also computationally efficient design solutions. Therefore, these new methods are capable of performing industrial-scale design optimization with numerous design variables.</p>

We present an Augmented Lagrangian Method (ALM) for solving image reconstruction problems with a cost function consisting of multiple regularization functions with a data fidelity constraint. The presented technique is used to solve inverse problems related to image reconstruction, including compressed sensing formulations. Our contributions include an improvement for reducing the number of computations required by an existing ALM method, an approach for obtaining the proximal mapping associated with p-norm based regularizers, and lastly a particular ALM for the constrained image reconstruction problem with a hybrid cost function including a weighted sum of the p-norm and the total variation of the image. We present examples from Synthetic Aperture Radar imaging and Computed Tomography.

The term, "Waveform inversion" (WFI), refers to a collection of techniques that use the information from seismic data to derive high-fidelity earth models for seismic imaging. The attractiveness of WFI lies mainly in its lack of approximations, at least in a theoretical sense, in contrast to other model determination techniques such as semblance or tomography. However, a whole raft of approximations must be made to make the technique viable with today'scomputing technology and restrictions of seismic acquisition. These are collectively referred to as "waveform inversion strategies" and in this paper we mainly discuss regularization and preconditioning strategies. Because the wavefields need to be accurately modeled to represent the kinematics of all the waves during WFI iterations, the effects of anisotropy often help to improve WFI results. In this paper, forward modeling and its adjoint computation are based on acoustic wave equations in vertical transversely isotropic (VTI)media. We introduce a multi-parameter inversion for P-wave velocity and anisotropy parameters. WFI is a highly nonlinear, ill-posed problem. We introduce additional information and turn the unconstrained optimization problem into a constrained optimization problem in order to reduce the ill-posedness. The geophysics of the problem leads to appropriate constraints, such as restriction of model parameters, or information from well logs. In this paper, we use well logs as constraints and solve the problem using the augmented Lagrangian method (ALM), a mathematical method that replaces a constrained optimization problem by a series of unconstrained problems. The ALM with well constraints aims at preserving velocity characteristics from welllogs and providing us with more reliable velocity updates. This paper presents the acoustic anisotropic WFI implementation using ALM with well constraints. It also discusses practical strategies for regularization and preconditioning and their influences on the models that are obtained from WFI. We illustrate these approaches on a 2D synthetic Marmousi example and another application to 3D VSOOBC data from the Green Canyon area of the Gulf of Mexico. From the results, we show that multi-parameter VTI WFI with ALM provides us with more useful and reliable model updates. To further evaluate our WFI results, we also compare offset gathers and RTM images and illustrate their significant improvement susing updated models generated from WFI.

Because balanced constraints can overcome the problems of trivial solutions of data classification via minimum cut method, many techniques with different balanced strategies have been proposed to improve data classification accuracy. However, their performances have not been compared comprehensively so far. In this paper, we investigate seven balanced classification methods under the discrete non-local total variational framework and compare their accuracy performances on graph. The two-class classification problem with equality constraints, inequality constraints and Ratio Cut, Normalized Cut, Cheeger Cut models are investigated. For cases of equality constraint, we firstly compare the Penalty Function Method (PFM) and the Augmented Lagrangian Method (ALM), which can transform the constrained problems into unconstrained ones, to show the advantages of ALM. The other cases are all solved using the ALM also. In order to make the comparison fairly, we solve all models using ALM method and using the same proportion of fidelity points and the same neighborhood size on graph. Experimental results demonstrate ALM with the equality balanced constraint has the best classification accuracy compared with other six constraints. 200 words.

Due to the possible lack of primal-dual-type error bounds, it was not clear whether the Karush–Kuhn–Tucker (KKT) residuals of the sequence generated by the augmented Lagrangian method (ALM) for solving convex composite conic programming (CCCP) problems converge superlinearly. In this paper, we resolve this issue by establishing the R-superlinear convergence of the KKT residuals generated by the ALM under only a mild quadratic growth condition on the dual of CCCP, with easy-to-implement stopping criteria for the augmented Lagrangian subproblems. This discovery may help to explain the good numerical performance of several recently developed semismooth Newton-CG based ALM solvers for linear and convex quadratic semidefinite programming.

The boundary element method (BEM) is now a versatile and powerful tool of computational mechanics which has become a popular alternative to the well-established finite element method (FEM). Recently, some authors indicated that the presence of any domain integrals undermines most advantages of the BEM. We believe that this is not true if the domain integral contains known quantities (such as body sources or initial values). The actual sources of advantages of the BEM formulation are not determined by the question of if it is necessary to discretize the domain or not. The BEM formulation for the solution of a boundary value problem exhibits pure boundary character if the solution at any internal point can be expressed in terms of the boundary integrals of relevant quantities and the domain integrals of known body sources and initial values, if any. In other words, the solution can be expressed at an internal point without the need to know the solution at any other internal point. If the boundary values of the relevant quantities were known exactly and the integrations were performed with absolute accuracy, the integral representation would present the exact solution of the boundary value problem. That is why in the BEM solution, extreme emphasis is put on the accuracy of the approximations of boundary values and computation of boundary integrals. Since in the domain discretization techniques (such as FEM, finite difference method, collocation methods) the unknowns at all nodal points are computed simultaneously, the computational error is accumulated.

The augmented Lagrangian method (ALM) is a classical optimization tool that solves a given “difficult” (constrained) problem via finding solutions of a sequence of “easier” (often unconstrained) subproblems with respect to the original (primal) variable, wherein constraints satisfaction is controlled via the so-called dual variables. ALM is highly flexible with respect to how primal subproblems can be solved, giving rise to a plethora of different primal–dual methods. The powerful ALM mechanism has recently proved to be very successful in various large-scale and distributed applications. In addition, several significant advances have appeared, primarily on precise complexity results with respect to computational and communication costs in the presence of inexact updates and design and analysis of novel optimal methods for distributed consensus optimization. We provide a tutorial-style introduction to ALM and its variants for solving convex optimization problems in large-scale and distributed settings. We describe control-theoretic tools for the algorithms’ analysis and design, survey recent results, and provide novel insights into the context of two emerging applications: federated learning and distributed energy trading.