Estimation of Optimal Well Controls Using the Augmented Lagrangian Function with Approximate Derivatives

Comparison of SQP and AL algorithms for deterministic constrained production optimization of hydrocarbon reservoirs

Duality in nonlinear programs using augmented Lagrangian functions

We present in this paper new results on the existence of saddle points of augmented Lagrangian functions for constrained nonconvex optimization. Four classes of augmented Lagrangian functions are considered: the essentially quadratic augmented Lagrangian, the exponential-type augmented Lagrangian, the modified barrier augmented Lagrangian, and the penalized exponential-type augmented Lagrangian. We first show that under second-order sufficiency conditions, all these augmented Lagrangian functions possess local saddle points. We then prove that global saddle points of these augmented Lagrangian functions exist under certain mild additional conditions. The results obtained in this paper provide a theoretical foundation for the use of augmented Lagrangians in constrained global optimization. Our findings also give new insights to the role played by augmented Lagrangians in local duality theory of constrained nonconvex optimization.

The paper describes a new approach to the optimal-power-flow problem based on Newton's method which it operates with an augmented Lagrangian function associated with the original problem. The function aggregates all the equality and inequality constraints. The first-order necessary conditions for optimality are reached by Newton's method, and by updating the dual variables and the penalty terms associated with the inequality constraints. The proposed approach does not have to identify the set of binding constraints and can be utilised for an infeasible starting point. The sparsity of the Hessian matrix of the augmented Lagrangian is completely exploited in the computational implementation. Tests results are presented to show the good performance of this approach.

Constrained optimization is central to large-scale machine learning, particularly in parallel and distributed environments. This paper presents a comprehensive study of augmented Lagrangian–based algorithms for such problems, including classical Lagrangian relaxation, the method of multipliers, the Alternating Direction Method of Multipliers (ADMM), Bertsekas’ algorithm, Tatjewski’s method, and the Separable Augmented Lagrangian Algorithm (SALA). We develop a unified theoretical framework, analyze convergence properties and decomposition strategies, and evaluate these methods on two representative classes of tasks: regularized linear systems and K-means clustering. Numerical experiments on synthetic and real-world datasets show that Bertsekas’ method consistently achieves the best balance of convergence speed and solution quality, while ADMM offers practical scalability under decomposition but struggles in high-dimensional or ill-conditioned settings. Tatjewski’s method benefits significantly from partitioning, whereas the classical Augmented Lagrangian approach proves computationally inefficient for large-scale problems. These findings clarify the trade-offs among augmented Lagrangian algorithms, highlighting Bertsekas’ method as the most effective for distributed optimization and providing guidance for algorithm selection in large-scale machine learning applications.

This article presents a numerical study of two augmented Lagrangian algorithms to solve continuous constrained global optimization problems. The algorithms approximately solve a sequence of bound constrained subproblems whose objective function penalizes equality and inequality constraints violation and depends on the Lagrange multiplier vectors and a penalty parameter. Each subproblem is solved by a population-based method that uses an electromagnetism-like (EM) mechanism to move points towards optimality. Three local search procedures are tested to enhance the EM algorithm. Benchmark problems are solved in a performance evaluation of the proposed augmented Lagrangian methodologies. A comparison with other techniques presented in the literature is also reported.

B034 PARALLEL WELL LOCATION OPTIMIZATION USING STOCHASTIC ALGORITHMS ON THE GRID COMPUTATIONAL FRAMEWORK 1 Hector Klie 1 Wolfgang Bangerth 1 2 Mary F. Wheeler 1 Manish Parashar 3 and Vincent Matossian 3 1 Center for Subsurface Modeling The University of Texas at Austin Austin TX. 2 Institute for Geophysics The University of Texas at Austin Austin TX. 3 The Applied Software Systems Laboratory Rutgers University Piscataway NJ. Abstract The determination of optimal well locations is a challenging problem in oil production since it depends on geological and fluid properties as well as on economic parameters. This work addresses the efficient

We introduce a twice differentiable augmented Lagrangian for nonlinear optimization with general inequality constraints and show that a strict local minimizer of the original problem is an approximate strict local solution of the augmented Lagrangian. A novel augmented Lagrangian method of multipliers (ALM) is then presented. Our method is originated from a generalization of the Hestenes-Powell augmented Lagrangian, and is a combination of the augmented Lagrangian and the interior-point technique. It shares a similar algorithmic framework with existing ALMs for optimization with inequality constraints, but it can use the second derivatives and does not depend on projections on the set of inequality constraints. In each iteration, our method solves a twice continuously differentiable unconstrained optimization subproblem on primal variables. The dual iterates, penalty and smoothing parameters are updated adaptively. The global and local convergence are analyzed. Without assuming any constraint qualification, it is proved that the proposed method has strong global convergence. The method may converge to either a Karush-Kuhn-Tucker (KKT) point or a singular stationary point when the converging point is a minimizer. It may also converge to an infeasible stationary point of nonlinear program when the problem is infeasible. Furthermore, our method is capable of rapidly detecting the possible infeasibility of the solved problem. Under suitable conditions, it is locally linearly convergent to the KKT point, which is consistent with ALMs for optimization with equality constraints. The preliminary numerical experiments on some small benchmark test problems demonstrate our theoretical results.

Optimization methods that employ the classical Powell-Hestenes-Rockafellar augmented Lagrangian are useful tools for solving nonlinear programming problems. Their reputation decreased in the last 10 years due to the comparative success of interior-point Newtonian algorithms, which are asymptotically faster. In this research, a combination of both approaches is evaluated. The idea is to produce a competitive method, being more robust and efficient than its ‘pure’ counterparts for critical problems. Moreover, an additional hybrid algorithm is defined, in which the interior-point method is replaced by the Newtonian resolution of a Karush-Kuhn-Tucker (KKT) system identified by the augmented Lagrangian algorithm. The software used in this work is freely available through the Tango Project web page:http://www.ime.usp.br/∼egbirgin/tango/

In this paper, we present complexity certification results for a distributed augmented Lagrangian (AL) algorithm used to solve convex optimization problems involving globally coupled linear constraints. Our method relies on the accelerated distributed AL (ADAL) algorithm, which can handle the coupled linear constraints in a distributed manner based on local estimates of the AL. We show that the theoretical complexity of ADAL to reach an $\epsilon$ -optimal solution both in terms of suboptimality and infeasibility is $O(\frac{1}{\epsilon })$ iterations. Moreover, we provide a valid upper bound for the optimal dual multiplier, which enables us to explicitly specify these complexity bounds. We also show how to choose the step-size parameter to minimize the bounds on the convergence rates. Finally, we discuss a motivating example, a model predictive control problem, involving a finite number of subsystems, which interact with each other via a general network.

An augmented Lagrangian formulation is proposed for large‐slip frictionless contact problems between deformable discretized bodies in two dimensions. Starting from a finite element discretization of the two bodies, a node‐on‐facet element is defined. A non‐linear gap vector and its first variation are derived in terms of the nodal displacements. The relevant action and reaction principle is stated. The gap distance is then related to the conjugate pressure by a (multivalued non‐differentiable) unilateral contact law. The resulting inequality constrained minimization problem is transformed into an unconstrained saddle point problem using an augmented Lagrangian function. Large slip over several facets is possible and the effects of target convexity or concavity are investigated. A generalized Newton method is used to solve the resulting piecewise differentiable equations necessary for equilibrium and contact. The proper tangent (Jacobian) matrices are calculated. The primal (displacements) and dual (contact forces) unknowns are simultaneously updated at each iteration.

An Augmented Lagrangian algorithm that uses Gauss-Newton approximations of the Hessian at each inner iteration is introduced and tested using a family of Hard-Spheres problems. The Gauss-Newton model convexifies the quadratic approximations of the Augmented Lagrangian function thus increasing the efficiency of the iterative quadratic solver. The resulting method is considerably more efficient than the corresponding algorithm that uses true Hessians. A comparative study using the well-known package LANCELOT is presented.

In this paper, we introduce two primal-dual active-set methods for solving large-scale constrained optimization problems. The first method minimizes a sequence of primal-dual augmented Lagrangian functions subject to bounds on the primal variables and artificial bounds on the dual variables. The basic structure is similar to the well-known optimization package Lancelot (Conn, et al. in SIAM J Numer Anal 28:545---572, 1991), which uses the traditional primal augmented Lagrangian function. Like Lancelot, our algorithm may use gradient projection-based methods enhanced by subspace acceleration techniques to solve each subproblem and therefore may be implemented matrix-free. The artificial bounds on the dual variables are a unique feature of our method and serve as a form of dual regularization. Our second algorithm is a two-phase method. The first phase computes iterates using our primal-dual augmented Lagrangian algorithm, which benefits from using cheap gradient projections and matrix-free linear CG calculations. The final iterate produced during this phase is then used as input for phase two, which is a stabilized sequential quadratic programming method (Gill and Robinson in SIAM J Opt 1---45, 2013). Obtaining superlinear local convergence under weak assumptions is an important benefit of the transition to a stabilized sequential quadratic programming algorithm. Interestingly, the bound-constrained subproblem used in phase one is equivalent to the stabilized subproblem used in phase two under certain assumptions. This fact makes our choice of algorithms a natural one.

Arriving at optimal signal timing parameters to improve the efficiency of traffic flow has been one of the major challenges faced by traffic engineers. The choice of a robust optimization framework and an accurate traffic model plays a significant role in determining optimal signal timing parameters. Though traffic flow is intuitively stochastic, few studies incorporate stochasticity in their optimization framework for traffic signal design. This study proposes two simulation-based stochastic optimization algorithms—an evolutionary algorithm-based framework and a simultaneous perturbation stochastic approximation (SPSA) algorithm-based framework for the optimal signal design of an isolated intersection using a calibrated microsimulation environment with reasonable accuracy. A software-in-the loop approach is used to control the traffic signals in the microsimulation environment. SPSA is a gradient descent algorithm with a powerful approach for approximating the gradient with just two function evaluations per gradient approximation. To evaluate the performance of the two frameworks, the study optimizes the signal timings for a case study on an isolated intersection in an urban arterial in Chennai. On comparing the two algorithms, it is found that SPSA performed better and took 100 function evaluations less than that taken by GA. A better (near optimal) initial solution is found to yield a faster rate of convergence for both algorithms. As the proposed optimization framework incorporates the stochastic nature of traffic in the optimization algorithm, it can accommodate the temporal variations in traffic and thereby provide traffic engineers a robust signal control strategy for improving the efficiency of traffic flow.

Exact augmented lagrangian function for nonlinear programming problems with inequality constraints