Numerical study of augmented Lagrangian algorithms for constrained global optimization

This paper presents the use of a constraint-handling technique, known as feasibility and dominance rules, in a electromagnetism-like (ELM) mechanism for solving constrained global optimization problems. Since the original ELM algorithm is specifically designed for solving bound constrained problems, only the inequality and equality constraints violation together with the objective function value are used to select points and to progress towards feasibility and optimality. Numerical experiments are presented, including a comparison with other methods recently reported in the literature.

In this paper, a convex filled function method for solving constrained global optimization problems is proposed. A convex filled function is proposed by combining the idea of filled function in unconstrained global optimization and the idea of penalty function in constrained optimization. Then a convex filled function method for obtaining a global minimizer or an approximate global minimizer of the constrained global optimization problem is presented. Some numerical results demonstrate the efficiency of this global optimization method for solving constrained global optimization problems.

Comparison of SQP and AL algorithms for deterministic constrained production optimization of hydrocarbon reservoirs

Constrained optimization is central to large-scale machine learning, particularly in parallel and distributed environments. This paper presents a comprehensive study of augmented Lagrangian–based algorithms for such problems, including classical Lagrangian relaxation, the method of multipliers, the Alternating Direction Method of Multipliers (ADMM), Bertsekas’ algorithm, Tatjewski’s method, and the Separable Augmented Lagrangian Algorithm (SALA). We develop a unified theoretical framework, analyze convergence properties and decomposition strategies, and evaluate these methods on two representative classes of tasks: regularized linear systems and K-means clustering. Numerical experiments on synthetic and real-world datasets show that Bertsekas’ method consistently achieves the best balance of convergence speed and solution quality, while ADMM offers practical scalability under decomposition but struggles in high-dimensional or ill-conditioned settings. Tatjewski’s method benefits significantly from partitioning, whereas the classical Augmented Lagrangian approach proves computationally inefficient for large-scale problems. These findings clarify the trade-offs among augmented Lagrangian algorithms, highlighting Bertsekas’ method as the most effective for distributed optimization and providing guidance for algorithm selection in large-scale machine learning applications.

Fuzzy programming offers a powerful means of handling optimization problems with fuzzy parameters. Fuzzy programming has been used in different ways in the past. The particle swarm optimization (PSO) has been applied successfully to continuous nonlinear constrained optimization problems, neural network, etc. But we have not been found to use PSO for fuzzy programming in literature. In this paper, we combined with fuzzy simulation, neural network and PSO to produce a hybrid intelligent algorithm. Based on this hybrid intelligent algorithm, we introduced for solving fuzzy expected value models. Some numerical examples are given to illustrate the algorithm is effective and powerful.

In this paper, we consider a general class of nonlinear mixed discrete programming problems. By introducing continuous variables to replace the discrete variables, the problem is first transformed into an equivalent nonlinear continuous optimization problem subject to original constraints and additional linear and quadratic constraints. Then, an exact penalty function is employed to construct a sequence of unconstrained optimization problems, each of which can be solved effectively by unconstrained optimization techniques, such as conjugate gradient or quasi-Newton methods. It is shown that any local optimal solution of the unconstrained optimization problem is a local optimal solution of the transformed nonlinear constrained continuous optimization problem when the penalty parameter is sufficiently large. Numerical experiments are carried out to test the efficiency of the proposed method.

This paper presents a new interval-based operator for continuous constrained global optimization. It is built upon a new filtering operator, named TEC, which constructs a bounded subtree using a Branch and Contract process and returns the parallel-to-axes hull of the leaf domains/boxes. Two extensions of TEC use the information contained in the leaf boxes of the TEC subtree to improve the two bounds of the objective function value: (i) for the lower bounding, a polyhedral hull of the (projected) leaf boxes replaces the parallel-to-axes hull, (ii) for the upper bounding, a good feasible point is searched for inside a leaf box of the TEC subtree, following a look-ahead principle. The algorithm proposed is an auto-adaptive version of TEC that plays with both extensions according to the outcomes of the upper and lower bounding phases. Experimental results show a significant speed-up on several state-of-the-art instances.

Estimation of Optimal Well Controls Using the Augmented Lagrangian Function with Approximate Derivatives

Clustering Search and Variable Mesh Algorithms for continuous optimization

Optimal location, number, and settings of unified power flow controllers (UPFC) using various multi-objective optimization algorithms is presented in this paper. The UPFC parameters, locations and number are computed to maximize the voltage stability margin and minimize the real power losses at the same time. For this, developed hierarchical optimization versions of three recent multi-objective algorithms are proposed namely: non-dominated genetic algorithms (NSGA-II), non-dominated sorting particle swarm optimization (NSPSO) and Strength Pareto Evolutionary Algorithm 2 (SPEA2). The fuzzy logic is proposed to extract the best compromise solution from the Pareto set. The proposed algorithms are applied to IEEE 30-bus power system. The line flow and load bus voltage limits are taken into account. The obtained results show that the installation of the UPFC in the power system minimizes the power losses, enhances the static voltage stability, and improves the voltage profiles. Furthermore, the proposed methods are able to solve a hard discrete---continuous constrained multi-objective optimization problem. In addition, they do not show any limitation on the number of objective functions under consideration.

In this study, we deal with the nonlinear constrained global optimization problems. First, we introduce a new smooth exact penalty function for constrained optimization problems. We combine the exact penalty function with the auxiliary function in regard to constrained global optimization. We present a new auxiliary function approach and the adapted algorithm for solving non-linear inequality constrained global optimization problems. Finally, we illustrate the efficiency of the algorithm on some numerical examples.

In this paper, we present complexity certification results for a distributed augmented Lagrangian (AL) algorithm used to solve convex optimization problems involving globally coupled linear constraints. Our method relies on the accelerated distributed AL (ADAL) algorithm, which can handle the coupled linear constraints in a distributed manner based on local estimates of the AL. We show that the theoretical complexity of ADAL to reach an $\epsilon$ -optimal solution both in terms of suboptimality and infeasibility is $O(\frac{1}{\epsilon })$ iterations. Moreover, we provide a valid upper bound for the optimal dual multiplier, which enables us to explicitly specify these complexity bounds. We also show how to choose the step-size parameter to minimize the bounds on the convergence rates. Finally, we discuss a motivating example, a model predictive control problem, involving a finite number of subsystems, which interact with each other via a general network.

Optimization methods that employ the classical Powell-Hestenes-Rockafellar augmented Lagrangian are useful tools for solving nonlinear programming problems. Their reputation decreased in the last 10 years due to the comparative success of interior-point Newtonian algorithms, which are asymptotically faster. In this research, a combination of both approaches is evaluated. The idea is to produce a competitive method, being more robust and efficient than its ‘pure’ counterparts for critical problems. Moreover, an additional hybrid algorithm is defined, in which the interior-point method is replaced by the Newtonian resolution of a Karush-Kuhn-Tucker (KKT) system identified by the augmented Lagrangian algorithm. The software used in this work is freely available through the Tango Project web page:http://www.ime.usp.br/∼egbirgin/tango/

The aim of this study to demonstrate that the previously developed ANGEL algorithm can be efficiently used for multiple constrained sizing-shaping truss optimization problems. The applied hybrid method ANGEL, which was originally developed for simple truss optimization problems combines ant colony optimization (ACO), genetic algorithm (GA), and local search strategy (LS). ACO and GA search alternately and cooperatively in the solution space. In ANGEL, the traditional stochastic mutation operator is replaced by the local search procedure as a deterministic counterpart of the stochastic mutation. The feasibility is measured by the maximal load intensity factor computed by a third order path-following method. The powerful LS algorithm, which is based on the local linearization of the set of the constraints and the objective function, is applied to yield a better feasible or less unfeasible solution when ACO or GA obtains a solution. In order to demonstrate the efficiency of ANGEL in the given application area, a well-known example is presented under multiple constraints.

Interior-point methods for the phase-field approach to brittle and ductile fracture