A Gauss–Newton Approach for Solving Constrained Optimization Problems Using Differentiable Exact Penalties

It was earlier demonstrated, by the so-called main (or simplest) problem of the Calculus of Variations, that the Theory of Exact Penalties allows one not only to derive fundamental results of the Calculus of Variations but also to construct new direct numerical methods for solving variational problems based on the notions of subgradient and hypogradient of the exact penalty function (which is essentially nonsmooth even if all initial data are smooth). In this article Exact Penalties are used to solve isoperimetric problems of the Calculus of Variations. New direct numerical methods are described (e.g. the method of hypodifferential descent). Several numerical examples are discussed.

In this paper, the classical exact absolute value penalty function method is used for solving a new class of nonconvex nonsmooth optimization problems. Nonconvex nondifferentiable optimization problems with both inequality and equality constraints are considered here, in which not all functions constituting them have the fundamental property of convex functions and most classes of generalized convex functions—namely, a stationary point of such a function is its global minimum. It is proved for such nonconvex optimization problems that there exists a finite threshold of penalty parameters equal to the largest absolute value of a Lagrange multiplier such that, for every penalty parameter exceeding this lower bound, there is the equivalence between an optimal solution in the original constrained minimization problem with ‐invex functions and a minimizer in its associated penalized optimization problem with the exact l1 penalty function. Further, under nondifferentiable ‐invexity assumptions, a characterization of a saddle point of the Lagrange function, defined for the considered constrained optimization problem in terms of minimizers of its associated exact penalized optimization problem with the exact l1 penalty function, is presented.

On an exact penalty function method for nonlinear mixed discrete programming problems and its applications in search engine advertising problems

In this paper, we study a new exact and smooth penalty function forsemi-infinite programming problems with continuous inequalityconstraints. Through this exact penalty function, we can transform asemi-infinite programming problem into an unconstrained optimizationproblem. We find that, under some reasonable conditions when thepenalty parameter is sufficiently large, the local minimizer of thispenalty function is the local minimizer of the primal problem.Moreover, under some mild assumptions, the local exactness propertyis explored. The numerical results demonstrate that it is aneffective and promising approach for solving constrainedsemi-infinite programming problems.

<p style='text-indent:20px;'>In this work, an exact objective penalty function and an exact objective filled penalty function are proposed to solve constrained optimization problems. There are two main innovations of these two functions: using the exact objective penalty function to find a locally optimal point; a better locally optimal point than the current one can be found by exact objective filled penalty function. A local search method and a global search method for constrained optimization problems are proposed. We can use this new global search method to compute an approximately globally optimal point for constrained optimization problems. The convergence of the sequences obtained by the new algorithms are also analyzed respectively. Finally, the rationality of these two search methods are illustrated by numerical experiments.</p>

ABSTRACTIn this article, the vector exact l1 penalty function method used for solving nonconvex nondifferentiable multiobjective programming problems is analyzed. In this method, the vector penalized optimization problem with the vector exact l1 penalty function is defined. Conditions are given guaranteeing the equivalence of the sets of (weak) Pareto optimal solutions of the considered nondifferentiable multiobjective programming problem and of the associated vector penalized optimization problem with the vector exact l1 penalty function. This equivalence is established for nondifferentiable invex vector optimization problems. Some examples of vector optimization problems are presented to illustrate the results established in the article.

In this chapter, an implementation of an interior point filter line-search algorithm for large-scale nonlinear programming proposed by Wachter and Biegler (2005a, b) is presented. As we know, to allow convergence from poor starting points and to enforce progress to the solution, interior point methods both in trust region and in line-search frameworks with exact penalty merit function have been developed. For example, KNITRO uses the l 1 exact penalty function (Byrd, Hribar, & Nocedal, 1999; Byrd, Gilbert, & Nocedal, 2000). On the other hand, Fletcher and Leyffer (2002) proposed filter methods as an alternative to merit functions, as a tool for global convergence guarantee in algorithms for nonlinear optimization. The idea of filter is that trial points are accepted if they improve the objective function value or improve the constraint violation instead of a combination of these two measures defined by the merit function. Even if the filter methods include different heuristics, they have been adapted to barrier methods in a number of ways. For example, Ulbrich et al. (2004) considered a trust-region filter method and accept the trial step on the basis of the norm of the optimality conditions. Benson et al. (2002a) proposed several heuristics using the idea of filter methods, for which improved efficiency is reported compared to their previous merit function approach. The global convergence of an interior point algorithm with a filter line search was analyzed by Wachter and Biegler (2001). Here, the assumptions made for the analysis of the global convergence are less restrictive than those made for line-search interior point methods for nonlinear programming developed, for example, by El-Bakry et al. (1996), Yamashita (1998), or Tits et al. (2003).

For inequality constrained minimization problem, we first propose a new exact nonsmooth objective penalty function and then apply a smooth technique to the penalty function to make it smooth. It is shown that any minimizer of the smoothing objective penalty function is an approximated solution of the original problem. Based on this, we develop a solution method for the inequality constrained minimization problem and prove its global convergence. Numerical experiments are provided to show the efficiency of the proposed method.

We study theoretical and computational aspects of an exact penalization approach to mathematical programs with equilibrium constraints (MPECs). In the first part, we prove that a Mangasarian--Fromovitz-type condition ensures the existence of a stable local error bound at the root of a real-valued nonnegative piecewise smooth function. A specification to nonsmooth formulations of equilibrium constraints, e.g., complementarity conditions or normal equations, provides conditions which guarantee the existence of a nonsmooth exact penalty function for MPECs. In the second part, we study a trust region minimization method for a class of composite nonsmooth functions which comprises exact penalty functions arising from MPECs. We prove a global convergence result for the general method and incorporate a penalty update rule. A further specification results in an SQP trust region method for MPECs based on an \(\ell_1\) penalty function.

An implementable algorithm for solving nonsmooth nonconvex constrained optimization is proposed by combining bundle ideas, proximity control, and the exact penalty function. We construct two kinds of approximations to nonconvex objective function; these two approximations correspond to the convex and concave behaviors of the objective function at the current point, which captures precisely the characteristic of the objective function. The penalty coefficients are increased only a finite number of times under the conditions of Slater constraint qualification and the boundedness of the constrained set, which limit the unnecessary penalty growth. The given algorithm converges to an approximate stationary point of the exact penalty function for constrained nonconvex optimization with weakly semismooth objective function. We also provide the results of some preliminary numerical testing to show the validity and efficiency of the proposed method.

In this paper, we will study optimality conditions of semi-infinite programs and generalized semi-infinite programs by employing lower order exact penalty functions and the condition that the generalized second-order directional derivative of the constraint function at the candidate point along any feasible direction for the linearized constraint set is non-positive. We consider three types of penalty functions for semi-infinite program and investigate the relationship among the exactness of these penalty functions. We employ lower order integral exact penalty functions and the second-order generalized derivative of the constraint function to establish optimality conditions for semi-infinite programs. We adopt the exact penalty function technique in terms of a classical augmented Lagrangian function for the lower-level problems of generalized semi-infinite programs to transform them into standard semi-infinite programs and then apply our results for semi-infinite programs to derive the optimality condition for generalized semi-infinite programs. We will give various examples to illustrate our results and assumptions.

In this paper we extend the theory of exact penalty functions for nonlinear programs whose objective functions and equality and inequality constraints are locally Lipschitz; arbitrary simple constraints are also allowed. Assuming a weak stability condition, we show that for all sufficiently large penalty parameter values an isolated local minimum of the nonlinear program is also an isolated local minimum of the exact penalty function. A tight lower bound on the parameter value is provided when certain first order sufficiency conditions are satisfied. We apply these results to unify and extend some results for convex programming. Since several effective algorithms for solving nonlinear programs with differentiable functions rely on exact penalty functions, our results provide a framework for extending these algorithms to problems with locally Lipschitz functions.

In this paper, we consider a general class of nonlinear mixed discrete programming problems. By introducing continuous variables to replace the discrete variables, the problem is first transformed into an equivalent nonlinear continuous optimization problem subject to original constraints and additional linear and quadratic constraints. Then, an exact penalty function is employed to construct a sequence of unconstrained optimization problems, each of which can be solved effectively by unconstrained optimization techniques, such as conjugate gradient or quasi-Newton methods. It is shown that any local optimal solution of the unconstrained optimization problem is a local optimal solution of the transformed nonlinear constrained continuous optimization problem when the penalty parameter is sufficiently large. Numerical experiments are carried out to test the efficiency of the proposed method.

In this paper we consider inequality constrained nonlinear optimization problems where the first order derivatives of the objective function and the constraints cannot be used. Our starting point is the possibility to transform the original constrained problem into an unconstrained or linearly constrained minimization of a nonsmooth exact penalty function. This approach shows two main difficulties: the first one is the nonsmoothness of this class of exact penalty functions which may cause derivative-free codes to converge to nonstationary points of the problem; the second one is the fact that the equivalence between stationary points of the constrained problem and those of the exact penalty function can only be stated when the penalty parameter is smaller than a threshold value which is not known a priori. In this paper we propose a derivative-free algorithm which overcomes the preceding difficulties and produces a sequence of points that admits a subsequence converging to a Karush–Kuhn–Tucker point of the constrained problem. In particular the proposed algorithm is based on a smoothing of the nondifferentiable exact penalty function and includes an updating rule which, after at most a finite number of updates, is able to determine a “right value” for the penalty parameter. Furthermore we present the results obtained on a real world problem concerning the estimation of parameters in an insulin-glucose model of the human body.

In this paper, a modified simple penalty function is proposed for a constrained nonlinear programming problem by augmenting the dimension of the program with a variable that controls the weight of the penalty terms. This penalty function enjoys improved smoothness. Under mild conditions, it can be proved to be exact in the sense that local minimizers of the original constrained problem are precisely the local minimizers of the associated penalty problem. MSC:47H20, 35K55, 90C30.