A CLASS OF NONLINEAR LAGRANGIANS: THEORY AND ALGORITHM

A novel nonlinear Lagrangian is presented for constrained optimization problems with both inequality and equality constraints, which is nonlinear with respect to both functions in problem and Lagrange multipliers. The nonlinear Lagrangian inherits the smoothness of the objective and constraint functions and has positive properties. The algorithm on the nonlinear Lagrangian is demonstrated to possess local and linear convergence when the penalty parameter is less than a threshold (the penalty parameter in the penalty method has to approximate zero) under a set of suitable conditions, and be super-linearly convergent when the penalty parameter is decreased following Lagrange multiplier update. Furthermore, the dual problem based on the nonlinear Lagrangian is discussed and some important properties are proposed, which fail to hold for the dual problem based on the classical Lagrangian. At last, the preliminary and comparing numerical results for several typical test problems by using the new nonlinear Lagrangian algorithm and the other two related nonlinear Lagrangian algorithms, are reported, which show that the given nonlinear Lagrangian is promising.

This paper studies the properties of a class of nonlinear Lagrangians for nonlinear programming with inequality constraints. It's shown that under a set of conditions this class of Lagrange algorithm is locally convergent when the penalty parameter is larger than a threshold. An error bound estimate of the solution, depending on the penalty, is also established. The paper also discusses the properties of the dual function associated with the proposed nonlinear Lagrangians. Finally, the dual algorithm corresponding to the proposed nonlinear Lagrangians is developed and used to solve some numerical examples by using the nonlinear Lagrangians in the literature. Numerical results suggest that the dual algorithm is effective for solving nonlinear programming.

We present a nonlinear Lagrangian method for nonconvex semidefinite programming. This nonlinear Lagrangian is generated by a Lowner operator associated with Log-Sigmoid function. Under a set of assumptions, we prove a convergence theorem, which shows that the nonlinear Lagrangian algorithm is locally convergent when the penalty parameter is less than a threshold and the error bound of the solution is proportional to the penalty parameter.

A nonlinear augmented Lagrangian for constrained minimax problems

A neural network for solving convex nonlinear programming problems is proposed in this paper. The distinguishing features of the proposed network are that the primal and dual problems can be solved simultaneously, all necessary and sufficient optimality conditions are incorporated, and no penalty parameter is involved. Based on Lyapunov, LaSalle and set stability theories, we prove strictly an important theoretical result that, for an arbitrary initial point, the trajectory of the proposed network does converge to the set of its equilibrium points, regardless of whether a convex nonlinear programming problem has unique or infinitely many optimal solutions. Numerical simulation results also show that the proposed network is feasible and efficient. In addition, a general method for transforming nonlinear programming problems into unconstrained problems is also proposed.

In this paper, the classical exact absolute value penalty function method is used for solving a new class of nonconvex nonsmooth optimization problems. Nonconvex nondifferentiable optimization problems with both inequality and equality constraints are considered here, in which not all functions constituting them have the fundamental property of convex functions and most classes of generalized convex functions—namely, a stationary point of such a function is its global minimum. It is proved for such nonconvex optimization problems that there exists a finite threshold of penalty parameters equal to the largest absolute value of a Lagrange multiplier such that, for every penalty parameter exceeding this lower bound, there is the equivalence between an optimal solution in the original constrained minimization problem with ‐invex functions and a minimizer in its associated penalized optimization problem with the exact l1 penalty function. Further, under nondifferentiable ‐invexity assumptions, a characterization of a saddle point of the Lagrange function, defined for the considered constrained optimization problem in terms of minimizers of its associated exact penalized optimization problem with the exact l1 penalty function, is presented.

A new exact penalty method for semi-infinite programming problems

A penalty adapting ant algorithm is presented in an attempt to eliminate the dependency of ant algorithms on the penalty parameter used for the solution of constrained optimization problems. The method uses an adapting mechanism for determination of the penalty parameter leading to elimination of the costly process of penalty parameter tuning. The method is devised on the basis of observation that for large penalty parameters, infeasible solutions will have a higher total cost than feasible solutions and vice versa. The method therefore uses the best feasible and infeasible solution costs of the iteration to adaptively adjust the penalty parameter to be used in the next iteration. The pheromone updating procedure of the max–min ant system is also modified to keep ants on and around the boundary of the feasible search space where quality solutions can be found. The sensitivity of the proposed method to the initial value of the penalty parameter is investigated and indicates that the method converges to optimal or near-optimal solutions irrespective of the initial starting value of the penalty parameter. This is significant as it eliminates the need for sensitivity analysis of the method with respect to the penalty factor, thus adding to the computational efficiency of ant algorithms. Furthermore, it is shown that the success rate of the search algorithm in locating an optimal solution is increased when a self-adapting mechanism is used. The presented method is applied to a benchmark pipe network optimization problem in the literature and the results are presented and compared with those of existing algorithms.

A newsboy problem model in which a vendor has limited resource is developed. It is assumed that the supplier will either sell the items to the vendor outright or offer the items to the vendor with return policy. In the latter case, the supplier buys back at certain percentage of original cost from the vendor the unsold items at the end of the selling season. The purpose of this study is to investigate how the vendor should replenish the items with return policy, constrained resources and changing procured price. Three numerical examples are provided. In one example with two constrained variables, an optimal solution is derived by using KKT (Karush-Kuhn-Tucker) conditions. The other two multiple variables examples with a minimum service level or a limited budget are solved using GA (genetic algorithm).

Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.

Inspired by the Lagrangian multiplier method with quadratic penalty function, which is widely used in Nonlinear Programming Theory, a Lagrange-type nonlinear programming neural network whose equilibria coincide with KKT pairs of the underlying nonlinear programming problem was devised with minor modification in regard to handling inequality constraints[1,2]. Of course, the structure of neural network must be elaborately conceived so that it is asymptotically stable. Normally this aim is not easy to be achieved even for the simple nonlinear programming problems. However, if the penalty parameters in these neural networks are taken as control variables and a control law is found to stabilize it, we may reasonably conjecture that the categories of solvable nonlinear programming problems will be greatly increased. In this paper, the conditions stabilizing the Lagrange-type neural network are presented and control-Lyapunov function approach is used to synthesize the adjusting laws of penalty parameters.KeywordsPenalty ParameterNonlinear Programming ProblemGlobal Asymptotic StabilityFeedback StabilizationHierarchical OptimizationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Augmented Lagrangian methods are effective tools for solving large-scale nonlinear programming problems. At each outer iteration, a minimization subproblem with simple constraints, whose objective function depends on updated Lagrange multipliers and penalty parameters, is approximately solved. When the penalty parameter becomes very large, solving the subproblem becomes difficult; therefore, the effectiveness of this approach is associated with the boundedness of the penalty parameters. In this paper, it is proved that under more natural assumptions than the ones employed until now, penalty parameters are bounded. For proving the new boundedness result, the original algorithm has been slightly modified. Numerical consequences of the modifications are discussed and computational experiments are presented.

A more general parallel dual-type method and application to state estimation

In this paper, a family of trust-region interior-point sequential quadratic programming (SQP) algorithms for the solution of a class of minimization problems with nonlinear equality constraints and simple bounds on some of the variables is described and analyzed. Such nonlinear programs arise, e.g., from the discretization of optimal control problems. The algorithms treat states and controls as independent variables. They are designed to take advantage of the structure of the problem. In particular they do not rely on matrix factorizations of the linearized constraints but use solutions of the linearized state equation and the adjoint equation. They are well suited for large scale problems arising from optimal control problems governed by partial differential equations. The algorithms keep strict feasibility with respect to the bound constraints by using an affine scaling method proposed, for a different class of problems, by Coleman and Li [SIAM J. Optim., 6 (1996), pp. 418--445] and they exploit trust-region techniques for equality-constrained optimization. Thus, they allow the computation of the steps using a variety of methods, including many iterative techniques. Global convergence of these algorithms to a first-order Karush--Kuhn--Tucker (KKT) limit point is proved under very mild conditions on the trial steps. Under reasonable, but more stringent, conditions on the quadratic model and on the trial steps, the sequence of iterates generated by the algorithms is shown to have a limit point satisfying the second-order necessary KKT conditions. The local rate of convergence to a nondegenerate strict local minimizer is q-quadratic. The results given here include, as special cases, current results for only equality constraints and for only simple bounds. Numerical results for the solution of an optimal control problem governed by a nonlinear heat equation are reported.

Purpose:To propose a new parallel‐friendly optimization algorithm, based on alternating direction method of multipliers (ADMM), for solving optimization problems emerging in treatment planning and imagingMethods:ADMM, emerging as a powerful tool for distributed optimization, is employed here for IMRT treatment planning. We modify the existing ADMM by integrating that with two optimization techniques known as Barzilai‐Borwein gradient method and line search. We apply original and integrated ADMM, with various penalty parameter values, on three IMRT treatment planning cases and we compare their performance. Each algorithm terminates once it found a solution within the relative error of 1E‐4 from the optimal solution. We also compare the performance of original and integrated ADMM against the commercial optimization software CPLEX.Results:Our experiments on three different cancer sites (prostate, head and neck, and GYN) indicate that integrated ADMM is much faster than original ADMM (5.8 times faster on average). Moreover, while the performance of original ADMM heavily depends on the selected penalty parameter, integrated ADMM performs very robust against the penalty parameter. Compared to the commercial software CPLEX, integrated ADMM turns out to be around 4–7 times faster.Conclusion:Integrated ADMM is a very efficient optimization algorithm to cope with the large‐scale optimization problems. It is a parallel‐friendly optimization algorithm and provides an idea platform for cloud‐based treatment planning and imaging.