The Dual Algorithm Based on a Class of Nonlinear Lagrangians for Nonlinear Programming

A novel nonlinear Lagrangian is presented for constrained optimization problems with both inequality and equality constraints, which is nonlinear with respect to both functions in problem and Lagrange multipliers. The nonlinear Lagrangian inherits the smoothness of the objective and constraint functions and has positive properties. The algorithm on the nonlinear Lagrangian is demonstrated to possess local and linear convergence when the penalty parameter is less than a threshold (the penalty parameter in the penalty method has to approximate zero) under a set of suitable conditions, and be super-linearly convergent when the penalty parameter is decreased following Lagrange multiplier update. Furthermore, the dual problem based on the nonlinear Lagrangian is discussed and some important properties are proposed, which fail to hold for the dual problem based on the classical Lagrangian. At last, the preliminary and comparing numerical results for several typical test problems by using the new nonlinear Lagrangian algorithm and the other two related nonlinear Lagrangian algorithms, are reported, which show that the given nonlinear Lagrangian is promising.

This paper establishes a theory framework of a class of nonlinear Lagrangians for solving nonlinear programming problems with inequality constraints. A set of conditions are proposed to guarantee the convergence of nonlinear Lagrangian algorithms, to analyze condition numbers of nonlinear Lagrangian Hessians as well as to develop the dual approaches. These conditions are satisfied by well-known nonlinear Lagrangians appearing in literature. The convergence theorem shows that the dual algorithm based on any nonlinear Lagrangian in the class is locally convergent when the penalty parameter is less than a threshold under a set of suitable conditions on problem functions and the error bound solution, depending on the penalty parameter, is also established. The paper also develops the dual problems based on the proposed nonlinear Lagrangians, and the related duality theorem and saddle point theorem are demonstrated. Furthermore, it is shown that the condition numbers of Lagrangian Hessians at optimal solutions are proportional to the controlling penalty parameters. We report some numerical results obtained by using nonlinear Lagrangians.

A nonlinear augmented Lagrangian for constrained minimax problems

We present a nonlinear Lagrangian method for nonconvex semidefinite programming. This nonlinear Lagrangian is generated by a Lowner operator associated with Log-Sigmoid function. Under a set of assumptions, we prove a convergence theorem, which shows that the nonlinear Lagrangian algorithm is locally convergent when the penalty parameter is less than a threshold and the error bound of the solution is proportional to the penalty parameter.

A new exact penalty method for semi-infinite programming problems

The modified barrier function approach for large-scale optimization

Stable isotope-assisted metabolic flux analysis (MFA) is a powerful method to estimate carbon flow and partitioning in metabolic networks. At its core, MFA is a parameter estimation problem wherein the fluxes and metabolite pool sizes are model parameters that are estimated, via optimization, to account for measurements of steady-state or isotopically-nonstationary isotope labeling patterns. As MFA problems advance in scale, they require efficient computational methods for fast and robust convergence. The structure of the MFA problem enables it to be cast as an equality-constrained nonlinear program (NLP), where the equality constraints are constructed from the MFA model equations, and the objective function is defined as the sum of squared residuals (SSR) between the model predictions and a set of labeling measurements. This NLP can be solved by using an algebraic modeling language (AML) that offers state-of-the-art optimization solvers for robust parameter estimation and superior scalability to large networks. When implemented in this manner, the optimization is performed with no distinction between state variables and model parameters. During each iteration of such an optimization, the system state is updated instead of being calculated explicitly from scratch, and this occurs concurrently with improvement in the model parameter estimates. This optimization approach starkly contrasts with traditional “shooting” methods where the state variables and model parameters are kept distinct and the system state is computed afresh during each iteration of a stepwise optimization. Our NLP formulation uses the MFA modeling framework of Wiechert et al. [1], which is amenable to incorporation of the model equations into an NLP. The NLP constraints consist of balances on either elementary metabolite units (EMUs) or cumomers. In this formulation, both the steady-state and isotopically-nonstationary MFA (inst-MFA) problems may be solved as an NLP. For the inst-MFA case, the ordinary differential equation (ODE) system describing the labeling dynamics is transcribed into a system of algebraic constraints for the NLP using collocation. This large-scale NLP may be solved efficiently using an NLP solver implemented on an AML. In our implementation, we used the reduced gradient solver CONOPT, implemented in the General Algebraic Modeling System (GAMS). The NLP framework is particularly advantageous for inst-MFA, scaling well to large networks with many free parameters, and having more robust convergence properties compared to the shooting methods that compute the system state and sensitivities at each iteration. Additionally, this NLP approach supports the use of tandem-MS data for both steady-state and inst-MFA when the cumomer framework is used. We assembled a software, eiFlux, written in Python and GAMS that uses the NLP approach and supports both steady-state and inst-MFA. We demonstrate the effectiveness of the NLP formulation on several examples, including a genome-scale inst-MFA model, to highlight the scalability and robustness of this approach. In addition to typical inst-MFA applications, we expect that this framework and our associated software, eiFlux, will be particularly useful for applying inst-MFA to complex MFA models, such as those developed for eukaryotes (e.g. algae) and co-cultures with multiple cell types.

In this paper, the classical exact absolute value penalty function method is used for solving a new class of nonconvex nonsmooth optimization problems. Nonconvex nondifferentiable optimization problems with both inequality and equality constraints are considered here, in which not all functions constituting them have the fundamental property of convex functions and most classes of generalized convex functions—namely, a stationary point of such a function is its global minimum. It is proved for such nonconvex optimization problems that there exists a finite threshold of penalty parameters equal to the largest absolute value of a Lagrange multiplier such that, for every penalty parameter exceeding this lower bound, there is the equivalence between an optimal solution in the original constrained minimization problem with ‐invex functions and a minimizer in its associated penalized optimization problem with the exact l1 penalty function. Further, under nondifferentiable ‐invexity assumptions, a characterization of a saddle point of the Lagrange function, defined for the considered constrained optimization problem in terms of minimizers of its associated exact penalized optimization problem with the exact l1 penalty function, is presented.

One class of effective methods for the optimization problem with inequality constraints are to transform the problem to a unconstrained optimization problem by constructing a smooth potential function. In this paper, we modifies a dual algorithm for constrained optimization problems and establishes a corresponding improved dual algorithm; It is proved that the improved dual algorithm has the local Q-superlinear convergence; Finally, we performed numerical experimentation using the improved dual algorithm for many constrained optimization problems, the numerical results are reported to show that it is valid in practical computation.

Inspired by the Lagrangian multiplier method with quadratic penalty function, which is widely used in Nonlinear Programming Theory, a Lagrange-type nonlinear programming neural network whose equilibria coincide with KKT pairs of the underlying nonlinear programming problem was devised with minor modification in regard to handling inequality constraints[1,2]. Of course, the structure of neural network must be elaborately conceived so that it is asymptotically stable. Normally this aim is not easy to be achieved even for the simple nonlinear programming problems. However, if the penalty parameters in these neural networks are taken as control variables and a control law is found to stabilize it, we may reasonably conjecture that the categories of solvable nonlinear programming problems will be greatly increased. In this paper, the conditions stabilizing the Lagrange-type neural network are presented and control-Lyapunov function approach is used to synthesize the adjusting laws of penalty parameters.KeywordsPenalty ParameterNonlinear Programming ProblemGlobal Asymptotic StabilityFeedback StabilizationHierarchical OptimizationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Nonlinear optimization with equation constraints has wide applications in intelligent control systems, acoustic signal processing, etc. Thus, effectively tackling the nonlinear optimization problems with equation constraints is of great significance for the advancement of these fields. Current discrete-time neurodynamics predominantly addresses unperturbed optimization scenarios, exhibiting inherent sensitivity to external noise, which limits the practical application of these methods. To address this issue, we propose a discrete-time noise-suppressed neurodynamics (DTNSN) model in this paper. First, the model integrates the static optimization stability of the gradient-based neurodynamics (GND) model with the time-varying tracking capability of the zeroing neurodynamics (ZND) model. Then, an integral feedback term is introduced to suppress external noise disturbances, thereby enhancing the robustness of the model. Additionally, to facilitate implementation on digital hardware, we employ an explicit linear three-step discretization method to obtain the proposed DTNSN model. Finally, the convergence performance, noise suppression capability, and practicality of the model are validated through theoretical analysis, numerical simulations, and acoustic source localization experiments. The model is applicable to the fields of intelligent control systems, acoustic signal processing, and industrial automation, providing new tools for real-time optimization in noisy environments.

Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.

A new nonmonotone line-search procedure is presented for the generally constrained case of nonlinear programming problems. The new algorithm is based on the use of standard penalty methods for the definition of merit functions used during line search to find the next iterate in algorithms generating a search direction iteratively. The key concept is the discretization of the penalty parameter used over a finite range of orders of magnitude and the provision of a memory list for each such order, as in standard nonmonotone line-search procedures used for unconstrained optimization. Nonmonotonicity helps in escaping from local minima, while the discretized penalty parameters overcome the difficulties in choosing a penalty parameter that varies, but having the same definition as the problem while not underpenalizing the constraints to arrive at the desired KKT point. An implementation within a customized logarithmic barrier algorithm for bounds' handling is presented with capabilities for very large scale applications; the algorithm uses exact first and second derivative information, derived symbolically, and the search direction is generated by solution of the Lagrange−Newton equations. The case studies presented demonstrate the capabilities of the new line-search procedure, and comparisons with other methods are discussed. It is noted that we found a significantly better solution in case study 5. The new nonmonotone line-search procedure is, at present, a heuristic and from the computational point of view: future work will focus on the investigation of both the theoretical properties of the method and new implementation aspects.

On an exact penalty function method for nonlinear mixed discrete programming problems and its applications in search engine advertising problems

In this paper we extend the theory of exact penalty functions for nonlinear programs whose objective functions and equality and inequality constraints are locally Lipschitz; arbitrary simple constraints are also allowed. Assuming a weak stability condition, we show that for all sufficiently large penalty parameter values an isolated local minimum of the nonlinear program is also an isolated local minimum of the exact penalty function. A tight lower bound on the parameter value is provided when certain first order sufficiency conditions are satisfied. We apply these results to unify and extend some results for convex programming. Since several effective algorithms for solving nonlinear programs with differentiable functions rely on exact penalty functions, our results provide a framework for extending these algorithms to problems with locally Lipschitz functions.