An exact penalty approach for general ℓ0-sparse optimization problems

We aim to compute lifted stationary points of a sparse optimization problem ( $$P_{0}$$ ) with complementarity constraints. We define a continuous relaxation problem ( $$R_{\nu }$$ ) that has the same global minimizers and optimal value with problem ( $$P_{0}$$ ). Problem ( $$R_{\nu }$$ ) is a mathematical program with complementarity constraints (MPCC) and a difference-of-convex objective function. We define MPCC lifted-stationarity of ( $$R_{\nu }$$ ) and show that it is weaker than directional stationarity, but stronger than Clarke stationarity for local optimality. Moreover, we propose an approximation method to solve ( $$R_{\nu }$$ ) and an augmented Lagrangian method to solve its subproblem ( $$R_{\nu ,\sigma }$$ ), which relaxes the equality constraint in ( $$R_{\nu }$$ ) with a tolerance $$\sigma $$ . We prove the convergence of our algorithm to an MPCC lifted-stationary point of problem ( $$R_{\nu }$$ ) and use a sparse optimization problem with vertical linear complementarity constraints to demonstrate the efficiency of our algorithm on finding sparse solutions in practice.

A Cluster-Based Competitive Particle Swarm Optimizer with a Sparse Truncation Operator for Multi-Objective Optimization

The weight optimization of reinforced concrete (RC) beams with material nonlinear response is formulated as a general nonlinear optimization problem. Incremental finite element procedures are used to integrate the structural response analysis and design sensitivity analysis in a consistent manner. In the finite element discretization, the concrete is modelled by plane stress elements and steel reinforcement is modelled by discrete truss elements. The cross-sectional areas of the steel and the thickness of the concrete are chosen as design variables, and design constraints can include the displacement, stress and sizing constraints. The objective function is the weight of the RC beams. The optimal design is performed by using the sequential linear programming algorithm for the changing process of design variables, and the gradient projection method for the calculations of the search direction. Three example problems are considered. The first two are demonstrated to show the stability and accuracy of the approaches by comparing previous results for truss and plane stress elements, separately. The last one is an example of an RC beam. Comparative cost objective functions are presented to prove the validity of the approach.

A model for optimal operation of water supply/irrigation systems of various water quality sources, with treatment plants, multiple water quality conservative factors, and dilution junctions is presented. The objective function includes water cost at the sources, water conveyance costs which account for the hydraulics of the network indirectly, water treatment cost, and yield reduction costs of irrigated crops due to irrigation with poor quality water. The model can be used for systems with supply by canals as well as pipes, which serve both drinking water demands of urban/rural consumers and field irrigation requirements. The general nonlinear optimization problem has been simplified by decomposing it to a problem with linear constraints and nonlinear objective function. This problem is solved using the projected gradient method. The method is demonstrated for a regional water supply system in southern Israel that contains 39 pipes, 37 nodes, 11 sources, 10 agricultural consumers, and 4 domestic consumers. The optimal operation solution is described by discharge and salinity values for all pipes of the network. Sensitivity of the optimal solution to changes in the parameters is examined. The solution was found to be sensitive to the upper limit on drinking water quality, with total cost being reduced by 5% as the upper limit increases from 260 to 600 mg Cl l−1. The effect of income from unit crop yield is more pronounced. An increase of income by a factor of 20 results in an increase of the total cost by a factor of 3, thus encouraging more use of fresh water as long as the marginal cost of water supply is smaller than the marginal decrease in yield loss. The effect of conveyance cost becomes more pronounced as its cost increases. An increase by a factor of 100 results in an increase of the total cost by about 14%. The network studied has a long pipe that connects two distinct parts of the network and permits the supply of fresh water from one part to the other. Increasing the maximum permitted discharge in this pipe from 0 to 200 m3 h−1 reduces the total cost by 11%. Increasing the maximum discharge at one of the sources from 90 to 300 m3 h−1 reduces the total cost by about 8%.

In this paper we present a Lagrange-multiplier formulation of discrete constrained optimization problems, the associated discrete-space first-order necessary and sufficient conditions for saddle points, and an efficient first-order search procedure that looks for saddle points in discrete space. Our new theory provides a strong mathematical foundation for solving general nonlinear discrete optimization problems. Specifically, we propose a new vector-based definition of descent directions in discrete space and show that the new definition does not obey the rules of calculus in continuous space. Starting from the concept of saddle points and using only vector calculus, we then prove the discrete-space first-order necessary and sufficient conditions for saddle points. Using well-defined transformations on the constraint functions, we further prove that the set of discrete-space saddle points is the same as the set of constrained local minima, leading to the first-order necessary and sufficient conditions for constrained local minima. Based on the first-order conditions, we propose a local-search method to look for saddle points that satisfy the first-order conditions.KeywordsLocal MinimumLagrange MultiplierSaddle PointLagrangian MethodFeasible PointThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An efficient method based on the sequential quadratic programming (SQP) algorithm for the linear antenna arrays pattern synthesis with prescribed nulls in the interference direction and minimum side lobe levels by the complex weights of each array element is presented. In general, the pattern synthesis technique that generates a desired pattern is a greatly nonlinear optimization problem. SQP method is a versatile method to solve the general nonlinear constrained optimization problems and is much simpler to implement. It transforms the nonlinear minimization problem to a sequence of quadratic subproblem that is easier to solve, based on a quadratic approximation of the Lagrangian function. Several numerical results of Chebyshev pattern with the imposed single, multiple, and broad nulls sectors are provided and compared with published results to illustrate the performance of the proposed method. © 2007 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2007.

We consider the general nonlinear optimization problem in 0-1 variables and provide an explicit equivalent positive semidefinite program in 2n-1 variables. The optimal values of both problems are identical. From every optimal solution of the former, one easily finds an optimal solution of the latter, and conversely, from every solution of the latter, one may construct an optimal solution of the former. For illustration, the equivalent positive semidefinite program is explicated for quadratic 0-1 programs and MAX-CUT in $\mathbb{R}^3$. For unconstrained 0-1 programs, a special representation in terms of a weighted sum of squares is provided.

A parallelized DPPQN based Expert System method and implementation

We present a collection of computer programs for the solution of linear and nonlinear semidefinite optimization problems. After briefly discussing the underlying algorithm, the generalized augmented Lagrangian method, we describe details of the specific programs for linear, bilinear and general nonlinear semidefinite optimization problems. For each of the programs we present typical application areas and examples.KeywordsPennonAugmented LagrangianCholesky MethodHessian-vector ProductsLinear Semidefinite Programming Problem (LSDP)These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this chapter we present an important theoretical development (Liu and Floudas, 1993) that allows us to extend the GOP global optimization approach from biconvex and bilinear problems into general constrained twice-differentiable nonlinear optimization problems. This theoretical development provides the link between the GOP algorithm and the αBB global optimization approach, which is our focus on Part III of this book. Section 10.1 presents the theoretical developments that lead from the GOP to the αBB global optimization approach. Section 10.2 presents a brief introduction to the calculation of the a parameters and one alternative for obtaining rigorous bounds on the a parameters for general constrained twice differentiable nonlinear optimization problems. The material presented in this chapter is based on the work of Liu and Floudas (1993) and Maranas and Floudas (1994b).

In this paper, we study complexity results of sparse optimization problems and reverse convex optimization problems. These problems are very important subjects of optimization problems. We prove that the complexity result of the sparsity constraint problem and sparse solution problem are all NP-hard in the strong sense and even testing feasibility of the sparsity constraint is NP-complete in the strong sense. Then the sparse optimization problem is NP-hard in the strong sense. We also prove that the reverse convex problem is NP-hard in the strong sense by transforming the sparsity constraint into a reverse convex constraint.

In this paper, a new sequential penalty algorithm, based on the Linfin exact penalty function, is proposed for a general nonlinear constrained optimization problem. The algorithm has the following characteristics: it can start from an arbitrary initial point; the feasibility of the subproblem is guaranteed; the penalty parameter is adjusted automatically; global convergence without any regularity assumption is proved. The update formula of the penalty parameter is new. It is proved that the algorithm proposed in this paper behaves equivalently to the standard SQP method after sufficiently many iterations. Hence, the local convergence results of the standard SQP method can be applied to this algorithm. Preliminary numerical experiments show the efficiency and stability of the algorithm.

This paper focuses on second-order necessary optimality conditions for constrained optimization problems on Banach spaces. For problems in the classical setting, where the objective function is \(C^2\)-smooth, we show that strengthened second-order necessary optimality conditions are valid if the constraint set is generalized polyhedral convex. For problems in a new setting, where the objective function is just assumed to be \(C^1\)-smooth and the constraint set is generalized polyhedral convex, we establish sharp second-order necessary optimality conditions based on the Fréchet second-order subdifferential of the objective function and the second-order tangent set to the constraint set. Three examples are given to show that the used hypotheses are essential for the new theorems. Our second-order necessary optimality conditions refine and extend several existing results.

In this article, a smooth mathematical program with complementarity constraints (MPCC) is reformulated as a non-smooth constrained optimization problem by using the Fischer-Burmeister function. A lower-order penalty method is applied to transform the resulted constrained optimization problem into unconstrained optimization problems. Lower-order penalty functions may not be locally Lipschitz. However, they require weaker conditions to guarantee an exact penalization property than the classical l 1 penalty functions. We derive optimality conditions for the penalty problems using a smooth approximate variational principle, and establish that the limit point of a sequence of points that satisfy the second-order necessary optimality conditions of penalty problems is a strongly stationary point (hence a B-stationary point) of the original MPCC if the limit point is feasible to MPCC, and a linear independence constraint qualification for MPCC and an upper level strict complementarity condition hold at the limit point. Furthermore, the limit point also satisfies a second-order necessary condition of MPCC. Numerical examples are presented to demonstrate and compare the effectiveness of the proposed methods.

We discuss possible scenarios of behaviour of the dual part of sequences generated by primal-dual Newton-type methods when applied to optimization problems with nonunique multipliers associated to a solution. Those scenarios are: (a) failure of convergence of the dual sequence; (b) convergence to a so-called critical multiplier (which, in particular, violates some second-order sufficient conditions for optimality), the latter appearing to be a typical scenario when critical multipliers exist; (c) convergence to a noncritical multiplier. The case of mathematical programs with complementarity constraints is also discussed. We illustrate those scenarios with examples, and discuss consequences for the speed of convergence. We also put together a collection of examples of optimization problems with constraints violating some standard constraint qualifications, intended for preliminary testing of existing algorithms on degenerate problems, or for developing special new algorithms designed to deal with constraints degeneracy.