On a Reformulation of Mathematical Programs with Cardinality Constraints

We consider a class of mathematical programs with complementarity constraints (MPCC) where the objective function involves a non-Lipschitz sparsity-inducing term. Due to the existence of the non-Lipschitz term, existing constraint qualifications for locally Lipschitz MPCC cannot ensure that necessary optimality conditions hold at a local minimizer. In this paper, we present necessary optimality conditions and MPCC-tailored qualifications for the non-Lipschitz MPCC. The proposed qualifications are related to the constraints and the non-Lipschitz term, which ensure that local minimizers satisfy these necessary optimality conditions. Moreover, we present an approximation method for solving the non-Lipschitz MPCC and establish its convergence. Finally, we use numerical examples of sparse solutions of linear complementarity problems and the second-best road pricing problem in transportation science to illustrate the effectiveness of our approximation method for solving the non-Lipschitz MPCC.

In this article, a smooth mathematical program with complementarity constraints (MPCC) is reformulated as a non-smooth constrained optimization problem by using the Fischer-Burmeister function. A lower-order penalty method is applied to transform the resulted constrained optimization problem into unconstrained optimization problems. Lower-order penalty functions may not be locally Lipschitz. However, they require weaker conditions to guarantee an exact penalization property than the classical l 1 penalty functions. We derive optimality conditions for the penalty problems using a smooth approximate variational principle, and establish that the limit point of a sequence of points that satisfy the second-order necessary optimality conditions of penalty problems is a strongly stationary point (hence a B-stationary point) of the original MPCC if the limit point is feasible to MPCC, and a linear independence constraint qualification for MPCC and an upper level strict complementarity condition hold at the limit point. Furthermore, the limit point also satisfies a second-order necessary condition of MPCC. Numerical examples are presented to demonstrate and compare the effectiveness of the proposed methods.

We aim to compute lifted stationary points of a sparse optimization problem ( $$P_{0}$$ ) with complementarity constraints. We define a continuous relaxation problem ( $$R_{\nu }$$ ) that has the same global minimizers and optimal value with problem ( $$P_{0}$$ ). Problem ( $$R_{\nu }$$ ) is a mathematical program with complementarity constraints (MPCC) and a difference-of-convex objective function. We define MPCC lifted-stationarity of ( $$R_{\nu }$$ ) and show that it is weaker than directional stationarity, but stronger than Clarke stationarity for local optimality. Moreover, we propose an approximation method to solve ( $$R_{\nu }$$ ) and an augmented Lagrangian method to solve its subproblem ( $$R_{\nu ,\sigma }$$ ), which relaxes the equality constraint in ( $$R_{\nu }$$ ) with a tolerance $$\sigma $$ . We prove the convergence of our algorithm to an MPCC lifted-stationary point of problem ( $$R_{\nu }$$ ) and use a sparse optimization problem with vertical linear complementarity constraints to demonstrate the efficiency of our algorithm on finding sparse solutions in practice.

This thesis aims at developing efficient mathematical methods for solving hierarchical dynamic optimization problems. The main motivation is to model processes in nature, for which there is evidence to assume that they run optimally. We describe models of such processes by optimal control problems (called optimal control models (OCMs)). However, an OCM typically includes unknown parameters that cannot be derived entirely on a theoretical basis, which is in particular the case for the cost function. Therefore, we develop parameter estimation techniques to estimate the unknowns in an OCM from observation data of the process. Mathematically, this leads to a hierarchical dynamic optimization problem with a parameter estimation problem on the upper level and an optimal control problem on the lower level. We focus on multi-stage equality and inequality constrained optimal control problems based on nonlinear ordinary differential equations. The main goal of this thesis is to derive numerically efficient mathematical methods for solving hierarchical dynamic optimization problems, and to use these methods to estimate parameters in high-dimensional OCMs from real-world measurement data. We develop parameter-dependent OCMs for the gait of cerebral palsy patients and able-bodied subjects. The unknown parameters in the OCMs are then estimated from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg by using the mathematical methods developed within this work. The main novelties and contributions of this thesis to the field of hierarchical dynamic optimization are summarized herein. - We establish a novel mathematical method, a so-called direct all-at-once approach, for solving hierarchical dynamic optimization problems based on the direct multiple shooting method and first-order optimality conditions. - Furthermore, we propose an efficient numerical algorithm for large-scale hierarchical dynamic optimization problems, which fully exploits the structures inherited from both the hierarchical setting and the discretization. - Pontryagin's maximum principle is used to analyze solution properties of hierarchical dynamic optimization problems like second-order optimality conditions of the lower-level problem. - In addition, we propose and discuss alternative methods for hierarchical dynamic optimization that are based on derivative-free optimization and a bundle approach. These methods keep the hierarchical problem setting and do not reformulate the lower-level problem using first-order optimality conditions. - We establish a novel lifting method for regularizing mathematical programs with complementarity constraints, which is discussed and numerically investigated by means of a well-known collection of benchmark problems. - Proofs of regularity and convergence results for sequential quadratic programming methods applied to lifted mathematical programs with complementarity constraints are provided. - Efficient state-of-the-art implementations of all mathematical methods derived in this thesis, as well as a benchmark collection of hierarchical dynamic optimization problems are presented. - High-dimensional optimal control gait models for cerebral palsy patients and able-bodied subjects are developed. The mathematical methods derived in this thesis are used to estimate the unknown model parameters from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg. The theoretical and practical results presented in this thesis can be considered an initial motivating step towards answering open questions in current medical research in fields like treatment planning, classification of gaits, or the evaluation of surgeries by means of hierarchical dynamic optimization.

In this paper we study constraint qualifications and optimality conditions for bilevel programming problems. We strive to derive checkable constraint qualifications in terms of problem data and applicable optimality conditions. For the bilevel program with convex lower level program we discuss drawbacks of reformulating a bilevel programming problem by the mathematical program with complementarity constraints and present a new sharp necessary optimality condition for the reformulation by the mathematical program with a generalized equation constraint. For the bilevel program with a nonconvex lower level program we propose a relaxed constant positive linear dependence (RCPLD) condition for the combined program.

Recently, nonconvex and nonsmooth models such as those using ‘norm’ have drawn much attention in the area of image restoration. This work investigates the local and global minimizers of the gradient regularized model with box constraints. There are four major ingredients. Firstly, we show that the set of local minimizers can be represented by solutions to some quadratic problems, which are independent of the fidelity parameter α. Based on this, every point satisfying the first-order necessary condition is a local minimizer. Secondly, any two local minimizers have different energy values under certain assumptions, implying the uniqueness of the global minimizer. Thirdly, there exists a uniform lower bound for nonzero gradients of the restored images. Finally, we show that the global minimizer set is piecewise constant in terms of α, and when A is of full column rank and α is large enough, the distance between the true image and the restored images is bounded by the noise level. The numerical examples perfectly demonstrate our theoretical analysis.

This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints are nonconvex and possibly complicated, but allow for a fast computation of projections onto this nonconvex set. Typical problem classes which satisfy this requirement are optimization problems with disjunctive constraints (like complementarity or cardinality constraints) as well as optimization problems over sets of matrices which have to satisfy additional rank constraints. The key idea behind our method is to keep these complicated constraints explicitly in the constraints and to penalize only the remaining constraints by an augmented Lagrangian function. The resulting subproblems are then solved with the aid of a problem-tailored nonmonotone projected gradient method. The corresponding convergence theory allows for an inexact solution of these subproblems. Nevertheless, the overall algorithm computes so-called Mordukhovich-stationary points of the original problem under a mild asymptotic regularity condition, which is generally weaker than most of the respective available problem-tailored constraint qualifications. Extensive numerical experiments addressing complementarity- and cardinality-constrained optimization problems as well as a semidefinite reformulation of MAXCUT problems visualize the power of our approach.

Given a suitably parametrized family of equilibrium models and a higher level criterion by which to measure an equilibrium state, mathematical programs with equilibrium constraints (MPECs) provide a framework for improving or optimizing the equilibrium state. An example is toll design in traffic networks, which attempts to reduce total travel time by choosing which arcs to toll and what toll levels to impose. Here, a Wardrop equilibrium describes the traffic response to each toll design. Communication networks also have a deep literature on equilibrium flows that suggest some MPECs. We focus on mathematical programs with complementarity constraints (MPCCs), a subclass of MPECs for which the lower level equilibrium system can be formulated as a complementarity problem and therefore, importantly, as a nonlinear program (NLP). Although MPECs and MPCCs are typically non-convex, which is a consequence of the upper level objective clashing with the users' objectives in the lower level equilibrium program, the last decade of research has paved the way for finding local solutions of MPCCs via standard NLP techniques.

MEMPSODE: A global optimization software based on hybridization of population-based algorithms and local searches

In this paper, optimization problems P with complementarity constraints are considered. Characterizations for local minimizers \bar{x} of P of Orders 1 and 2 are presented. We analyze a parametric smoothing approach for solving these programs in which P is replaced by a perturbed problem P_{\tau} depending on a (small) parameter \tau. We are interested in the convergence behavior of the feasible set \cal{F}_{\tau} and the convergence of the solutions \bar{x}_{\tau} of P_{\tau} for \tau\to 0. In particular, it is shown that, under generic assumptions, the solutions \bar{x}_{\tau} are unique and converge to a solution \bar{x} of P with a rate \cal{O}(\sqrt{\tau}). Moreover, the convergence for the Hausdorff distance d(\cal{F}_{\tau}, \cal{F}) between the feasible sets of P_{\tau} and P is of order \cal{O}(\sqrt{\tau}).

The multiobjective bilevel program is a sequence of two optimization problems, with the upper-level problem being multiobjective and the constraint region of the upper level problem being determined implicitly by the solution set to the lower-level problem. In the case where the Karush-Kuhn-Tucker (KKT) condition is necessary and sufficient for global optimality of all lower-level problems near the optimal solution, we present various optimality conditions by replacing the lower-level problem with its KKT conditions. For the general multiobjective bilevel problem, we derive necessary optimality conditions by considering a combined problem, with both the value function and the KKT condition of the lower-level problem involved in the constraints. Most results of this paper are new, even for the case of a single-objective bilevel program, the case of a mathematical program with complementarity constraints, and the case of a multiobjective optimization problem.

Continuous network design problem (CNDP) is searching for a transportation network configuration to minimize the sum of the total system travel time and the investment cost of link capacity expansions by considering that the travellers follow a traditional Wardrop user equilibrium (UE) to choose their routes. In this paper, the CNDP model can be formulated as mathematical programs with complementarity constraints (MPCC) by describing UE as a non-linear complementarity problem (NCP). To address the difficulty resulting from complementarity constraints in MPCC, they are substituted by the Fischer-Burmeister (FB) function, which can be smoothed by the introduction of the smoothing parameter. Therefore, the MPCC can be transformed into a well-behaved non-linear program (NLP) by replacing the complementarity constraints with a smooth equation. Consequently, the solver such as LINDOGLOBAL in GAMS can be used to solve the smooth approximate NLP to obtain the solution to MPCC for modelling CNDP. The numerical experiments on the example from the literature demonstrate that the proposed algorithm is feasible.

For a long time, the bilevel programming problem has essentially been considered as a special case of mathematical programs with equilibrium constraints, in particular when the so-called KKT reformulation is in question. Recently though, this widespread believe was shown to be false in general. In this paper, other aspects of the difference between both problems are revealed as we consider the KKT approach for the nonsmooth bilevel program. It turns out that the new inclusion (constraint) which appears as a consequence of the partial subdifferential of the lower-level Lagrangian (PSLLL) places the KKT reformulation of the nonsmooth bilevel program in a new class of mathematical program with both set-valued and complementarity constraints. While highlighting some new features of this problem, we attempt here to establish close links with the standard optimistic bilevel program. Moreover, we discuss possible natural extensions for C-, M-, and S-stationarity concepts. Most of the results rely on a coderivati...

Using standard nonlinear programming (NLP) theory, we establish formulas for first and second order directional derivatives of optimal value functions of parametric mathematical programs with complementarity constraints (MPCCs). The main point is that under a linear independence condition on the active constraint gradients, optimal value sensitivity of MPCCs is essentially the same as for nonlinear programs, in spite of the combinatorial nature of the MPCC feasible set. Unlike NLP however, second order directional derivatives of the MPCC optimal value function show combinatorial structure.

We consider the general nonlinear optimization problem where the objective function has an additional term defined by the ℓ 0 -quasi-norm in order to promote sparsity of a solution. This problem is highly difficult due to its nonconvexity and discontinuity. We generalize some recent work and present a whole class of reformulations of this problem consisting of smooth nonlinear programmes. This reformulated problem is shown to be equivalent to the original ℓ 0 -sparse optimization problem both in terms of local and global minima. The reformulation contains a complementarity constraint, and exploiting the particular structure of this reformulated problem, we introduce several problem-tailored constraint qualifications, first- and second-order optimality conditions and develop an exact penalty-type method which is shown to work extremely well on a whole bunch of different applications.