On Decomposition Methods for a Class of Partially Separable Nonlinear Programs

Multidisciplinary design optimization (MDO) problems are engineering design problems that require the consideration of the interaction between several design disciplines. Due to the organizational aspects of MDO problems, decomposition algorithms are often the only feasible solution approach. Decomposition algorithms reformulate the MDO problem as a set of independent subproblems, one per discipline, and a coordinating master problem. A popular approach to MDO problems is bilevel decomposition algorithms. These algorithms use nonlinear optimization techniques to solve both the master problem and the subproblems. In this paper, we propose two new bilevel decomposition algorithms and analyze their properties. In particular, we show that the proposed problem formulations are mathematically equivalent to the original problem and that the proposed algorithms converge locally at a superlinear rate. Our computational experiments illustrate the numerical performance of the algorithms.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

This article describes an approach for the dynamic optimal power flow problem using interior point method (IPM) and Benders decomposition considering both active and reactive constraints. An efficient predictor-corrector primal-dual interior-point algorithm is used to solve the linearized dynamic OPF problem. The inclusion of active and reactive security constraints will assure a subsequent feasible solution for the dynamic OPF problem. The dynamic OPF problem is decomposed into a master problem and subproblems for checking the feasibility of the active and reactive constraints. The master problem is formulated and solved without active and reactive constraints to avoid complexity. The obtained dynamic OPF schedule from the master problem is applied to the active and reactive subproblems to minimize the violations. In case the current mix of scheduled units cannot remove the violations, additional constraints will be introduced in the master problem for rescheduling dynamic OPF. The proposed approach has been evaluated on an IEEE 118-bus test system. The results obtained with the proposed approach are presented and compared favorably with results of other stochastic techniques.

Collaborative robots are progressively employed to support human workers in assembly tasks or independently assigned to one station to complete assembly tasks in U-shaped assembly lines. This study develops two exact methods, based on constraint programming (CP) model and Combinatorial Benders’ decomposition algorithm (CBDA), to tackle the U-shaped assembly line balancing problem with collaborative robots. The CBDA is based on distinct decomposition strategies that address the master problem and the subproblem, formulated by a mixed integer linear programming (MILP) model and a CP model, which is designed to obtain a real objective value and generate cuts that refine the master problem. Additionally, MILP-based and CP-based local search techniques are proposed to improve the algorithm's performance. The computational study demonstrates that both the proposed methods, namely CP and CBDA, outperform the current state-of-the-art methods (the MILP model and simulated annealing algorithm). Furthermore, the CP outperforms the CBDA in terms of the number of optimal solutions, achieving the optimal solutions for 574 out of 1500 test instances, and might be regarded as the new state-of-the-art methodology.

We consider a single product maritime inventory routing problem in which the production and consumption rates are constant over the planning horizon. The problem involves a heterogeneous fleet and multiple production and consumption ports with limited storage capacity. Maritime transportation is characterized by high levels of uncertainty, and sailing times can be severely influenced by varying and unpredictable weather conditions. To deal with the uncertainty, this paper investigates the use of adaptable robust optimization where the sailing times are assumed to belong to the well-known budget polytope uncertainty set. In the recourse model, the routing, the order of port visits, and the quantities to load and unload are fixed before the uncertainty is revealed, while the visit time to ports and the stock levels can be adjusted to the scenario. We propose a decomposition algorithm that iterates between a master problem that considers a subset of scenarios and an adversarial separation problem that searches for scenarios that make the solution from the master problem infeasible. Several improvement strategies are proposed aiming at reducing the running time of the master problem and reducing the number of iterations of the decomposition algorithm. An iterated local search heuristic is also introduced to improve the decomposition algorithm. A computational study is reported based on a set of real instances.

Interior point methods in the year 2025

In this chapter, we discuss a data-sparse approximation to the Schur complement matrix defined on polygonal domains in the case of Laplace equation. First, we introduce the matrix compression techniques in the case of canonical (rectangular) boundaries. It is based on the truncated block-Fourier representation to the Schur complement in 2D (the approach can be easily extended to the case of parallelepiped/cylinder type geometries in 3D). An arbitrary polygonal domain is then decomposed into rectangular and right triangular subdomains by the refined interface. Reducing the FE system of equations to the refined interface and solving the resultant interface equation by multilevel iterations, we arrive at the fast matrix-vector multiplication with the corresponding Schur complement matrix. This yields the linear-logarithmic complexity O(N log3 N) with respect to the number of boundary degrees of freedom N. An extension to the case of refined meshes and exterior domains will be also addressed.KeywordsNest SelectionSparse ApproximationExterior ProblemInterface EquationNodal Basis FunctionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In the preceding paper [24], a method is described for an explicit hierarchical (ℋ-matrix) approximation to the inverse of an elliptic differential operator with piecewise constant/smooth coefficients in ℝ d . In the present paper, we proceed with the ℋ-matrix approximation to the Green function. Here, it is represented by a sum of an ℋ-matrix and certain correction term including the product of data-sparse matrices of hierarchical formats based on the so-called boundary concentrated FEM [26]. In the case of jumping coefficients with respect to non-overlapping domain decomposition, the approximate inverse operator is obtained as a direct sum of local inverses over subdomains and the Schur complement inverse on the interface corresponding to the boundary concentrated FEM. Our Schur complement matrix provides the cheap spectrally equivalent preconditioner to the conventional interface operator arising in the iterative substructuring methods by piecewise linear finite elements.

In this work, large-scale structural optimization problems involving non-ordinal categorical design variables and continuous variables are investigated. The aim is to minimize the weight of a structure with respect to cross-section areas, with materials and stiffening principles selection. First, the problem is formulated using a bi-level decomposition involving master and slave problems. The master problem is given by a first-order-like approximation that helps to drastically reduce the combinatorial explosion raised by the categorical variables. Continuous variables are handled in a slave problem solved using a gradient-based approach, where the categorical variables are driven by the master problem. The proposed algorithm is tested on three different structural optimization test cases. A comparison to state-of-the-art algorithms emphasize efficiency of the proposed algorithm in terms of the optimum quality, the computation cost, and the scaling with respect to the problem dimension.

In most full-waveform inversion (FWI) problems, sufficient prior information is available to constrain the velocity of certain parts of the model, e.g., the water column or, in some cases, near-surface velocities. We take advantage of this situation and develop a fast Schur-complement-based forward modeling and inversion approach by partitioning the velocity model into two parts. The first part consists of the constrained zone that does not change during the inversion, whereas the second part is the anomalous zone to be updated during the inversion. For this decomposition, we partially factorize the governing system of linear equations by computing a Schur complement for the anomalous zone. The Schur complement system is then solved to compute the fields in the anomalous zone, which are then back substituted to compute the fields in the constrained region. For each successive modeling steps with new anomalous zone velocities, the corresponding Schur complement is easily computed using simple algebra. Because the anomalous part of the model is comparatively smaller than the whole model, considerable computational savings can be achieved using our Schur approach. Additionally, we showed that the Schur complement method maintains the accuracy of standard frequency-domain finite difference formulations, but this comes at a slightly higher peak memory requirement. Our FWI workflow shows reduced runtime by 15%–57% depending upon the depth of the water column without losing any accuracy compared to the standard method.

: In many situations it is advantageous to solve the dual of a linear program rather than the primal. For example, an approach to solve two stage stochastic linear programs with recourse employs Bender's decomposition (Dantzig and Glynn 1990). Within this algorithm, cuts (necessary conditions for the second stage costs) are added sequentially to the master problem. In the primal version of the master problem, cuts appear as new rows. In each iteration of Bender's decomposition algorithm, when a new row is added, the pervious solution of the master problem gets infeasible. However, when solving the dual of the master problem, a cut appears as a column. When adding a new column, the previous solution of the master problem remains feasible and the new optimal solution can be obtained within a few simplex iterations. This report describes the changes made by the author in the MINOS file MI35INPT, so MINOS could transform the given primal linear program, specified in an MPS file, to its dual, solve the dual and in addition, write another MPS file which contains the specification of the dual problem.

Two-stage recoverable robust optimization for an integrated location–allocation and evacuation planning problem

This paper presents an approach for maximizing a GENCO's profit in a constrained power market. The proposed approach considers the Interior Point Method (IPM) and Benders decomposition for solving the security-constrained optimal generation scheduling (SC-GS) problem. The master problem represents the economic dispatch problem for a GENCO which intends to optimize its profit. The formulation of the master problem does not bear any transmission network constraints. The subproblem will be used by the same GENCO to check the viability of its proposed bidding strategy in the presence of transmission network constraints. In this case if the subproblem does not yield a certain level of financial return for the GENCO or if the subproblem results in an infeasible solution of the GENCO's proposed bidding strategy, the GENCO will modify its proposed solution according to the Benders cuts that stem out of the subproblem. The study shows a more flexible scheduling paradigm for a GENCO in a competitive arena. The proposed approach proves practical for modeling the impact of transmission congestion on a GENCO's expected profit in a competitive environment.

Macro-elementwise preconditioning methods

Polygeneration, a multi-input multioutput energy conversion process which typically involves the coproduction of electricity and liquid synthetic fuels, is a promising technology which offers real potential toward the reduction of excessive energy consumption and consequent greenhouse gas emissions. The optimal design of such a complex and nonlinear process system under inevitable and unpredictable future uncertainty poses great challenges in terms of both modeling and corresponding solution strategies. In this paper, we propose a stochastic programming framework for the optimal design under uncertainty of polygeneration energy systems. On the basis of a detailed mixed-integer nonlinear programming (MINLP) model, proposed in our previous work, a two-stage stochastic programming problem is formulated, which is then converted into a large-scale multiperiod MINLP problem by employing cubature based integration and sampling techniques. A decomposition algorithm is utilized for the efficient solution of the multiperiod problem, which involves iterations between a set of nonlinear subproblems of much smaller size and a master mixed-integer linear programming problem. A case study is then presented, where detailed computational results and comparisons between optimal designs obtained for both the stochastic and deterministic cases are shown.