• Home
  • Search
  • Solution of Large-scale Structured Optimization Problems with Schur-complement and Augmented Lagrangian Decomposition Methods
  • Cite Icon1
  • https://doi.org/10.25394/pgs.8210243.v1Copy DOI Icon

Solution of Large-scale Structured Optimization Problems with Schur-complement and Augmented Lagrangian Decomposition Methods

Show More
  • Abstract
  • Literature Map
  • Citations
  • Similar Papers
Abstract

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

Similar Papers
  • Research Article
  • Citations10

Scalable preconditioning of block-structured linear algebra systems using ADMM

  • Jun 18, 2019
  • Computers & Chemical Engineering
  • Jose S Rodriguez +2
  • Research Article
  • Citations30

On the Asymptotic Linear Convergence Speed of Anderson Acceleration Applied to ADMM

  • Jun 25, 2021
  • Journal of Scientific Computing
  • Dawei Wang +2
  • Conference Article
  • Citations5

A comparison study of LDPC decoding using accelerated ADMM and over-relaxed ADMM

  • Oct 01, 2016
  • Xiaopeng Jiao +2
  • Research Article

Parallel and distributed Machine Learning on Augmented Lagrangian Algorithms

  • Oct 22, 2025
  • International Journal of Electronics and Telecommunications
  • Anthony Nwachukwu +1
  • Research Article
  • Citations272

Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMM

  • Feb 01, 2017
  • IEEE Transactions on Automatic Control
  • Pontus Giselsson +1
  • Research Article
  • Citations6

Investigating Customization Strategies and Convergence Behaviors of Task-Specific ADMM.

  • Jan 01, 2021
  • IEEE Transactions on Image Processing
  • Risheng Liu +2
  • Research Article
  • Citations1

SU‐F‐BRB‐11: An Integrated Alternating Direction Method of Multipliers for Treatment Planning Optimization

  • Jun 01, 2015
  • Medical Physics
  • M Zarepisheh +2
  • Conference Article
  • Citations3

Graph-aware Weighted Hybrid ADMM for Fast Decentralized Optimization

  • Oct 01, 2018
  • Meng Ma +1
  • Research Article
  • Citations217

An Augmented Lagrangian Based Algorithm for Distributed NonConvex Optimization

  • Jan 01, 2016
  • SIAM Journal on Optimization
  • Boris Houska +2
  • Conference Article
  • Citations4

Fast Classification Algorithms via Distributed Accelerated Alternating Direction Method of Multipliers

  • Nov 01, 2019
  • Huihui Wang +3
  • Conference Article
  • Citations7

Constrained quadratic optimization for radiation treatment planning by use of graph form ADMM

  • Jul 01, 2016
  • Xinmin Liu +3
  • Conference Article
  • Citations30

ADMM-based distributed model predictive control: Primal and dual approaches

  • Dec 01, 2017
  • Ramin Rostami +2
  • Research Article

On the \(O(1/K)\) convergence rate of alternating direction method of multipliers in a complex domain

  • Oct 07, 2018
  • ANZIAM Journal
  • Lu Li +2
  • Research Article
  • Citations6

Lowering the error floor of ADMM penalized decoder for LDPC codes

  • Aug 01, 2016
  • China Communications
  • Xiaopeng Jiao +1
  • Book Chapter
  • Citations6

Fast Communication Structure for Asynchronous Distributed ADMM Under Unbalance Process Arrival Pattern

  • Jan 01, 2018
  • Shuqing Wang +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.