Learning Regionally Decentralized AC Optimal Power Flows With ADMM

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

In light of the soaring uptake of distributed energy resources, distributed methods are attracting increased focus. This paper proposes an adaptive scheme to improve the convergence of the alternating direction of multipliers method (ADMM) on the component-based dual decomposition of the AC optimal power flow (OPF). The method incorporates a local curvature approximation scheme, with underlying parameter update steps inspired by the local residual balancing scheme, to automatically tune the penalty parameters locally, without central oversight. Substantial speed-ups over the vanilla ADMM, as high as 10.6 times, are demonstrated on real-world test systems with more than 13 600 buses, as well as on other difficult test cases. Furthermore, scalability is assessed by showing that, unlike in the centralized computation, the distributed implementation of the proposed adaptive method is independent of system size.

The alternating direction method of multipliers (ADMM) with over-relaxation and the accelerated ADMM are two effective methods to improve the convergence rate of ADMM. In this paper, an efficient linear programming (LP) decoder based on accelerated ADMM for low-density parity-check (LDPC) codes is proposed. Then it is shown that the ADMM update steps for decoding LDPC codes using the accelerated ADMM and the over-relaxed ADMM are quite similar. Therefore, it is impossible to obtain a more efficient ADMM decoder by combining the two improved ADMM methods. Simulation results also validate that the average number of iterations for LDPC decoding using the accelerated ADMM and the over-relaxed ADMM are close to each other.

The alternating direction method of multipliers (ADMM) is an algorithm for solving large-scale data optimization problems in machine learning. In order to reduce the communication delay in a distributed environment, asynchronous distributed ADMM (AD-ADMM) was proposed. However, due to the unbalance process arrival pattern existing in the multiprocessor cluster, the communication of the star structure used in AD-ADMM is inefficient. Moreover, the load in the entire cluster is unbalanced, resulting in a decrease of the data processing capacity. This paper proposes a hierarchical parameter server communication structure (HPS) and an asynchronous distributed ADMM (HAD-ADMM). The algorithm mitigates the unbalanced arrival problem through process grouping and scattered updating global variable, which basically achieves load balancing. Experiments show that the HAD-ADMM is highly efficient in a large-scale distributed environment and has no significant impact on convergence.

Alternating Direction Method of Multiplier (ADMM) has been a popular algorithmic framework for separable optimization problems with linear constraints. For numerical ADMM fail to exploit the particular structure of the problem at hand nor the input data information, leveraging task-specific modules (e.g., neural networks and other data-driven architectures) to extend ADMM is a significant but challenging task. This work focuses on designing a flexible algorithmic framework to incorporate various task-specific modules (with no additional constraints) to improve the performance of ADMM in real-world applications. Specifically, we propose Guidance from Optimality (GO), a new customization strategy, to embed task-specific modules into ADMM (GO-ADMM). By introducing an optimality-based criterion to guide the propagation, GO-ADMM establishes an updating scheme agnostic to the choice of additional modules. The existing task-specific methods just plug their task-specific modules into the numerical iterations in a straightforward manner. Even with some restrictive constraints on the plug-in modules, they can only obtain some relatively weaker convergence properties for the resulted ADMM iterations. Fortunately, without any restrictions on the embedded modules, we prove the convergence of GO-ADMM regarding objective values and constraint violations, and derive the worst-case convergence rate measured by iteration complexity. Extensive experiments are conducted to verify the theoretical results and demonstrate the efficiency of GO-ADMM.

Convex optimization is at the core of many of today's analysis tools for large datasets, and in particular machine learning methods. This thesis will develop approximate versions of the alternating directrion of multipliers (ADMM) for the general setting of minimizing the sum of two convex functions. The alternating direction method of multipliers is a form of augmented Lagrangian algorithm that has experienced a renaissance in recent years due to its applicability to optimization problems arising from ``big data'' and image processing applications, and the relative ease with which it may be implemented in parallel and distributed computational environments. There are two fundamental approaches for proving the convergence of the ADMM, each based on a different form of two-way emph{splitting}, that is, expressing a mapping as the sum of two simpler mappings. The first approach is based on Douglas-Rachford operator splitting theory, and yields considerable insight into the convergence of the ADMM. The second convergence proof approach is at its core based on the Lagrangian splitting analysis. We present three new approximate versions of ADMM based on both convergence analyses, all of which require only knowledge of subgradients of the subproblem objectives, rather bounds on the distance to the exact subproblem solution. One version, which applies only to certain common special cases, is based on combining the operator splitting analysis of the ADMM with a relative-error proximal point algorithm of Solodov and Svaiter. A byproduct of this analysis is a new, relative-error version of the Douglas-Rachford splitting algorithm for monotone operators. The other two approximate versions of the ADMM are more general and based on the Lagrangian splitting analysis of the ADMM: one uses a summable absolute error criterion, and the other uses a relative error criterion and an auxiliary iterate sequence. We experimentally compare our new algorithms to an essentially exact form of the ADMM and to an inexact form that can be easily derived from prior theory (but again applies only to certain common special cases). These experiments show that our methods can significantly reduce total computational effort when iterative methods are used to solve ADMM subproblems.

A survey on applications of Alternating Direction Method of Multipliers in smart power grids

We discuss an online decentralized decision making problem where the agents are coupled with affine inequality constraints. Alternating Direction Method of Multipliers (ADMM) is used as the computation engine and we discuss the convergence of the algorithm in an online setting. To be specific, when decisions have to be made sequentially with a fixed time step, there might not be enough time for the ADMM to converge before the scenario changes and the decision needs to be updated. In this case, a suboptimal solution is employed and we analyze the optimality gap given the convergence condition. Moreover, in many cases, the decision making problem changes gradually over time. We propose a warm-start scheme to accelerate the convergence of ADMM and analyze the benefit of the warm-start. The proposed method is demonstrated in a decentralized multiagent control barrier function problem with simulation.

Purpose:To propose a new parallel‐friendly optimization algorithm, based on alternating direction method of multipliers (ADMM), for solving optimization problems emerging in treatment planning and imagingMethods:ADMM, emerging as a powerful tool for distributed optimization, is employed here for IMRT treatment planning. We modify the existing ADMM by integrating that with two optimization techniques known as Barzilai‐Borwein gradient method and line search. We apply original and integrated ADMM, with various penalty parameter values, on three IMRT treatment planning cases and we compare their performance. Each algorithm terminates once it found a solution within the relative error of 1E‐4 from the optimal solution. We also compare the performance of original and integrated ADMM against the commercial optimization software CPLEX.Results:Our experiments on three different cancer sites (prostate, head and neck, and GYN) indicate that integrated ADMM is much faster than original ADMM (5.8 times faster on average). Moreover, while the performance of original ADMM heavily depends on the selected penalty parameter, integrated ADMM performs very robust against the penalty parameter. Compared to the commercial software CPLEX, integrated ADMM turns out to be around 4–7 times faster.Conclusion:Integrated ADMM is a very efficient optimization algorithm to cope with the large‐scale optimization problems. It is a parallel‐friendly optimization algorithm and provides an idea platform for cloud‐based treatment planning and imaging.

Low-Rank Tensor Recovery (LRTR), the higher order generalization of Low-Rank Matrix Recovery (LRMR), is especially suitable for analyzing multi-linear data with gross corruptions, outliers and missing values, and it attracts broad attention in the fields of computer vision, machine learning and data mining. This paper considers a generalized model of LRTR and attempts to recover simultaneously the low-rank, the sparse, and the small disturbance components from partial entries of a given data tensor. Specifically, we first describe generalized LRTR as a tensor nuclear norm optimization problem that minimizes a weighted combination of the tensor nuclear norm, the l1-norm and the Frobenius norm under linear constraints. Then, the technique of Alternating Direction Method of Multipliers (ADMM) is employed to solve the proposed minimization problem. Next, we discuss the weak convergence of the proposed iterative algorithm. Finally, experimental results on synthetic and real-world datasets validate the efficiency and effectiveness of the proposed method.

Distributed optimization has gained popularity in many areas because it can to cope with the increasing volume of data, and the corresponding demand for computational resources. A popular distributed solver is the alternating direction method of multipliers (ADMM). Fully decentralized (D) ADMM arises when node-to-node communications have to abide by the underlying network connectivity. DADMM’s convergence however, slows down as the network diameter grows large. To deal with this challenge, the recently proposed hybrid (H) ADMM provides considerable performance boost over DADMM by exploiting local topology information. But HADMM only applies to unweighted graphs. The present contribution develops a weighted hybrid (WH) consensus-based ADMM approach that can deal with weighted graphs. The resultant scheme further improves the performance of HADMM through graph-aware weight tuning. Theoretical analysis offers convergence guarantees and establishes linear convergence rate, while numerical tests on various graphs demonstrate the WHADMM merits.

Data collection plays an important role in wireless sensor networks. Recovery of spatio-temporal data from incomplete sensing data is vital to the network lifetime. Many works have utilized the spatial and temporal correlations to achieve satisfactory data recovery results. However, these methods introduce large computational overhead at the fusion center. In this paper, we develop an ADMM-Net framework for correlated spatio-temporal data recovery. Both the spatial correlation and temporal correlation of sensing data are considered in a convex optimization problem, which is solved by the alternating direction method of multipliers (ADMM) algorithm. We then unfold the ADMM algorithm into a fixed-length neural network that reduces the iterations dramatically and does not require additional location information of nodes. Experimental results on a realworld dataset demonstrate that the proposed method can achieve faster convergence speed than the baseline ADMM algorithm with slight accuracy loss.

Distributed machine learning has gained lots of attention due to the rapid growth of data. In this paper, we focus regularized empirical risk minimization problems, and propose two novel Distributed Accelerated Alternating Direction Method of Multipliers (D-A2DM2) algorithms for distributed classification. Based on the framework of Alternating Direction Method of Multipliers (ADMM), we decentralize the distributed classification problem as a global consensus optimization problem with a series of sub-problems. In D-A2DM2, we exploit ADMM with variance reduction for sub-problem optimization in parallel. Taking global update and local update into consideration respectively, we propose two acceleration mechanisms in the framework of D-A2DM2. In particular, inspired by Nesterov's accelerated gradient descent, we utilize it for global update to further improve time efficiency. Moreover, we also introduce Nesterov's acceleration for local update, and develop the corrected local update and symmetric dual update to accelerate the convergence with only a little change in the computational effort. Theoretically, D-A2DM2 has a linear convergence rate. Empirically, experimental results show that D-A2DM2 converge faster than existing distributed ADMM-based classification, and could be a highly efficient algorithm for practical use.

The alternating direction method of multipliers (ADMM) is applied to constrained quadratic optimization of intensity-modulated radiation therapy (IMRT) planning. The proximal operator for the ADMM algorithm is developed, and the algorithm is applied to different clinical treatment sites (liver, prostate, and head & neck). The results are compared with the solutions obtained by several other commercial and non-commercial optimizers. The ADMM solver is about one to two orders of magnitude faster than the other solvers, moreover, it requires less computer memory.

Distributed model predictive control (DMPC) is a widely used approach especially for large-scale systems like transportation, traffic control or power distribution systems where solving the MPC problem in a centralized manner is not desirable or even not possible due to the size of the problem, maintenance issues, cabling burden, etc. Distributed optimization methods, which are being employed to solve the DMPC problems, normally suffer from having an enormous number of communications to reach a consensus between the local controllers. Therefore trying to find distributed optimization schemes that can realize an agreement among the local controllers with a lower communication load deserves attention. The alternating direction method of multipliers (ADMM) is a well known method to solve consensus problems. Since the local problems in ADMM can be handled with second-order methods, the convergence will be reached with a lower number of iterations in comparison to first-order methods, e.g. the fast gradient method. In this paper, ADMM is formulated and applied to a DMPC problem first in the primal domain. The main contribution of this paper is, however, the application of ADMM to the dual of the DMPC problem, introducing an augmented connection graph, which shows considerable improvement in convergence speed and therefore requires a lower number of communications in comparison with primal ADMM.