Distributed control and constraint preconditioners

The numerical solution of optimization problems with partial differential equation (PDE) constraints is vital to a growing number of science and engineering applications. The development of robust and efficient algorithms for the solution of these optimization problems presents many challenges that arise out of, e.g., the intricate mathematical structure of these problems, the complicated interactions between numerical methods for PDE and optimization, the large-scale of the optimization problems, and the increasing complexity of applications. To identify and overcome these challenges an integrated approach is needed that builds on a variety of mathematical sub-disciplines, such as theory of PDEs, distributed parameter systems, numerical solution of PDEs, numerical optimization, and numerical linear algebra. This international workshop has brought together some of the leading experts in the fast developing field of optimization problems with PDE constraints to present recent developments in this area as well as to identify open problems and further research needs. Among the themes of this workshop were the design and analysis of approaches for the solution of PDE constrained optimization problems with additional point-wise constraints on controls and states (the solution of the governing PDE). State constrained problems are particularly challenging because of the low regularity properties of the Lagrange multipliers associated with point-wise constraints on the states. A second theme was the development of adaptive methods for the solution of PDE constrained optimization problems and, more generally, the development of optimization level model reduction techniques for these problems. The goal here is to develop models (through, e.g., mesh adaptation or proper orthogonal model reduction) of the PDE constrained optimization problems that capture the relevant features of the optimization problems with a specified accuracy, but involve as few degrees of freedom as possible and, hence, are computationally less expensive to work with. A third theme was The efficient solution of linear systems arising in optimization algorithms for discretized PDE constrained optimization problems represented another theme. Finally, a number of talks presented advances and challenges in the solution of PDE constrained optimization problems arising in important science and industrial applications.

For engineering design optimization, the full-space formulation offers the potential for greater efficiency than the more commonly used reduced-space formulation. This potential is greater when the numerical model involves discretized partial differential equations or coupled disciplines. However, the full-space formulation results in a larger optimization problem with at least a factor of two increase in the number of optimization variables and equality constraints. Using Newton-type methods to solve such problems involves solving a large-scale and, often, ill-conditioned Karush–Kuhn–Tucker linear system at each optimization iteration. This can be time-consuming to solve even with a Krylov solver. If the number of iterations is reduced, the full-space formulation could be applied to a broader class of problems. This paper presents an inexact quasi-Newton algorithm with an adaptive extension for solving large-scale equality-constrained optimization problems. The new algorithm inexactly solves the Karush–Kuhn–Tucker system using new inexactness criteria that are derived to ensure a descent direction. The adaptive extension chooses the stopping condition of the Krylov solver by also taking its convergence rate into account. The paper presents results of numerical experiments applying this algorithm to three types of problems: six constrained optimization problems from the widely used CUTEst test suite, a bar thickness optimization problem, and a two-dimensional topology optimization problem. For all problems, the new algorithm consistently shows a roughly 50% reduction in the total number of Krylov solver iterations and a minimum of roughly 15% reduction in the optimization time. Moreover, the proposed approach for selecting the Krylov solver tolerance shows an improvement in all cases, whereas the existing forcing-parameter approach shows an increase in the number of Krylov iterations in some cases. These results indicate that this new method for selecting solver tolerances is effective and robust, and a good choice in algorithms that use a Krylov solver for solving the Karush–Kuhn–Tucker linear system.

Contributors

Deep mixed residual method for solving PDE-constrained optimization problems

A class of optimal shape design problems is studied in this paper. The boundary shape of a domain is determined such that the solution of the underlying partial differential equation matches, as well as possible, a given desired state. In the original optimal shape design problem, the variable domain is parameterized by a class of functions in such a way that the optimal design problem is changed to an optimal control problem on a fixed domain. Then, the resulting distributed control problem is embedded in a measure theoretical form, in fact, an infinite-dimensional linear programming problem. The optimal measure representing the optimal shape is approximated by a solution of a finite-dimensional linear programming problem. The method is evaluated via a numerical example.

Each slab entering to the reheating furnace has an optimal and unique reheating curve. The process of obtaining the optimal reheating curve is to solve the typical Partial differential equations (PDE) constrained optimization problem. Obviously, the solution of optimization problem is determined by both the precision of the mathematical PDE model and the numerical method. Firstly, the more accurate mathematical PDE model, in which some key parameters are reconsidered as temperature-dependent, is built for the reheating furnace. Secondly, the first-optimize-then-discretize approach is introduced to solve this PDE-constrained optimization problem. The analysis of the Fr<italic>&#x00E9;</italic>chet gradient of the cost functional is given and we can prove the gradient is Lipschitz continuous. Then, an improved conjugate gradient method is proposed to solve this problem. Finally, numerical simulations and experiment examples are given and analyzed. The results can prove the effectiveness of the proposed strategy.

The distributed control architecture becomes more and more important in future gas turbine engine control systems, in which, the sensors and actuators will be connected to the controllers via a network. Therefore, the control problem of network-enabled high-performance distributed engine control (DEC) has come to play an important role in modern gas turbine control systems. While, due to the properties of the network, the packet-dropouts must be considered. This study introduces a distributed control system architecture based on a networked cascade control system (NCCS). Typical turboshaft engine-distributed controllers are designed based on the NCCS framework with a H ∞ state feedback under random packet-dropouts. The sufficient robust exponential mean-stable conditions are derived via the Lyapunov stability theory and linear matrix inequality approach. Simulations illustrate the effectiveness of the presented method.

The distributed control architecture becomes more and more important in future gas turbine engine control systems, in which the sensors and actuators will be connected to the controllers via a network. Therefore, the control problem of network-enabled high-performance distributed engine control (DEC) has come to play an important role in modern gas turbine control systems, while, due to the properties of the network, the packet dropouts must be considered. This study introduces a distributed control system architecture based on a networked cascade control system (NCCS). Typical turboshaft engine distributed controllers are designed based on the NCCS framework with H∞ state feedback under random packet dropouts. The sufficient robust stable conditions are derived via the Lyapunov stability theory and linear matrix inequality approach. Simulations illustrate the effectiveness of the presented method.

We use a hybrid local meshless method to solve the distributed optimal control problem of a system governed by parabolic partial differential equations with Caputo fractional time derivatives of order α ∈ (0, 1]. The presented meshless method is based on the linear combination of moving least squares and radial basis functions in the same compact support, this method will change between interpolation and approximation. The aim of this paper is to solve the system of coupled fractional partial differential equations, with necessary and sufficient conditions, for fractional distributed optimal control problems using a combination of moving least squares and radial basis functions. To keep matters simple, the problem has been considered in the one-dimensional case, however the techniques can be employed for both the two- and three-dimensional cases. Several test problems are employed and results of numerical experiments are presented. The obtained results confirm the acceptable accuracy of the proposed method.

In this paper, we formulate for the first time an interactive input-output (I-O) distributed control problem in dynamic energy state space. The primary difficulty in distributed control problem by component i is finding consistent relations between control specifications on the output of interest y <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</inf> and shared output variables by the neighboring components z <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">j</inf> . The basic idea in the paper is to align these seemingly inconsistent objectives by imposing generalized Tellegen's theorem conditions in energy state-space at the component interfaces. The shared variable z <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">j</inf> is then shown to take on a physical meaning of power and the rate of change of reactive power. Using this modeling approach, we derive sufficient general conditions to guarantee that interconnected components form a feasible system. We elucidate steps to test whether the interconnected components will converge to an equilibrium. We show that the derived I-O feedback control stabilizes and regulates the outputs of interest by exchanging information about interaction variables between the neighboring components. Finally, we illustrate the first-of-its-kind combined feed-forward and feedback control in energy space capable of following an exogenous time-varying power reference by a controllable voltage source in an RLC circuit. Notably, the system is representative of inverter control of DC microgrids.

Rapid advances in information processing, communication and sensing technologies have enabled more and more devices to be provided with embedded processors, networking capabilities and sensors. For the field of estimation and control, it is now possible to consider an architecture in which many simple components communicate and cooperate to achieve a joint team goal. This distributed (or networked) architecture promises much in terms of performance, reliability and simplicity of design; however, at the same time, it requires extending the traditional theories of control, communication and computation and, in fact, looking at a unified picture of the three fields. A systematic theory of how to design distributed systems is currently lacking. This dissertation takes the first steps towards understanding the effects of imperfect information flow in distributed systems from an estimation and control perspective and coming up with new design principles to counter these effects. Designing networked systems is difficult because such systems challenge two basic assumptions of traditional control theory - presence of a central node with access to all the information about the system and perfect transmission of information among components. We formulate and solve many problems that deal with the removal of one, or both, of these assumptions. The chief idea explored in this dissertation is the joint design of information flow and the control law. While traditional control design has concentrated on calculating the optimal control input by assuming a particular information flow between the components, our approach seeks to synthesize the optimal information flow along with the optimal control law that satisfies the constraints of the information flow. Thus besides the question of 'What should an agent do?', the questions of 'Whom should an agent talk to?', 'What should an agent communicate?', 'When should an agent communicate?' and so on also have to be answered. The design of the information flow represents an important degree of freedom available to the system designer that has hitherto largely been ignored. As we demonstrate in the dissertation, the joint design of information flow and the optimal control input satisfying the constraints of that information flow yields large improvements in performance over simply trying to fit traditional design theories on distributed systems. We begin by formulating a distributed control problem in which many agents in a formation need to cooperate to minimize a joint cost function. We provide numerical algorithms to synthesize the optimal constrained control law that involve solving linear equations only and hence are free from numerical issues plaguing the other approaches proposed in the literature. We then provide and analyze a model to understand the issue of designing the topology according to which the agents interact. The results are very surprising since there are cases when allowing communication to happen between two agents may, in fact, be detrimental to the performance. We then move on to consider the effects of communication channels on control performance. To counter such effects, we propose the idea of encoding information for the purpose of estimation and control prior to transmission. Although information theoretic techniques are not possible in our problem, we are able to solve for a recursive yet optimal encoder / decoder structure in many cases. This information flow design oriented approach has unique advantages such as being optimal for any packet drop pattern, being able to include the effect of known but random delays easily, letting us escape the limits set by reliability for transmission of data across a network by using intermediate nodes as 'repeaters' similar to a digital communication network and so on. We finally take a look at combining the effects of multiple sources of information and communication channels on estimation and control. We look at a distributed estimation problem in which, at every time step, only a subset out of many sensors can transmit information to the estimator. This is also a representative resource allocation problem. We propose the idea of stochastic communication patterns that allows us to include the effects of communication channels explicitly during system design. Thus, instead of tree-search based algorithms proposed in the literature, we provide stochastic scheduling algorithms that can take into account the random packet drop effect of the channels. We also consider a distributed control problem with switching topologies and solve for the optimal controller. The tools that we develop are applicable to many other scenarios and we demonstrate some of them in the dissertation. Along the way, we look at many other related problems in the dissertation. As an example, we provide initial results about the issue of robustness of a distributed system design to a malfunctioning agent. This notion is currently lacking in the control and estimation community, but has to be a part of any effective theory for designing networked or distributed systems.

We address the parallelization of the determination of all optimal solutions (DAOS) for the 1D array partitioning problem, an easy combinatorial optimization problem that may be solved by dynamic programming. It turns out that the designed DAOS algorithm is a polyhedral algorithm (PA), i.e. a FOR-loop nest with affine loop bounds, whose iteration space that has to be scanned may be of exponential cardinality in the array size hence too time consuming. Thus, after a cardinality reducing procedure leading to an improved PA (IPA), we propose to parallelize the IPA through a specific approach. This latter first consists in a dependence analysis within the IPA leading to detect that all its loops are parallel. We then apply a loop interchange on the IPA in order to have a new IPA whose first loop has the largest number of parallel iterations. Another alternative consists in moving to the first position the loop whose number of iterations is the closest to the number of available processors. We detail an analysis of the theoretical performances of the derived parallel IPA by proposing several versions fitting the number of available processors and reducing the overhead due to processor synchronizations.

Given a linear system Ax = b , one can construct a related “normal equations” system AA T y = b, x = A T y . Björck and Elfving have shown that the SSOR algorithm, applied to the normal equations, can be accelerated by the conjugate gradient algorithm (CG). The resulting algorithm, called CGMN, is error-reducing and in theory it always converges even when the equation system is inconsistent and/or nonsquare. SSOR on the normal equations is equivalent to the Kaczmarz algorithm (KACZ), with a fixed relaxation parameter, run in a double (forward and backward) sweep on the original equations. CGMN was tested on nine well-known large and sparse linear systems obtained by central-difference discretization of elliptic convection-diffusion partial differential equations (PDEs). Eight of the PDEs were strongly convection-dominated, and these are known to produce very stiff systems with large off-diagonal elements. CGMN was compared with some of the foremost state-of-the art Krylov subspace methods: restarted GMRES, Bi-CGSTAB, and CGS. These methods were tested both with and without various preconditioners. CGMN converged in all the cases, while none of the preceding algorithm/preconditioner combinations achieved this level of robustness. Furthermore, on varying grid sizes, there was only a gradual increase in the number of iterations as the grid was refined. On the eight convection-dominated cases, the initial convergence rate of CGMN was better than all the other combinations of algorithms and preconditioners, and the residual decreased monotonically. The CGNR algorithm was also tested, and it was as robust as CGMN, but slower.

The two papers in this issue share a common concern: smoothing. In the first paper, it is time series that are being smoothed, and in the second paper it is multigrid methods that do the smoothing for PDE-constrained optimization problems. In biology, medicine, finance, economics, geophysics, and social sciences one frequently needs to discern “trends” in data sets. Mathematically, this problem is known as filtering, smoothing, or time series analysis, and many different methods have been devised, including moving average filters, bandpass filters, and smoothing splines. For a given set of scalars $y_t$, one wants to find another set of scalars $x_t$ that vary smoothly, but that are close to the original set. The new points $x_t$ are considered to represent the underlying trend. The question now is, how do we define what it means to “vary smoothly”? Seung-Jean Kim, Kwangmoo Koh, Stephen Boyd, and Dimitry Gorinevsky answer this question in the one-norm. In their paper “$\ell_1$ Trend Filtering,” they minimize an expression containing a one-norm, so that the resulting $x_t$ are points of a piecewise linear function. The minimization is formulated as a convex quadratic program and solved by an interior point method. This paper should be of interest to many readers, because of its connections to $\ell_1$ regularization in geophysics, signal processing, statistics, and sparse approximation. In their paper “Multigrid Methods for PDE Optimization,” Alfio Borzì and Volker Schulz review multigrid methods for solving infinite-dimensional optimization problems whose constraints are expressed in terms of partial differential equations (PDEs). Such optimization problem arise in optimal control, shape design, and shape optimization. Multigrid methods, very informally, solve PDEs by discretizing them iteratively on a hierarchy of grids, so as to capture all frequencies. So-called smoothers are responsible for high frequencies, while lower frequencies are resolved on coarser grids. This paper is essentially self-contained. It starts by introducing terminology for PDE-constrained optimization problems and reviewing multigrid methods. Subsequent discussions focus on multigrid SQP, Schur-complement-based multigrid smoothers, and collective smoothing multigrid, as well as applications to optimal control problems governed by hyperbolic, elliptic, and parabolic PDEs.

In the paper, the method of straight lines approximately solves one class of optimal control problems for systems, the behavior of which is described by a nonlinear equation of parabolic type and a set of ordinary differential equations. Control is carried out using distributed and lumped parameters. Distributed control is included in the partial differential equation, and lumped controls are contained both in the boundary conditions and in the right-hand side of the ordinary differential equation. The convergence of the solutions of the approximating boundary value problem to the solution of the original one is proved when the step of the grid of straight lines tends to zero, and on the basis of this fact, the convergence of the approximate solution of the approximating optimal problem with respect to the functional is established. A constructive scheme for constructing an optimal control by a minimizing sequence of controls is proposed. The control of the process in the approximate solution of a class of optimization problems is carried out on the basis of the Pontryagin maximum principle using the method of straight lines. For the numerical solution of the problem, a gradient projection scheme with a special choice of step is used, this gives a converging sequence in the control space. The numerical solution of one variational problem of the mentioned type related to a one-dimensional heat conduction equation with boundary conditions of the second kind is presented. An inequality-type constraint is imposed on the control function entering the right-hand side of the ordinary differential equation. The numerical results obtained on the basis of the compiled computer program are presented in the form of tables and figures. The described numerical method gives a sufficiently accurate solution in a short time and does not show a tendency to «dispersion». With an increase in the number of iterations, the value of the functional monotonically tends to zero