Convergence Analysis of Nonconvex Distributed Stochastic Zeroth-order Coordinate Method

The distributed optimal coordinated control problem of heterogeneous linear multi-agent systems is studied in this paper. Specifically, each agent in the system under consideration has a local private cost function, which is assumed to be strongly convex with a Lipschitz continuous gradient. The agents aim to achieve output consensus on the optimal solution of a global cost function, which is the sum of local cost functions. The communication topology among agents is assumed to be a weight-unbalanced and strongly connected directed graph. To solve such a control problem, a tracking-based approach is suggested and utilized, where an auxiliary system is designed to seek the optimal solution. Based on the output regulation technique, distributed tracking control inputs are proposed for the agents, which ensure that the outputs of the agents track the trajectories generated by the auxiliary system. Under the assumptions on the local cost functions, the communication topology and system matrices, it is proved that with the proposed control scheme, the outputs of all agents can achieve consensus on the optimal solution. Finally, two numerical simulation examples are presented to illustrate the effectiveness of the proposed tracking control scheme.

In this chapter, we focus on introducing a distributed online optimization problem for a set of nodes communicating on a time-varying unbalanced directed network, while considering the problem of how to preserve the privacy of their local cost functions. The main goal of this set of nodes is to cooperatively minimize the sum of all locally known convex cost functions (global cost function). We propose a differentially private distributed stochastic subgradient-push algorithm, named DP-DSSP, to solve such an optimization problem in a collaborative and distributed manner. The algorithm ensures that nodes interact with their in-neighbors and collectively optimize the global cost function. Unlike most existing distributed algorithms that do not consider privacy issues, DP-DSSP successfully protects the privacy of participating nodes through a differential privacy strategy, which is more practical in applications involving sensitive information, such as military affairs or medical treatment. An important feature of DP-DSSP is that it handles distributed online optimization problems in the context of time-varying unbalanced directed networks. Theoretical analysis shows that DP-DSSP can effectively protect differential privacy and can achieve sublinear regret. The tradeoff between the level of privacy and accuracy of DP-DSSP is also revealed. Moreover, DP-DSSP is able to handle arbitrarily large but uniformly bounded delays in the communication link. Finally, simulation experiments confirm the usefulness of DP-DSSP and the findings of this chapter.KeywordsDistributed online optimizationDifferential privacyStochastic subgradient-push algorithmTime-varying directed networksCommunication delays

The paper mainly talks about the distributed optimal issue under the continuous-time high-order multi-agent system with event-triggered communication. Every agent possesses its local convex cost function only known by itself. and our aim is to minimize the sum of all the local cost functions and make the agents convergent to the optimal point through the communication between the agents. Utilizing the knowledge of convex analysis and graph theory, we propose a distributed protocol with event-triggered communication under a undirected graph for the high-order multi-agent system. Through Lyapunov method, the agents gradually converging to the optimal point is testified. In addition, we provide an example to validate the proposed protocol.

In this paper, we consider the problem of multi-agent systems where each agent aims to establish a line formation in a distributed manner. In constructing an efficient line formation, finding a line with the closest total distance from every agent is essential. We propose a formation control using least squares estimation (LSE) performed by each agent with only the local information that consists of the corresponding agent’s and neighbors’ positions. Each agent calculates the local cost function, which is the squared distance from the LSE line to the related agent’s and its neighbors’ positions. Our goal is to minimize the global cost function, which is the sum of these local cost functions. To achieve this, we employ distributed optimization to the global cost function of the overall system using the subgradient method performed by each agent locally. We evaluate our proposed method using numerical simulation, and the result complies with our goal of this paper

This paper presents a convergence analysis on a distributed Alternating Direction Method of Multipliers (ADMM) algorithm which solves online convex optimization problems under linear constraints. The goal is to distributively optimize a global objective function over a network of decision makers. The global objective function is composed of convex cost functions associated with each agent. The local cost functions can be broken down into two convex functions, one of which is revealed over time to the decision makers and one known a priori. We extend an online ADMM algorithm to a distributed setting based on dual-averaging. We then explore the rate of convergence of the performance of the sequence of decisions generated by the algorithm to the best fixed decision in hindsight. This performance metric is called regret. An upper bound on the regret of the proposed algorithm is presented as a function of the underlying network topology and linear constraints. The online distributed ADMM algorithm is then applied to a formation acquisition problem.

This paper considers the problem of multi-agent distributed optimization. In this problem, there are multiple agents in the system, and each agent only knows its local cost function. The objective for the agents is to collectively compute a common minimum of the aggregate of all their local cost functions. In principle, this problem is solvable using a distributed variant of the traditional gradient-descent method, which is an iterative method. However, the speed of convergence of the traditional gradient-descent method is highly influenced by the conditioning of the optimization problem being solved. Specifically, the method requires a large number of iterations to converge to a solution if the optimization problem is ill-conditioned. In this paper, we propose an iterative pre-conditioning approach that can significantly attenuate the influence of the problem's conditioning on the convergence-speed of the gradient-descent method. The proposed pre-conditioning approach can be easily implemented in distributed systems and has minimal computation and communication overhead. For now, we only consider a specific distributed optimization problem wherein the individual local cost functions of the agents are quadratic. Besides the theoretical guarantees, the improved convergence speed of our approach is demonstrated through experiments on a real data-set.

This paper considers distributed nonconvex optimization with the cost functions being distributed over agents. Noting that information compression is a key tool to reduce the heavy communication load for distributed algorithms as agents iteratively communicate with neighbors, we propose three distributed primal–dual algorithms with compressed communication. The first two algorithms are applicable to a general class of compressors with bounded relative compression error and the third algorithm is suitable for two general classes of compressors with bounded absolute compression error. We show that the proposed distributed algorithms with compressed communication have comparable convergence properties as state-of-the-art algorithms with exact communication. Specifically, we show that they can find first-order stationary points with sublinear convergence rate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/T)$</tex-math></inline-formula> when each local cost function is smooth, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the total number of iterations, and find global optima with linear convergence rate under an additional condition that the global cost function satisfies the Polyak–Łojasiewicz condition. Numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Recently, increasing penetration of renewable energy generation has created challenges for power system operators to perform efficient power generation daily scheduling, due to the intermittent nature of the renewable generation and discrete decisions of each generation unit. Among all aspects to be considered, a unit commitment polytope is fundamental and embedded in the models at different stages of power system planning and operations. In this article, we focus on deriving polynomial-time algorithms for the unit commitment problems with a general convex cost function and piecewise linear cost function, respectively. We refine an time, where T represents the number of time periods, algorithm for the deterministic single-generator unit commitment problem with a general convex cost function and accordingly develop an extended formulation in a higher-dimensional space that can provide an integral solution, in which the physical meanings of the decision variables are described. This means the original problem can be solved as a convex program instead of a mixed-integer convex program. Furthermore, for the case in which the cost function is piecewise linear, by exploring the optimality conditions, we derive more efficient algorithms for both deterministic (i.e., time) and stochastic (i.e., time, where N represents the number of nodes in the stochastic scenario tree) single-generator unit commitment problems. We also develop the corresponding extended formulations for both deterministic and stochastic single-generator unit commitment problems that solve the original mixed-integer linear programs as linear programs. Similarly, physical meanings of the decision variables are explored to show the insights of the new modeling approach.

This paper deals with the distributed time-varying optimization problem over a digragh (or directed graph). Motivated by the time-varying nature present in the cost functions, we model the time-varying features using an exosystem and formulate the problem of minimizing a global cost function, which is the sum of the local time-varying cost functions. We design a distributed algorithm for each agent that only utilizes the information of its own cost function and the information obtained through a network represented by a strongly connected and weight-balanced digraph. Convergence analysis is carried out to show that the decision variables of all agents converge to the time-varying optimal solution with time. Simulation results verify the theoretical conclusions.

Given two probability measures $\mu$ and $\nu$ in “convex order” on $\mathbb{R}^{d}$, we study the profile of one-step martingale plans $\pi$ on $\mathbb{R}^{d}\times\mathbb{R}^{d}$ that optimize the expected value of the modulus of their increment among all martingales having $\mu$ and $\nu$ as marginals. While there is a great deal of results for the real line (i.e., when $d=1$), much less is known in the richer and more delicate higher-dimensional case that we tackle in this paper. We show that many structural results can be obtained, provided the initial measure $\mu$ is absolutely continuous with respect to the Lebesgue measure. One such a property is that $\mu$-almost every $x$ in $\mathbb{R}^{d}$ is transported by the optimal martingale plan into a probability measure $\pi_{x}$ concentrated on the extreme points of the closed convex hull of its support. This will be established for the distance cost $c(x,y)=\vert x-y\vert $ in the two-dimensional case, and also for any $d\geq3$ as long as the marginals are in “subharmonic order.” In some cases, $\pi_{x}$ is supported on the vertices of a $k(x)$-dimensional polytope, such as when the target measure is discrete. Duality plays a crucial role in our approach, even though, in contrast to standard optimal transports, the dual extremal problem may not be attained in general. We show however that “martingale supporting” Borel subsets of $\mathbb{R}^{d}\times\mathbb{R}^{d}$ can be decomposed into a collection of mutually disjoint components by means of a “convex paving” of the source space, in such a way that when the martingale is optimal for a general cost function, each of the components then supports a restricted optimal martingale transport whose dual problem is attained. This decomposition is used to obtain structural results in cases where global duality is not attained. On the other hand, it shows that certain “optimal martingale supporting” Borel sets can be viewed as higher-dimensional versions of Nikodym-type sets. The paper focuses on the distance cost, but much of the results hold for general Lipschitz cost functions.

Various distributed optimization methods have been developed for solving problems which have simple local constraint sets and whose objective function is the sum of local cost functions of distributed agents in a network. Motivated by emerging applications in smart grid and distributed sparse regression, this paper studies distributed optimization methods for solving general problems which have a coupled global cost function and have inequality constraints. We consider a network scenario where each agent has no global knowledge and can access only its local mapping and constraint functions. To solve this problem in a distributed manner, we propose a consensus-based distributed primal-dual perturbation (PDP) algorithm. In the algorithm, agents employ the average consensus technique to estimate the global cost and constraint functions via exchanging messages with neighbors, and meanwhile use a local primal-dual perturbed subgradient method to approach a global optimum. The proposed PDP method not only can handle smooth inequality constraints but also non-smooth constraints such as some sparsity promoting constraints arising in sparse optimization. We prove that the proposed PDP algorithm converges to an optimal primal-dual solution of the original problem, under standard problem and network assumptions. Numerical results illustrating the performance of the proposed algorithm for a distributed demand response control problem in smart grid are also presented.

Motivated by large-scale signal processing and machine learning applications, this paper considers the distributed multi-agent optimization problem for a linearly constrained non-convex problem. Each of the agents owns a local cost function and local variable, but are coupled with each other due to the linear constraint. Most of the existing methods are either applicable for convex problems only or are developed under the non-convex setting subject to a specific type of linear constraint. There still lacks a distributed method for solving the linear constrained problem under the general and non-convex setting. In this paper, we propose such a method, called the proximal dual consensus (PDC) method, that combines a proximal technique and the dual consensus method. Theoretical analysis shows that the proposed PDC method can yield a Karush-Kuhn-Tucker solution of the linearly constrained non-convex problem and it has an O(1/e) iteration complexity, where e is a solution accuracy. The practical behavior of the proposed method is examined by numerical results.

The paper “An Optimal Control Framework for Online Job Scheduling with General Cost Functions,” by Etesami devises online speed-augmented competitive algorithms for minimizing the generalized completion time on a single and multiple unrelated machines for a very general class of cost functions. To that end, a novel optimal control formulation for the offline version of the problem is developed. Such a formulation allows using tools from optimal control theory, such as the minimum principle and Hamilton-Jacobi-Bellman equation, to set the dual variables as close as possible to the optimal dual variables and leverage them to design primal-dual online algorithms as an iterative application of the offline problem. The analysis can achieve state-of-the-art competitive ratios for several special cases and provide new competitive ratios which are the first in their settings. In particular, the analysis offers a principled method of estimating dual variables in a general setting of online job scheduling.

This paper studies safe Reinforcement Learning (safe RL) with linear function approximation and under hard instantaneous constraints where unsafe actions must be avoided at each step. Existing studies have considered safe RL with hard instantaneous constraints, but their approaches rely on several key assumptions: (i) the RL agent knows a safe action set for every state or knows a safe graph in which all the state-action-state triples are safe, and (ii) the constraint/cost functions are linear. In this paper, we consider safe RL with instantaneous hard constraints without assumption (i) and generalize (ii) to Reproducing Kernel Hilbert Space (RKHS). Our proposed algorithm, LSVI-AE, achieves O(√{d³H⁴K}) regret and O(H √{dK}) hard constraint violation when the cost function is linear and O(H?ₖ √{K}) hard constraint violation when the cost function belongs to RKHS. Here K is the learning horizon, H is the length of each episode, and ?ₖ is the information gain w.r.t the kernel used to approximate cost functions. Our results achieve the optimal dependency on the learning horizon K, matching the lower bound we provide in this paper and demonstrating the efficiency of LSVI-AE. Notably, the design of our approach encourages aggressive policy exploration, providing a unique perspective on safe RL with general cost functions and no prior knowledge of safe actions, which may be of independent interest.

Optimization of the profile and distribution of absorption material in sonic black holes