Distributed Nash equilibrium seeking: A gossip-based algorithm

This article considers a distributed Nash equilibrium seeking problem, where the players only have partial access to other players' actions, such as their neighbors' actions. Thus, the players are supposed to communicate with each other to estimate other players' actions. To solve the problem, a leader-following consensus gradient-free distributed Nash equilibrium seeking algorithm is proposed. This algorithm utilizes only the measurements of the player' local cost function without the knowledge of its explicit expression or the requirement on its smoothness. Hence, the algorithm is gradient-free during the entire updating process. Moreover, the analysis on the convergence of the Nash equilibrium is studied for the algorithm with both diminishing and constant step-sizes, respectively. Specifically, in the case of diminishing step-size, it is shown that the players' actions converge to the Nash equilibrium almost surely, while in the case of fixed step-size, the convergence to the neighborhood of the Nash equilibrium is achieved. The performance of the proposed algorithm is verified through numerical simulations.

In this paper, the distributed Nash equilibrium seeking problem of aggregative games, with players been presented by uncertain nonlinear systems over directed graphs, is investigated. The problem aims at designing a distributed algorithm for each player such that the player's state is bounded, while the player's output converges to the Nash equilibrium point. A dynamic compensator is firstly proposed by utilizing gradient-based protocol and average consensus strategy. Then the distributed Nash equilibrium seeking problem is converted into a tracking problem. Finally, the resulting tracking problem is addressed by developing a novel output feedback controller. It is shown that the concerned distributed Nash equilibrium seeking problem can be solved by the distributed algorithm composed of the dynamic compensator and the output feedback controller.

A random walk is a path that is comprised of random steps taken in random directions. Random walks present themselves in a variety of environments, and for this reason they have been a subject of relevance spanning disciplines ranging from economics to biology. A cycle in a random walk consists of the choice of a random direction, followed by a step forward, occurring for some duration. We investigated isotropic and non-isotropic random walks involving the following four cycle variations: 1. constant time, constant step size 2. constant time, varied step size 3. varied time, constant step size 4. varied time, varied step size. In biological terms, these cases of a random walks are of interest in modeling dispersal of organisms. In this study we have attempted to establish a mechanism to better study cases of non-normal diffusion in a population using a simplistic model.

In this paper, we consider distributed Nash equilibrium (NE) seeking in potential games over a multi-agent network, where each agent can not observe the actions of all its rivals. Based on the best response dynamics, we design a distributed NE seeking algorithm by incorporating the non-smooth finite-time average tracking dynamics, where each agent only needs to know its own action and exchange information with its neighbours through a communication graph. We give a sufficient condition for the Lipschitz continuity of the best response mapping for potential games, and then prove the convergence of the proposed algorithm based on the Lyapunov theory. Numerical simulations are given to verify the result and illustrate the effectiveness of the algorithm.

In wireless sensor networks, clustering is a very crucial problem. Basically clustering means grouping some specific objects based on their behavior and functionality. Clustering can be formulated for different optimization problems, such as nonsmooth, nonconvex problems. This paper is based on the review of the optimization algorithm that was proposed in the paper A Convergent Incremental Gradient Method With Constant Step Size by Blatt et al called Incremental Aggregate Gradient method. A novel algorithm called Incremental Clustered Aggregate Gradient Method was proposed in this paper to counter the shortcomings of the previous one. It has many similarities with the earlier method but it is more efficient for wireless sensor networks. The main aim of Incremental Gradient Method was to minimize the sum of continuously differentiable functions and also it required a single gradient evaluation per iteration and used a constant step size. For quadratic functions, a global linear rate of convergence was proved. It was claimed that it is more suitable for sensor networks. Although the experiments performed in this work confirm the convergence properties of it, it was found that it is not suitable for sensor networks. The proposed method addresses the flaws of the previous method as regards to sensor networks. When both algorithms operate with their respective optimal step sizes, they require approximately the same number of gradient evaluations for convergence.

In this article, the prescribed-time distributed Nash equilibrium (NE) seeking problem for multicoalition games with heterogeneous general linear players is investigated. In such games, there exist multiple coalitions that consist of heterogeneous players. The players have to seek the NE of the entire coalition while being limited to access only their individual cost functions. The proposed algorithm involves a two-layer structure composed of the NE seeking part and the tracking control part. To address the challenge of global information dependency, the leader-following-based estimates are designed. The NE seeking part is designed based on the distributed estimates and concurrently performed over the weight-balanced digraphs. Subsequently, by introducing the time-varying parametric Lyapunov equation, the strategies of the players are driven to the NE solution within a manually prescribed settling time, and the prescribed-time convergence is proved. Finally, the proposed algorithm is executed on the heterogeneous coalitions made up of quadrotor unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) in a group formation scenario. Both the simulation and the experiment results are presented to show the effectiveness of the algorithm.

Distributed Nash Equilibrium (NE) seeking for networked games has been widely investigated in recent years. In particular, convergence speed and communication cost are two of the critical concerns for the design of the seeking algorithms. To achieve fast convergence while saving communication resources, this paper investigates the appointed-time distributed NE seeking problem for networked games under a discrete-time communication scenario. By utilizing the idea of motion planning, a new continuous-time distributed NE seeking algorithm, which ensures convergence to NE at the prescribed time, is proposed based on sampled-data information exchange. Compared with the existing related works, the established algorithm has advantages on both the communication cost and convergence speed (i.e., appointed-time convergence). Finally, the effectiveness of the proposed method is verified via numerical simulations.

This paper deals with a new adaptive step-size overlay dedicated to Least Mean-Square (LMS) algorithms for noise cancellation with reference. LMS algorithms are often used in real-time estimations due to their robustness and simplicity, and are guided by their constant step size. Their transient and asymptotic performances are both linked to the step size, which leads to the well-known trade-off between speed and accuracy. In this paper, we propose a LMS algorithm with a new self-adaptive overlay based on a Multiplicative Update with Forgetting Factor (LMS-MUFF). The adaptive overlay improves the speed and the reactivity of the algorithm. In order to tune the algorithm parameters, we express the scalar case semi-analytical relationship between the LMS-MUFF parameters and a false-alarm probability, as defined from unwanted variations in the step-size during the asymptotic mode. We compare our method to reference methods in the literature and show that it offers better speed and reactivity with the same asymptotic performance.

We design a distributed algorithm for learning Nash equilibria over time-varying communication networks in a partial-decision information scenario, where each agent can access its own cost function and local feasible set, but can only observe the actions of some neighbors. Our algorithm is based on projected pseudo-gradient dynamics, augmented with consensual terms. Under strong monotonicity and Lipschitz continuity of the game mapping, we provide a very simple proof of linear convergence, based on a contractivity property of the iterates. Compared to similar solutions proposed in literature, we also allow for a time-varying communication and derive tighter bounds on the step sizes that ensure convergence. In fact, in our numerical simulations, our algorithm outperforms the existing gradient-based methods, when the step sizes are set to their theoretical upper bounds. Finally, to relax the assumptions on the network structure, we propose a different pseudo-gradient algorithm, which is guaranteed to converge on time-varying balanced directed graphs.

In this paper, we study the problem of distributed Nash equilibrium seeking of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -player games with single integrator dynamics subject to disturbances generated by an uncertain exosystem. Similar problems have been studied for games with single integrator dynamics subject to disturbances generated by a known exosystem. In addition, this paper offers some other new features. First, the existing results only apply to static, connected, and undirected graphs while the result of this paper applies to jointly strongly connected switching graphs, which can be disconnected at every time instant. Second, the existing results are asymptotic or ultimately uniformly bounded, while our result guarantees the exponential convergence when the exosystem is known. Third, the validity of existing results relies on an inequality involving some Lipschitz constant and the smallest nonzero eigenvalue of the Laplacian of the network graph. In contrast, the result of this paper is unconditional other than satisfying the same assumptions on the cost functions as those in the literature. Our design is illustrated by an application to velocity-actuated robots in sensor networks.

This work extends the distributed Nash equilibrium seeking algorithm to address multi-agent games under time-varying, signed, and undirected communication networks. Initially, based on uniform complete observability for linear time-varying systems, a new analytical framework is developed to determine the necessary and sufficient condition for achieving globally uniformly exponential stability of the error dynamics of the distributed estimator. It is shown that exponential stability is realized if and only if the joint <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(\epsilon,T)$</tex-math></inline-formula>-connectivity and the negative-link assumption of time-varying signed graphs are satisfied. Subsequently, it is demonstrated that under some mild conditions, the seeking algorithm ensures the global uniform exponential convergence of players' strategies towards the Nash equilibrium.

Conventional least-mean-square (LMS) algorithm is one of the most popular algorithms, which is widely used for adaptive beamforming. But the performance of the LMS algorithm degrades significantly because the constant step size is not suitable for varying signal-to-noise ratio (SNR) scenarios. Although numerous variable-step-size LMS (VSS-LMS) algorithms were proposed to improve the performance of the LMS algorithm; however, most of these VSS-LMS algorithms are either computationally complex or not reliable in practical scenarios since they depend on many parameters that are not easy to tune manually. In this letter, a fast and robust VSS-LMS algorithm is proposed for adaptive beamforming. The VSS is obtained based on normalized sigmoid function, where the sigmoid function is calculated by using the mean of instantaneous error first and then normalized by the squared cumulative sum of instantaneous error and estimated signal power. The proposed algorithm can update the step size adaptively without tuning any parameter and outperform state-of-the-art algorithms with low computational complexity. The simulation results show better performance of the proposed algorithm.

Asynchronous progressive iterative approximation method for least squares fitting

In this paper, we study the bias and high-order error bounds of the Linear Stochastic Approximation (LSA) algorithm with Polyak-Ruppert (PR) averaging under Markovian noise. We focus on the version of the algorithm with constant step size and propose a novel decomposition of the bias via a linearization technique. We analyze the structure of the bias and show that the leading-order term is linear in the step size and cannot be eliminated by PR averaging. To address this, we apply the Richardson-Romberg (RR) extrapolation procedure, which effectively cancels the leading bias term. We derive high-order moment bounds for the RR iterates and show that the leading error term aligns with the asymptotically optimal covariance matrix of the vanilla averaged LSA iterates. We validate applicability of our findings for the temporal difference algorithm in reinforcement learning.

The steepest descent method for large linear systems is well-known to often converge very slowly, with the number of iterations required being about the same as that obtained by utilizing a gradient descent method with the best constant step size and growing proportionally to the condition number. Faster gradient descent methods must occasionally resort to significantly larger step sizes, which in turn yields a rather non-monotone decrease pattern in the residual vector norm. We show that such faster gradient descent methods in fact generate chaotic dynamical systems for the normalized residual vectors. Very little is required to generate chaos here: simply damping steepest descent by a constant factor close to 1 will do. Several variants of the family of faster gradient descent methods are investigated, both experimentally and analytically. The fastest practical methods of this family in general appear to be the known, chaotic, two-step ones. Our results also highlight the need of better theory for existing faster gradient descent methods.