Query Complexity of Approximate Equilibria in Anonymous Games

We investigate an iterative algorithm for computing an exact Nash equilibrium in two-player zero-sum extensive-form games with imperfect information. The approach uses the sequence-form representation of extensive-form games and the double-oracle algorithmic framework. The main idea is to restrict the game by allowing the players to play only some of the sequences of available actions, then iteratively solve this restricted game, and exploit fast best-response algorithms to add additional sequences to the restricted game for the next iteration. In this paper we (1) extend the sequence-form double-oracle method to be applicable on non-deterministic extensive-form games, (2) present more efficient methods for maintaining valid restricted game and computing best-response sequences, and finally we (3) provide theoretical guarantees of the convergence of the algorithm to a Nash equilibrium. We experimentally evaluate our algorithm on two types of games: a search game on a graph and simplified variants of Poker. The results show significant running-time improvements compared to the previous variant of the double-oracle algorithm, and demonstrate the ability to find an exact solution of much larger games compared to solving full linear program for the complete game.

Extensive-form games are a powerful framework for modeling sequential multi-agent interactions. In extensive-form games with imperfect information, Nash equilibria are generally used as a solution concept, but computing a Nash equilibrium can be intractable in large games. Instead, a variety of techniques are used to find strategies that approximate Nash equilibria. Traditionally, an approximate Nash equilibrium strategy is evaluated by measuring the strategy's worst-case performance, or exploitability. However, because exploitability fails to capture how likely the worst-case is to be realized, it provides only a limited picture of strategy strength, and there is extensive empirical evidence showing that exploitability can correlate poorly with one-on-one performance against a variety of opponents. In this thesis, we introduce a class of adaptive opponents called pretty-good responses that exploit a strategy but only have limited exploitative power. By playing a strategy against a variety of counter-strategies created with pretty-good responses, we get a more complete picture of strategy strength than that offered by exploitability alone. In addition, we show how standard no-regret algorithms can me modified to learn strategies that are strong against adaptive opponents. We prove that this technique can produce optimal strategies for playing against pretty-good responses. We empirically demonstrate the effectiveness of the technique by finding static strategies that are strong against Monte Carlo opponents who learn by sampling our strategy, including the UCT Monte Carlo tree search algorithm.

Finding Nash equilibrium in the domain of imperfect information games as a challenging problem has received much attention. Neural Fictitious Self-Play (NFSP) is a popular model-free machine learning algorithm and has computed approximate Nash equilibrium on such games. However, the deep reinforcement learning method used to approximate the best response in NFSP requires reaching a fully observable Markov state, while the states in imperfect information games are partially observable and non-Markovian, which results in a poor approximation of the best response. Thus, NFSP needs more iterations to converge. In this study, we present a new reinforcement learning method that is inspired by counterfactual regret minimization to relax the Markov requirement by iteratively updating policy according to the regret matching process. Combining this new reinforcement learning algorithm with fictitious play, we further present a novel algorithm to find approximate Nash equilibrium in zero-sum imperfect information games. Experimental results in three benchmark games show that this new algorithm can find approximate Nash equilibrium effectively and converge much faster compared with baseline.

We consider a model of game-theoretic network design initially studied by Anshelevich et al. (Proceedings of the 45th Annual Symposium on Foundations of Computer Science (FOCS), pp. 295–304, 2004), where selfish players select paths in a network to minimize their cost, which is prescribed by Shapley cost shares. If all players are identical, the cost share incurred by a player for an edge in its path is the fixed cost of the edge divided by the number of players using it. In this special case, Anshelevich et al. (Proceedings of the 45th Annual Symposium on Foundations of Computer Science (FOCS), pp. 295–304, 2004) proved that pure-strategy Nash equilibria always exist and that the price of stability—the ratio between the cost of the best Nash equilibrium and that of an optimal solution—is Θ(log k), where k is the number of players. Little was known about the existence of equilibria or the price of stability in the general weighted version of the game. Here, each player i has a weight w i ≥1, and its cost share of an edge in its path equals w i times the edge cost, divided by the total weight of the players using the edge. This paper presents the first general results on weighted Shapley network design games. First, we give a simple example with no pure-strategy Nash equilibrium. This motivates considering the price of stability with respect to α-approximate Nash equilibria—outcomes from which no player can decrease its cost by more than an α multiplicative factor. Our first positive result is that O(log w max )-approximate Nash equilibria exist in all weighted Shapley network design games, where w max is the maximum player weight. More generally, we establish the following trade-off between the two objectives of good stability and low cost: for every α=Ω(log w max ), the price of stability with respect to O(α)-approximate Nash equilibria is O((log W)/α), where W is the sum of the players’ weights. In particular, there is always an O(log W)-approximate Nash equilibrium with cost within a constant factor of optimal. Finally, we show that this trade-off curve is nearly optimal: we construct a family of networks without o(log w max / log log w max )-approximate Nash equilibria, and show that for all α=Ω(log w max /log log w max ), achieving a price of stability of O(log W/α) requires relaxing equilibrium constraints by an Ω(α) factor.

We prove the existence of e-Nash equilibrium strategies with support logarithmic in the number of pure strategies. We also show that the payoffs to all players in any (exact) Nash equilibrium can be e-approximated by the payoffs to the players in some such logarithmic support e-Nash equilibrium. These strategies are also uniform on a multiset of logarithmic size and therefore this leads to a quasi-polynomial algorithm for computing an e-Nash equilibrium. To our knowledge this is the first subexponential algorithm for finding an e-Nash equilibrium. Our results hold for any multiple-player game as long as the number of players is a constant (i.e., it is independent of the number of pure strategies). A similar argument also proves that for a fixed number of players m, the payoffs to all players in any m-tuple of mixed strategies can be e-approximated by the payoffs in some m-tuple of constant support strategies.We also prove that if the payoff matrices of a two person game have low rank then the game has an exact Nash equilibrium with small support. This implies that if the payoff matrices can be well approximated by low rank matrices, the game has an e-equilibrium with small support. It also implies that if the payoff matrices have constant rank we can compute an exact Nash equilibrium in polynomial time.

We study the two-party communication complexity of finding an approximate Brouwer fixed point of a composition of two Lipschitz functions g o f: [0,1] <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> → [0,1] <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> , where Alice holds f and Bob holds g. We prove an exponential (in n) lower bound on the deterministic communication complexity of this problem. Our technical approach is to adapt the Raz-McKenzie simulation theorem (FOCS 1999) into geometric settings, thereby "smoothly lifting" the deterministic query lower bound for finding an approximate fixed point (Hirsch, Papadimitriou and Vavasis, Complexity 1989) from the oracle model to the two-party model. Our results also suggest an approach to the well-known open problem of proving strong lower bounds on the communication complexity of computing approximate Nash equilibria. Specifically, we show that a slightly "smoother" version of our fixed-point computation lower bound (by an absolute constant factor) would imply that: The deterministic two-party communication complexity of finding an ∈ = Ω(1/log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> N)-approximate Nash equilibrium in an N × N bimatrix game (where each player knows only his own payoff matrix) is at least N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">γ</sup> for some constant γ > 0. (In contrast, the nondeterministic communication complexity of this problem is only O(log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">6</sup> N)). ; The deterministic (Number-In-Hand) multiparty communication complexity of finding an ∈ = Ω(1)-Nash equilibrium in a k-player constant-action game is at least 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Ω(k/log k)</sup> (while the nondeterministic communication complexity is only O(k)).

In view of the intractability of finding a Nash equilibrium, it is important to understand the limits of approximation in this context. A subexponential approximation scheme is known [LMM03], and no approximation better than $1\over 4$ is possible by any algorithm that examines equilibria involving fewer than logn strategies [Alt94]. We give a simple, linear-time algorithm examining just two strategies per player and resulting in a $1\over 2$ -approximate Nash equilibrium in any 2-player game. For the more demanding notion of well-supported approximate equilibrium due to [DGP06] no nontrivial bound is known; we show that the problem can be reduced to the case of win-lose games (games with all utilities 0–1), and that an approximation of $5\over 6$ is possible contingent upon a graph-theoretic conjecture.

We give exponential lower bounds on the Price of Stability (PoS) of weighted congestion games with polynomial cost functions. In particular, for any positive integer $d$ we construct rather simple games with cost functions of degree at most $d$ which have a PoS of at least $\varOmega(Φ_d)^{d+1}$, where $Φ_d\sim d/\ln d$ is the unique positive root of equation $x^{d+1}=(x+1)^d$. This almost closes the huge gap between $\varTheta(d)$ and $Φ_d^{d+1}$. Our bound extends also to network congestion games. We further show that the PoS remains exponential even for singleton games. More generally, we provide a lower bound of $\varOmega((1+1/α)^d/d)$ on the PoS of $α$-approximate Nash equilibria for singleton games. All our lower bounds hold for mixed and correlated equilibria as well. On the positive side, we give a general upper bound on the PoS of $α$-approximate Nash equilibria, which is sensitive to the range $W$ of the player weights and the approximation parameter $α$. We do this by explicitly constructing a novel approximate potential function, based on Faulhaber's formula, that generalizes Rosenthal's potential in a continuous, analytic way. From the general theorem, we deduce two interesting corollaries. First, we derive the existence of an approximate pure Nash equilibrium with PoS at most $(d+3)/2$; the equilibrium's approximation parameter ranges from $\varTheta(1)$ to $d+1$ in a smooth way with respect to $W$. Secondly, we show that for unweighted congestion games, the PoS of $α$-approximate Nash equilibria is at most $(d+1)/α$.

We focus on the problem of computing approximate Nash equilibria and well-supported approximate Nash equilibria in random bimatrix games, where each player's payoffs are bounded and independent random variables, not necessarily identically distributed, but with common expectations. We show that the completely mixed uniform strategy profile, i.e. the combination of mixed strategies (one per player) where each player plays with equal probability each one of her available pure strategies, is with high probability a √ln n/n -Nash equilibrium and a √3 lnn/n -well supported Nash equilibrium, where n is the number of pure strategies available to each player. This asserts that the completely mixed, uniform strategy profile is an almost Nash equilibrium for random bimatrix games, since it is, with high probability, an e-well-supported Nash equilibrium where e tends to zero as n tends to infinity.

The economic dispatch problem for a Microgrid with independent prosumers, presents a market scenario where the prosumers act strategically maximizing their own benefits, and restricting strategic information such as energy production costs. In this context, the power allocation needs to follow a bidding process instead of a conventional economic dispatch process. Based on a game theory approach, a new average bidding algorithm is proposed, which achieves that the prosumer bids converge to an exact and efficient Nash equilibrium (ENE). At the ENE the power allocation minimizes the total energy cost in the Microgrid, and the set of bids maximizes the individual profit of each prosumer. Convergence conditions of the algorithm are presented, and it is shown that unlike of previous approaches the accuracy of the ENE does not depend on the step-size of the iterations. The performance of the bidding process is verified in numerical simulations, finding advantages of the proposed strategy respect to the Nash equilibrium seeking algorithm in terms of accuracy, and applicability for online operation.

In this thesis, we study the problem of computing approximate equilibria in several classes of games. In particular, we study approximate Nash equilibria and approximate well-supported Nash equilibria in polymatrix and bimatrix games and approximate equilibria in Lipschitz games, penalty games and biased games. We construct algorithms for computing approximate equilibria that beat the cur- rent best algorithms for these problems. In Chapter 3, we present a distributed method to compute approximate Nash equilibria in bimatrix games. In contrast to previous approaches that analyze the two payoff matrices at the same time (for example, by solving a single LP that combines the two players’ payoffs), our algorithm first solves two independent LPs, each of which is derived from one of the two payoff matrices, and then computes an approximate Nash equilibrium using only limited communication between the players. In Chapter 4, we present an algorithm that, for every δ in the range 0 < δ ≤ 0.5, finds a (0.5+δ)-Nash equilibrium of a polymatrix game in time polynomial in the input size and 1 . Note that our approximation guarantee does not depend on δ the number of players, a property that was not previously known to be achievable for polymatrix games, and still cannot be achieved for general strategic-form games. In Chapter 5, we present an approximation-preserving reduction from the problem of computing an approximate Bayesian Nash equilibrium (e-BNE) for a two-player Bayesian game to the problem of computing an e-NE of a polymatrix game and thus show that the algorithm of Chapter 4 can be applied to two-player Bayesian games. Furthermore, we provide a simple polynomial-time algorithm for computing a 0.5-BNE. In Chapter 5, we study games with non-linear utility functions for the players. Our key insight is that Lipschitz continuity of the utility function allows us to provide algorithms for finding approximate equilibria in these games. We begin by studying Lipschitz games, which encompass, for example, all concave games with Lipschitz continuous payoff functions. We provide an efficient algorithm for computing approximate equilibria in these games. Then we turn our attention to penalty games, which encompass biased games and games in which players take risk into account. Here we show that if the penalty function is Lipschitz continuous, then we can provide a quasi-polynomial time approximation scheme. Finally, we study distance biased games, where we present simple strongly poly- nomial time algorithms for finding best responses in L1, L2, and L∞ biased games, and then use these algorithms to provide strongly polynomial algorithms that find 2/3, 5/7, and 2/3 approximations for these norms, respectively.

Nash Equilibrium Seeking for Dynamic Systems with Non-Quadratic Payoffs

The arbitrary-sharing connection game is prominent in the network formation game literature [1]. An undirected graph with positive edge weights is given, where the weight of an edge is the cost of building it. An edge is built if agents contribute a sufficient amount for its construction. For agent i, the goal is to contribute the least possible amount while assuring that the source node si is connected to the terminal node ti . In this paper, we study the special case of this game in which there are only two source nodes. In this setting, we prove that there exists a 2-approximate Nash equilibrium that is socially optimal. We also consider the further special case in which there are no auxiliary nodes (i.e., every node is a terminal or source node). In this further special case, we show that there exists a 3 2 -approximate Nash equilibrium that is socially optimal. Moreover, we show that it is computable in polynomial time.

In an $$\epsilon $$∈-Nash equilibrium, a player can gain at most $$\epsilon $$∈ by changing his behaviour. Recent work has addressed the question of how best to compute $$\epsilon $$∈-Nash equilibria, and for what values of $$\epsilon $$∈ a polynomial-time algorithm exists. An $$\epsilon $$∈-well-supported Nash equilibrium ($$\epsilon $$∈-WSNE) has the additional requirement that any strategy that is used with non-zero probability by a player must have payoff at most $$\epsilon $$∈ less than a best response. A recent algorithm of Kontogiannis and Spirakis shows how to compute a 2/3-WSNE in polynomial time, for bimatrix games. Here we introduce a new technique that leads to an improvement to the worst-case approximation guarantee.

Pure strategy Nash equilibria of large finite-player games and their relationship to non-atomic games