A differentiable homotopy method to compute perfect equilibria

Differentiable homotopy methods for gradually reinforcing the training of fully connected neural networks

In this chapter I examine the traditional position in game theory concerning the adequacy of the normal form representations of extensive form games. According to this traditional view, an extensive form game can be reduced to a normal form game without loss of strategically relevant information about the structure of the game. Many game theorists have already raised questions about such a reduction from a game in extensive form to strategic form by considering solution concepts that explicitly use the extensive form structure of the game. In the same vein, I evaluate the traditional view of the equivalence of extensive and normal form games in a systematic way. Within the traditional game-theoretic framework, I show that two extensive form games with the same reduced normal form may lead to different outcomes, and thus are not strategically equivalent. In addition, I demonstrate that the reduced normal form of an extensive form game cannot sufficiently represent certain reasonable strategic reasoning in the game of extensive form.

[Introduction] To a large extent the cooperative theory of games has an altogether different appearance from the noncooperative theory. The noncooperative theory generally deals with games in either extensive form or normal form, while the cooperative theory is usually described in characteristic function form. One of the central concepts in the cooperative theory is that of the core, which is the set of utility allocations which no coalition can improve upon. This notion of the core and of the characteristic function form of a game depends heavily on the existence of a utility representation for players' preferences. Recently Gale and Mas-Colell [3] and Shafer and Sonnenschein [6] have proven theorems on the existence of a Nash equilibrium for noncooperative games in normal form in which the players' preferences over strategy vectors are not necessarily complete or transitive and so may fail to have a utility representation. Thus it might appear that the noncooperative theory is applicable in environments where the cooperative theory is not. In order to formulate theorems in the cooperative theory of games which can be applied to environments in which players may have nonordered preferences, the characteristic function must be reformulated in terms of physical outcomes as opposed to utility outcomes. The players' preferences can then be expressed in terms of the physical outcomes without the use of a utility function.

This paper presents a survey of the use of homotopy methods in game theory. Homotopies allow for a robust computation of game-theoretic equilibria and their refinements. Homotopies are also suitable to compute equilibria that are selected by various selection theories. We present the relevant techniques underlying homot- opy algorithms. We give detailed expositions of the Lemke-Howson algorithm and the van den Elzen-Talman algorithm to compute Nash equilibria in 2-person games, and the Herings-van den Elzen, Herings-Peeters, and McKelvey-Palfrey algorithms to compute Nash equilibria in general n-person games. We explain how the main ideas can be extended to compute equilibria in extensive form and dynamic games, and how homotopies can be used to compute all Nash equilibria.

Further Results on Verification Problems in Extensive-Form Games

The complexity of computing a (quasi-)perfect equilibrium for an n-player extensive form game

The most profound reason why the Nash equilibrium concept has to be refined lies in the fact that a Nash equilibrium of an extensive form game may prescribe irrational behavior at unreached parts of the game tree. Until now, however, we have Mainly studied games in normal form. This comprehensive study yielded a deeper insight into the relations between various refinements of the Nash equilibrium concept. Our goal in this chapter is to investigate to what extend this study has yielded results which are also relevant for extensive form games. It will turn out that the insights we obtained are valuable, but that many results we proved for normal form games cannot be generalized to extensive form games.

The computational study of strategic interaction situations has recently deserved a lot of attention in multi--agent systems. A number of results on strategic--form games and zero--sum extensive--form games are known in the literature, while general--sum extensive--form games are not studied in depth. We focus on the problem to decide whether or not a solution is a refinement of the Nash equilibrium (NE) for extensive--form games. Refinements are needed because the NE concept is not satisfactory for this game class. While verifying whether a solution is an NE is in P, verifying whether it is a NE refinement may be not (all the results known so far show NP--hardness). In this paper, we provide the first positive result, showing that verifying a sequential equilibrium with any number of agents and a quasi perfect equilibrium with two agents are in P. We show also that when the input is expressed in (non--perturbed) sequence form even the problem to verify a subgame perfect equilibrium is NP--complete and that sequence form, if applicable, must be rethought to verify (and therefore to compute) an extensive--form perfect equilibrium.

Chapter summary In strategic-form games, a mixed strategy extends the player's possibilities by allowing him to choose a pure strategy randomly. In extensive-form games random choices can be executed in two ways. The player can randomly choose a pure strategy for the whole play at the outset of the game; this type of randomization yields in fact the concept of mixed strategy in an extensive-form game. Alternatively, at every one of his information sets, the player can randomly choose one of his available actions; this type of randomization yields the concept of behavior strategy , which is the subject of this chapter. We study the relationship between behavior strategies and mixed strategies in extensive-form games. To this end we define an equivalence relation between strategies and we show by examples that there are games in which some mixed strategies do not have equivalent behavior strategies, and there are games in which some behavior strategies do not have equivalent mixed strategies. We then introduce the concept of perfect recall : a player has perfect recall in an extensive-form game if along the play of the game he does not forget any information that he knew in the past (regarding his moves, the other players' moves, or chance moves). We prove Kuhn's Theorem, which states that if a player has perfect recall, then any one of his behavior strategies is equivalent to a mixed strategy, and vice versa.

This paper considers characterizations of perfect recall in extensive form games. It is shown that perfect recall can be expressed in terms of choices without any reference to infomation sets. When information sets are taken into account, it is decomposable into an ordering of information sets and that players do not forget what they knew nor what they did. Thus, if information sets are partially ordered, then perfect recall is implied by the player's inability to refine her information from the memory.

Chapter 2 Computation of equilibria in finite games

Among other solution concepts, the notion of the pure Nash equilibrium plays a central role in Game Theory. Pure Nash equilibria in a game characterize situations with non-cooperative deterministic players in which no player has any incentive to unilaterally deviate from the current situation in order to achieve a higher payoff. Unfortunately, it is well known that there are games that do not have pure Nash equilibria. Furhermore, even in games where the existence of equilibria is guaranteed, their computation can be a computationally hard task. Such negative results significantly question the importance of pure Nash equilibria as solution concepts that characterize the behavior of rational players. Approximate pure Nash equilibria, which characterize situations where no player can significantly improve her payoff by unilaterally deviating from her current strategy, could serve as alternative solution concepts provided that they exist and can be computed efficiently. In this letter, we discuss recent positive algorithmic results for approximate pure Nash equilibria in congestion games.

Imperfect-recall games—in which players may forget previously acquired information—have found many practical applications, ranging from game abstractions to team games and testing AI agents. In this paper, we quantify the utility gain by endowing a player with perfect recall, which we call the value of recall (VoR). While VoR can be unbounded in general, we parameterize it in terms of various game properties, namely the structure of chance nodes and the degree of absentmindedness (the number of successive times a player enters the same information set). Further, we identify several pathologies that arise with VoR, and show how to circumvent them. We also study the complexity of computing VoR, and how to optimally apportion partial recall. Finally, we connect VoR to other previously studied concepts in game theory, including the price of anarchy. We use that connection in conjunction with the celebrated smoothness framework to characterize VoR in a broad class of games.

This article reviews basic insights about compliance and "hard" enforcement that can be derived from various non-cooperative equilibrium concepts, and evaluates the Marrakesh Accords in light of these insights. Five different notions of equilibrium are considered – the Nash equilibrium, the subgame perfect equilibrium, the renegotiation proof equilibrium, the coalition proof equilibrium, and the perfect Bayesian equilibrium. These various types of equilibrium have a number of implications for effective enforcement: (1) Consequences of non-compliance should be more than proportionate. (2) Punishment needs to take place on the Pareto frontier, rather than by reversion to some suboptimal state. (3) An effective enforcement system must be able to curb collective as well as individual incentives to cheat. (4) A fully transparent enforcement regime is not unconditionally a good thing. It is concluded that constructing an effective system for "hard" enforcement of the Kyoto Protocol is a formidable task that has only partially been accomplished by the Marrakesh Accords. In practice, however, the design of the compliance system for the climate regime had to balance a desire to minimize non-compliance against other important goals, including the need for due process.

Equilibrium notions for games with unawareness in the literature cannot be interpreted as steady-states of a learning process because players may discover novel actions during play. In this sense, many games with unawareness are “self-destroying” as a player's representation of the game may change after playing it once. We define discovery processes where at each state there is an extensive-form game with unawareness that together with the players' play determines the transition to possibly another extensive-form game with unawareness in which players are now aware of actions that they have discovered. A discovery process is rationalizable if players play extensive-form rationalizable strategies in each game with unawareness. We show that for any game with unawareness there is a rationalizable discovery process that leads to a self-confirming game that possesses a self-confirming equilibrium in extensive-form rationalizable conjectures. This notion of equilibrium can be interpreted as a steady-state of both a discovery and learning process.