Iterated backward inference

An agent, or a coalition of agents, faces an ethical dilemma between several statements if she is forced to make a conscious choice between which of these statements will be true. This paper proposes to capture ethical dilemmas as a modality in strategic game settings with and without limit on sacrifice and for perfect and imperfect information games. The authors show that the dilemma modality cannot be defined through the earlier proposed blameworthiness modality. The main technical result is a sound and complete axiomatization of the properties of this modality with sacrifice in games with perfect information.

Rethinking common belief, revision, and backward induction

The Logic of Backward Induction

We investigate the verification of Multi-agent Systems against strategic properties expressed in Alternating-time Temporal Logic under the assumptions of imperfect information and perfect recall. To this end, we develop a three-valued semantics for concurrent game structures upon which we define an abstraction method. We prove that concurrent game structures with imperfect information admit perfect information abstractions that preserve three-valued satisfaction. Further, we present a refinement procedure to deal with cases where the value of a specification is undefined. We illustrate the overall procedure in a variant of the Train Gate Controller scenario under imperfect information and perfect recall.

Common Belief and the Theory of Games with Perfect Information

Given an agent with limited sensing capabilities, we analyze whether it is possible to deploy a new agent in the operational space of the preexisting agent in a safe manner. One approach for modeling the interaction of the introduced agent with its environment, which contains the preexisting agent, is through a two-player game of imperfect information. However, the computational cost of solving this game is prohibitive. Restricting the preexisting agent's strategy to just memoryless strategies and assuming that the introduced agent has perfect information alleviates the computational cost while still modeling realistic environments. The proposed algorithm for solving the game finds a winning strategy for the introduced agent by solving a quantified Boolean formula (QBF) for the game. We justify this approach by establishing a matching PSPACE lower bound. We also show that this result holds even when the preexisting agent uses bounded history to condition its play.

In logics for strategic reasoning the main challenge is represented by their verification in contexts of imperfect information and perfect recall strategies. In this work, we show the combination of two techniques to approximate the verification of Alternating-time Temporal Logic (ATL∗ ) under imperfect information and perfect recall, which is known to be undecidable. Given a model M and a formula φ, we propose a verification procedure that generates sub-models of M in which each sub-model M′ satisfies a sub-formula φ′ of φ and the verification of φ′ in M′ is decidable. Then, we use CTL∗ model checking to provide a verification result of φ on M. In case the previous step does not give a final result, we exploit a runtime verification mechanism to provide some intermediate result. We prove that our procedure is sound and in the same complexity class of ATL∗ model checking under perfect information and perfect recall. Moreover, we present a tool that uses our procedure and provide experimental results.

In logics for strategic reasoning the main challenge is represented by their verification in contexts of imperfect information and perfect recall strategies. In this work, we show a technique to approximate the verification of Alternating-time Temporal Logic (ATL*) under imperfect information and perfect recall, which is known to be undecidable. Given a model M and a formula phi, we propose a verification procedure that generates sub-models of M in which each sub-model M' satisfies a sub-formula phi' of phi and the verification of phi' in M' is decidable. Then, we use CTL* model checking to provide a verification result of phi on M. We prove that our procedure is sound and in the same complexity class of ATL* model checking under perfect information and perfect recall. Moreover, we present a tool that uses our procedure and provide experimental results.

We consider the problem of playing a repeated two-player zero-sum game safety: that is, guaranteeing at least the value of the game per period in expectation regardless of the strategy used by the opponent. Playing a stage-game equilibrium strategy at each time step clearly guarantees safety, and prior work has (incorrectly) stated that it is impossible to simultaneously deviate from a stage-game equilibrium (in hope of exploiting a suboptimal opponent) and to guarantee safety. We show that such profitable deviations are indeed possible specifically in games where certain types of “gift” strategies exist, which we define formally. We show that the set of strategies constituting such gifts can be strictly larger than the set of iteratively weakly-dominated strategies; this disproves another recent assertion which states that all noniteratively weakly dominated strategies are best responses to each equilibrium strategy of the other player. We present a full characterization of safe strategies, and develop efficient algorithms for exploiting suboptimal opponents while guaranteeing safety. We also provide analogous results for extensive-form games of perfect and imperfect information, and present safe exploitation algorithms and full characterizations of safe strategies for those settings as well. We present experimental results in Kuhn poker, a canonical test problem for game-theoretic algorithms. Our experiments show that (1) aggressive safe exploitation strategies significantly outperform adjusting the exploitation within stage-game equilibrium strategies only and (2) all the safe exploitation strategies significantly outperform a (nonsafe) best response strategy against strong dynamic opponents.

The author discusses games of both perfect and imperfect information at two levels of structural detail: players’ local actions, and their global powers for determining outcomes of the game. Matching logical languages are proposed for both. In particular, at the ‘action level’, imperfect information games naturally model a combined ‘dynamic‐epistemic language’– and correspondences are found between special axioms in this language and particular modes of playing games with their information dynamics. At the ‘outcome level’, the paper presents suitable notions of game equivalence, and some simple representation results.

We propose a theory of the interaction between knowledge and games. Epistemic game theory is of course a well-developed subject but there is also a need for a theory of how some agents can affect the outcome of a game by affecting the knowledge which other agents have and thereby affecting their actions. We concentrate on games of incomplete or imperfect information, and study how conservative, moderate, or aggressive players might play such games. We provide models for the behavior of a knowledge manipulator who seeks to manipulate the knowledge states of active players in order to affect their moves and to maximize her own payoff even while she herself remains inactive, except for influencing the states of knowledge of the other players.

The logic of backward induction

We consider the representation and evaluation of team decision making under uncertainty using influence diagrams. We assume that all team members agree on common beliefs and preferences, but complete sharing of information is generally impossible. As a result, the team can be represented as a single rational individual with imperfect recall, and the optimal solution with perfect recall might not be achievable, except in special cases we can recognize. An alternative solution concept is a stable solution that integrates the notion of optimality with that of equilibrium from game theory. We extend this concept from individual decisions to sets of decisions, and introduce the Strategy Improvement and its variation, Uniform Strategy Improvement, as the corresponding solution methods. We also provide a variety of simplifying transformations to the influence diagram by exploiting its graphical structure. The result is a requisite influence diagram, one that requires minimum assessment and creates additional opportunities for optimality.

Imperfect information game in multiplayer no-limit Texas Hold'em is a critical challenge in AI research. Recent advanced solving approaches, such as deep CounterFactual Value networks(CFVnet) combined with continual resolving, provide a way to conduct depth-limited search in imperfect-information games. However, CFVnet has limited deployment in Heads-Up No-Limit Texas Hold'em, and is hard to scale to multiplayer setting. In this paper, we propose a novel algorithm, mean approximation, that effectively converting multi-agent interactions to two-agent interactions, and introduce a useful trick virtual action generation to solve conflicts occur in this conversion. Furthermore, we introduce several improvements to deep CFVnet applied in Texas Hold'em Poker. We combined all above improvements and extensions of CFVnet to create our poker AI MuCFVnet, unlocking the potential of deep CFVnet. Experimental results show that MuCFVnet has strong performance, successfully beats some available public multiplayer poker AI.

Finding Nash equilibrium in the domain of imperfect information games as a challenging problem has received much attention. Neural Fictitious Self-Play (NFSP) is a popular model-free machine learning algorithm and has computed approximate Nash equilibrium on such games. However, the deep reinforcement learning method used to approximate the best response in NFSP requires reaching a fully observable Markov state, while the states in imperfect information games are partially observable and non-Markovian, which results in a poor approximation of the best response. Thus, NFSP needs more iterations to converge. In this study, we present a new reinforcement learning method that is inspired by counterfactual regret minimization to relax the Markov requirement by iteratively updating policy according to the regret matching process. Combining this new reinforcement learning algorithm with fictitious play, we further present a novel algorithm to find approximate Nash equilibrium in zero-sum imperfect information games. Experimental results in three benchmark games show that this new algorithm can find approximate Nash equilibrium effectively and converge much faster compared with baseline.