Time and Space: Why Imperfect Information Games are Hard

An agent, or a coalition of agents, faces an ethical dilemma between several statements if she is forced to make a conscious choice between which of these statements will be true. This paper proposes to capture ethical dilemmas as a modality in strategic game settings with and without limit on sacrifice and for perfect and imperfect information games. The authors show that the dilemma modality cannot be defined through the earlier proposed blameworthiness modality. The main technical result is a sound and complete axiomatization of the properties of this modality with sacrifice in games with perfect information.

Imperfect information game in multiplayer no-limit Texas Hold'em is a critical challenge in AI research. Recent advanced solving approaches, such as deep CounterFactual Value networks(CFVnet) combined with continual resolving, provide a way to conduct depth-limited search in imperfect-information games. However, CFVnet has limited deployment in Heads-Up No-Limit Texas Hold'em, and is hard to scale to multiplayer setting. In this paper, we propose a novel algorithm, mean approximation, that effectively converting multi-agent interactions to two-agent interactions, and introduce a useful trick virtual action generation to solve conflicts occur in this conversion. Furthermore, we introduce several improvements to deep CFVnet applied in Texas Hold'em Poker. We combined all above improvements and extensions of CFVnet to create our poker AI MuCFVnet, unlocking the potential of deep CFVnet. Experimental results show that MuCFVnet has strong performance, successfully beats some available public multiplayer poker AI.

Finding Nash equilibrium in the domain of imperfect information games as a challenging problem has received much attention. Neural Fictitious Self-Play (NFSP) is a popular model-free machine learning algorithm and has computed approximate Nash equilibrium on such games. However, the deep reinforcement learning method used to approximate the best response in NFSP requires reaching a fully observable Markov state, while the states in imperfect information games are partially observable and non-Markovian, which results in a poor approximation of the best response. Thus, NFSP needs more iterations to converge. In this study, we present a new reinforcement learning method that is inspired by counterfactual regret minimization to relax the Markov requirement by iteratively updating policy according to the regret matching process. Combining this new reinforcement learning algorithm with fictitious play, we further present a novel algorithm to find approximate Nash equilibrium in zero-sum imperfect information games. Experimental results in three benchmark games show that this new algorithm can find approximate Nash equilibrium effectively and converge much faster compared with baseline.

Given an agent with limited sensing capabilities, we analyze whether it is possible to deploy a new agent in the operational space of the preexisting agent in a safe manner. One approach for modeling the interaction of the introduced agent with its environment, which contains the preexisting agent, is through a two-player game of imperfect information. However, the computational cost of solving this game is prohibitive. Restricting the preexisting agent's strategy to just memoryless strategies and assuming that the introduced agent has perfect information alleviates the computational cost while still modeling realistic environments. The proposed algorithm for solving the game finds a winning strategy for the introduced agent by solving a quantified Boolean formula (QBF) for the game. We justify this approach by establishing a matching PSPACE lower bound. We also show that this result holds even when the preexisting agent uses bounded history to condition its play.

Auto-encoder neural network based prediction of Texas poker opponent's behavior

Phantom Go is a derivative of Go with imperfect information. It is challenging in AI field due to its great uncertainty of the hidden information and high game complexity inherited from Go. To deal with this imperfect information game with large game tree complexity, a general search framework named belief-state Monte Carlo tree search (BS-MCTS) is put forward in this paper. BS-MCTS incorporates belief-states into Monte Carlo Tree Search, where belief-state is a notation derived from philosophy to represent the probability that speculation is in accordance with reality. In BS-MCTS, a belief-state tree, in which each node is a belief-state, is constructed and search proceeds in accordance with beliefs. Then, Opponent Guessing and Opponent Predicting are proposed to illuminate the learning mechanism of beliefs with heuristic information. The beliefs are learned by heuristic information during search by specific methods, and we propose Opponent Guessing and Opponent Predicting to illuminate the learning mechanism. Besides, some possible improvements of the framework are investigated, such as incremental updating and all moves as first (AMAF) heuristic. Technical details are demonstrated about applying BS-MCTS to Phantom Go, especially on inference strategy. We examine the playing strength of the BS-MCTS and AMAF-BS-MCTS in Phantom Go by varying search parameters, also testify the proposed improvements.

The perfection of information is an important notion in games when considering sequential and simultaneous game. It is a key concept when analyzing the possibility of strategies and decision making for the next move. Scrabble board game is an imperfect information game where each opponent is unaware of the letters on the opposite rack. It is board game of luck, vocabulary and strategy. Our computer board game strategy focuses on maximizing the score and minimizing response time while searching for the best word possible from letters through the dictionary of words where best word could be a word containing all letters of rack and with best scoring letters (Vyas and Gahlot in A theoretical approach to reinforcement learning algorithm and Nash equilibrium to maximize the score in imperfect information game. Springer, Singapore, 2021) [1].KeywordsDecision makingImperfect information gameScrabble

Computers have just overshadowed the degree of human play in imperfect information games like Scrabble, yet there remains an opportunity to get better. Specifically, there is a lot to be acquired about the rival's tiles and moves. Reinforcement learning is one of the parts of Machine Learning. It is tied in with making an appropriate move to increase reward in a specific circumstance. The Nash Equilibrium is a hypothesis in game theory that communicates that a player can achieve the ideal outcome by not wandering from their initial strategy. Reinforcement learning algorithm and game theory would be best technique for training the agent. It will learn from as much as it plays, such that our agent can play and win the game with different persons with their different playing strategy they use for Scrabble game. To do this, our agents need to find out about other players' strategies and win every time they play against the human.

We consider the problem of playing a repeated two-player zero-sum game safety: that is, guaranteeing at least the value of the game per period in expectation regardless of the strategy used by the opponent. Playing a stage-game equilibrium strategy at each time step clearly guarantees safety, and prior work has (incorrectly) stated that it is impossible to simultaneously deviate from a stage-game equilibrium (in hope of exploiting a suboptimal opponent) and to guarantee safety. We show that such profitable deviations are indeed possible specifically in games where certain types of “gift” strategies exist, which we define formally. We show that the set of strategies constituting such gifts can be strictly larger than the set of iteratively weakly-dominated strategies; this disproves another recent assertion which states that all noniteratively weakly dominated strategies are best responses to each equilibrium strategy of the other player. We present a full characterization of safe strategies, and develop efficient algorithms for exploiting suboptimal opponents while guaranteeing safety. We also provide analogous results for extensive-form games of perfect and imperfect information, and present safe exploitation algorithms and full characterizations of safe strategies for those settings as well. We present experimental results in Kuhn poker, a canonical test problem for game-theoretic algorithms. Our experiments show that (1) aggressive safe exploitation strategies significantly outperform adjusting the exploitation within stage-game equilibrium strategies only and (2) all the safe exploitation strategies significantly outperform a (nonsafe) best response strategy against strong dynamic opponents.

The author discusses games of both perfect and imperfect information at two levels of structural detail: players’ local actions, and their global powers for determining outcomes of the game. Matching logical languages are proposed for both. In particular, at the ‘action level’, imperfect information games naturally model a combined ‘dynamic‐epistemic language’– and correspondences are found between special axioms in this language and particular modes of playing games with their information dynamics. At the ‘outcome level’, the paper presents suitable notions of game equivalence, and some simple representation results.

We propose a theory of the interaction between knowledge and games. Epistemic game theory is of course a well-developed subject but there is also a need for a theory of how some agents can affect the outcome of a game by affecting the knowledge which other agents have and thereby affecting their actions. We concentrate on games of incomplete or imperfect information, and study how conservative, moderate, or aggressive players might play such games. We provide models for the behavior of a knowledge manipulator who seeks to manipulate the knowledge states of active players in order to affect their moves and to maximize her own payoff even while she herself remains inactive, except for influencing the states of knowledge of the other players.

An important approach to game theory is to examine the consequences of beliefs that agents may have about each other. This paper investigates respect for public preferences. Consider an agent A who believes that B strictly prefers an option a to an option b. Then A respects B's preference if A assigns probability 1 to the choice of a given that B chooses a or b. Respect for public preferences requires that if it is common belief that B prefers a to b, then it is common belief that all other agents respect that preference. Along the lines of Blume, Brandenburger and Dekel [3] and Asheim [1], I treat respect for public preferences as a constraint on lexicographic probability systems. The main result is that given respect for public preferences and perfect recall, players choose in accordance with Iterated Backward Inference. Iterated Backward Inference is a procedure that generalizes standard backward induction reasoning for games of both perfect and imperfect information. From Asheim's characterization of proper rationalizability [1] it follows that properly rationalizable strategies are consistent with respect for public preferences; hence strategies eliminated by Iterated Backward Inference are not properly rationalizable.

The enforcement and pricing problems of the effective supply of public products have always been a hot academic topic, but the basic research paradigm is always principal-agent framework of central and local governments. This paradigm has neglected the ordinary households’ preferences which are the actual public demand, which is the argument foundation of our study. So using the public rental housing as the analysis object, the households’ preferences as the foundation for analyzing the government behavior, through principal-agent dynamic game with imperfect information and signal game with mixed strategy, we research which levels of incentives should the central government choose and how does the reputation effect of local government influent the long-term supply of public rental housing. Results show that under the premises that the local government pursue the maximization of achievements utility and economic utility, the enforcement of the supply of public rental housing significantly related to the levels of central government’s incentives, strong incentives have marginal amplification effect and weak incentives have marginal narrow effect. When information is so incomplete that households cannot distinguish the quality of public rental housing in advance, the equilibrium rent will be influented by the government reputation, that is the construction quality of public rental housing. The government reputation is closely related to the gap between the rent determined by the local government and the rent that households willing to pay.

Other work has shown that adaptive learning can be highly successful in developing programs which are able to play games at a level similar to human players and, in some cases, exceed the ability of a vast majority of human players. This study uses poker to investigate how adaptation can be used in games of imperfect information. An internal learning value is manipulated which allows a poker playing agent to develop its playing strategy over time. The results suggest that the agent is able to learn how to play poker, initially losing, before winning as the players strategy becomes more developed. The evolved player performs well against opponents with different playing styles. Some limitations of previous work are overcome, such as deal rotation to remove the bias introduced by one player always being the last to act. This work provides encouragement that this is an area worth exploring more fully in our future work.

During recent years, AI found its ways to games. Here, imperfect information games, such as Thousand Schnapsen, bring about major challenges. In this work, the rules and characteristics of this game have been described. Next, an overview of existing literature, focusing on similar problems, has been presented, followed by a summary of selected methods for finding an optimal strategy along with applied modifications. Finally, the results of the experiments are discussed.