The Complexity of Solving Stochastic Games on Graphs

We show that a Simple Stochastic Game (SSG) can be formulated as an LP-type problem. Using this formulation, and the known algorithm of Sharir and Welzl [SW] for LP-type problems, we obtain the first strongly subexponential solution for SSGs (a strongly subexponential algorithm has only been known for binary SSGs [L]). Using known reductions between various games, we achieve the first strongly subexponential solutions for Discounted and Mean Payoff Games. We also give alternative simple proofs for the best known upper bounds for Parity Games and binary SSGs. To the best of our knowledge, the LP-type framework has been used so far only in order to yield linear or close to linear time algorithms for various problems in computational geometry and location theory. Our approach demonstrates the applicability of the LP-type framework in other fields, and for achieving subexponential algorithms.

We define the first nontrivial polynomially recognizable subclass of P-matrix Generalized Linear Complementarity Problems (GLCPs) with a subexponential pivot rule. No such classes/rules were previously known. We show that a subclass of Shapley turn-based stochastic games, subsuming Condon’s simple stochastic games, is reducible to the new class of GLCPs. Based on this we suggest the new strongly subexponential combinatorial algorithms for these games.KeywordsInductive HypothesisLinear Programming ProblemLinear Complementarity ProblemStochastic GameDiagonal DominanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this paper, we consider two-player zero-sum stochastic mean payoff games with perfect information modeled by a digraph with black, white, and random vertices. These BWR-games games are polynomially equivalent with the classical Gillette games, which include many well-known subclasses, such as cyclic games, simple stochastic games, stochastic parity games, and Markov decision processes. They can also be used to model parlor games such as Chess or Backgammon. It is a long-standing open question if a polynomial algorithm exists that solves BWR-games. In fact, a pseudo-polynomial algorithm for these games with an arbitrary number of random nodes would already imply their polynomial solvability. Currently, only two classes are known to have such a pseudo-polynomial algorithm: BW-games (the case with no random nodes) and ergodic BWR-games (in which the game's value does not depend on the initial position) with constant number of random nodes. In this paper, we show that the existence of a pseudopolynomial algorithm for BWR-games with constant number of random vertices implies smoothed polynomial complexity and the existence of absolute and relative polynomial-time approximation schemes. In particular, we obtain smoothed polynomial complexity and derive absolute and relative approximation schemes for BW-games and ergodic BWR-games (assuming a technical requirement about the probabilities at the random nodes).

Simple stochastic games are turn-based 2.5-player games with a reachability objective. The basic question asks whether one player can ensure reaching a given target with at least a given probability. A natural extension is games with a conjunction of such conditions as objective. Despite a plethora of recent results on the analysis of systems with multiple objectives, the decidability of this basic problem remains open. In this paper, we present an algorithm approximating the Pareto frontier of the achievable values to a given precision. Moreover, it is an anytime algorithm, meaning it can be stopped at any time returning the current approximation and its error bound.

We revisit the deterministic graphical gamesof Washburn. A deterministic graphical game can be described as a simple stochastic game (a notion due to Anne Condon), except that we allow arbitrary real payoffs but disallow moves of chance. We study the complexity of solving deterministic graphical games and obtain an almost-linear time comparison-based algorithm for computing an equilibrium of such a game. The existence of a linear time comparison-based algorithm remains an open problem.

A short certificate of the number of universal optimal strategies for stopping simple stochastic games

Pure Strategy Equilibria in Symmetric Two-Player Zero-Sum Games

We introduce unique sink orientations of grids as digraph models for many well-studied problems, including linear programming over products of simplices, generalized linear complementarity problems over P-matrices (PGLCP), and simple stochastic games. We investigate the combinatorial structure of such orientations and develop randomized algorithms for finding the sink. We show that the orientations arising from PGLCP satisfy the Holt-Klee condition known to hold for polytope digraphs, and we give the first expected linear-time algorithms for solving PGLCP with a fixed number of blocks.

Game solving is a relatively less explored area in general game playing. This paper introduces a translation from the Game Description Language GDL to Quantified Boolean Formulas (QBF) that lets us leverage QBF solvers to compute winning strategies in two-player games described in GDL. We implement this approach and measure the computation time needed by state-of-the-art QBF solvers on a range of two-player zero-sum turn-taking games. We introduce a variety of optimizations to the translation and evaluate them experimentally. Our empirical analysis establishes that our proposed approach is suitable for solving small games and can potentially help general game players evaluate endgame positions.

An undecidable statement regarding zero-sum games

Chapter 1 - A Very Short Tour of Game Theory

Forn-person perfect information stochastic games and forn-person stochastic games with Additive Rewards and Additive Transitions (ARAT) we show the existence of pure limiting average equilibria. Using a similar approach we also derive the existence of limiting average e-equilibria for two-person switching control stochastic games. The orderfield property holds for each of the classes mentioned, and algorithms to compute equilibria are pointed out.

Complex systems with components or subsystems having game-like relationships are probably the most complex ones that we encounter everyday. Much progress has been made over the past half century on differential games which are used as a tool in modeling conflicts in the context of dynamic systems, however, almost all of the current literature assume that both the parameters and the structure of the game are known to the players. Since in many practical situations, the players may have unknown parameters, which motivate us to investigate a class of two-player zero-sum linear-quadratic stochastic differential games in [1] with unknown parameters. In this paper, we will further consider a class of two-player nonzero-sum linear quadratic stochastic differential games, with unknown parameters to both players. We will design adaptive strategies and prove that they will converge to the optimal ones under some natural conditions on the true parameters of the system.

We investigate an iterative algorithm for computing an exact Nash equilibrium in two-player zero-sum extensive-form games with imperfect information. The approach uses the sequence-form representation of extensive-form games and the double-oracle algorithmic framework. The main idea is to restrict the game by allowing the players to play only some of the sequences of available actions, then iteratively solve this restricted game, and exploit fast best-response algorithms to add additional sequences to the restricted game for the next iteration. In this paper we (1) extend the sequence-form double-oracle method to be applicable on non-deterministic extensive-form games, (2) present more efficient methods for maintaining valid restricted game and computing best-response sequences, and finally we (3) provide theoretical guarantees of the convergence of the algorithm to a Nash equilibrium. We experimentally evaluate our algorithm on two types of games: a search game on a graph and simplified variants of Poker. The results show significant running-time improvements compared to the previous variant of the double-oracle algorithm, and demonstrate the ability to find an exact solution of much larger games compared to solving full linear program for the complete game.

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.