Stochastic adaptive dynamical games

A command and control (C/sup 2/) problem for military air operations is addressed. Specifically, we consider C/sup 2/ problems for air vehicles against ground-based targets and defensive systems. The problem is viewed as a stochastic game. We restrict our attention to the C/sup 2/ level where the problem may consist of a few unmanned combat air vehicles (UCAVs) or aircraft (or possibly teams of vehicles), less than say, a half-dozen enemy surface-to-air missile air defense units (SAMs), a few enemy assets (viewed as targets from our standpoint), and some enemy decoys (assumed to mimic SAM radar signatures). At this low level, some targets are mapped out and possible SAM sites that are unavoidably part of the situation are known. One may then employ a discrete stochastic game problem formulation to determine which of these SAMs should optimally be engaged (if any), and by what series of air vehicle operations. We provide analysis, numerical implementation, and simulation for full state-feedback and measurement feedback control within this C/sup 2/ context. Sensitivity to parameter uncertainty is discussed. Some insight into the structure of optimal and near-optimal strategies for C/sup 2/ is obtained. The analysis is extended to the case of observations which may be affected by adversarial inputs. A heuristic based on risk-sensitive control is applied, and it is found that this produces improved results over more standard approaches.

Chapter 3 - Moving from Static to Dynamic Games

We consider a non-cooperative multi-stage game with discrete-time state dynamics. Players have their own decoupled state dynamics and each player wishes to minimize its own expected total cost. The salient aspect of our model is that each player's stage cost includes a payment (e.g., to a public utility) proportional to the magnitude of the player's decision. The coefficient multiplying each player's decision, called the price, is the same for all players and is determined as a function of the average of all player's decisions at that stage. Hence, each player's cost depends on the decisions of the other players only through the price. Here, we provide a stochastic and dynamic generalization of an equilibrium concept adopted in the economics literature, called the price-taking equilibrium, at which each player has no incentive to unilaterally deviate from its equilibrium strategy provided that the player ignores the effect of its own decisions on the price. In our setup, we allow for stochasticity in the price process and players observe only the past price realizations in addition to their own state realizations and their own past decisions. At a price-taking equilibrium, if players are given the distribution of the price process as if the price process is exogenous, they would have no incentive to unilaterally deviate from their equilibrium strategies. The main contribution of this paper is to establish such a stochastic and dynamic game generalization of price taking equilibria. We first derive the conditions for the existence of a price-taking equilibrium in the special case where the state dynamics are linear, the stage cost are quadratic, and the price function is linear. In this special case, our existence results are constructive for both finite-horizon and infinite horizon-problems. In the case where the number of players is taken to infinity, a price taking equilibrium exists which in turn is a mean-field equilibrium and is thus actually a Bayesian Nash equilibrium unlike the setup with a finite number of players. Finally, non-constructive existence results for price-taking equilibria and asymptotic equivalence with Nash equilibria are obtained for the case where the state and action sets are finite.

We consider a risk-sensitive non-zero sum game problem of the stochastic system derived by backward stochastic differential equations (BSDEs). The domain of the control is assumed to be convex. Necessary conditions in the form of Pontryagin's maximum principle for open-loop equilibrium point are obtained for the risk-sensitive backward non-zero sum game problem. An interesting feature is that the necessary conditions for the equilibrium point do not depend on the risk-sensitive parameter. We also obtain a verification theorem, which provides a sufficient condition for the equilibrium point. A backward stochastic linear-quadratic risk-sensitive game problem is given to illustrate the theoretical results.

In this paper, we consider the problem of inverse dynamic games: given the observed behaviour of players during a dynamic game in equilibrium, how can we determine the underlying objective functions of the game? Whereas previous work in the literature has focused on inverse static games, our work focuses on inverse dynamic games. In particular, we address the problem of estimating the unknown parameters of the objective function of a two-player zero-sum dynamic game in open-loop Nash equilibrium. We exploit necessary conditions for equilibrium in a two-player zero-sum dynamic game to develop sufficient conditions for solving the two-player zero-sum inverse dynamic game problem. The sufficient conditions hold under assumptions on the control constraints and convexity of the game dynamics, and transform the inverse two-player zero-sum dynamic game problem into the problem of solving a system of linear equations. We apply our results to a linear quadratic two-player zero-sum game, and illustrate the recovery of objective function parameters from state and control equilibrium trajectories.

This paper studies the modeling problem of event- triggered networked hybrid dynamic games (HDGs). By considering the influence of an event-triggering mechanism, the evolution process of dynamic games with hybrid characteristics is analyzed, and we point out the complexity and technical difficulties in the analysis of such a game problem. From the perspective of network science, we give a viewpoint on network-based modeling of event-triggered HDGs. On the basis of the state-space model with established, we first give the normal form of HDGs by a seven-tuple. We then establish the directed dynamic network model of HDGs for the first time, involving a graph-based tree structure form, which can well describe the distinctive features of the continuous-time and discrete-event dynamic game process on both sides, and has great advantages in evolutionary analysis. An example of the evolution of event-triggered HDGs show the innovation of the proposed model.

This letter studies the distributed adaptive consensus problem for multi-agent systems (MASs) subject to nonlinearly parameterized uncertainties. In the existing research, an adaptive consensus scheme was developed for the same scenario but under the assumption that the uncertainties must be linearly parameterized in their unknown parameters. In this letter, the assumption is removed by using a novel distributed adaptive update law, which makes the scheme applicable for more general nonlinear MASs. This letter contains a general design paradigm using the certainty equivalence principle and specific implementation of it on nonlinear second-order MASs. The results are supported by rigorous mathematical analysis and numerical simulation.

In the development of any educational game the educational and entertainment values it provides should be balanced. In this contribution we propose that the adaptation rules in Adaptive Educational Games are based on both the player profile, to motivate the student in the use of the tool, and the learning style, to help determine the best teaching/learning method to use and what learning objects and services that best suit the learning process for each particular student are.

A stochastic game is a multi-stage game played in discrete time where, at each stage, the stage game played depends upon a parameter called state. The value of the state evolves as a function of its current value and the actions of the players. Let I be the finite set of players and S be the set of states. For each state z in S, an I-player normal form game is specified by action sets A i(z) for each player i in I and reward functions r i (z,.), i in I, from the set of action profiles at z, A (z) = Π iɛI A i(z) to the reals, ℝ. In addition, for any pair consisting of a state z in S and an action profile a in A(z), a probability p(.|z,a) on S describes the random transition.

Presents a sample path optimization technique, as an alternative to the ensemble average in formulating and solving stochastic minimax dynamic games: The stochastic games are nonlinear, discrete-time, and partially observable. The controlled state and observation processes are subject to non-Gaussian inputs and square-summable disturbances.

I Foundations, Concepts, and Structure.- Stable Schedule Matching under Revealed Preference.- Banzhaf Permission Values for Games with a Permission Structure.- Moral Hazard in Teams Revisited.- Endogenous Determination of Utility Functions: An Evolutionary Approach.- N-person Prisoner's Dilemma with Mutual Choice by Agent-based Modeling.- Randomly-Furcating Stochastic Differential Games.- On Modulo 2 Game.- Bargaining in Dynamic Games.- II Equilibrium Properties.- Extensions of Hart and Mas-Colell's Consistency to Efficient, Linear, and Symmetric Values for TU-Games.- On the Stability of Cooperation Structures.- Time-Consistent Imputation Distribution Procedure for Multistage Game.- Rationality of Final Decisions Leads to Sequential Equilibrium.- The Core in the Presence of Externalities.- Network Topology and the Efficiency of Equilibrium.- Essential Components of the Set of Weakly Pareto-Nash Equilibria for Multiobjective Generalized Games.- III Applications to the Natural and Social Sciences.- Discretization of Information Collecting Situations and Continuity of Compensation Rules.- Some Variation Calculus Problems in Dynamic Games on 2D Surfaces.- On The Chinese Postman Game.- Farsighted Stability in Duopoly Markets with Product Differentiation.- International Emissions Trading with Agent-Based Simulation and Web-Based Gaming.- Comparison of two Economic Models for a Business-to-Business Exchange.- IV Computational Aspects of Games.- Computability of Nash Equilibrium.- Numerical Algorithm for Solving Cross-Coupled Multiparameter Algebraic Riccati Equations of Multimodeling Systems Related to Nash Games.- Effects of Symmetry on Paradoxes in Non-Cooperative Distributed Computing.- Computation of Stackelberg Trajectories in a Class of Linear Differential Games on Plane.- A Game Theoretic Approach to Value Information in Data Mining.

In the process of evacuation of pedestrian flow, the dynamic strategy choosing among pedestrians greatly influence evacuation efficiency. To address the problem of this dynamic evolutionary game, the direction fuzzy visual field of a moving pedestrian is defined, which fully takes into account the differences between pedestrians in terms of visual field selection. Based on the direction fuzzy visual field, this paper constructs a dynamic evolutionary game model and analyzes the formation and dynamic development of pedestrian flow strategy choosing. It presents the payoff matrix in which heterogeneous pedestrians adjust their strategy according to different payoff. The study results show that the model can effectively demonstrate macroscopic self-organization phenomena of pedestrian flow. The dynamic game equilibrium is related to the radius of the direction fuzzy visual field, the cost of strategic adjustment, the pedestrian density, and the system scale. The radius of the direction fuzzy visual field is bigger, the pedestrians can have more information, and they more likely to adopt competition strategy. There is a critical value for pedestrian density. When pedestrian density is greater than this critical value, the pedestrians will feel crowded and tend to adopt competition strategy. The cost of strategic adjustment is smaller, the pedestrians are more likely to adopt cooperation strategy, a few competitors will gather near the walls. When the cost of strategic adjustment is bigger, the pedestrians tend to adopt competition strategy, a small number of cooperators are gathering near the walls. Therefore, the walls play a very important role in pedestrian flow evacuation. The cellular system scale is bigger, pedestrians are more likely to cooperate.

This paper develops an algorithm for solving the generalized Nash equilibrium problem (GNEP) in non-cooperative games. The problem involves a set of players, each with a cost function that depends on their own decision as well as the decisions of other players. The goal is to find a decision vector that minimizes the cost for each player. Unlike most of the existing algorithms for GNEP, which require full information exchange among all players, this paper considers a more realistic scenario where players can only communicate with a subset of players through a connectivity graph. The proposed algorithm enables each player to estimate the decisions of other players and update their own and others’ estimates through local communication with their neighbors. By introducing a network Lagrangian function and applying the Douglas-Rachford splitting method (DR), the GNEP is reformulated as a zero-finding problem. It is shown that the DR method can find the generalized Nash equilibrium (GNE) of the original problem under some mild conditions.

Chapter 1 - A Very Short Tour of Game Theory

With the development of robot control theory and the requirement for robot to provide better performance in uncertain working environment, more and more research interests have been attracted to the two research topics of dealing with kinematic uncertainty existing in the robot system and considering actuator dynamics into the controller design during the past one decade. However, the studies in these two research areas have been developing separately in the literature. This thesis is dedicated to solve the adaptive control problems of robot in the presence of all uncertainties in the kinematics, manipulator dynamics and actuator dynamics. The main contribution of the thesis is that a novel adaptive regressor design method is proposed which enables the adaptation of different kinds of uncertainties simultaneously during the control process. Based on the novel regressor design, adaptive control methods have been proposed for regulation and tracking control of robot with uncertain kinematics, manipulator and actuator dynamics. These approaches constitute the first results in the research area of robot control to deal with both kinematic and actuator dynamic uncertainties. Through Lyapunov analysis, the stability of the closed-loop systems are shown to be guaranteed and the sufficient conditions to be satisfied are explicitly presented.