Open-loop and closed-loop local and remote stochastic nonzero-sum game with inconsistent information structure

We consider a risk-sensitive non-zero sum game problem of the stochastic system derived by backward stochastic differential equations (BSDEs). The domain of the control is assumed to be convex. Necessary conditions in the form of Pontryagin's maximum principle for open-loop equilibrium point are obtained for the risk-sensitive backward non-zero sum game problem. An interesting feature is that the necessary conditions for the equilibrium point do not depend on the risk-sensitive parameter. We also obtain a verification theorem, which provides a sufficient condition for the equilibrium point. A backward stochastic linear-quadratic risk-sensitive game problem is given to illustrate the theoretical results.

Sequential decomposition and policy iteration schemes for M-player games with partial weak coupling

Linear–quadratic stochastic two-person nonzero-sum differential games: Open-loop and closed-loop Nash equilibria

In this paper, we present a class of learning dynamics for distributed adaptive pricing in affine congestion games. We consider the setting where a large population of users is faced with the problem of choosing between a finite number of available resources, each resource having a particular cost function that depends only on the share of users using that particular resource. Since the mass of users is constant, their individual decisions affect the performance of all the available resources, thus generating a population game where each resource can be seen as a particular strategy in the game. Given the well-known fact that Nash equilibria in population games may not be socially optimal, a social planner is faced with the challenge of designing incentive mechanisms that induce a socially optimal Nash equilibrium. To achieve this, we present in this paper a class of model-free distributed pricing algorithms that guarantee convergence to the set of optimal tolls that induce a socially optimal Nash equilibrium. Our results allow us to consider populations of users that react instantaneously to tolls, as well as populations with social dynamics. Since the algorithms are distributed and data-driven, they can be implemented in settings where full information of the game is not available. By combining tools from game theory, robust set-valued dynamical systems, and adaptive control, a convergence result is established.

Non-zero sum quadratic differential game of BSDEs and multi-dimensional diagonally quadratic BSDE

Open-loop and closed-loop Nash equilibria for the LQ stochastic difference game

A new Mertens decomposition of [formula omitted]-submartingale systems. Application to BSDEs with weak constraints at stopping times

In this paper, we present a game-theoretic approach for the purpose of deriving the problem of joint beamforming and power control in cognitive radio (CR) multiple-input multiple-output (MIMO) broadcast channels (CR MIMO-BCs), where the primary users (PUs) coexist with the secondary users (SUs) and they share the same spectrum. The cognitive base station (CBS), which is equipped with multiple antennas, is capable of transmitting data to the SU’s multiple-antenna receiver by employing the technology of beamforming. The proposed approach is an application of separable games, which are formally stated by the subgames of beamforming and power control. Furthermore, based on the model of noncooperative separate games, separable cost functions for the parameters of beamforming and power control are also proposed, showing that these cost functions are convex. Therefore, the convex theory of a noncooperative game can be employed to investigate the best response strategies as well as existence of Nash equilibrium solutions. Finally, we propose an iterative algorithm to achieve the optimal Nash equilibrium of the proposed joint beamforming subgame and power control subgame. Numerical results verify both the convergence and the tracking properties of the proposed algorithm for variant scenarios.

The Nash equilibria of two two-person nonzero-sum differential games with hard constraints on the controls are studied. For both games the open-loop as well as the closed-loop solutions, and their relationships, are discussed. As is well-known for "smooth" nonzero-sum games, these solutions are generally different. Because of the constraints, the optimal controls are of the bang-bang type, and the solutions of the two problems under consideration are nonsmooth. One deals with non-Lipschitzian differential equations (considering the problem as an optimal control problem for one player while the bang-bang feedback control of the other player is assumed to be fixed), and the corresponding value functions possess singular surfaces. General conditions for the existence and uniqueness of the feedback solutions in this framework are not yet known. It is shown that in the two examples the open-loop and closed-loop solutions differ. As a by-product, the paper aims at a modest exploration of singular surfaces in nonzero-sum games.

We are interested in two-person games whose structure is far from zero-sum. We study the iterated Traveler's Dilemma (TD) which is a two-player, non-zero sum game that, depending on the exact values of its critical parameters, may offer plenty of incentives for cooperation. We first briefly summarize the results of a round-robin tournament with 36 competing strategies that was motivated by the work by Axelrod et al. on the iterated Prisoner's Dilemma. We then generalize the default version of Iterated TD with respect to two important game parameters, the bonus value and the granularity of the allowable bids. We analytically show the impact of the ratio of these two parameters on the game structure. Third, we re-run the 36-player round-robin tournament and investigate how varying the bonus-to-granularity ratio affects relative performances of various types of strategies in the tournament. We draw some conclusions based on those results and outline some promising ways forward in further investigating games whose structures seem to defy the prescriptions of classical game theory.

Leadership in singleton congestion games: What is hard and what is easy

Correlated relaxed equilibria in nonzero-sum linear differential games

This article investigates the distributed online optimization problem in a zero-sum game between two distinct time-varying multiagent networks. At each iteration, the agents not only communicate with their neighbors but also gather information about agents in the opposing network through a time-varying network, assigning weights accordingly. Moreover, we consider quantized communication and bandit feedback mechanisms, with agents transmitting quantized information and adopting one-point estimators. At each iteration, agents make and submit decisions and then receive the cost function values near their decision points rather than the full cost function information. To guarantee the payoff of each network, we design an algorithm named quantized distributed online bandit optimization in two-network (QDOBO-TN). We use dynamic Nash equilibrium regret to measure the positive payoff discrepancy between the decision sequence produced by Algorithm QDOBO-TN and the Nash equilibrium sequence. Furthermore, we propose a multiepoch version of Algorithm QDOBO-TN. The regret bounds for both algorithms are sublinear with respect to the iteration count T. Finally, we conduct a series of simulation experiments that further validate the effectiveness of the algorithms.

Coordination is an important issue in multi-agent systems when agents want to maximize their revenue. Often coordination is achieved through communication, however communication has its price. We are interested in finding an approach where the communication between the agents is kept low, and a global optimal behavior can still be found.In this paper we report on an efficient approach that allows independent reinforcement learning agents to reach a Pareto optimal Nash equilibrium with limited communication. The communication happens at regular time steps and is basically a signal for the agents to start an exploration phase. During each exploration phase, some agents exclude their current best action so as to give the team the opportunityto look for a possibly better Nash equilibrium. This technique of reducing the action space by exclusions was only recently introduced for finding periodical policies in games of conflicting interests. Here, we explore this technique in repeated common interest games with deterministic or stochastic outcomes.

Price differentiation and control in the Kelly mechanism