Optimizing Prosumer Policies in Periodic Double Auctions Inspired by Equilibrium Analysis

We study stochastic dynamic games with a large number of players, where players are coupled via their cost functions. A standard solution concept for stochastic games is Markov perfect equilibrium (MPE). In MPE, each player's strategy is a function of its own state as well as the state of the other players. This makes MPE computationally prohibitive as the number of players becomes large. An approximate solution concept called oblivious equilibrium (OE) was introduced, where each player¿s decision depends only on its own state and the "long-run average" state of other players. This makes OE computationally more tractable than MPE. It was shown that, under a set of assumptions, as the number of players become large, OE closely approximates MPE. In this paper we relax those assumptions and generalize that result to cases where the cost functions are unbounded. Furthermore, we show that under these relaxed set of assumptions, the OE approximation result can be applied to large population linear quadratic Gaussian (LQG) games.

This paper explores how the persistence of past choices creates incentives in a continuous time stochastic game involving a large player (e.g., a firm) and a sequence of small players (e.g., customers). The large player faces moral hazard and her actions are distorted by a Brownian motion. Persistence refers to how actions impact a payoff‐relevant state variable (e.g., product quality depends on past investment). I characterize actions and payoffs in Markov perfect equilibria (MPE) for a fixed discount rate, show that the perfect public equilibrium (PPE) payoff set is the convex hull of the MPE payoff set, and derive sufficient conditions for a MPE to be the unique PPE. Persistence can serve as an effective channel for intertemporal incentives in a setting where traditional channels fail. Applications to persistent product quality and policy targeting demonstrate the impact of persistence on equilibrium behavior.

We study an infinitely-repeated Bertrand competition game among a fixed number of firms in a market of both stochastic entry and stochastic demand. A firm’s entry into market in the next period is possible by making a positioning investment with stochastic success rate. The market demand in the next period is also stochastic and will not be realized until the firm enters the market. A successful investment allows a firm to participate in the Bertrand competition and an unsuccessful investment prevents a firm from entering the market, for the next period. We characterize the symmetric Markov perfect Nash Equilibrium (SMPNE) of such a dynamic game, where a firm’s strategy consists of two components: positioning strategy and pricing strategy. In examples with 1, 2, and 3 firms, we show the stage game market outcome, present the dynamic process of market structure, solve for the steady state of the dynamic system, and discuss about the speed of convergence to the steady state. Our work contributes to the dynamic oligopoly literature by allowing for two dimensions of stochastic uncertainty in firms’ decision-making.

We present a Nash equilibrium analysis for single-buyer single-seller multi-unit k-double auctions for scaling-based bidding strategies. We then design a Deep Deterministic Policy Gradient (DDPG) based learning strategy, DDPGBBS, for a participating agent to suggest bids that approximately achieve the above Nash equilibrium. We expand DDPGBBS to be helpful in more complex settings with multiple buyers/sellers trading multiple units in a Periodic Double Auction (PDA), such as the wholesale market in smart-grids. We demonstrate the efficacy of DDPGBBS with Power Trading Agent Competition's (PowerTAC) wholesale market PDA as a testbed.

Quality, Upgrades, and (the Loss of) Market Power in a Dynamic Monopoly Model

Can negotiations prevent fish wars?

We study Markov perfect equilibria (MPE) of two-player multi-mode differential games with controlled state dynamics, where one player controls the transition between modes. Different types of MPE are characterized distinguishing between delay equilibria, inducing for some initial conditions mode switches after a positive finite delay, and now or never equilibria, under which, depending on the initial condition, a mode switch occurs immediately or never. These results are applied to analyze the MPE of a game capturing the dynamic interaction between two incumbent firms among which one has to decide when to extend its product range by introducing a new product. The market appeal of the new product can be influenced over time by both firms through costly investments. Under a wide range of market introduction costs a now or never equilibrium coexists with a continuum of delay equilibria, each of them inducing a different time of product introduction.

Recursive Lexicographical Search: Finding All Markov Perfect Equilibria of Finite State Directional Dynamic Games

Markov Perfect Equilibrium: I. Observable Actions

In this paper, we empirically analyze weekly advertising policies of manufacturing firms in consumer goods markets. We assume firms engage in persuasive advertising, thus policies of firms affect the goodwill of a brand. We introduce a demand and a goodwill production function. A simple transformation of the demand function allows us to identify not only the demand parameters but also the parameters of the goodwill production function. We reconstruct the unobserved goodwill levels using these parameters and past advertising levels. We restrict our attention to Markov Perfect Equilibrium (MPE) strategies which are functions of payoff relevant state variables. Without imposing further restrictions on the dynamic competitive environment, we investigate the relationship between observed advertising strategies—which are assumed to be MPE—and payoff relevant state variables by means of several reduced form specifications. The most important determinant of advertising intensity turns out to be goodwill. We demonstrate that controlling for an advertising campaign significantly improves the explanatory power of the model.

In the spectrum auction, bidding strategy of telecommunication operation enterprise is very important. Considing the telecommunication operation enterprise how to choose the optimal bidding strategy as the train of thought. According to the spectrum of value distribution characteristics to the enterprise, using incomplete information static game theory to construct the operating enterprise bidding game model based on the partial distribution, and obtained model’s Nash equilibrium through the simulation with numerical solutions of the form. At the same time, the equilibrium analysis found: enterprise’s valuation central tendency or fluctuation can positive motivate its equilibrium bid; Along with the enterprise central tendency of valuation and fluctuation amplitude of valuation increasing ceaselessly, increasing unit valuation incentive equilibrium bid increment continuously increase.

Nash Equilibrium and Public Policy

In this article, we study the problem of online market clearing where there is one commodity in the market being bought and sold by multiple buyers and sellers whose bids arrive and expire at different times. The auctioneer is faced with an online clearing problem of deciding which buy and sell bids to match without knowing what bids will arrive in the future. For maximizing profit , we present a (randomized) online algorithm with a competitive ratio of ln( p max − p min ) + 1, when bids are in a range [ p min , p max ], which we show is the best possible. A simpler algorithm has a ratio twice this, and can be used even if expiration times are not known. For maximizing the number of trades, we present a simple greedy algorithm that achieves a factor of 2 competitive ratio if no money-losing trades are allowed. We also show that if the online algorithm is allowed to subsidize matches---match money-losing pairs if it has already collected enough money from previous pairs to pay for them---then it can actually be 1-competitive with respect to the optimal offline algorithm that is not allowed subsidy. That is, for maximizing the number of trades, the ability to subsidize is at least as valuable as knowing the future. We also consider objectives of maximizing buy or sell volume and social welfare. We present all of these results as corollaries of theorems on online matching in an incomplete interval graph.We also consider the issue of incentive compatibility, and develop a nearly optimal incentive-compatible algorithm for maximizing social welfare. For maximizing profit , we show that no incentive-compatible algorithm can achieve a sublinear competitive ratio, even if only one buy bid and one sell bid are alive at a time. However, we provide an algorithm that, under certain mild assumptions on the bids, performs nearly as well as the best fixed pair of buy and sell prices, a weaker but still natural performance measure. This latter result uses online learning methods, and we also show how such methods can be used to improve our “optimal” algorithms to a broader notion of optimality. Finally, we show how some of our results can be generalized to settings in which the buyers and sellers themselves have online bidding strategies, rather than just each having individual bids.

We revisit the evidence for the contribution of the long-lived radioactive nuclides 44Ti, 55Fe, 56Co, 57Co, and 60Co to the UVOIR light curve of SN 1987A. We show that the V-band luminosity constitutes a roughly constant fraction of the bolometric luminosity between 900 and 1900 days, and we obtain an approximate bolometric light curve out to 4334 days by scaling the late time V-band data by a constant factor where no bolometric light curve data is available. Considering the five most relevant decay chains starting at 44Ti, 55Co, 56Ni, 57Ni, and 60Co, we perform a least squares fit to the constructed composite bolometric light curve. For the nickel isotopes, we obtain best fit values of M(56Ni) = (7.1 +- 0.3) x 10^{-2} Msun and M(57Ni) = (4.1 +- 1.8) x 10^{-3} Msun. Our best fit 44Ti mass is M(44Ti) = (0.55 +- 0.17) x 10^{-4} Msun, which is in disagreement with the much higher (3.1 +- 0.8) x 10^{-4} Msun recently derived from INTEGRAL observations. The associated uncertainties far exceed the best fit values for 55Co and 60Co and, as a result, we only give upper limits on the production masses of M(55Co) < 7.2 x 10^{-3} Msun and M(60Co) < 1.7 x 10^{-4} Msun. Furthermore, we find that the leptonic channels in the decay of 57Co (internal conversion and Auger electrons) are a significant contribution and constitute up to 15.5% of the total luminosity. Consideration of the kinetic energy of these electrons is essential in lowering our best fit nickel isotope production ratio to [57Ni/56Ni]=2.5+-1.1, which is still somewhat high but is in agreement with gamma-ray observations and model predictions.

This paper proposes a reinforcement learning (RL) algorithm for the security problem of state estimation of cyber-physical system (CPS) under denial-of-service (DoS) attacks. The security of CPS will inevitably decline when faced with malicious cyber attacks. In order to analyze the impact of cyber attacks on CPS performance, a Kalman filter, as an adaptive state estimation technology, is combined with an RL method to evaluate the issue of system security, where estimation performance is adopted as an evaluation criterion. Then, the transition of estimation error covariance under a DoS attack is described as a Markov decision process, and the RL algorithm could be applied to resolve the optimal countermeasures. Meanwhile, the interactive combat between defender and attacker could be regarded as a two-player zero-sum game, where the Nash equilibrium policy exists but needs to be solved. Considering the energy constraints, the action selection of both sides will be restricted by setting certain cost functions. The proposed RL approach is designed from three different perspectives, including the defender, the attacker and the interactive game of two opposite sides. In addition, the framework of Q-learning and state–action–reward–state–action (SARSA) methods are investigated separately in this paper to analyze the influence of different RL algorithms. The results show that both algorithms obtain the corresponding optimal policy and the Nash equilibrium policy of the zero-sum interactive game. Through comparative analysis of two algorithms, it is verified that the differences between Q-Learning and SARSA could be applied effectively into the secure state estimation in CPS.