Stability and robustness analysis of minmax solutions for differential graphical games

This article studies a distributed minmax strategy for multiplayer games and develops reinforcement learning (RL) algorithms to solve it. The proposed minmax strategy is distributed, in the sense that it finds each player's optimal control policy without knowing all the other players' policies. Each player obtains its distributed control policy by solving a distributed algebraic Riccati equation in a multiplayer noncooperative game. This policy is found against the worst policies of all the other players. We guarantee the existence of distributed minmax solutions and study their L2 and asymptotic stabilities. Under mild conditions, the resulting minmax control policies are shown to improve robust gain and phase margins of multiplayer systems compared to the standard linear-quadratic regulator controller. Distributed minmax solutions are found using both model-based policy iteration and data-driven off-policy RL algorithms. Simulation examples verify the proposed formulation and its computational efficiency over the nondistributed Nash solutions.

In this paper, the multiagent pursuit-evasion (MPE) games are solved in order to obtain optimal strategic policies for all players. In these games, multiple pursuers attempt to intercept multiple evaders who try to avoid capture. A graph-theoretic approach is employed to study the interactions of the agents with limited sensing capabilities, such that distributed control policies are obtained for every agent. Furthermore, the minimization of performance indices associated with the goals of the agents is guaranteed. Nash equilibrium among the players is obtained by means of optimal policies that use the solutions of the Hamilton–Jacobi–Isaacs (HJI) equations of the game. Minmax strategies are also proposed to guarantee a security-level performance when the solutions of the HJI equations for Nash equilibrium do not exist. Scenarios for finite-time capture and for asymptotic rendezvous are analyzed, and emergent behaviors are obtained by means of modifications of the proposed general-case performance indices. The containment control results are shown to be special cases of the solutions of the MPE games.

The modified second-order sliding mode algorithm is used for controller tuning. Namely, the modified suboptimal algorithm-based test (modified SOT) and non-parametric tuning rules for proportional-integral-derivative (PID) controllers are presented in this article. In the developed method of test and tuning, the idea of coordinated selection of the test parameters and the controller tuning parameters is introduced. The proposed approach allows for the formulation of simple non-parametric tuning rules for PID controllers that provide desired amplitude or phase margins exactly. In the modified SOT, the frequency of the self-excited oscillations can be generated equal to either the phase crossover frequency or the magnitude crossover frequency of the open-loop system frequency response (including a future PID controller) – depending on the tuning method choice. The first option will provide tuning with specification on gain margin, and the second option will ensure tuning with specification on phase margin. Tuning rules for a PID controller and simulation examples are provided.

Aiming at the static unstable Hypersonic vehicle, a method for designing H∞ autopilot to satisfy gain margin and phase margin is proposed. Structured H∞ synthesis is used to synthesize the autopilot under the fixed structure. The gain margin and phase margin were constrained by using H∞ norm of the complementary of the scaled plant. The requirement for the gain margin and phase margin is represented by the distance between the open loop Nyquist curve and the circle that represents the gain margin and phase margin in the complex plane. The distance is adjusted by tuning the parameter of performance specifications automatically to reduce the conservatism of the controller. The present method is applicable to both static stable vehicle and static unstable vehicle, and a three loops acceleration tracking autopilot is design. The numerical simulations have demonstrated that the autopilot satisfys the performance specifications, gain margin and phase margin simultaneously.

A hybrid optimization approach for PI controller tuning based on gain and phase margin specifications

This paper presents an approach for finding regions Of specified gain margins (GM) and phase margins (PM) in the parameter space of a single-input single-output (SISO) control system with adjustable parameters. The method uses the Nyquist stability criterion of encirclement of a point in the polar plot. The necessary of the specified GM or PM regions in the parameter space are fully treated by mapping a point related to specified GM or PM in the polar plane into the parameter space. A general problem with previous mapping techniques is identified where points in the parameter space do not necessarily satisfy the required GM or PM. A searchbased technique is proposed to address this problem. Three examples illustrate the theoretical development. I. Introduction G AIN margin (GM) and phase margin (PM) are perhaps the two most important and widespread accepted criteria in analysis and design of practical control systems. These criteria are related to the open-loop transfer function of systems. In the robust stabilization problem of control systems, however, parameter space methods generally consider the closed-loop characteristic equation of the system.13 Parameter space techniques were initially studied by Vishnegradesky for a third-order system using a graphical technique that was popularized by Nimark and was referred to as D-partition or D-decomposition46 in its early development. This method actually establishes a direct correlation between the variable parameters of the closed-loop characteristic equation and the various important stability regions, e.g., open left half of the complex plane (OLHP). The method is directly suitable for graphical representation, in which case it is convenient to consider two parameters at a time. The idea of correlating the free parameters of the open-loop transfer function of a system and the relative stability regions appears to have been initially given in Ref. 7. Unfortunately , the method was not developed in detail, and the idea of isolating the GM region in the parameter plane was not correct. A recent improvement to the idea was presented by introducing the gain-phase margin tester into the system,8 which attempted to isolate specified GM and PM regions in the parameter space to allow a choice of the adjustable parameters. However, it is argued here that in this latter approach all the possible of the relative stability regions are not in fact considered, and some and possibly all points in the relative stability region do not necessarily satisfy the actual specified relative stability margin. In particular, the so called singular boundaries of the D-partition method6 need to be considered. Finally, arising through frequency loci with re-entrant or convoluted characteristics also need to be considered with care. In this paper, a method for accounting for the

Game theory studies mathematical models of interactions amongst self-interested entities. A means a description of the outcome of a game, and it is important that it should be defined in such a way that a solution always exists (every game should have an outcome). Nash's famous theorem that mixed-strategy equilibria are guaranteed to exist, resulted in Nash equilibrium being the prominent solution concept in game theory. As a result, computational challenges of the form a game, find a solution, have the property that we are searching for something whose existence is guaranteed (they are total search problems). Moreover, these solutions belong to the complexity class NP, since it is usually straightforward to check whether a proposed solution is correct (an incorrect one will admit a profitable deviation by one or more of the players, and this is usually easy to find). However, in versions of the problem that appear to be computationally hard, we cannot apply NP-completeness, due to a result of Megiddo saying that total search problems cannot be NP-complete unless NP is equal to co-NP. In this tutorial, which is intended for people familiar with NP-completeness, I give an overview of the alternative notions of computational hardness that apply to game-theoretic solution concepts. I discuss the complexity class PPAD (introduced by Papadimitriou) which captures the computational complexity of various classes of games that don’t seem to be solvable in polynomial time. I also mention the complexity classes PLS and FIXP, and the kinds of games that they apply to. Suppose, alternatively, that we have a polynomial-time algorithm that applies to some given class of games. A follow-up question is whether there exist algorithms that find a solution via processes that reflect decentralised selfish behaviour. This is because a solution concept arguably remains unrealistic if it can be efficiently computed, but only using a highly centralised algorithm. In the second half of the tutorial I present some results on learning dynamics for equilibrium computation, and mention recent work on communication complexity and query complexity. I discuss some research directions and open problems, such as the following. What are the prospects for proving that PPAD is as hard as NP? How about algorithms that find improved approximate Nash equilibria? 2-player games are easy to solve in practice, using the Lemke-Howson algorithm, so is there a satisfying mathematical sense in which 2-player games are easy to solve? (For example, a sense in which Lemke-Howson works most of the time?)

This paper proposes a novel method to graphically compute all feasible gain and phase margin specifications-oriented robust and optimal fractional PID controllers to stabilize time delay processes (TD) of any order. The gain-phase margin tester method is utilized to guarantee the concerned systems certain robust safety margins and test the constant gain margin boundary, constant phase margin boundary, and stability boundary in the parameter plane. The overlapping region of these plotted boundaries, which represents all feasible robust stabilizing PID controllers and allows the considered TD processes to retain pre-specified safety margins, is graphically determined as the gain and phase margin specifications-oriented region (GPMSOR). The variations of the GPMSOR and the stability region with respect to the variations of the noninteger orders of the integral and derivative terms have been investigated. According this, it is possible to slightly enlarge the GPMSOR by altering the noninterger order of the integral and derivative terms. This modifications in GPMSOR through altering the noninteger orders of the integral and derivative terms allows us to find the robust and optimal PID controller gain set for the considered TD systems. In other words, the selected fractional order PID controllers from the GPMSOR not only retain the prescribed GM and PM specifications but minimize the integral of the absolute error (IAE) or the integral of the squared error (ISE) performance criterion. Finally, an illustrative example cited from the literature with computer simulations is provided to demonstrate the effectiveness and confirm the validity of the proposed methodology.

In this paper, distributed algorithms are designed to search the Nash equilibrium (NE) for an $N$ -player game in continuous-time. The agents are not assumed to have direct access of other agents' states, and instead, they estimate other agents' states by communicating with their neighbours. Advanced consensus algorithms are implemented for such purposes, and consequently the game is decentralised into $N$ subsystems interacting over a communication network. It is proved that, for any communication network with a connected graph, a fixed-time consensus is achieved, independent of the initial conditions, based on which an NE can be obtained asymptotically by the gradient descent term for the fixed-time consensus-based algorithm. Then, the results are extended to a fixed-time NE seeking with modifications of the gradient terms, where both the consensus and the optimisation can be obtained in fixed time, and the upper bound of the settling time is established by the Lyapunov theory. A simulation example is presented to verify the effectiveness of the theoretical development, where some comparisons with other works are studied to demonstrate the advantages of the proposed algorithms.

This paper investigates the single-evader and multi-pursuer (SEMP) reach-avoid differential graphical (RADG) games of nonlinear heterogeneous players subject to saturated input and limited communication channels. The evader’s objective is to reach a designated target area while avoiding the pursuers, whose aim is to intercept the evader. First, we reformulate the SEMP RADG game as an optimal control problem within a weighted communication topology graph by incorporating the interception and control errors. Then, optimal control strategies that account for input saturation are derived by solving the coupled Hamilton-Jacobi (HJ) equations. These strategies are shown to constitute the Nash Equilibrium (NE) of the game. In addition, four types of pursuers, namely, isolated, passive, invisible, and regular pursuers, are defined based on the communication topology, and the conditions for achieving an Interactive Nash Equilibrium (INE), which is proposed for SEMP RADG games, are analyzed. Moreover, a single-network approximate dynamic programming (ADP) algorithm using concurrent learning (CL) is proposed to provide the near-optimal solutions to the coupled HJ equations. Asymptotic capture conditions are established through an examination of equilibrium points, and extensions to general pursuit-evasion (PE) games and half-space targets are further discussed. Our results are validated through numerical simulations.

Abstract: Beyond what Colman is suggesting, some residual indeterminacy of Nash equilibrium may remain even after individual rationality is amended. Although alternative solution concepts can expand the positive scope (explanatory power) of game theory, they tend to reduce its accuracy of predictions (predictive power). Moreover, the appeal of alternative solutions may be context-specific, as illustrated by the Stackelberg solution.

About a symmetric three-players zero-sum game we will show the following results. 1. A modified version of Sion's minimax theorem with the coincidence of the maximin strategy and the minimax strategy are proved by the existence of a symmetric Nash equilibrium. 2. The existence of a symmetric Nash equilibrium is proved by the modified version of Sion's minimax theorem with the coincidence of the maximin strategy and the minimax strategy. Thus, they are equivalent. If a zero-sum game is asymmetric, maximin strategies and minimax strategies of players do not correspond to Nash equilibrium strategies. If it is symmetric, the maximin strategies and the minimax strategies constitute a Nash equilibrium. However, without the coincidence of the maximin strategy and the minimax strategy there may exist an asymmetric equilibrium in a symmetric three-players zero-sum game.

The weak sequential core is a solution concept to coalition formation under uncertainty when only non-binding agreements can be formed ex ante. Here the case where any deviation from an allocation must be robust to communication both within, and between, coalitions at the ex ante stage is considered. The communication-proof weak sequential core is advanced as an alternative solution concept. Examples, including a production economy, are used to demonstrate the differences between the solutions. Finally, sufficient conditions for the non-emptiness of the communication-proof weak sequential core are provided.

The economic management of a microgrid can greatly benefit from energy storage systems (ESSs), which may act as virtual load deferral systems to take advantage of the fluctuations of energy prices and accommodate for demand-production mismatches caused by the scarce predictability of renewable sources. In a distributed energy management scenario, an ESS may serve multiple users, a setting which calls for the development of suitable resource allocation policies for the storage capacity. In particular, distributed control policies are of interest, where each user operates independently with the least exchange of information with the other users. A methodology is developed in the paper for such purpose, based on an iterative resource allocation mechanism, realized by means of a negotiation process among users, resembling stock exchange dynamics. The resulting distributed strategy for the management of the shared resource comes close to optimality at a low computational cost, which is affordable in large scale practical applications. It is also robust to communication failures between users.

Spatial pillage game