Best-Response Dynamics in Continuous Potential Games: Non-Convergence to Saddle Points

We prove the existence of a pure-strategy trembling-hand perfect equilibrium in upper semicontinuous potential games, and we show that generic potential games possess pure-strategy strictly perfect and essential equilibria. We also establish a more powerful result: the set of maximizers of an upper semicontinuous potential contains a strategically stable set of pure-strategy Nash equilibria.

Game theoretical learning in multi-agent systems is a rapidly developing area of research. It gained popularity since the wide range of optimization problems in multi-agent systems can be reformulated in terms of potential games, where the set of potential function maximizers represents the set of optimal system states. The crucial point of such approach is the design of a distributed algorithm that is guaranteed to converge to the set of potential function maximizers. Various learning algorithms, whose features depend on system properties, have been proposed so far. However, there is currently no learning algorithm that can be efficiently executed in a multi-agent system in which uncoupled agents update their actions synchronously and do not take into account the past history of actions. In this paper, we fill this gap by introducing a new learning algorithm for potential games with uncoupled dynamics and memoryless players who act synchronously. We prove the probabilistic convergence of this algorithm to potential function maximizers, which correspond to the optimal system states under appropriate game settings.

Distributed constraint optimization problems (DCOPs) are important in many areas of computer science and optimization. In a DCOP, each variable is controlled by one of many autonomous agents, who together have the joint goal of maximizing a global objective function. A wide variety of techniques have been explored to solve such problems, and here we focus on one of the main families, namely iterative approximate best-response algorithms used as local search algorithms for DCOPs. We define these algorithms as those in which, at each iteration, agents communicate only the states of the variables under their control to their neighbours on the constraint graph, and that reason about their next state based on the messages received from their neighbours. These algorithms include the distributed stochastic algorithm and stochastic coordination algorithms, the maximum-gain messaging algorithms, the families of fictitious play and adaptive play algorithms, and algorithms that use regret-based heuristics. This family of algorithms is commonly employed in real-world systems, as they can be used in domains where communication is difficult or costly, where it is appropriate to trade timeliness off against optimality, or where hardware limitations render complete or more computationally intensive algorithms unusable. However, until now, no overarching framework has existed for analyzing this broad family of algorithms, resulting in similar and overlapping work being published independently in several different literatures. The main contribution of this paper, then, is the development of a unified analytical framework for studying such algorithms. This framework is built on our insight that when formulated as non-cooperative games, DCOPs form a subset of the class of potential games. This result allows us to prove convergence properties of iterative approximate best-response algorithms developed in the computer science literature using game-theoretic methods (which also shows that such algorithms can also be applied to the more general problem of finding Nash equilibria in potential games), and, conversely, also allows us to show that many game-theoretic algorithms can be used to solve DCOPs. By so doing, our framework can assist system designers by making the pros and cons of, and the synergies between, the various iterative approximate best-response DCOP algorithm components clear.

We investigate the speed of convergence of congestion games with linear latency functions under best response dynamics. Namely, we estimate the social performance achieved after a limited number of rounds, during each of which every player performs one best response move. In particular, we show that the price of anarchy achieved after krounds, defined as the highest possible ratio among the total latency cost, that is the sum of all players latencies, and the minimum possible cost, is $O(\sqrt[2^{k-1}] {n})$, where nis the number of players. For constant values of ksuch a bound asymptotically matches the $\Omega(\sqrt[2^{k-1}] {n}/k)$ lower bound that we determine as a refinement of the one in [7]. As a consequence, we prove that order of loglognrounds are not only necessary, but also sufficient to achieve a constant price of anarchy, i.e. comparable to the one at Nash equilibrium. This result is particularly relevant, as reaching an equilibrium may require a number of rounds exponential in n. We then provide a new lower bound of $\Omega(\sqrt[2^k-1] {n})$ for load balancing games, that is congestion games in which every strategy consists of a single resource, thus showing that a number of rounds proportional to loglognis necessary and sufficient also under such a restriction. Our results thus solve the important left open question of the polynomial speed of convergence of linear congestion games to constant factor solutions.

The uncertainty in the new power system has increased, leading to limitations in traditional stability analysis methods. Therefore, considering the perspective of the three-dimensional static security region (SSR), we propose a novel approach for system static stability analysis. To address the slow training speed of traditional deep learning algorithms using batch gradient descent (BGD), we introduce an improved stochastic–batch gradient descent (S-BGD) search method that combines the advantages of stochastic gradient descent (SGD) in fast training. This method ensures both speed and precision in parameter training. Moreover, to tackle the problem of getting trapped in local optima and saddle points during parameter training, we draw inspiration from kinematic theory and propose a gradient pile (GP) training method. By utilizing accumulated gradients as parameter corrections, this method effectively avoids getting stuck in local optima and saddle points, thereby enhancing precision. Finally, we apply the proposed methods to construct the static security region for the IEEE-118 new power system using its data as samples, demonstrating the effectiveness of our approach.

This paper investigates the problem of co-tier interference mitigation for dynamic small-cell networks, in which the load of each small-cell varies with the number of active associated small-cell users (SUs).Due to the fact that most small-cell base stations (SBSs) are deployed in an ad-hoc manner, the problem of reducing co-tier interference caused by dynamic loads in a distributed fashion is quite challenging.First, we propose a new distributed channel allocation method for small-cells with dynamic loads and define a dynamic interference graph.Based on this approach, we formulate the problem as a dynamic interference graph game and prove that the game is a potential game and has at least one pure strategy Nash equilibrium (NE) point.Moreover, we show that the best pure strategy NE point minimizes the expectation of the aggregate dynamic co-tier interference in the small-cell network.A distributed dynamic learning algorithm is then designed to achieve NE of the game, in which each SBS is unaware of the probability distributions of its own and other SBSs' dynamic loads.Simulation results show that the proposed approach can mitigate dynamic co-tier interference effectively and significantly outperform random channel selection.

SummaryIn this paper, we study the full‐duplex relay selection strategy based on a potential game in a cognitive cooperative network under the interference power constraint from secondary users to the primary receivers, the total available transmission power constraint for the secondary system, and the self‐interference constraint at each secondary relay. The relay selection problem is modeled as a non‐cooperative game where the total rate of a cognitive cooperative network has common utility. Then, we prove that the game is a potential game that has at least a pure strategy Nash equilibrium (NE), and the optimal strategy set that able to maximize cognitive cooperative system rate is also a pure strategy NE of the proposed game model. On the premise of having no information of infeasible strategy sets, we solve the feasibility conditions of the pure NE in the proposed game. Furthermore, we propose a cognitive full‐duplex relay iterative algorithm that can achieve a pure strategy NE, and the complexity and the convergence of the proposed algorithm are studied. Simulation results show that the proposed algorithm can achieve optimal or near optimal rate performance with low complexity and offers significant performance gain compared with the traditional half‐duplex mode.

Exact solutions of self-consistent field (SCF) equations are obtained for a class of coupled oscillators (CO). For some CO models the solutions localized in various domains of configurational space are found. These local solutions (LS) correspond to the critical points (CP) of SCF energy functionals of various types, e.g. local minima, saddle points and others which are Thom's catastrophes. The symmetry properties of LS are discussed. Broken symmetry LS and a new class of LS with symmetry different from the symmetry of the CO Hamiltonian are also found.

Robustness of dynamics in games: A contraction mapping decomposition approach

We extend earlier works on continuous potential games to the most general case: stochastic time varying environment, stochastic rewards, non-reduced form and constrained state-action sets. We provide conditions for a Markov Nash equilibrium (MNE) of the game to be equivalent to the solution of a single control problem. Then, we address the problem of learning this MNE when the reward and state transition models are unknown. We follow a reinforcement learning approach and extend previous algorithms for working with constrained state-action subsets of real vector spaces. As an application example, we simulate a network flow optimization model, in which the relays have batteries that deplete with a random factor. The results obtained with the proposed framework are close to optimal.

Partially overlapping filtered multitone with reconfigurable antennas in uncoordinated networks

In this paper, we consider distributed Nash equilibrium (NE) seeking in potential games over a multi-agent network, where each agent can not observe the actions of all its rivals. Based on the best response dynamics, we design a distributed NE seeking algorithm by incorporating the non-smooth finite-time average tracking dynamics, where each agent only needs to know its own action and exchange information with its neighbours through a communication graph. We give a sufficient condition for the Lipschitz continuity of the best response mapping for potential games, and then prove the convergence of the proposed algorithm based on the Lyapunov theory. Numerical simulations are given to verify the result and illustrate the effectiveness of the algorithm.

A group of collaborative nodes can efficiently complete spatial long-distance transmission tasks using beamforming technology. However, a high sidelobe level interferes with communication quality, and uneven energy consumption of nodes affects network lifetime. This paper proposes an energy-efficient collaborative transmission algorithm based on potential game theory for beamforming. First, the minimum number of cooperative nodes is determined in accordance with the energy consumption and spacing limitation condition. A group of nodes satisfying the node spacing condition is selected as cooperative nodes based on the ring array to minimize communication interference among nodes. Second, a potential game model is proposed as a joint method for optimizing the collaborative parameters of the cooperative nodes and their energy consumption balancing features. Finally, the game process is continuously executed until the Nash equilibrium is reached. According to simulation results, the sidelobe level caused by the cooperative nodes is reduced and the transmission conflicts are lessened. Thus, the quality of communication links in between nodes in the network is improved. Energy efficiency is also promoted because a balancing of energy consumption is involved in the proposed potential game model, and network lifetime is effectively prolonged accordingly.

In this paper, we provide a theoretical analysis of log-linear learning algorithm that can be used for studying decision processes and solving control problems related to potential games. So far, for this algorithm the convergence of collective behavior to some state in a potential game has been proven to be specified by a chosen parameter and does not imply the convergence in probability to potential function maximizers. Tending the parameter and time to infinity simultaneously, we investigate the probabilistic convergence of joint actions based on the log-linear learning algorithm. We formulate conditions under which the convergence does not take place at all. Nevertheless, we explain how the parameter and utility functions should be designed to guarantee the probabilistic convergence of system behavior with some stationary distribution. Moreover, for such a setup in potential games the stable states, i.e. those having positive probability in the limit stationary distribution, are from the set of Nash equilibria and maximize the potential function.

A main target of the next generation wireless networks is to integrate multiple wireless access technologies to provide seamless mobility to mobile users with high-speed wireless connectivity. Network selection is one of the most significant challenges for load balancing to avoid network congestion and performance degradation because of the heterogeneous wireless access environment in next-generation wireless heterogenous networks. In this paper, we present an efficient network selection mechanism to guarantee mobile users selecting a most appropriate wireless network to connect from the heterogenous wireless networks using the theory of games. We prove the network selection game is a potential game, and the solution to the sum rate maximization problem constitutes a feasible pure strategy Nash equilibrium (NE) of the formulated game. Furthermore, a distributed algorithm to learn the NE of the proposed game with limited feedback is presented based on the concept of learning automata. Simulations show that this algorithm which outperforms random selection algorithm in a highly efficient manner can converge to the NE with our designed utility function and achieve near optimal or optimal sum rate performance.