Evolutionary Synthesis of Stable Normative Systems

The key technology concerning the dispatch and control of the cyber-physical-social systems (CPSS) integration and group machine learning (ML) based Web-of-Cells (WoC) is systematically investigated, aiming to develop an intelligent dispatching system that has a high penetration of distributed generations (DG). Based on practical engineering demands and the weakly-centralized WoC, which is characterized by self-organized co-evolution, high independence, high-efficiency synergy, and autonomous learning, a variety of advanced theoretical tools such as complex network theory, group ML, evolutionary game theory, and CPSS-based parallel system theory have been adopted to address the following key issue: How can we achieve overall optimal dispatching and control decision-making in a class of complex systems relying on a large number of group cells with characteristics of limited information, weak controllability, small capacity, and wide distribution? Starting from this, four basic scientific issues are discussed: 1) a modeling method for a self-organized coupled network of the CPSS integration-based WoC; 2) a stability analysis and stability control system of the self-organized evolution of the WoC; 3) a highly autonomous group intelligent decision (GID) method of an independent cell; 4) multi-cell synergetic evolutionary game and GID theory. Hence, an innovative breakthrough on the intersection of complex network game theory and group ML is expected to be obtained, contributing to the emergence of group knowledge in complex circumstances and a significant improvement in the level of GID. Lastly, based on the theoretical investigation in this paper, explorations on the development of the CPSS platform for the WoC as well as its engineering practice in application are conducted. Furthermore, the in-depth development of the WoC, as well as its anticipated technical challenges, are prospected and analyzed with the hope of applying it on practical smart distribution grid demonstration projects in the future.

Juglone is a 1,4-naphthoquinone (1,4-NQ) and the allelochemical responsible for the well-known toxic effects of black walnut (Juglans nigra) and other members of the Juglandaceae. Juglone affects a variety of weed species via a mode of action unlike any commercially available herbicides, and thus has the potential to be used as a new natural product-based herbicide. However, lack of knowledge about its metabolism precludes introducing juglone biosynthesis traits into resistant crops through biotechnology. Herein, we established that juglone is derived from the phylloquinone pathway at the level of the intermediate 1,4-dihydroxy-2-naphthoic acid (DHNA). Phylloquinone is a primary 1,4-NQ made by all plants for photosynthetic electron transport. Despite the fundamental importance of phylloquinone, there are still unanswered questions about the subcellular architecture of the phylloquinone pathway. In chapter 3, we show that o-succinylbenzoate CoA-ligase is localized to both chloroplasts and peroxisomes and that its activity is vital in both organelles. The required dual localization of CoA ligase activity is a theme common to other plant pathways with CoA metabolic steps occurring in peroxisomes and thus leads us to propose a revised model of the phylloquinone pathway. Lastly, given the potential of introducing juglone biosynthesis as part of novel weed management strategies, we investigated the circumstances, costs, and benefits of producing allelochemicals in crops using an evolutionary game theory model. Together, this work (i) shows that the phylloquinone pathway provides crops with the biosynthetic framework to produce juglone, (ii) sheds new light on the phylloquinone pathway architecture, and (iii) reveals the circumstances in which producing an allelochemical will be an evolutionarily stable strategy. We envision these results will assist biotechnological efforts to utilize juglone as a novel, natural product-based herbicide.

We consider an integro-PDE model for a population structured by the spatial variables and a trait variable affecting the dispersal coefficients. Competition for resource is local in spatial variables, but nonlocal in the trait variable. We focus on the asymptotic profile of positive steady state solutions. Our result shows that in the limit of small mutation rate, the solution remains regular in the spatial variables and yet concentrates in the trait variable and forms Dirac concetrations (i) at one boundary point; (ii) the interior; or (iii) at both boundary points. In particular, evolutionary branching is found in spatially heteogeneous but temporally constant environment. Other connections to notions and concepts in evolutionary game theory will also be discussed. This is joint work with Wenrui Hao (MBI) and Yuan Lou (Ohio State).

Open complex systems such as financial markets evolve in highly dynamic and uncertain environments. They are often subject to significant fluctuations due to unanticipated behaviours and information. Modelling and simulating these systems by means of multi-agent systems, i.e., through artificial markets is a valuable approach. Initial Public Offering (IPO) is a process based on finding a reasonable offering price or the price of the first assets sale by a firm to public. The study of the firms' strategic choices at the IPO requires the use of formal tools like game theory. This article is about the study, analysis and simulation of a firm's dynamic evolution at IPO using the EGT (Evolutionary Game Theory) as a formal framework for IPO strategies (offering prices) through modeling a financial market by multi-agent system. The firm in the system is a cognitive agent built around a classifier system. Simulations with our prototype allow us to deduce the factors that cause IPO under pricing.

Multi-agent learning is a crucial method to control or find solutions for systems, in which more than one entity needs to be adaptive. In today's interconnected world, such systems are ubiquitous in many domains, including auctions in economics, swarm robotics in computer science, and politics in social sciences. Multi-agent learning is inherently more complex than single-agent learning and has a relatively thin theoretical framework supporting it. Recently, multi-agent learning dynamics have been linked to evolutionary game theory, allowing the interpretation of learning as an evolution of competing policies in the mind of the learning agents. The dynamical system from evolutionary game theory that has been linked to Q-learning predicts the expected behavior of the learning agents. Closer analysis however allows for two interesting observations: the predicted behavior is not always the same as the actual behavior, and in case of deviation, the predicted behavior is more desirable. This discrepancy is elucidated in this article, and based on these new insights Frequency Adjusted Q- (FAQ-) learning is proposed. This variation of Q-learning perfectly adheres to the predictions of the evolutionary model for an arbitrarily large part of the policy space. In addition to the theoretical discussion, experiments in the three classes of two-agent two-action games illustrate the superiority of FAQ-learning.

Chapter 28 Game theory and evolutionary biology

L’article s’attache au problème de la genèse et de la dynamique des normes chez Hayek. Il s’appuie sur la théorie des jeux évolutionnistes pour présenter un modèle général d’apprentissage des agents et offrir une interprétation économique du processus par lequel les normes sont produites et sélectionnées.

With the increasing popularization and application of the smart grid, the harm of the data silo issue in the smart grid is more and more prominent. Therefore, it is especially critical to promote data interoperability and sharing in the smart grid. Existing data-sharing schemes generally lack effective incentive mechanisms, and data holders are reluctant to share data due to privacy and security issues. Because of the above issues, a dynamic incentive mechanism for smart grid data sharing based on evolutionary game theory is proposed. Firstly, several basic assumptions about the evolutionary game model are given, and the evolutionary game payoff matrix is established. Then, we analyze the stabilization strategy of the evolutionary game based on the payoff matrix, and propose a dynamic incentive mechanism for smart grid data sharing based on evolutionary game theory according to the analysis results, aiming to encourage user participation in data sharing. We further write the above evolutionary game model into a smart contract that can be invoked by the two parties involved in data sharing. Finally, several factors affecting the sharing of data between two users are simulated, and the impact of different factors on the evolutionary stabilization strategy is discussed. The simulation results verify the positive or negative incentives of these parameters in the data-sharing game process, and several factors influencing the users’ data sharing are specifically analyzed. This dynamic incentive mechanism scheme for smart grid data sharing based on evolutionary game theory provides new insights into effective incentives for current smart grid data sharing.

The optimal solution for the cooperation win-win task for Multi-agent system (MAS) was studied and a automatic negotiation reciprocal mechanism was build in this paper for the typical case of multi- agent foraging. The equivalences between agent in MAS and player in game theory, optimal solution in MAS and equilibrium in game theory, and cooperation task in MAS and multi-stage Markov game were then established. To the difficulties of equilibriums selection, a hawk dove game model, which is a uncorrelated asymmetry game, is build from the point view of evolutionary game theory. And a self-enforcing cooperative algorithm based on replicator dynamic was proposed to find the evolutionary stable strategy of hawk dove game. The simulation verified the effectiveness of the proposed approach.

- The analysts of the processes of public bodies decision - taking have long been discussing on the establishment of proper strategies to manage "environmental conflicts" - above all the so-called problems of "undesired location" of public works and facilities - efficiently (i.e. on a short-period basis so as to grant decision and agreement stability) and fairly (the parties' satisfaction is itself a further guarantee of decision and agreement stability). Each strategy, anyway, is still in progress, like a "universe to create and explore". Therefore, in this paper, we will focus on the analysis of the problem and provide as well some theoretical proposals to arrange "a new interpreting model of public bodies decision-taking processes" based on the achievements of two new subject-matters: evolutionary game theory and cognitive economy. Both sciences share their investigation field with law and economic science.Key words: Collective decision analysis, public decision-taking theory, conflict (re)solution, game theory, cooperative games, evolutionary game theory, neuroeconomics, bargaining theory.Parole chiave: Analisi delle decisioni collettive, teoria delle scelte pubbliche, risoluzione dei conflitti, teoria dei giochi, teoria della contrattazione, giochi cooperativi, teoria evolutiva dei giochi, neuroeconomia.JEL classifications: D700, D710, D740, C700, C710, C730, D870.

This article shows that seemingly diverse implementations of multi-agent reinforcement learning share the same basic building block in their learning dynamics: a mathematical term that is closely related to the gradient of the expected reward. Specifically, two independent branches of multi-agent learning research can be distinguished based on their respective assumptions and premises. The first branch assumes that the value function of the game is known to all players, which is then used to update the learning policy based on Gradient Ascent. Notable algorithms in this branch include Infinitesimal Gradient Ascent (IGA) [7], the variation Win or Learn Fast IGA (WoLF) [3] and the Weighted Policy Learner [1]. The second branch of multi-agent learning is concerned with learning in unknown environments, using interaction-based Reinforcement Learning, and contains those algorithms which have been shown to be formally connected to the replicator equations of Evolutionary Game Theory. In this case, the learning agent updates its policy based on a sequence of pairs that indicate the quality of the actions taken. Notable algorithms include Cross Learning (CL) [4], Regret Minimization (RM) [6], and Frequency Adjusted Q-learning (FAQ) [5]. This article demonstrates inherent similarities between these diverse families of algorithms by comparing their underlying learning dynamics, derived as the continuous time limit of their policy updates. These dynamics have already been investigated for algorithms from each family separately [1, 2, 3, 5, 6, 7], however, they have not yet been discussed in context of the relation to each other, and the origin of their similarity has not been discussed satisfactorily. In addition to the formal derivation, directional field plots of the learning dynamics in representative classes of two-player two-action games illustrate the similarities and strengthen the theoretical findings.

The Peer-to-Peer network paradigm is drawing the attention of both final users and researchers for its features. P2P networks shift from the classic client-server approach to a high level of decentralization where there is no central control and all the nodes should be able not only to require services, but to provide them to other peers as well. While on one hand such high level of decentralization might lead to interesting properties like scalability and fault tolerance, on the other hand it implies many new problems to deal with. A key feature of many P2P systems is openness, meaning that everybody is potentially able to join a network with no need for subscription or payment systems. The combination of openness and lack of central control makes it feasible for a user to free-ride, that is to increase its own benefit by using services without allocating resources to satisfy other peers’ requests. One of the main goals when designing a P2P system is therefore to achieve cooperation between users. Given the nature of P2P systems based on simple local interactions of many peers having partial knowledge of the whole system, an interesting way to achieve desired properties on a system scale might consist in obtaining them as emergent properties of the many interactions occurring at local node level. Two methods are typically used to face the problem of cooperation in P2P networks: 1) engineering emergent properties when designing the protocol; 2) study the system as a game and apply Game Theory techniques, especially to find Nash Equilibria in the game and to reach them making the system stable against possible deviant behaviors. In this work we present an evolutionary framework to enforce cooperative behaviour in P2P networks that is alternative to both the methods mentioned above. Our approach is based on an evolutionary algorithm inspired by computational sociology and evolutionary game theory, consisting in having each peer periodically trying to copy another peer which is performing better. The proposed algorithms, called SLAC and SLACER, draw inspiration from tag systems originated in computational sociology, the main idea behind the algorithm consists in having low performance nodes copying high performance ones. The algorithm is run locally by every node and leads to an evolution of the network both from the topology and from the nodes’ strategy point of view. Initial tests with a simple Prisoners’ Dilemma application show how SLAC is able to bring the network to a state of high cooperation independently from the initial network conditions. Interesting results are obtained when studying the effect of cheating nodes on SLAC algorithm. In fact in some cases selfish nodes rationally exploiting the system for their own benefit can actually improve system performance from the cooperation formation point of view. The final step is to apply our results to more realistic scenarios. We put our efforts in studying and improving the BitTorrent protocol. BitTorrent was chosen not only for its popularity but because it has many points in common with SLAC and SLACER algorithms, ranging from the game theoretical inspiration (tit-for-tat-like mechanism) to the swarms topology. We discovered fairness, meant as ratio between uploaded and downloaded data, to be a weakness of the original BitTorrent protocol and we drew inspiration from the knowledge of cooperation formation and maintenance mechanism derived from the development and analysis of SLAC and SLACER, to improve fairness and tackle freeriding and cheating in BitTorrent. We produced an extension of BitTorrent called BitFair that has been evaluated through simulation and has shown the abilities of enforcing fairness and tackling free-riding and cheating nodes.

In this chapter, we discuss the links between evolutionary game theory and personality research. Evolutionary game theory is a set of research methods used to investigate the evolution of social behavior, whereas personality research is mainly focused on comprehensively describing individual differences and their correlates. Until recently, these research domains developed independently, and scholars from one domain only scarcely referred to insights obtained in the other domain. This is changing, however, due to two developments. First, models of evolutionary game theory reveal that personality differences may have an adaptive explanation. These models generate new insights into the origins of personality, and some of these models produce novel and testable hypotheses. Second, it is becoming clear that individual differences matter for the course and outcome of evolution. Accordingly, insights from evolutionary game theory (and experiments based on these insights) can be misleading if personality differences are neglected. Here, we review these recent developments, emphasizing both the evolutionary causes and the evolutionary consequences of personality differences. Overall, we argue that the behavioral sciences would profit from a closer integration of mechanistic and functional perspectives.

Evolutionary Game Theory is the study of strategic interactions among large populations of agents who base their decisions on simple, myopic rules. A major goal of the theory is to determine broad classes of decision procedures which both provide plausible descriptions of selfish behaviour and include appealing forms of aggregate behaviour. For example, properties such as the correlation between strategies' growth rates and payoffs, the connection between stationary states and the well-known game theoretic notion of Nash equilibria, as well as global guarantees of convergence to equilibrium, are widely studied in the literature. Our paper can be seen as a quick introduction to Evolutionary Game Theory, together with a new research result and a discussion of many algorithmic and complexity open problems in the area. In particular, we discuss some algorithmic and complexity aspects of the theory, which we prefer to view more as Game Theoretic Aspects of Evolution rather than as Evolutionary Game Theory, since the term “evolution” actually refers to strategic adaptation of individuals' behaviour through a dynamic process and not the traditional evolution of populations. We consider this dynamic process as a self-organization procedure which, under certain conditions, leads to some kind of stability and assures robustness against invasion. In particular, we concentrate on the notion of the Evolutionary Stable Strategies (ESS). We demonstrate their qualitative difference from Nash Equilibria by showing that symmetric 2-person games with random payoffs have on average exponentially less ESS than Nash Equilibria. We conclude this article with some interesting areas of future research concerning the synergy of Evolutionary Game Theory and Algorithms.

Recently, game theory and evolutionary game theory - mathematical frameworks from economics and biology designed to model and explain interactive behavior - have proved fruitful tools for philosophers in areas such as ethics, philosophy of language, social epistemology, and political philosophy. This methodological osmosis is part of a trend where philosophers have blurred disciplinary lines to import the best epistemic tools available. In this vein, experimental philosophers have drawn on practices from the social sciences, and especially from psychology, to expand philosophy's grasp on issues from morality to consciousness. We argue that the recent prevalence of formal work on human interaction in philosophy opens the door for new methods in experimental philosophy. In particular, we discuss methods from experimental economics, focusing on a small literature we have been developing investigating signaling and communication in humans. We describe results from a novel experiment showing how environmental structure can shape signaling behavior.