Independent Log-Linear Learning in Potential Games With Continuous Actions

Game theoretical learning in multi-agent systems is a rapidly developing area of research. It gained popularity since the wide range of optimization problems in multi-agent systems can be reformulated in terms of potential games, where the set of potential function maximizers represents the set of optimal system states. The crucial point of such approach is the design of a distributed algorithm that is guaranteed to converge to the set of potential function maximizers. Various learning algorithms, whose features depend on system properties, have been proposed so far. However, there is currently no learning algorithm that can be efficiently executed in a multi-agent system in which uncoupled agents update their actions synchronously and do not take into account the past history of actions. In this paper, we fill this gap by introducing a new learning algorithm for potential games with uncoupled dynamics and memoryless players who act synchronously. We prove the probabilistic convergence of this algorithm to potential function maximizers, which correspond to the optimal system states under appropriate game settings.

Toward better approximation for the minimum-weighted vertex cover (MWVC) problem in multiagent systems, we present a distributed algorithm from the perspective of learning in games. For self-organized coordination and optimization, we see each vertex as a potential game player who makes decisions using local information of its own and the immediate neighbors. The resulting Nash equilibrium is classified into two categories, i.e., the inferior Nash equilibrium (INE) and the dominant Nash equilibrium (DNE). We show that the optimal solution must be a DNE. To achieve better approximation ratios, local rules of perturbation and weighted memory are designed, with the former destroying the stability of an INE and the latter facilitating the refinement of a DNE. By showing the existence of an improvement path from any INE to a DNE, we prove that when the memory length is larger than 1, our algorithm converges in finite time to DNEs, which could not be improved by exchanging the action of a selected node with all its unselected neighbors. Moreover, additional freedom for solution efficiency refinement is provided by increasing the memory length. Finally, intensive comparison experiments demonstrate the superiority of the presented methodology to the state of the art, both in solution efficiency and computation speed.

Consensus conditions for heterogeneous multi-agent sampled-data control systems with nonlinearities and inter-agent communication delay.

Learning from experience is a powerful technique used by humans to improve their problem-solving ability. It is considered to be an inseparable part of intelligence. Since the ability of learning is fundamental for every intelligent system that should improve its problem solving, it should become a key element in design and development of distributed artificial intelligence (DAI) systems.Learning in a distributed system can be divided into two parts, namely learning of individual agents and learning of the system as a whole. Learning of individual agents stems from classical methods of machine learning and involves acquisition of new knowledge, development of abilities for problem solving based on previous experience (using methods of inductive learning, learning from analogy). In this way the activity and problem-solving process of individual agents can be improved. However, it does not mean that the behaviour of the whole distributed system must necessarily improve. Satisfying this requirement may be reached by application of learning methods to the system as a whole. The paper focuses on basic issues of learning in multi-agent systems. Learning of individual agents and of the system as a whole is discussed. There is suggested possible architecture for incorporating learning features into the distributed system DISCIM and simulation experiments for evaluation of the methodology.KeywordsIndividual AgentMessage PassingCooperative LearningLearning AgentInductive LearningThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Motivated by the problem of decentralized direction-tracking, we consider the general problem of cooperative learning in multi-agent systems with time-varying connectivity and intermittent measurements. We propose a distributed learning protocol capable of learning an unknown vector μ from noisy measurements made independently by autonomous nodes. Our protocol is completely distributed and able to cope with the time-varying, unpredictable, and noisy nature of inter-agent communication, and intermittent noisy measurements of μ. Our main result bounds the learning speed of our protocol in terms of the size and combinatorial features of the (time-varying) network connecting the nodes.

Distributed constrained optimization for multi-agent networks with nonsmooth objective functions

We consider several basic consensus problems in multi-agents systems with switching interconnection graphs, where the connections between any two agents are assumed bidirectional for notation simplicity. When the switching interconnection graphs are always connected, it has been shown in the literature that the Laplacian based algorithm always achieves the average consensus. We first extend the discussion to the case where disconnected interconnection graphs are involved, by showing that the average consensus is still achieved if the dwell time ratio between connected graphs and disconnected ones satisfies a specified condition. Next, we consider the case where there is no connected graph but the combination of a set of graphs is connected, and propose two switching strategies for achieving the average consensus. Several numerical examples are provided to show the algorithms.

Multi-agent system (MAS) requires coordination mechanisms to facilitate dynamic collaboration of the intelligent components, with the goal of meeting local and global objectives. This paper deals with the issue of using dynamic epistemic default logic to offer a natural way of communication policies for the management of inter-agent exchanges in MAS. We first explore the communication protocols in MAS that operate in dynamic and highly uncertain environments, and then we add the constrained default sets to realize the extension of dynamic epistemic logic theory and restrict the agent’s inference behavior via constrained epistemic default reasoning. We also specify and reason the characteristic of the dynamic updating when agent meets incompatible knowledge in the logical framework that show the usefulness of logical tools carried out in the dynamic process of information acquisition.

This chapter studies the way to apply stochastic approximation procedure, known as the Robbins–Monro procedure [RM51], to distributed non-convex optimization as well as to communication- and payoff-based learning in potential games.

A fundamental problem in multi agent systems is conflict resolution. A conflict occurs in general when the agents have to deal with inconsistent goals, such as a demand for shared resources. We investigate some theoretical game approaches as efficient methods to examine a class of conflicts in multi agent systems.

As a new area of research, consensus problems for multi-agent systems have been studied extensively in the literature. It is usually assumed that all agents share a common one-dimensional state space. The authors of this study concentrate on the consensus problems of multi-agent systems in the framework of high-dimensional state space. As a basis for the analysis, a general model is first introduced by presenting two typical examples along with some necessary and sufficient conditions for the system to solve a consensus problem. Then the structures of consensus functions are investigated and some algebraic characterisations are given. Finally, a class of switched multi-agent systems are discussed and some sufficient conditions for the possession of the consensus property are provided. Several examples to illustrate the effectiveness of theoretical results are given.

We investigate the fixed-time containment control (FCC) problem of multi-agent systems (MASs) under discontinuous communication. A saturation function is used in the controller to achieve the containment control in MASs. One difference from using a symbolic function is that it avoids the differential calculation process for discontinuous functions, which further ensures the continuity of the control input. Considering the discontinuous communication, a dynamic variable is constructed, which is always non-negative between any two communications of the agent. Based on the designed variable, the dynamic event-triggered algorithm is proposed to achieve FCC, which can effectively reduce controller updating. In addition, we further design a new event-triggered algorithm to achieve FCC, called the team-trigger mechanism, which combines the self-triggering technique with the proposed dynamic event trigger mechanism. It has faster convergence than the proposed dynamic event triggering technique and achieves the tradeoff between communication cost, convergence time and number of triggers in MASs. Finally, Zeno behavior is excluded and the validity of the proposed theory is confirmed by simulation.

Learning in multi-agent systems: a case study of construction claims negotiation

Most of the methods proposed to improve the learning ability in multiagent systems are not appropriate to more complex multiagent learning problems because the state space of each learning agent grows exponentially in terms of the number of partners present in the environment. We propose a novel and robust multiagent architecture to handle these problems. The architecture is based on a learning fuzzy controller whose rule base is partitioned into several different modules. Each module deals with a particular agent in the environment and the fuzzy controller maps the input fuzzy sets to the output fuzzy sets that represent the state space of each learning module and the action space, respectively. Also, each module uses an internal model table to estimate the action of the other agents. Experimental results show the robustness and effectiveness of the proposed approach.

In this paper we have investigated a group of multi agent systems (MAS) in which the agents change their environment and this change has the potential to trigger behaviors in other agents of the group in another time or another position in the environment. The structure makes it possible to conceptualize the group as a super organism incorporating the agents and the environment such that new behaviors are observed from the whole group as a result of the specific distribution of agents in that environment. This distribution exists in many aspects like a super memory (or even a super brain) that exists in the environment and is not limited to memories of the individuals. There is a distributed decision making that is done by the group of agents which, in a higher level consists of both individual and group decision makings, and can be viewed as emergent rather than consciously planned. As the agents change the environment, they decrease the error for the group and hence a distributed learning is also forming between the agents of the group. This implicit learning is related to the implicit memory existing in the environment. These two interpretations of memory and learning are assessed with experimental results where two robots perform a task while they are not aware of their global behavior.