Evolving Coordinated Agents

Multi-agent systems consist of a number of interacting autonomous agents, each of which is capable of sensing its environment (including other agents) and deciding to act in order to achieve its own objectives. In order to guarantee the overall design objectives of multi-agent systems, the behavior of individual agents and their interactions need to be regulated and coordinated [23,29,30]. The development of multi-agent systems therefore requires programming languages that facilitate the implementation of individual agents as well as mechanisms that control and regulate individual agents' behaviors. It also requires computational tools to test and verify programs that implement multi-agent systems [7].

The study of multi-agent systems (MAS) focuses on systems in which many intelligent agents interact within an environment. The agents are considered to be autonomous entities. MAS can be used to solve problems that are difficult or impossible for an individual agent to solve. The main feature which is achieved when developing MAS, if they work, is flexibility, since MAS can be added to, modified and reconstructed, without the need for detailed rewriting of the application. MAS can manifest self-organization as well as self-steering related complex behaviors even when the individual strategies of all their agents are simple. The goal of MAS research is to find methods that allow us to build complex systems composed of autonomous agents who, while operating on local knowledge and possessing only limited abilities, are nonetheless capable of enacting the desired global behaviors. We want to know how to take a description of what a system of agents should do and break it down into individual agent behaviors. This thesis investigates the problem when discrete-time MAS are consensusable under undirected graph. A discussion is provided to show how the problem differs from continuous time system. Then a consensusability condition is derived in terms of the Mahler measure of the agent system for single input single out systems (SISO) and result shows that there is an improved consensusability by a power of two. An algorithm is proposed for distributed consensus feedback control law when the consensusability holds. Also the case of output feedback is considered in which the consensusability problem becomes more complicated. To solve this we decompose the problem into two parts i.e. state feedback and state estimation. Simulation results demonstrate the effectiveness of the established results.

: The biologically-inspired swarm paradigm is being used to design self-organizing systems of locally interacting agents. A major difficulty in designing swarms with desired characteristics is understanding the causal relation between individual agent and collective behaviors. Mathematical analysis of swarm dynamics can address this difficulty to gain insight into system design. This project developed a formal framework for mathematical modeling and analysis of multi-agent swarms. Though the behavior of an individual agent can be considered to be stochastic and unpredictable, collective behavior of a swarm can have a simple probabilistic description. We showed that a class of mathematical models that describe the dynamics of collective behavior of multi-agent systems can be written down from the details of the individual agent controller. We have successfully applied this formalism to study collective behavior of distributed robot systems for which a body of experimental and simulations data exists.

This paper reviews the proposed opponent modeling algorithms within the soccer simulation domain. RoboCup soccer simulation 2D is a rich multi agent environment where opponent modeling plays a crucial role. In multi agent systems with adversarial and cooperative agents, team agents should be adapted to the current environment and opponent in order to propose appropriate and effective counteractions. Predicting the opponent's future behaviors during competition allows for more informed decisions. We divide opponent modeling into two categories of individual agent behaviors and team behaviors. Individual behaviors concern modeling the low-level behaviors of individual opponent agents, however in team behaviors, the high-level strategy of the entire team like formation, offensive and defensive system, is recognized. Several methods have been proposed to create different models of opponents to improve the performance of teams in an essential aspect. In this paper, we review the approaches to the problem of opponent modeling published from 2000 to 2010.

Like all systems, multi-agent systems need to be verified during the design cycle. But the test of multi-agent systems is more diffi-cult than the test of classic software. Indeed, it implies to test more than agent functionalities: individual agent behavior, inter-action between agents and global system need to be tested. Sever-al testing methods compensate these difficulties, but do not cover all the aspects of multi-agent systems. In this paper, we compare software testing and multi-agent systems testing and particularly in embedded context. Then major multi-agent system testing techniques are analyzed, with the AEIO facets for multi-agent systems. Finally, we propose a strategy to conduct the testing activity for embedded MAS.

Adaptation and Evolution in Dynamic Persistent Environments

A key difficulty in the design of multi-agent robotic systems is the size and complexity of the space of possible designs. In order to make principled design decisions, an understanding of the many possible system configurations is essential. To this end, we present a taxonomy that classifies multi-agent systems according to communication, computational and other capabilities. We survey existing efforts involving multi-agent systems according to their positions in the taxonomy. We also present additional results concerning multi-agent systems, with the dual purposes of illustrating the usefulness of the taxonomy in simplifying discourse about robot collective properties, and also demonstrating that a collective can be demonstrably more powerful than a single unit of the collective.

This paper studies a consensus protocol design for leader-following multi-agent systems (MASs) via stochastic sampling information. Unlike traditional sampled-data control, this paper is focused on the stochastically varying sample intervals with a given probability by the Bernoulli distribution. Based on the Lyapunov–Krasovskii functional and reciprocally convex technique, the sufficient conditions are derived for the stochastic sampled-data protocol design of the error system, which guarantees that the following agent's states can reach an agreement on the leader's state. Finally, the numerical examples are provided to demonstrate the effectiveness of the developed theoretical results.

Most agent architectures implement autonomous agents that use extensive interaction protocols and social laws to control interactions in order to ensure that the correct behaviors result during run-time. These agents, organized into multi-agent systems in which all agents adhere to predefined interaction protocols, are well suited to the analysis, design and implementation of complex systems in environments where it is possible to predict interactions during the analysis and design phases. In these multi-agent systems, intelligence resides in individual autonomous agents, rather than in the collective behavior of the individual agents. These agents are commonly referred to as “next-generation” or intelligent components, which are difficult to implement using current component-based architectures. In most distributed environments, such as the Internet, it is not possible to predict interactions during analysis and design. For a complex system to be able to adapt in such an uncertain and non-deterministic environment, we propose the use of agencies, consisting of simple agents, which use probabilistic reasoning to adapt to their environment. Our agents collectively implement distributed Bayesian networks, used by the agencies to control behaviors in response to environmental states. Each agency is responsible for one or more behaviors, and the agencies are structured into heterarchies according to the topology of the underlying Bayesian networks. We refer to our agents and agencies as “Bayesian agents” and “Bayesian agencies.”

This paper proposes a methodology for developing multi-agent systems. Several design considerations for robotic agents are discussed. A script specifies a multi-agent system by defining the behavior of each agent. An agent is modeled by an extended finite state automaton. By using the state transition rules as the building blocks, an agent as well as a multi-agent system can be developed efficiently. This methodology is further applied to multi-agent robotic systems in which reactivity and interaction are important issues. A script-based architecture is developed to implement the agents in multi-agent robotic systems. This architecture considers not only the control architecture of a single agent, but also the interactions among multiple agents and their environments. A script for the object-sorting task problem is utilized for demonstration.

The development of theoretical-based methods for the assessment of multi-agent systems properties is of critical importance. This work is a preliminary investigation on methodologies for modeling, analysis and design of multi-agent systems. Multi-agent systems are regarded as discrete-event dynamic systems and Petri nets are used as a modeling tool to assess the structural properties of the multi-agent system. Our methodology consists of defining a simple multi-agent system based on the abstract architecture for intelligent agents. The abstract architecture is modeled as a discrete-event system using Petri nets and structural analysis of the net provides an assessment of the communication and coordination properties of the multi-agent system. Deadlock avoidance in the multi-agent system is considered as an initial key property, and it is evaluated using liveness and boundedness properties of the Petri net model.

The modeling and design of multi-agent systems is imperative for applications in the evolving intelligence of unmanned systems. In this paper, we propose a multi-agent system design that is used to build a system for training a team of unmanned surface vehicles (USVs) where no historical data concerning the behavior is available. In this approach, agents are built as the physical controller of each USV and their cooperative decisions used for the USVs’ group coordination. To make our multi-agent system intelligently coordinate USVs, we built a multi-agent-based learning system. First, an agent-based data collection platform is deployed to gather competition data from agents’ observation for on-line learning tasks. Second, we design a genetic-based fuzzy rule training algorithm that is capable of optimizing agents’ coordination decisions in an accumulated manner. The simulation results of this study demonstrate that our proposed training approach is feasible and able to converge to a stable action selection policy towards efficient multi-USVs’ cooperative decision making.

: Controls research under this program concentrated on the development of a network thermodynamic stabilization framework for hybrid control design of large-scale and multiagent aerospace systems. In particular, we concentrate on hybrid control, hierarchical control, impulsive dynamical systems, nonnegative dynamical systems, compartmental systems, large-scale systems, nonlinear switching control, cooperative control, and adaptive control. Application areas include large flexible interconnected space structures, spacecraft stabilization, cooperative control of unmanned air vehicles, network systems, swarms of air and space vehicle formations, and pharmacological systems.

This paper focuses the development of an adaptive observer-based distributed fault estimation observer (DFEO) for multi-agent nonlinear time-delay systems under a directed communication topology. The process involves constructing a fault estimation observer for each agent based on their relative output estimation errors. An adaptive DFEO design is proposed for multi-agent systems with a directed communication topology. Furthermore, investigating the effect of faulty agents on the criteria of fault detection on healthy agents. The proposed design is represented with Linear Matrix Inequalities (LMIs) to establish a systematic design procedure. The study also includes simulation results to demonstrate the effectiveness of the proposed design techniques.

The demand for creating increasingly dynamic, autonomous and proactive software systems is challenging for the traditional Multi-agent Systems (MASs) approaches. Such requirement has given rise to adaptive software agents approaches. At the same time, norm is an essential and challenging feature that still tends to be addressed in adaptive MAS. In fact, norms to regulate agent behavior is still a vague concept that has not been properly investigated in terms of modeling and implementation. Even though many researchers have proposed modeling languages to deal with different abstractions, these languages fail to support the modeling of abstractions, such as adaptation and norms. Even more severe is the fact that little has been done to support the systematic design of Adaptive Normative Multi-Agent Systems (ANMASs). To facilitate the design and development of ANMASs, this paper presents a new metamodel, as well as language support, as means to provide tools to enable software developers. The proposed metamodel fosters a better understanding of the way agents are able to change their behaviors to deal with norms and captures interactions between agent’s norms and adaptation. To this end, our research is organized into five steps: (i) a literature review to identify the limitations of existing approaches related to ANMAS modeling; (ii) propose a new metamodel to support adaptative and normative concepts; (iii) propose a new language for modeling ANMASs; (iv) perform a qualitative and quantitative evaluation of the proposed language using a real case scenario, and (v) an empirical evaluation. The proposed metamodel and its associated modeling language advances the state of the art in modeling MASs and the approach is assessed in terms of correctness, time and difficulty. Our initial results revealed that our approach can be feasibly applied in a real world application, and is less difficult to apply and requires less time in comparison with a traditional approach. As software applications become more dynamic and adaptive, we believe it is essential to support developers to model MASs with abstractions such as adaptive agents, norms and their relationships. Such information can be foundational to steer future research on modeling adaptive agents capable of understanding and dealing with norms and adaptation.