Layered Disclosure: Revealing Agents’ Internals

Research suggests that humans and autonomous agents can be more effective when working together as a combined unit rather than as individual entities. However, most research has focused on autonomous agent design characteristics while ignoring the importance of social interactions and team dynamics. Two experiments examined how the perception of teamwork among human–human and human–autonomous agents and the application of team building interventions could enhance teamwork outcomes. Participants collaborated with either a human or an autonomous agent. In the first experiment, it was revealed that manipulating team structure by considering your human and autonomous partner as a teammate rather than a tool can increase affect and behavior, but does not benefit performance. In the second experiment, participants completed goal setting and role clarification (team building) with their teammate prior to task performance. Team building interventions led to significant improvements for all teamwork outcomes, including performance. Across both studies, participants communicated more substantially with human partners than they did with autonomous partners. Taken together, these findings suggest that social interactions between humans and autonomous teammates should be an important design consideration and that particular attention should be given to team building interventions to improve affect, behavior, and performance.

From its beginning, Artificial Intelligence (AI) has tried to synthesize intelligent behavior by reducing it to artifacts. Two families of approaches have, however, appeared [Zie97].On one hand, Knowledge-Based AI (also named Classical AI or Top-Down AI, closely related to the classical view of cognitive sciences) has developed methodologies for manipulating internal representation models of the world, using notions like knowledge or information. This trend has focused on building expert systems (see [FMN88] for an overview) that own and manage ‘knowledge’ in a sufficiently narrow problem domain. Thus, research in this field has concentrated on knowledge representation and knowledge engineering. Several systems have shown good performance in domains like medical diagnosis and theorem proving.On the other hand, Behavior-Based AI (also named Bottom-Up AI) considers interaction with an environment as an essential feature for intelligent behavior. Research in this area has studied systems that have ‘life’-like attributes, in particular autonomous agents that persist within an environment. Autonomous agents are therefore defined as embodied systems that are designed to fulfill internal goals by their own actions in continuous long-term interaction with the environment in which they are situated [Zie97], [Bee95]. Autonomous agents possess two essential properties: autonomy and embod- iment. Autonomy basically means that an agent acts on its own, without being driven by another agent (possibly a human): it has its own control over its actions and its internal state. Embodiment refers to the fact that an autonomous agent has a “body” that delineates it from its environment in which the agent is situated. It is on this environment that an agent senses and acts through its own specific means (sensors and effectors). The degrees of embodiment define several autonomous agent types: physical embodied agents have a physical body, sensors and effectors (they are robots); simulated embodied agents roam in a simulated physical environment; and software agents lack a body but persist and interact in a complex software environment.Interest has rapidly shifted from the single agent case towards the multiagent case, in which several agents share a common environment and interact with one another. Thismeans that an agent participates in a society of agents, a so-called multi-agent system. These systems result therefore from the or- ganization of multiple agents within an environment, whereby the state of each agent individually changes as a result of its sensing and its own behavior rules. As no global control is applied to the participating agents, every agent has a subjective view of the evolution of the world. Multi-agent systems are therefore interested in an interaction-centered perspective. Dealing with interactions leads naturally to the concept of emergence of behavior, which essentially means that a system’s behavior can only be specified using descriptive categories which are not to be used to describe the behavior of the constituent components (the agents) [For90].KeywordsAutonomous AgentTarget ClassCoordination ModelSubjective CoordinationIntelligent BehaviorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The actions and interactions of independently acting agents in a multi-agent system must be managed if the agents are to function effectively in their shared environment. Norms, which define the obligatory, prohibited and permitted actions for an agent to perform, have been suggested as a possible method for regulating the actions of agents. Norms are local rules designed to govern the actions of individual agents whilst also allowing the agents to achieve a coherent global behaviour. However, there appear to be very few instances of norm-governed multi-agent systems beyond theoretical examples. We describe an implementation strategy for allowing autonomous agents to take a set of norms into account when determining their actions. These norms are implemented using directives, which are local rules specifying actions for an agent to perform depending on its current state. Agents using directives are implemented in a simulation test bed, called Sinatra. Using Sinatra, we investigate the ability of directives to manage agent actions. We begin with directives to manage agent interactions. We find that when agents rely on only local rules they will encounter situations where the local rules are unable to achieve the desired global behaviour. We show how a centralised control mechanism can be used to manage agent interactions that are not successfully handled by directives. Controllers, with a global view of the interaction, instruct the individual agents how to act. We also investigate the use of an existing planning tool to implement the resolution mechanism of a controller. We investigate the ability of directives to coordinate the actions of agents in order to achieve a global objective more effectively. Finally, we present a case study of how directives can be used to determine the actions of autonomous mobile robots.

In simple terms, one can say that team coaching in adversarial domains consists of providing advice to distributed players to help the team to respond effectively to an adversary. We have been researching this problem to find that creating an autonomous coach is indeed a very challenging and fascinating endeavor. This paper reports on our extensive empirical study of coaching in simulated robotic soccer. We can view our coach as a special agent in our team. However, our coach is also capable of coaching other teams other than our own, as we use a recently developed universal coach language for simulated robotic soccer with a set of predefined primitives. We present three methods that extract models from past games and respond to an ongoing game: (i) formation learning, in which the coach captures a team’s formation by analyzing logs of past play; (ii) set-play planning, in which the coach uses a model of the adversary to direct the players to execute a specific plan; (iii) passing rule learning, in which the coach learns clusters in space and conditions that define passing behaviors. We discuss these techniques within the context of experimental results with different teams. We show that the techniques can impact the performance of teams and our results further illustrate the complexity of the coaching problem.

In this paper we improve the performance of an evolutionary method for obtaining team strategies in simulated robot soccer. In the previous method each team strategy was evaluated based on the goals and the goals against of a single game. It is possible for a good team strategy to be eliminated from the population in the evolutionary method as there is a high degree of uncertainty in the simulated soccer field. In order to tackle the problem of uncertainty, we propose a robust evaluation method using match history. The performance of team strategies in the proposed method is measured by the average goals and average goals against. Through a series of computer simulations, we show the effectiveness of our robust evaluation method.

Anticipatory looking on mindreading tasks can indicate our expectation of an agent's action. The challenge is that social situations are often more complex, involving instances where we need to track an agent's false belief to successfully identify the outcome to which an action is directed. If motor processes can guide how action goals are understood, it is conceivable—where that kind of goal ascription occurs in false-belief tasks—for motor representations to account for someone's belief-like state. Testing adults (N = 42) in a real-time interactive helping scenario, we discovered that participants' early mediolateral motor activity (leftwards–rightwards leaning on balance board) foreshadowed the agent's belief-based action preparation. These results suggest fast belief-tracking can modulate motor representations generated in the course of one's interaction with an agent. While adults' leaning, and anticipatory looking, revealed the contribution of fast false-belief tracking, participants did not correct the agent's mistake in their final helping action. These discoveries suggest that adults may not necessarily use another's belief during overt social interaction or find reflecting on another's belief as being normatively relevant to one's own choice of action. Our interactive task design offers a promising way to investigate how motor and mindreading processes may be variously integrated.

When human adults make decisions (e.g., wearing a seat belt), we often consider the negative consequences that would ensue if our actions were to fail, even if we have never experienced such a failure. Do the same considerations guide our understanding of other people’s decisions? In this paper, we investigated whether adults, who have many years of experience making such decisions, and 6- and 7-year-old children, who have less experience and are demonstrably worse at judging the consequences of their own actions, conceive others’ actions as motivated both by reward (how good reaching one’s intended goal would be), and by what we call “danger” (how badly one’s action could end). In two pre-registered experiments, we tested whether adults and 6- and 7-year-old children tailor their predictions and explanations of an agent’s action choices to the specific degree of danger and reward entailed by each action. Across four different tasks, we found that children and adults expected others to negatively appraise dangerous situations and minimize the danger of their actions. Children’s and adults’ judgments varied systematically in accord with both the degree of danger the agent faced and the value the agent placed on the goal state it aimed to achieve. However, children did not calibrate their inferences about how much an agent valued the goal state of a successful action in accord with the degree of danger the action entailed, and adults calibrated these inferences more weakly than inferences concerning the agent’s future action choices. These results suggest that from childhood, people use a degree of danger and reward to make quantitative, fine-grained explanations and predictions about other people’s behavior, consistent with computational models on theory of mind that contain continuous representations of other agents’ action plans.

The Xpilot-AI video game platform allows the creation of artificially intelligent and autonomous control agents. At the same time, the Xpilot environment is highly complex, with very many state variables and action choices. Basic reinforcement learning (RL) techniques are somewhat limited in their application when dealing with such large state- and action-spaces, since the repetition of exposure that is key to their value updates can proceed very slowly. To solve this problem, state abstractions are often generated, allowing learning to move more quickly, but often requiring the programmer to hand-craft state representations, reward functions, and action choices in an ad hoc manner. We apply an automated technique for generating useful abstractions for learning, adaptive Kanerva coding. This method employs a small sub-set of the original states as a proxy for the full environment, updating values over the abstract representative prototype states in a manner analogous to Q-learning. Over time, the set of prototypes is adjusted to provide more effective coverage and abstraction, again automatically. Our results show that this technique allows a simple learning agent to double its survival time when navigating the Xpilot environment, using only a small fraction of the full state-space as a stand-in and greatly increasing the potential for more rapid learning.

Active inference provides a general framework for behavior and learning in autonomous agents. It states that an agent will attempt to minimize its variational free energy, defined in terms of beliefs over observations, internal states and policies. Traditionally, every aspect of a discrete active inference model must be specified by hand, i.e. by manually defining the hidden state space structure, as well as the required distributions such as likelihood and transition probabilities. Recently, efforts have been made to learn state space representations automatically from observations using deep neural networks. In this paper, we present a novel approach of learning state spaces using quantum physics-inspired tensor networks. The ability of tensor networks to represent the probabilistic nature of quantum states as well as to reduce large state spaces makes tensor networks a natural candidate for active inference. We show how tensor networks can be used as a generative model for sequential data. Furthermore, we show how one can obtain beliefs from such a generative model and how an active inference agent can use these to compute the expected free energy. Finally, we demonstrate our method on the classic T-maze environment.

This paper presents the architecture and key components of a simulated humanoid robot soccer team, UT Austin Villa, which was designed to compete in the RoboCup 3D simulation competition. These key components include (1) an omnidirectional walk engine and associated walk parameter optimization framework, (2) an inverse kinematics based kicking architecture, and (3) a dynamic role assignment and positioning system. UT Austin Villa won the RoboCup 2011 3D simulation competition in convincing fashion by winning all 24 games it played. During the course of the competition the team scored 136 goals while conceding none. We analyze the effect of each component in isolation and show through extensive experiments that the complete team significantly outperforms all the other teams from the competition.

Interactive teaching methods in contemporary higher education

With the emergence of fast and standardized communication infrastructures over which separately designed agents of different organizations can interact in real-time, there is an increasing demand for cooperation mechanisms that allow to carry out inter-organizational cooperations in a safe way. The lack of external control over an agent's decisions, resources and actions hamper the usage of traditional transaction and workflow technology to make self-interested agents cooperate, i.e., agents cannot not be forced from a mediating cooperation instance to continue a cooperation. The challenge is, therefore, to design a cooperation mechanism that motivates cooperating agents to carry out a specified contract and, in case of unilateral defection, ensures that none of the cooperators can benefit from the situation. In this paper we present a domain independent framework how non-enforceable cooperations can be made safe against unilateral defection. We have developed a utility-based scheduling algorithm that keeps a cooperation in equilibrium and that motivates agents to continue a cooperation as long as it is for all participants beneficial.

As it achieves a goal on behalf of its human user, an autonomous agent's actions may have side effects that change features of its environment in ways that negatively surprise its user. An agent that can be trusted to operate safely should thus only change features the user has explicitly permitted. We formalize this problem, and develop a planning algorithm that avoids potentially negative side effects given what the agent knows about (un)changeable features. Further, we formulate a provably minimax-regret querying strategy for the agent to selectively ask the user about features that it hasn't explicitly been told about. We empirically show how much faster it is than a more exhaustive approach and how much better its queries are than those found by the best known heuristic.

The Meca Sapiens Blueprint is a System Architecture to build conscious machines. A system based on the Meca Sapiens Blueprint will be a unique and autonomous entity that is self-aware and capable of intentional mutations. It will interact with humans as an independent and self-directed being. The Blueprint is complete (at the system architecture level) and ready for design and implementation. Its content is unique and entirely original. Using the Blueprint, a development team that is familiar with conventional software techniques can transform an autonomous agent into a conscious synthetic being. The Meca Sapiens Blueprint is primarily intended for software developers. However, its content is readily accessible to the informed layman. It will be of interest to anyone interested in Artificial Intelligence or concerned about the impacts of this emerging technology on society. By discovering the mechanisms that generate synthetic consciousness, all readers will also gain a renewed understanding of human consciousness itself.

In multi-agent systems, anticipating the behavior of other agents constitutes a difficult problem. In this paper we present the case where a cognitive agent is inserted into an unknown environment composed of different kinds of other objects and agents; our cognitive agent needs to incrementally learn a model of the environment dynamics, doing it only from its interaction experience; the learned model can then be used to define a policy of actions. It is relatively easy to do so when the agent interacts with static objects, with simple mobile objects, or with trivial reactive agents; however, when the agent deals with other complex agents that may change their behaviors according to some non-directly observable internal properties (like emotional or intentional states), the construction of a model becomes significantly harder. The complete system can be described as a Factored and Partially Observable Markov Decision Process (FPOMDP); our agent implements the Constructivist Anticipatory Learning Mechanism (CALM) algorithm, and the experiment (called mept) shows that the induction of non-observable variables enable the agent to learn a deterministic model of most of the system events (if it represents a well-structured universe), allowing it to anticipate other agents actions and to adapt to them, even if some interactions appear as non-deterministic in a first sight.KeywordsFactored Partially Observable MDPsAnticipatory Learning MechanismsModel-Based RLConstructivist Artificial IntelligenceAgent Intersubjectivity