Analyzing the performance of distributed algorithms

Coordinated multi-agent reinforcement learning (MARL) provides a promising approach to scaling learning in large cooperative multi-agent systems. Distributed constraint optimization (DCOP) techniques have been used to coordinate action selection among agents during both the learning phase and the policy execution phase (if learning is off-line) to ensure good overall system performance. However, running DCOP algorithms for each action selection through the whole system results in significant communication among agents, which is not practical for most applications with limited communication bandwidth. In this paper, we develop a learning approach that generalizes previous coordinated MARL approaches that use DCOP algorithms and enables MARL to be conducted over a spectrum from independent learning (without communication) to fully coordinated learning depending on agents' communication bandwidth. Our approach defines an interaction measure that allows agents to dynamically identify their beneficial coordination set (i.e., whom to coordinate with) in different situations and to trade off its performance and communication cost. By limiting their coordination set, agents dynamically decompose the coordination network in a distributed way, resulting in dramatically reduced communication for DCOP algorithms without significantly affecting overall learning performance. Essentially, our learning approach conducts co-adaptation of agents' policy learning and coordination set identification, which outperforms approaches that sequence them.

Distributed Constraint Optimization (DCOP) is rapidly emerging as a prominent technique for multiagent coordination. Unfortunately, rigorous quantitative evaluations of privacy loss in DCOP algorithms have been lacking despite the fact that agent privacy is a key motivation for applying DCOPs in many applications. Recently, Maheswaran et al. [3, 4] introduced a framework for quantitative evaluations of privacy in DCOP algorithms, showing that early DCOP algorithms lose more privacy than purely centralized approaches and questioning the motivation for applying DCOPs. Do state-of-the art DCOP algorithms suffer from a similar shortcoming? This paper answers that question by investigating the most efficient DCOP algorithms, including both DPOP and ADOPT.

The establishment of a common operating picture through shared situation awareness is critical for the management of first responder operations. Situation awareness is the knowledge of the state and physical location of people, objects and terrain within an area of interest. First responder operations can be quite complex, involving the coordination of and collaboration among teams distributed over large areas. Adding to this complexity is the use of heterogeneous networks with diverse radios, traffic flows and applications. This paper describes some initial results using distributed constraint optimization (DCOP) techniques to facilitate situation awareness for first responders. A brief background on DCOP is presented, along with how DCOP can be applied to enable situation awareness. An example demonstrating the application of DCOP algorithms to a particular situation awareness problem is presented. Finally, some critical issues that arise when utilizing DCOP are described, along with some approaches to addressing these issues. By integrating these DCOP techniques with situation awareness tools, we believe that many of the information distribution, interoperability, and networking difficulties faced in the first responder environment can be overcome.

This paper proposes a system that supports effective evacuation from danger using a Distributed Constraint Optimization Algorithm. The use of the algorithm facilitates the assisted optimization of people's evacuation timing, by estimating the location of evacuees. This system enables assistance in terms of evacuation guidance to be given to relieve congestion, by calculating evacuation routes via an ad-hoc network of evacuees' mobile devices (phones, PCs, etc.), intercommunication function, and location information.

Distributed Constraint Optimization Problems (DCOPs) can be used to model a number of multi-agent coordination problems. The conventional DCOP model assumes that the subproblem that each agent is responsible for (i.e. the mapping of nodes in the constraint graph to agents) is part of the model description. While this assumption is often reasonable, there are many applications where there is some flexibility in making this assignment. In this paper, we focus on this gap and make the following contributions: (1) We formulate this problem as an optimization problem, where the goal is to find an assignment that minimizes the completion time of the DCOP algorithm (e.g. Action-GDL or Max-Sum) that operates on this mapping. (2) We propose a novel heuristic, called MNA, that can be executed in a centralized or decentralized manner. (3) Our empirical evaluation illustrates a substantial reduction in completion time, ranging from 16% to 40%, without affecting the solution quality of the algorithms, compared to the current state of the art. In addition, we observe empirically that the completion time obtained from our approach is near-optimal; it never exceeds more than 10% of what can be achieved from the optimal node-to-agent mapping.

Complete algorithms have been proposed to solve problems modelled as distributed constraint optimization (DCOP). However, there are only few attempts to address real world scenarios using this formalism, mainly because of the complexity associated with those algorithms. In the present work we compare three complete algorithms for DCOP, aiming at studying how they perform in complex and dynamic scenarios of increasing sizes. In order to assess their performance we measure not only standard quantities such as number of cycles to arrive to a solution, size and quantity of exchanged messages, but also computing time and quality of the solution which is related to the particular domain we use. This study can shed light in the issues of how the algorithms perform when applied to problems other than those reported in the literature (graph coloring, meeting scheduling, and distributed sensor network).

The Distributed Constraint Optimization Problem (DCOP) has been studied as a fundamental optimization problem that represents various problems on multiagent systems. We focus on the asymmetric DCOPs where each objective function is differently defined as an evaluation of an agent. This class of problems is studied as a multi-objective problem for the preferences of individual agents. In this work, we investigate the possibility of a solution framework based on relaxation methods as a scalable and inexact solution approach for this class of problems. We address a bottleneck problem that minimizes the worst-case cost value. As the first study, we apply a penalty method to the minimization problems of the maximum cost values.

Reducing the information revealed by agents is an important requirement in multiagent systems. Privacy preservation has been studied in several research areas including automated negotiating agents and Distributed Constraint Optimization Problems (DCOPs). Although one of recent approaches for privacy preservation is applying secure computation to negotiation process, publishing some information during the negotiation is necessary in the cases where agents should evaluate the reason of their agreement. We propose a negotiation framework consisting of two types of Asymmetric Multi-Objective DCOPs that are repeatedly solved to gradually publish the utility values of agents until an agreement is reached. One problem defines the selection of utility values that are gradually published by agents while another is the problem consist of the published utility values and related assignments to variables. With the general formalization of DCOPs, there are opportunities to represent several classes of negotiation problems. We define several constraints for heuristic strategies that choose the utility values to be published in the next negotiation step. The criterion of social welfare to consider the multiple objectives among individual agents are also introduced. Moreover, we apply two additional preprocessing methods to adjust the utilities among agents by transferring some amount of utility values and to reduce the complexity of the original problem by approximating several relationships of agent pairs. We experimentally evaluate the effect and influence of our proposed approaches including the heuristics to select the published utility information and the additional preprocessing methods.KeywordsPrivacyDecentralized problem solvingAsymmetric constraint optimizationMulti-objectiveSocial welfareNegotiationMultiagent system

A vessel rotation is a sequence in which a vessel visits the different terminals in a large port for loading and unloading containers. Constructing a vessel rotation involves making a lot of appointments through communication. Nowadays, the communication between vessel operator and terminal operator is done through telephone, fax, and e-mail based on the operator’s knowledge and experience. With this practice, it happens frequently that appointments cannot be met either by the vessel operator or by the terminal operator. In this paper we propose a technique to solve the vessel rotation planning problem in an automated way. For this, we propose to model the problem as a distributed constraint optimization problem (DCOP). DCOP has generated much interest in the artificial intelligence and constraint programming communities, and a number of algorithms have been developed for solving DCOPs. However, DCOP algorithms have been rarely used in practice. This paper presents a first approach based on DCOP for generating reasonable and optimized rotation plans for vessel operators. To evaluate the performance of the proposed approach, different DCOP algorithms are considered. We evaluate the solution quality and computational and communication costs of these algorithms when solving the vessel rotation planning problem.

We propose a novel incomplete cooperative algorithm for distributed constraint optimization problems (DCOPs) denoted as Cooperative Constraint Approximation (CoCoA). The key strategy of the algorithm is to use a semi-greedy approach in which knowledge is distributed amongst neighboring agents, and assigning a value only once instead of an iterative approach. Furthermore, CoCoA uses a unique-first approach to improve the solution quality. It is designed such that it can solve DCOPs as well as Asymmetric DCOPS, with only few messages being communicated between neighboring agents. Experimentally, through evaluating graph coloring problems, randomized (A)DCOPs, and a sensor network communication problem, we show that CoCoA is able to very quickly find solutions of high quality with a smaller communication overhead than state-of-the-art DCOP solvers such as DSA, MGM-2, ACLS, MCS-MGM and Max-Sum. In our asymmetric use case problem of a sensor network, we show that CoCoA not only finds the best solution, but also finds this solution faster than any other algorithm.

Distributed Constraint Optimization Problems (DCOPs) have been used to model a number of multi-agent coordination problems. In DCOPs, agents are assumed to have complete information about the utility of their possible actions. However, in many real-world applications, such utilities are stochastic due to the presence of exogenous events that are beyond the direct control of the agents. This paper addresses this issue by extending the standard DCOP model to Expected Regret DCOP (ER-DCOP) for DCOP applications with uncertainty in constraint utilities. Different from other approaches, ER-DCOPs aim at minimizing the overall expected regret of the problem. The paper proposes the ER-DPOP algorithm for solving ER-DCOPs, which is complete and requires a linear number of messages with respect to the number of agents in the problem. We further present two implementations of ER-DPOP---GPU- and ASP-based implementations---that orthogonally exploit the problem structure and present their evaluations on random networks and power network problems.

Distributed constraint optimization problems (DCOPs) are well-suited for modeling multi-agent coordination problems where the primary interactions are between local subsets of agents. However, one limitation of DCOPs is the assumption that the constraint rewards are without uncertainty. Researchers have thus extended DCOPs to Stochastic DCOPs (SDCOPs), where rewards are sampled from known probability distribution reward functions, and introduced algorithms to find solutions with the largest expected reward. Unfortunately, such a solution might be very risky, that is, very likely to result in a poor reward. Thus, in this paper, we make three contributions: (1) we propose a stricter objective for SDCOPs, namely to find a solution with the most stochastically dominating probability distribution reward function; (2) we introduce an algorithm to find such solutions; and (3) we show that stochastically dominating solutions can indeed be less risky than expected reward maximizing solutions.

The Distributed Pseudo-tree Optimization Procedure (DPOP) is a well-known message passing algorithm that provides optimal solutions to Distributed Constraint Optimization Problems (DCOPs) in cooperative multi-agent systems. However, the traditional DCOP formulation does not consider constraints that must be satisfied (hard constraints), rather it concentrates only on constraints that place no restriction on satisfaction (soft constraints). This is a serious shortcoming as many real-world applications involve both types of constraints. Traditional DPOP algorithms are not able to benefit from the existence of hard constraints, where an additional calculation is required to handle such constraints. This results in longer runtimes. Thus scalability remains an issue. Additionally, in the standard DPOP, the agents are arranged as a Depth First Search (DFS) pseudo-tree, but recent work has shown that the construction of pseudo-trees in this way often leads to chain-like communication structures that greatly impair the algorithm’s performance. To address these issues, we develop an algorithm that speeds up the DPOP algorithm by reducing the size of the messages exchanged and increases parallelism in the pseudo tree. For this purpose, initially, we improve the path for exchanging messages. Next, we introduce a new form of constraint propagation, which we call cross-edge consistency. Our theoretical evaluation shows that our proposed algorithm is complete and correct. In empirical evaluations, our algorithm achieves a significant reduction in the runtime, ranging from 4% to 96%, compared to the state-of-the-art.

Multi-agent systems have already been successfully applied to a variety of traffic control problems and demonstrated the potential to lower travel times and environmental impact. Sharing this goal, we have developed iCO2, an online tool for training eco-friendly driving in a multi-user three-dimensional environment. iCO2 supports eco-driving practice by instructing computer-controlled agents, such as traffic lights and other vehicles, to create traffic situations that make eco-driving more difficult. Hence the agents take the role of "opponents" that try to achieve the optimal challenge level for the skill level of each user. The research challenge is to find the optimal challenge level for all user drivers in a shared simulation space that (1) involves both controllable entities ("opponents") and non-controllable entities (users) and (2) is highly dynamic, with dependencies between entities being created and destroyed in real time. We try to solve this problem by modeling the scenario as a distributed constraint optimization problem (DCOP). The main contribution of our paper is the application of a DCOP algorithm to such a new type of application scenario. We evaluate our approach by running scenarios both in terms of speed and optimality of the solutions proposed by the DCOP algorithm.

Distributed Constraint Optimization Problems (DCOPs) are a powerful tool to model multi-agent coordination problems that are distributed by nature. The formulation is suitable for problems where variables are discrete and constraint utilities are represented in tabular form. However, many real-world applications have variables that are continuous and tabular forms thus cannot accurately represent constraint utilities. To overcome this limitation, researchers have proposed the Continuous DCOP (C-DCOP) model, which are DCOPs with continuous variables. But existing approaches usually come with some restrictions on the form of constraint utilities and are without quality guarantees. Therefore, in this paper, we (i) propose an exact algorithm to solve a specific subclass of C-DCOPs; (ii) propose an approximation method with quality guarantees to solve general C-DCOPs; (iii) propose additional C-DCOP algorithms that are more scalable; and (v) empirically show that our algorithms outperform existing state-of-the-art C-DCOP algorithms when given the same communication limitations.