Speeding up distributed pseudo-tree optimization procedures with cross edge consistency to solve DCOPs

Coordinated multi-agent reinforcement learning (MARL) provides a promising approach to scaling learning in large cooperative multi-agent systems. Distributed constraint optimization (DCOP) techniques have been used to coordinate action selection among agents during both the learning phase and the policy execution phase (if learning is off-line) to ensure good overall system performance. However, running DCOP algorithms for each action selection through the whole system results in significant communication among agents, which is not practical for most applications with limited communication bandwidth. In this paper, we develop a learning approach that generalizes previous coordinated MARL approaches that use DCOP algorithms and enables MARL to be conducted over a spectrum from independent learning (without communication) to fully coordinated learning depending on agents' communication bandwidth. Our approach defines an interaction measure that allows agents to dynamically identify their beneficial coordination set (i.e., whom to coordinate with) in different situations and to trade off its performance and communication cost. By limiting their coordination set, agents dynamically decompose the coordination network in a distributed way, resulting in dramatically reduced communication for DCOP algorithms without significantly affecting overall learning performance. Essentially, our learning approach conducts co-adaptation of agents' policy learning and coordination set identification, which outperforms approaches that sequence them.

The establishment of a common operating picture through shared situation awareness is critical for the management of first responder operations. Situation awareness is the knowledge of the state and physical location of people, objects and terrain within an area of interest. First responder operations can be quite complex, involving the coordination of and collaboration among teams distributed over large areas. Adding to this complexity is the use of heterogeneous networks with diverse radios, traffic flows and applications. This paper describes some initial results using distributed constraint optimization (DCOP) techniques to facilitate situation awareness for first responders. A brief background on DCOP is presented, along with how DCOP can be applied to enable situation awareness. An example demonstrating the application of DCOP algorithms to a particular situation awareness problem is presented. Finally, some critical issues that arise when utilizing DCOP are described, along with some approaches to addressing these issues. By integrating these DCOP techniques with situation awareness tools, we believe that many of the information distribution, interoperability, and networking difficulties faced in the first responder environment can be overcome.

A vessel rotation is a sequence in which a vessel visits the different terminals in a large port for loading and unloading containers. Constructing a vessel rotation involves making a lot of appointments through communication. Nowadays, the communication between vessel operator and terminal operator is done through telephone, fax, and e-mail based on the operator’s knowledge and experience. With this practice, it happens frequently that appointments cannot be met either by the vessel operator or by the terminal operator. In this paper we propose a technique to solve the vessel rotation planning problem in an automated way. For this, we propose to model the problem as a distributed constraint optimization problem (DCOP). DCOP has generated much interest in the artificial intelligence and constraint programming communities, and a number of algorithms have been developed for solving DCOPs. However, DCOP algorithms have been rarely used in practice. This paper presents a first approach based on DCOP for generating reasonable and optimized rotation plans for vessel operators. To evaluate the performance of the proposed approach, different DCOP algorithms are considered. We evaluate the solution quality and computational and communication costs of these algorithms when solving the vessel rotation planning problem.

We propose a novel incomplete cooperative algorithm for distributed constraint optimization problems (DCOPs) denoted as Cooperative Constraint Approximation (CoCoA). The key strategy of the algorithm is to use a semi-greedy approach in which knowledge is distributed amongst neighboring agents, and assigning a value only once instead of an iterative approach. Furthermore, CoCoA uses a unique-first approach to improve the solution quality. It is designed such that it can solve DCOPs as well as Asymmetric DCOPS, with only few messages being communicated between neighboring agents. Experimentally, through evaluating graph coloring problems, randomized (A)DCOPs, and a sensor network communication problem, we show that CoCoA is able to very quickly find solutions of high quality with a smaller communication overhead than state-of-the-art DCOP solvers such as DSA, MGM-2, ACLS, MCS-MGM and Max-Sum. In our asymmetric use case problem of a sensor network, we show that CoCoA not only finds the best solution, but also finds this solution faster than any other algorithm.

Distributed Constraint Optimization Problems (DCOPs) have been used to model a number of multi-agent coordination problems. In DCOPs, agents are assumed to have complete information about the utility of their possible actions. However, in many real-world applications, such utilities are stochastic due to the presence of exogenous events that are beyond the direct control of the agents. This paper addresses this issue by extending the standard DCOP model to Expected Regret DCOP (ER-DCOP) for DCOP applications with uncertainty in constraint utilities. Different from other approaches, ER-DCOPs aim at minimizing the overall expected regret of the problem. The paper proposes the ER-DPOP algorithm for solving ER-DCOPs, which is complete and requires a linear number of messages with respect to the number of agents in the problem. We further present two implementations of ER-DPOP---GPU- and ASP-based implementations---that orthogonally exploit the problem structure and present their evaluations on random networks and power network problems.

Distributed Constraint Optimization Problems (DCOPs) can be used to model a number of multi-agent coordination problems. The conventional DCOP model assumes that the subproblem that each agent is responsible for (i.e. the mapping of nodes in the constraint graph to agents) is part of the model description. While this assumption is often reasonable, there are many applications where there is some flexibility in making this assignment. In this paper, we focus on this gap and make the following contributions: (1) We formulate this problem as an optimization problem, where the goal is to find an assignment that minimizes the completion time of the DCOP algorithm (e.g. Action-GDL or Max-Sum) that operates on this mapping. (2) We propose a novel heuristic, called MNA, that can be executed in a centralized or decentralized manner. (3) Our empirical evaluation illustrates a substantial reduction in completion time, ranging from 16% to 40%, without affecting the solution quality of the algorithms, compared to the current state of the art. In addition, we observe empirically that the completion time obtained from our approach is near-optimal; it never exceeds more than 10% of what can be achieved from the optimal node-to-agent mapping.

Complete algorithms have been proposed to solve problems modelled as distributed constraint optimization (DCOP). However, there are only few attempts to address real world scenarios using this formalism, mainly because of the complexity associated with those algorithms. In the present work we compare three complete algorithms for DCOP, aiming at studying how they perform in complex and dynamic scenarios of increasing sizes. In order to assess their performance we measure not only standard quantities such as number of cycles to arrive to a solution, size and quantity of exchanged messages, but also computing time and quality of the solution which is related to the particular domain we use. This study can shed light in the issues of how the algorithms perform when applied to problems other than those reported in the literature (graph coloring, meeting scheduling, and distributed sensor network).

Distributed constraint optimization problems (DCOPs) are well-suited for modeling multi-agent coordination problems where the primary interactions are between local subsets of agents. However, one limitation of DCOPs is the assumption that the constraint rewards are without uncertainty. Researchers have thus extended DCOPs to Stochastic DCOPs (SDCOPs), where rewards are sampled from known probability distribution reward functions, and introduced algorithms to find solutions with the largest expected reward. Unfortunately, such a solution might be very risky, that is, very likely to result in a poor reward. Thus, in this paper, we make three contributions: (1) we propose a stricter objective for SDCOPs, namely to find a solution with the most stochastically dominating probability distribution reward function; (2) we introduce an algorithm to find such solutions; and (3) we show that stochastically dominating solutions can indeed be less risky than expected reward maximizing solutions.

The Distributed Constraint Optimization Problem (DCOP) has been studied as a fundamental optimization problem that represents various problems on multiagent systems. We focus on the asymmetric DCOPs where each objective function is differently defined as an evaluation of an agent. This class of problems is studied as a multi-objective problem for the preferences of individual agents. In this work, we investigate the possibility of a solution framework based on relaxation methods as a scalable and inexact solution approach for this class of problems. We address a bottleneck problem that minimizes the worst-case cost value. As the first study, we apply a penalty method to the minimization problems of the maximum cost values.

Reducing the information revealed by agents is an important requirement in multiagent systems. Privacy preservation has been studied in several research areas including automated negotiating agents and Distributed Constraint Optimization Problems (DCOPs). Although one of recent approaches for privacy preservation is applying secure computation to negotiation process, publishing some information during the negotiation is necessary in the cases where agents should evaluate the reason of their agreement. We propose a negotiation framework consisting of two types of Asymmetric Multi-Objective DCOPs that are repeatedly solved to gradually publish the utility values of agents until an agreement is reached. One problem defines the selection of utility values that are gradually published by agents while another is the problem consist of the published utility values and related assignments to variables. With the general formalization of DCOPs, there are opportunities to represent several classes of negotiation problems. We define several constraints for heuristic strategies that choose the utility values to be published in the next negotiation step. The criterion of social welfare to consider the multiple objectives among individual agents are also introduced. Moreover, we apply two additional preprocessing methods to adjust the utilities among agents by transferring some amount of utility values and to reduce the complexity of the original problem by approximating several relationships of agent pairs. We experimentally evaluate the effect and influence of our proposed approaches including the heuristics to select the published utility information and the additional preprocessing methods.KeywordsPrivacyDecentralized problem solvingAsymmetric constraint optimizationMulti-objectiveSocial welfareNegotiationMultiagent system

Traffic congestion is a widespread epidemic that continually wreaks havoc in urban areas. Traffic jams, car wrecks, construction delays, and other causes of congestion, can turn even the biggest highways into a parking lot. Several congestion mitigation strategies are being studied, many focusing on micro-simulation of traffic to determine how modifying road structures will affect the flow of traffic and the networking perspective of vehicle-to-vehicle communication. Vehicle routing on a network of roads and intersections can be modeled as a distributed constraint optimization problem and solved using a range of centralized to decentralized techniques. In this paper, we present a constraint optimization model of a traffic routing problem. We produce congestion data using a sinusoidal wave pattern and vary its amplitude (saturation) and frequency (vehicle waves through a given intersection). Through empirical evaluation, we show how a centralized and decentralized solution each react to unknown congestion information that occurs after the initial route planning period.Keywordsroutingcoordinationconstraint optimization

Distributed Constraint Optimization (DCOP) is rapidly emerging as a prominent technique for multiagent coordination. Unfortunately, rigorous quantitative evaluations of privacy loss in DCOP algorithms have been lacking despite the fact that agent privacy is a key motivation for applying DCOPs in many applications. Recently, Maheswaran et al. [3, 4] introduced a framework for quantitative evaluations of privacy in DCOP algorithms, showing that early DCOP algorithms lose more privacy than purely centralized approaches and questioning the motivation for applying DCOPs. Do state-of-the art DCOP algorithms suffer from a similar shortcoming? This paper answers that question by investigating the most efficient DCOP algorithms, including both DPOP and ADOPT.

Emergency responders are faced with a number of significant challenges when managing major disasters. First, the number of rescue tasks posed is usually larger than the number of responders (or agents) and the resources available to them. Second, each task is likely to require a different level of effort in order to be completed by its deadline. Third, new tasks may continually appear or disappear from the environment, thus requiring the responders to quickly recompute their allocation of resources. Fourth, forming teams or coalitions of multiple agents from different agencies is vital since no single agency will have all the resources needed to save victims, unblock roads and extinguish the fires which might erupt in the disaster space. Given this, coalitions have to be efficiently selected and scheduled to work across the disaster space so as to maximize the number of lives and the portion of the infrastructure saved. In particular, it is important that the selection of such coalitions should be performed in a decentralized fashion in order to avoid a single point of failure in the system. Moreover, it is critical that responders communicate only locally given they are likely to have limited battery power or minimal access to long-range communication devices. Against this background, we provide a novel decentralized solution to the coalition formation process that pervades disaster management. More specifically, we model the emergency management scenario defined in the RoboCup Rescue disaster simulation platform as a coalition formation with spatial and temporal constraints (CFST) problem where agents form coalitions to complete tasks, each with different demands. To design a decentralized algorithm for CFST, we formulate it as a distributed constraint optimization problem and show how to solve it using the state-of-the-art Max-Sum algorithm that provides a completely decentralized message-passing solution. We then provide a novel algorithm (F-Max-Sum) that avoids sending redundant messages and efficiently adapts to changes in the environment. In empirical evaluations, our algorithm is shown to generate better solutions than other decentralized algorithms used for this problem.

Distributed Constraint Optimization Problems (DCOPs) are a powerful tool to model multi-agent coordination problems that are distributed by nature. The formulation is suitable for problems where variables are discrete and constraint utilities are represented in tabular form. However, many real-world applications have variables that are continuous and tabular forms thus cannot accurately represent constraint utilities. To overcome this limitation, researchers have proposed the Continuous DCOP (C-DCOP) model, which are DCOPs with continuous variables. But existing approaches usually come with some restrictions on the form of constraint utilities and are without quality guarantees. Therefore, in this paper, we (i) propose an exact algorithm to solve a specific subclass of C-DCOPs; (ii) propose an approximation method with quality guarantees to solve general C-DCOPs; (iii) propose additional C-DCOP algorithms that are more scalable; and (v) empirically show that our algorithms outperform existing state-of-the-art C-DCOP algorithms when given the same communication limitations.

Wireless Power Transfer (WPT) technologies aim at getting rid of cables used by consumer devices for energy provision. As long distance WPT is becoming mature, the health impact of WPT becomes increasingly important to consider. In this paper we look at how to maximize the wireless power transfer to remote devices, while maintaining a safe level of electromagnetic radiation (EMR) for humans that are in the vicinity of the energy transmitters. Classically, this problem can be described as a centralized optimization problem of finding the optimal set of safe power levels at locations of human presence. Instead, we advocate to formulate this problem as an agent-based Distributed Constraint Optimization Problem (DCOP). As a solution to this problem we introduce CoCoA_WPT, a variant of the DCOP solver CoCoA. CoCoA_WPT provides a solution of similar quality to centralized solver even for a large scale network involving over a thousand nodes. Based on CoCoA_WPT, we propose a self-adaptive charging system: Transferring Energy Safely by Self-Adaptation (TESSA). TESSA keeps the charging network safe even when it is perturbed by environmental dynamics. We show that TESSA can reach on average up to 85% of the theoretical optimal maximum total transmitted power (calculated using centralized solution) while satisfying the EMR safety constraints.