A Study of Relaxation Approaches for Asymmetric Constraint Optimization Problems

Reducing the information revealed by agents is an important requirement in multiagent systems. Privacy preservation has been studied in several research areas including automated negotiating agents and Distributed Constraint Optimization Problems (DCOPs). Although one of recent approaches for privacy preservation is applying secure computation to negotiation process, publishing some information during the negotiation is necessary in the cases where agents should evaluate the reason of their agreement. We propose a negotiation framework consisting of two types of Asymmetric Multi-Objective DCOPs that are repeatedly solved to gradually publish the utility values of agents until an agreement is reached. One problem defines the selection of utility values that are gradually published by agents while another is the problem consist of the published utility values and related assignments to variables. With the general formalization of DCOPs, there are opportunities to represent several classes of negotiation problems. We define several constraints for heuristic strategies that choose the utility values to be published in the next negotiation step. The criterion of social welfare to consider the multiple objectives among individual agents are also introduced. Moreover, we apply two additional preprocessing methods to adjust the utilities among agents by transferring some amount of utility values and to reduce the complexity of the original problem by approximating several relationships of agent pairs. We experimentally evaluate the effect and influence of our proposed approaches including the heuristics to select the published utility information and the additional preprocessing methods.KeywordsPrivacyDecentralized problem solvingAsymmetric constraint optimizationMulti-objectiveSocial welfareNegotiationMultiagent system

Distributed Constraint Optimization Problems (DCOPs) can be used to model a number of multi-agent coordination problems. The conventional DCOP model assumes that the subproblem that each agent is responsible for (i.e. the mapping of nodes in the constraint graph to agents) is part of the model description. While this assumption is often reasonable, there are many applications where there is some flexibility in making this assignment. In this paper, we focus on this gap and make the following contributions: (1) We formulate this problem as an optimization problem, where the goal is to find an assignment that minimizes the completion time of the DCOP algorithm (e.g. Action-GDL or Max-Sum) that operates on this mapping. (2) We propose a novel heuristic, called MNA, that can be executed in a centralized or decentralized manner. (3) Our empirical evaluation illustrates a substantial reduction in completion time, ranging from 16% to 40%, without affecting the solution quality of the algorithms, compared to the current state of the art. In addition, we observe empirically that the completion time obtained from our approach is near-optimal; it never exceeds more than 10% of what can be achieved from the optimal node-to-agent mapping.

We propose a novel incomplete cooperative algorithm for distributed constraint optimization problems (DCOPs) denoted as Cooperative Constraint Approximation (CoCoA). The key strategy of the algorithm is to use a semi-greedy approach in which knowledge is distributed amongst neighboring agents, and assigning a value only once instead of an iterative approach. Furthermore, CoCoA uses a unique-first approach to improve the solution quality. It is designed such that it can solve DCOPs as well as Asymmetric DCOPS, with only few messages being communicated between neighboring agents. Experimentally, through evaluating graph coloring problems, randomized (A)DCOPs, and a sensor network communication problem, we show that CoCoA is able to very quickly find solutions of high quality with a smaller communication overhead than state-of-the-art DCOP solvers such as DSA, MGM-2, ACLS, MCS-MGM and Max-Sum. In our asymmetric use case problem of a sensor network, we show that CoCoA not only finds the best solution, but also finds this solution faster than any other algorithm.

Coordinated multi-agent reinforcement learning (MARL) provides a promising approach to scaling learning in large cooperative multi-agent systems. Distributed constraint optimization (DCOP) techniques have been used to coordinate action selection among agents during both the learning phase and the policy execution phase (if learning is off-line) to ensure good overall system performance. However, running DCOP algorithms for each action selection through the whole system results in significant communication among agents, which is not practical for most applications with limited communication bandwidth. In this paper, we develop a learning approach that generalizes previous coordinated MARL approaches that use DCOP algorithms and enables MARL to be conducted over a spectrum from independent learning (without communication) to fully coordinated learning depending on agents' communication bandwidth. Our approach defines an interaction measure that allows agents to dynamically identify their beneficial coordination set (i.e., whom to coordinate with) in different situations and to trade off its performance and communication cost. By limiting their coordination set, agents dynamically decompose the coordination network in a distributed way, resulting in dramatically reduced communication for DCOP algorithms without significantly affecting overall learning performance. Essentially, our learning approach conducts co-adaptation of agents' policy learning and coordination set identification, which outperforms approaches that sequence them.

A vessel rotation is a sequence in which a vessel visits the different terminals in a large port for loading and unloading containers. Constructing a vessel rotation involves making a lot of appointments through communication. Nowadays, the communication between vessel operator and terminal operator is done through telephone, fax, and e-mail based on the operator’s knowledge and experience. With this practice, it happens frequently that appointments cannot be met either by the vessel operator or by the terminal operator. In this paper we propose a technique to solve the vessel rotation planning problem in an automated way. For this, we propose to model the problem as a distributed constraint optimization problem (DCOP). DCOP has generated much interest in the artificial intelligence and constraint programming communities, and a number of algorithms have been developed for solving DCOPs. However, DCOP algorithms have been rarely used in practice. This paper presents a first approach based on DCOP for generating reasonable and optimized rotation plans for vessel operators. To evaluate the performance of the proposed approach, different DCOP algorithms are considered. We evaluate the solution quality and computational and communication costs of these algorithms when solving the vessel rotation planning problem.

Distributed Constraint Optimization Problems (DCOPs) have been used to model a number of multi-agent coordination problems. In DCOPs, agents are assumed to have complete information about the utility of their possible actions. However, in many real-world applications, such utilities are stochastic due to the presence of exogenous events that are beyond the direct control of the agents. This paper addresses this issue by extending the standard DCOP model to Expected Regret DCOP (ER-DCOP) for DCOP applications with uncertainty in constraint utilities. Different from other approaches, ER-DCOPs aim at minimizing the overall expected regret of the problem. The paper proposes the ER-DPOP algorithm for solving ER-DCOPs, which is complete and requires a linear number of messages with respect to the number of agents in the problem. We further present two implementations of ER-DPOP---GPU- and ASP-based implementations---that orthogonally exploit the problem structure and present their evaluations on random networks and power network problems.

Distributed constraint optimization problems (DCOPs) are well-suited for modeling multi-agent coordination problems where the primary interactions are between local subsets of agents. However, one limitation of DCOPs is the assumption that the constraint rewards are without uncertainty. Researchers have thus extended DCOPs to Stochastic DCOPs (SDCOPs), where rewards are sampled from known probability distribution reward functions, and introduced algorithms to find solutions with the largest expected reward. Unfortunately, such a solution might be very risky, that is, very likely to result in a poor reward. Thus, in this paper, we make three contributions: (1) we propose a stricter objective for SDCOPs, namely to find a solution with the most stochastically dominating probability distribution reward function; (2) we introduce an algorithm to find such solutions; and (3) we show that stochastically dominating solutions can indeed be less risky than expected reward maximizing solutions.

Wireless Power Transfer (WPT) technologies aim at getting rid of cables used by consumer devices for energy provision. As long distance WPT is becoming mature, the health impact of WPT becomes increasingly important to consider. In this paper we look at how to maximize the wireless power transfer to remote devices, while maintaining a safe level of electromagnetic radiation (EMR) for humans that are in the vicinity of the energy transmitters. Classically, this problem can be described as a centralized optimization problem of finding the optimal set of safe power levels at locations of human presence. Instead, we advocate to formulate this problem as an agent-based Distributed Constraint Optimization Problem (DCOP). As a solution to this problem we introduce CoCoA_WPT, a variant of the DCOP solver CoCoA. CoCoA_WPT provides a solution of similar quality to centralized solver even for a large scale network involving over a thousand nodes. Based on CoCoA_WPT, we propose a self-adaptive charging system: Transferring Energy Safely by Self-Adaptation (TESSA). TESSA keeps the charging network safe even when it is perturbed by environmental dynamics. We show that TESSA can reach on average up to 85% of the theoretical optimal maximum total transmitted power (calculated using centralized solution) while satisfying the EMR safety constraints.

The Distributed Pseudo-tree Optimization Procedure (DPOP) is a well-known message passing algorithm that provides optimal solutions to Distributed Constraint Optimization Problems (DCOPs) in cooperative multi-agent systems. However, the traditional DCOP formulation does not consider constraints that must be satisfied (hard constraints), rather it concentrates only on constraints that place no restriction on satisfaction (soft constraints). This is a serious shortcoming as many real-world applications involve both types of constraints. Traditional DPOP algorithms are not able to benefit from the existence of hard constraints, where an additional calculation is required to handle such constraints. This results in longer runtimes. Thus scalability remains an issue. Additionally, in the standard DPOP, the agents are arranged as a Depth First Search (DFS) pseudo-tree, but recent work has shown that the construction of pseudo-trees in this way often leads to chain-like communication structures that greatly impair the algorithm’s performance. To address these issues, we develop an algorithm that speeds up the DPOP algorithm by reducing the size of the messages exchanged and increases parallelism in the pseudo tree. For this purpose, initially, we improve the path for exchanging messages. Next, we introduce a new form of constraint propagation, which we call cross-edge consistency. Our theoretical evaluation shows that our proposed algorithm is complete and correct. In empirical evaluations, our algorithm achieves a significant reduction in the runtime, ranging from 4% to 96%, compared to the state-of-the-art.

A large class of problems in multiagent systems can be solved by distributed constraint optimization (DCOP). Several algorithms have been created to solve these problems, however, no extensive evaluation of current DCOP algorithms on live networks exists in the literature. This paper uses DCOPolis---a framework for comparing and deploying DCOP software in heterogeneous environments---to contribute an analysis of two state-of-the-art DCOP algorithms solving a number of different problem types. Then, we use this empirical validation to evaluate the use of both cycle-based runtime and concurrent constraint checks.

Multi-agent systems have already been successfully applied to a variety of traffic control problems and demonstrated the potential to lower travel times and environmental impact. Sharing this goal, we have developed iCO2, an online tool for training eco-friendly driving in a multi-user three-dimensional environment. iCO2 supports eco-driving practice by instructing computer-controlled agents, such as traffic lights and other vehicles, to create traffic situations that make eco-driving more difficult. Hence the agents take the role of "opponents" that try to achieve the optimal challenge level for the skill level of each user. The research challenge is to find the optimal challenge level for all user drivers in a shared simulation space that (1) involves both controllable entities ("opponents") and non-controllable entities (users) and (2) is highly dynamic, with dependencies between entities being created and destroyed in real time. We try to solve this problem by modeling the scenario as a distributed constraint optimization problem (DCOP). The main contribution of our paper is the application of a DCOP algorithm to such a new type of application scenario. We evaluate our approach by running scenarios both in terms of speed and optimality of the solutions proposed by the DCOP algorithm.

Distributed Constraint Optimization Problems (DCOPs) are a powerful tool to model multi-agent coordination problems that are distributed by nature. The formulation is suitable for problems where variables are discrete and constraint utilities are represented in tabular form. However, many real-world applications have variables that are continuous and tabular forms thus cannot accurately represent constraint utilities. To overcome this limitation, researchers have proposed the Continuous DCOP (C-DCOP) model, which are DCOPs with continuous variables. But existing approaches usually come with some restrictions on the form of constraint utilities and are without quality guarantees. Therefore, in this paper, we (i) propose an exact algorithm to solve a specific subclass of C-DCOPs; (ii) propose an approximation method with quality guarantees to solve general C-DCOPs; (iii) propose additional C-DCOP algorithms that are more scalable; and (v) empirically show that our algorithms outperform existing state-of-the-art C-DCOP algorithms when given the same communication limitations.

Virtual Networks (VN) offer a flexible and economic approach to deploy customer suited networks. However, defining how resources of a physical network are used to support VNs demands is a NP-hard problem. For this reason, heuristics have been proposed for providing satisfactory allocation of resources in an efficient way. This work presents a modeling of the VN allocation problem using Distributed Constraint Optimization Problem (DCOP) with factor graphs. The max-sum algorithm is employed to solve the DCOP. Correctness criteria for this approach are discussed and verifications going to be conducted through model checking.

This paper introduces a new agent-based algorithm for scheduling in CSCWD. Distributed Artificial Intelligence provides a lot of research areas, including CSCWD, with efficient decentralized optimization and problem solving techniques. In this paper we focus on how DCOP (Distributed Constraint Optimization Problem) can be used for scheduling in CSCWD, discussing a new algorithm. Our algorithm is almost-complete, providing the best solution most the times. However, it saves a lot of computational resources, being useful in situations where other state of the art algorithms are not feasible. The results are quite encouraging, showing that our algorithm outperforms easily two well-known DCOP algorithms in terms of runtime, number of messages, size of messages and throughput.

This paper addresses distributed task allocation in complex scenarios modeled using the distributed constraint optimization problem (DCOP) formalism. It is well known that DCOP, when used to model complex scenarios, generates problems with exponentially growing number of parameters. However, those scenarios are becoming ubiquitous in real-world applications. Therefore, approximate solutions are necessary. We propose and evaluate an algorithm for distributed task allocation. This algorithm, called Swarm-GAP, is based on theoretical models of division of labor in social insect colonies. It uses a probabilistic decision model. Swarm-GAP is experimented both in a scenario from RoboCup Rescue and an abstract simulation environment. We show that Swarm-GAP achieves similar results as other recent proposed algorithm with a reduction in communication and computation. Thus, our approach is highly scalable regarding both the number of agents and tasks.