Market-Based Dynamic Task Allocation Using Heuristically Accelerated Reinforcement Learning

Multi-robot systems comprising of heterogeneous autonomous vehicles on land, air, water are being increasingly used to assist or replace humans in different hazardous missions. Two crucial aspects in such multi-robot systems are to: a) explore an initially unknown region of interest to discover tasks, and, b) allocate and share the discovered tasks between the robots in a coordinated manner using a multi-robot task allocation (MRTA) algorithm. In this paper, we describe results from our research on multi-robot terrain coverage and MRTA algorithms within an autonomous landmine detection scenario, done as part of the COMRADES project. Each robot is equipped with a different type of landmine detection sensor and different sensors, even of the same type, can have different degrees of accuracy. The landmine detection-related operations performed by each robot are abstracted as tasks and multiple robots are required to complete a single task. First, we describe a distributed and robust terrain coverage algorithm that employs Voronoi partitions to divide the area of interest among the robots and then uses a single-robot coverage algorithm to explore each partition for potential landmines. Then, we describe MRTA algorithms that use the location information of discovered potential landmines and employ either a greedy strategy, or, an opportunistic strategy to allocate tasks among the robots while attempting to minimize the time (energy) expended by the robots to perform the tasks. We report experimental results of our algorithms using accurately-simulated Corobot robots within the Webots simulator performing a multi-robot, landmine detection operation.

Many application domains require search and retrieval, which is also known in the robotic domain as foraging. For example, in a search and rescue domain, a disaster area needs to be explored and transportation of survivors to a safe area needs to be arranged. Performing such a search and retrieval task by more than one robot increases performance if they are able to distribute their workload efficiently and evenly. In this work, we study the Multi-Robot Task Allocation (MRTA) problem in the search and retrieval domain, where a team of robots is required to cooperatively search for targets of interest in an environment and also retrieve them back to a home base. In comparison with typical foraging tasks, we look at a more general search and retrieval task in which the targets are distinguished with various types, and task allocation also requires taking into account temporal constraints on the team goal. As usual, robots have no prior knowledge about the location of targets in the environment but in addition they need to deliver targets to the home base in a specific order according to their types, which significantly increases the complexity of a foraging problem. We first use a graph-based model to analyse the search and retrieval problem and the dynamics of exploration and retrieval within a cooperative team. We then proceed to present an extended auction-based approach, as well as a prediction approach. The essential difference between these two approaches is that the task allocation and execution procedures in the auction approach are running in parallel, whereas a robot in the prediction approach only needs to choose a task to perform when it has no thing to do. The auction approach uses a winner determination mechanism to allocate tasks to each robot, whereas the robots in the prediction approach implicitly coordinate their activities by team reasoning that leads to consensuses about task allocation. We use the Blocks World for Teams (BW4T) simulator to evaluate the two approaches in our experimental study.

Task allocation is an important aspect of many multi-robot systems. The features and complexity of multi-robot task allocation (MRTA) problems are dictated by the requirements of the particular domain under consideration. These problems can range from those involving instantaneous distribution of simple, independent tasks among members of a homogenous team, to those requiring the time-extended scheduling of complex interrelated multi-step tasks for members of a heterogenous team related by several constraints. The existing widely used taxonomy for task allocation in multi-robot systems was designed for problems with independent tasks and does not deal with problems with interrelated utilities and constraints. While that taxonomy was a ground-breaking contribution to the MRTA literature, a survey of recent work in MRTA reveals that it is no longer a sufficient taxonomy, due to the increasing importance of interrelated utilities and constraints in realistic MRTA problems under consideration. Thus, in this paper, we present a new, comprehensive taxonomy, iTax, that explicitly takes into consideration the issues of interrelated utilities and constraints. Our taxonomy maps categories of MRTA problems to existing mathematical models from combinatorial optimization and operations research, and hence draws important parallels between robotics and these fields.

Task allocation is one of the research focuses of multi-robot system. On the base of presenting the utility values matrix of robots relative to tasks and analyzing the characteristics of multi-robot task allocation, we build the multi-robot task allocation model based on robotic utility value. In order to prevent the basic particle swarm optimization (PSO) algorithm from converging on local optimum, this paper proposes a modified particle swarm optimization (MPSO) algorithm by introducing the linear decrease mechanism of inertia weight and the concept of adjustment operator and adjustment sequence. With the evolution of velocity in the MPSO algorithm, particle not only studies from the historical optimum individual of itself and population, but also studies from the other stochastic individuals with some probability. Finally, the MPSO algorithm is used to solve the task allocation problem of RoboCup 2D soccer robot system, the efficiency of this modified algorithm is proved through simulation results.

A multirobot system has cooperative team of robots designed to enhance efficiency of its operations. One of the critically investigated problems of multirobot system is the multirobot task allocation (MRTA) issue. The main objective of MRTA is to assign tasks to the most suitable robot based on its functions and capability as well as availability. In this paper, a task-oriented, auction-based task allocation framework is presented and tested through simulations and real-world experiments. The developed framework consists of a novel heuristic-based task allocation algorithm and communication module. It is implemented in a multirobot system, allowing tasks to be dynamically assigned to the robots as they achieve given tasks. The implemented framework shows robustness in its flexibility to the task and environment requirements such as resource and energy requirements and size of the environment. The framework involved a task allocation algorithm, which consists of bid generation and bid selection process, and a TCP/IP-based client-server communication module. The results from both simulations and real-world experiments matched, producing optimum results in task allocation.

Multirobot systems (MRS) hold the promise of improved performance and increased fault tolerance for large-scale problems. One of the most important aspects in the design of MRS is the allocation of tasks among the robots in a productive and efficient manner. Optimal solutions to multirobot task allocation (MRTA) can be found through an exhaustive search. Since there are ways in which m tasks can be assigned to n robots, an exhaustive search is often not possible. Task allocation methodologies must ensure that not only the global mission is achieved, but also the tasks are well distributed among the robots. This paper presents task allocation methodologies for MRS by considering their capability in terms of time and space. A two-phase solution methodology is used to solve the MRTA problem wherein the task capacity of the robots is determined during the first phase and the task allocation optimization is done during the second phase using linear programming (LP).

Multi-Robot Multi-Station Cooperative Spot Welding Task Allocation Based on Stepwise Optimization: An Industrial Case Study

With the advent of Industry 4.0, the seamless integration of industrial systems and unmanned technologies has significantly accelerated the development of smart industries. However, the research on task allocation for railway maintenance robots remains limited, particularly with respect to optimizing costs and efficiency within smart railway systems. To address this gap, the present study explores multi-robot task allocation for automated orbital bolt maintenance, aiming to enhance operational efficiency by minimizing both makespan and total travel distance for all robots. To achieve this, an improved hybrid algorithm combining NSGA-II and MOPSO is proposed. Initially, a dynamic task planning method, tailored to the specific conditions of railway operations, is developed. This method uses the coordinates of track bolts to extract environmental features, enabling the dynamic partitioning of task areas. Subsequently, a multi-elite archive strategy is introduced, along with an adaptive mechanism for adjusting crossover and mutation probabilities. This ensures the preservation and maintenance of multiple solutions across various Pareto fronts, effectively mitigating the premature convergence commonly observed in traditional NSGA-II algorithms. Moreover, the integration of the MOPSO algorithm strikes a balance between local and global search capabilities, thereby enhancing both optimization efficiency and solution quality. Finally, a series of experiments, conducted with varying task sizes and robot quantities during the railway maintenance window, validate the effectiveness and improved performance of the proposed algorithm in addressing the multi-robot task allocation problem.

PurposeThe multi-robot task allocation (MRTA) problem is a challenging issue in the robotics area with plentiful practical applications. Expanding the number of tasks and robots increases the size of the state space significantly and influences the performance of the MRTA. As this process requires high computational time, this paper aims to describe a technique that minimizes the size of the explored state space, by partitioning the tasks into clusters. In this paper, the authors address the problem of MRTA by putting forward a new automatic clustering algorithm of the robots' tasks based on a dynamic-distributed double-guided particle swarm optimization, namely, ACD3GPSO.Design/methodology/approachThis approach is made out of two phases: phase I groups the tasks into clusters using the ACD3GPSO algorithm and phase II allocates the robots to the clusters. Four factors are introduced in ACD3GPSO for better results. First, ACD3GPSO uses the k-means algorithm as a means to improve the initial generation of particles. The second factor is the distribution using the multi-agent approach to reduce the run time. The third one is the diversification introduced by two local optimum detectors LODpBest and LODgBest. The last one is based on the concept of templates and guidance probability Pguid.FindingsComputational experiments were carried out to prove the effectiveness of this approach. It is compared against two state-of-the-art solutions of the MRTA and against two evolutionary methods under five different numerical simulations. The simulation results confirm that the proposed method is highly competitive in terms of the clustering time, clustering cost and MRTA time.Practical implicationsThe proposed algorithm is quite useful for real-world applications, especially the scenarios involving a high number of robots and tasks.Originality/valueIn this methodology, owing to the ACD3GPSO algorithm, task allocation's run time has diminished. Therefore, the proposed method can be considered as a vital alternative in the field of MRTA with growing numbers of both robots and tasks. In PSO, stagnation and local optima issues are avoided by adding assorted variety to the population, without losing its fast convergence.

Robotic Mobile Fulfillment System (RMFS) is a new type of parts-to-picker order picking system and has become the development trend of e-commerce logistics distribution centers. There are usually a large number of tasks need to be allocated to many robots and the picking time for e-commerce orders is usually very tight, which puts forward higher requirements for the efficiency of multirobot task allocation (MRTA) in e-commerce RMFS. Current researches on MRTA in RMFS seldom consider task correlation and the balance among picking stations. In this paper, a task time cost model considering task correlation is built according to the characteristics of the picking process. Then, a multirobot task allocation model minimizing the overall picking time is established considering both the picking time balance of picking stations and the load balance of robots. Finally, a four-stage balanced heuristic auction algorithm is designed to solve the task allocation model and the tasks with execution sequence for each robot are obtained. By comparing with the traditional task time cost model and the algorithm without considering the balance among picking stations, it is found that the proposed model and algorithm can significantly shorten the overall picking time.

When it comes to Multi-Robot Systems (MRS), Multi-Robot System Task Allocation (MRTA) is one of the key issues in realizing multi-robot systems for real-world applications. In this paper, we propose an optimization-based Hungarian algorithm to solve the task allocation problem of multi-robot systems. Firstly, each robot is allowed to autonomously evaluate the task cost and send it to the central processor and an equilibrium compensation approach is proposed to compensate the constraints of the algorithm for the cost matrix to be square. Then a task cache pool is used to avoid the problem from degenerating into Single-Robot (SR) task allocation, and a prioritization strategy considering time is also designed to prevent the task points with lower generation values from lingering for a long time in the dynamic task allocation. Through simulation experiments, we demonstrate that the algorithm has a positive effect on the efficiency of assigned tasks in both dynamic and static situations.

Multirobot task allocation is a hot issue in the field of robot research. A new emotional model is used with the self-interested robot, which gives a new way to measure self-interested robots' individual cooperative willingness in the problem of multirobot task allocation. Emotional cooperation factor is introduced into self-interested robot; it is updated based on emotional attenuation and external stimuli. Then a multirobot pursuit task allocation algorithm is proposed, which is based on emotional cooperation factor. Combined with the two-step auction algorithm recruiting team leaders and team collaborators, set up pursuit teams, and finally use certain strategies to complete the pursuit task. In order to verify the effectiveness of this algorithm, some comparing experiments have been done with the instantaneous greedy optimal auction algorithm; the results of experiments show that the total pursuit time and total team revenue can be optimized by using this algorithm.

Edge computing (EC) extends the ability of cloud computing to the network edge to support diverse resource-sensitive and performance-sensitive IoT applications. However, due to the limited capacity of edge servers (ESs) and the dynamic computing requirements, the system needs to dynamically update the task allocation policy according to real-time system states. Service migration is essential to ensure service continuity when implementing dynamic task allocation. Therefore, this article investigates the long-term dynamic task allocation and service migration (DTASM) problem in edge-cloud IoT systems where users’ computing requirements and mobility change over time. The DTASM problem is formulated to achieve the long-term performance of minimizing the load forwarded to the cloud while fulfilling the seamless migration constraint and the latency constraint at each time of implementing the DTASM decision. First, the DTASM problem is divided into two subproblems: 1) the user selection problem on each ES and 2) the system task allocation problem. Then, the DTASM problem is formulated as a Markov decision process (MDP) and an approach based on deep reinforcement learning (DRL) is proposed. To tackle the challenge of vast discrete action spaces for DTASM task allocation in the system with a mass of IoT users, a training architecture based on the twin-delayed deep deterministic policy gradient (DDPG) is employed. Meanwhile, each action is divided into a differentiable action for policy training and one mapped action for implementation in the IoT system. Simulation results demonstrate that the proposed DRL-based approach obtains the long-term optimal system performance compared to other benchmarks while satisfying seamless service migration.

On a dynamic and decentralized energy-aware technique for multi-robot task allocation

Continuous timely repair and replacement of infrastructures, equipment and utilities play an important role in maintaining the smooth-running of a city or local community. Thereby, to help individuals and businesses go about their daily activities with ease, it is vital to develop a proper method for automatically identifying and assigning capable workers for tasks. This paper defines the community management service task allocation problem as CMS-TAP and hence an end-to-end “recommendation + allocation” network, i.e. a task recommender and allocation optimization network (denoted as TROpt-NET), is then developed for handling such problem. TROpt-NET consists of two layers, namely one for predicting worker ability and the other for allocating tasks which are TR Layer and TA Layer, corresponding to “recommendation” and “allocation” of tasks. Different from operations research approaches where workers are assigned to jobs based on their pre-labelled skills and fixed locations, we propose a task recommender and allocation optimization network. The TR layer is a task recommender system designed to learn implicit worker abilities for different tasks using Neural Collaborative Filtering (NCF) by mining a historical dataset of worker task completion. Whereas in the TA layer a differential optimization approach for allocation is used because of its differentiable property and ability to allow for backpropagation to the prediction layer. In this study, we first formulate the CMS-TAP problem as a recommendation +optimization problem and then propose and end-to-end network architecture that tackles the problem in a real-world setting. TROpt-NET curbs uncertainty and assumptions in optimization by learning to more accurately approximate worker ability across different tasks. Additionally, the network can learn implicit worker abilities enabling optimal utilization of workers across a wide range of tasks, which is often ignored in task allocation problems. We find that normalizing worker ability across all tasks improves the implicit learning capability of the network and that good approximations don’t always lead to optimal allocation but learning allocations by backpropagating through recommendations improves the allocation objective. Offline experiments on a real-world large-scale dataset demonstrate the effectiveness of our proposed TROpt-NET.