Multi-Agent Reinforcement Learning with Temporal Logic Specifications

This paper proposes a novel version of the vehicle routing problem (VRP). Instead of servicing all the customers, feasible solutions of the VRP instance are forced to satisfy a set of complex high-level tasks given as a metric temporal logic (MTL) specification, which allows complex quantitative timing constraints to be incorporated into the problem. For the resulting vehicle routing problem with metric temporal logic specifications (VRPMTL), a mixed-integer linear programming (MILP) based algorithm is provided that solves the problem to optimality. Examples for optimal multi-UAV mission planning are provided where MTL is used as a high level language to specify complex mission tasks.

The normal operation of synchronous modules may be temporarily suspended or finally aborted due to requests of their environment. Hence, if a temporal logic specification has already been verified for a synchronous module, then the available verification result can typically only be used if neither suspension nor abortion will take place. Also, the simulation of synchronous modules has to be finally aborted so that temporal logic specifications referring to infinite behaviors cannot be completely answered. In this paper, we therefore define transformations on temporal logic specifications to lift available verification results for synchronous modules without suspension or abortion to refined temporal logic specifications that take care of these preemption statements. This way, one can establish simulation and modular verification of synchronous modules in contexts where preemptions are used.KeywordsModel CheckTransition SystemTemporal LogicLinear Temporal LogicLinear Temporal Logic FormulaThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The fundamental idea of this work is to synthesize reactive controllers such that closed-loop execution trajectories of the system satisfies desired specifications that ensure correct system behaviors, while optimizing a desired performance criteria. We define the correctness of a system's behavior according to the system's relation to the environment, for example, the output trajectories of the system terminate in a goal set without entering an unsafe set. Using Metric Temporal Logic (MTL) specifications we can capture complex system behaviors and timing requirements, such as the output trajectories must pass through a number of way-points within a certain time frame before terminating in the goal set. Given a Mixed Logical Dynamical (MLD) system and system specifications in terms of MTL formula or simpler reach-avoid specifications, our goal is to find a closed-loop trajectory that satisfies the specifications, in non-deterministic environments. Using an MILP framework we search over the space of input signals to obtain such valid trajectories of the system, by adding constraints to satisfy the MTL formula only when necessary, to avoid the exponential complexity of solving MILP problems. We also present experimental results for planning a path in real-time for a mobile robot through a dynamically changing environment with a desired task specification.

In this paper, we investigate fully-automated controller synthesis of coordination strategies for multi-robot warehouse robotic systems. We consider the scenario where a group of automated guided vehicles (AGVs) deliver goods from starting locations to destinations in a grid-abstracted workspace. We propose a distributed framework with prioritized-based coordination strategy to fulfill the entire planning task. Specifically, the task of each AGV is specified by a Linear Temporal Logic (LTL) specification. The local control strategies are obtained by solving the corresponding LTL reactive synthesis problems. In order to avoid conflicts, we use the prioritized-based coordination strategy to guarantee that the overall systems is safe and live, i.e., AGVs will collide with each other and there is no deadlock and livelock. The proposed design methodology is correct-by-construction with the formal guarantee and is amenable to handle temporal tasks in a flexible manner. Furthermore, the proposed framework is distributed; hence computationally more efficient than the centralized approach. Finally, we demonstrate the proposed distributed framework by simulation examples.

Manipulation planning from high-level task specifications, even though highly desirable, is a challenging problem. The large dimensionality of manipulators and complexity of task specifications make the problem computationally intractable. This work introduces a manipulation planning framework with linear temporal logic (LTL) specifications. The use of LTL as the specification language allows the expression of rich and complex manipulation tasks. The framework deals with the state-explosion problem through a novel abstraction technique. Given a robotic system, a workspace consisting of obstacles, manipulable objects, and locations of interest, and a co-safe LTL specification over the objects and locations, the framework computes a motion plan to achieve the task through a synergistic multi-layered planning architecture. The power of the framework is demonstrated through case studies, in which the planner efficiently computes plans for complex tasks. The case studies also illustrate the ability of the framework in intelligently moving away objects that block desired executions without requiring backtracking.

We present an energy storage controller synthesis method for power systems with respect to metric temporal logic (MTL) specifications. The power systems with both constant impedance loads and constant power loads are modeled as a set of differential–algebraic equations. After a fault is cleared, with uncertainties in the fault clearing time, the generator machine angles and rotor speed deviations will enter a set of postfault initial states. We use the robust neighborhood approach to cover this set using the initial robust neighborhoods of finitely many simulated postfault trajectories. These simulated postfault trajectories meet the frequency regulation requirements specified in MTL as they are driven by the optimal control input signals obtained through a functional gradient descent approach. In this way, all the possible postfault trajectories with the given uncertainties in the fault clearing time are guaranteed to satisfy the MTL specification. Furthermore, we learn a piecewise linear control law from the data of the simulated trajectories to generate a feedback controller.

In control theory, models of physical processes, such as systems of differential equations, are usually checked against specifications, such as stability and set invariance. In formal methods, rich specifications, such as languages and formulae of temporal logics, are checked against models of software programs and digital circuits, such as finite transition graphs. With the development and integration of cyber physical and safety critical systems, there is an increasing need for computational tools for verification and control of complex systems from rich, temporal logic specifications. The formal verification and synthesis problems have been shown to be undecidable even for very simple classes of infinitespace continuous and hybrid systems. However, provably correct but conservative approaches, in which the satisfaction of a property by a dynamical system is implied by the satisfaction of the property by a finite over-approximation (abstraction) of the system, have received a lot of attention in recent years. Some classes of systems allowing for computationally efficient verification and control from temporal logic specifications are reviewed. For continuous and discrete-time linear systems and continuous-time multi-linear systems, it is shown that finite abstractions can be constructed through polyhedral operations only. By using techniques from model checking and automata games, this allows for verification and control from specifications given as Linear Temporal Logic (LTL) formulae over linear predicates in the state variables. A connection between the existence of Lyapunov functions and finite bisimulations is established for discrete-time linear and switched linear systems. Finally, optimality and correctness requirements are combined in a model predictive approach to generate control strategies for discrete-time linear systems. The usefulness of these computational tools is illustrated with various examples such as verification and synthesis of biological circuits in synthetic biology and motion planning and control in robotics.

In this paper, a motion planning system based on predictive control and temporal logic specifications is established for the welding dual-arm robot with gantry. Firstly, based on the spatial structure of the dual-arm robot with gantry, a novel mathematical model is presented between the joint variables and the terminal attitude and the quaternion replaces the Euler Angle to describe the posture for avoiding gimbal lock. Then a constraint model predictive controller is constructed for the multi-variable, nonlinear and coupling model of the robot and the constraints of the environment. Meanwhile, the temporal logic formula is used for describing the complex welding tasks and reducing the workload of repeated programming. Next the expectation values and constraints are converted by the temporal logic formula will be sent to the predictive controller one by one. Finally, the simulation result shows that the system can effectively solve the problem of motion planning for the welding dual-arm robot with gantry.

This paper presents an approach for building SoftWareImplemented Fault Injection (SWIFI) tools for real-time systems starting from temporal logic specifications. Temporal logic formulas are provided describing the behavior of the main SWIFI mechanisms, both for the injection of faults and for the observation of the target system. Besides conventional fault injectors (i.e., that flip bits in memory and in system calls’ parameters), we have also considered fault injectors based on the notion of saboteur. Concerning the observation mechanisms, we have defined on-line software probes aimed at observing not only the failure modes of the system under test, but also the real-time behavior of both the application and the underlying operating system. The specifications of all such mechanisms are then translated into programs that are executed on-line by a virtual machine. This approach was applied to the development of MAFALDA-RT, a SWIFI tool for real-time systems. A case study exemplifies how the proposed approach allows for the SWIFI mechanisms to be defined and applied.

This paper considers the problem of motion planning for a\nhybrid robotic system with complex and nonlinear dynamics\nin a partially unknown environment given a temporal logic\nspecification. We employ a multi-layered synergistic framework\nthat can deal with general robot dynamics and combine\nit with an iterative planning strategy. Our work allows us\nto deal with the unknown environmental restrictions only\nwhen they are discovered and without the need to repeat\nthe computation that is related to the temporal logic specification.\nIn addition, we define a metric for satisfaction of\na specification. We use this metric to plan a trajectory that\nsatisfies the specification as closely as possible in cases in\nwhich the discovered constraint in the environment renders\nthe specification unsatisfiable. We demonstrate the efficacy\nof our framework on a simulation of a hybrid second-order\ncar-like robot moving in an office environment with unknown\nobstacles. The results show that our framework is successful\nin generating a trajectory whose satisfaction measure of the\nspecification is optimal. They also show that, when new obstacles\nare discovered, the reinitialization of our framework\nis computationally inexpensive.

In this paper, the problem of motion planning for a mobile robot is addressed in which motion planning scenarios such as reachability, invariance, surveillance and sequencing missions are formulated with Linear Temporal Logic (LTL). First, by using the idea of invariant set theory, a novel algorithm is proposed for checking the satisfaction of temporal logic planning specification in the offline phase. Then, an optimal control strategy is designed in the presence of bounded additive disturbances to meet the given temporal logic specification and system constraints. Illustrative case studies for a robot with a double integrator model are presented to demonstrate the applicability of the proposed approach in motion planning tasks.

We consider the problem of computing the set of initial states of a dynamical system such that there exists a control strategy to ensure that the trajectories satisfy a temporal logic specification with probability 1 (almost-surely). We focus on discrete-time, stochastic linear dynamics and specifications given as formulas of the Generalized Reactivity(1) fragment of Linear Temporal Logic over linear predicates in the states of the system. We propose a solution based on iterative abstraction-refinement, and turn-based 2-player probabilistic games. While the theoretical guarantee of our algorithm after any finite number of iterations is only a partial solution, we show that if our algorithm terminates, then the result is the set of satisfying initial states. Moreover, for any (partial) solution our algorithm synthesizes witness control strategies to ensure almost-sure satisfaction of the temporal logic specification. We demonstrate our approach on an illustrative case study.

In recent years, event and self-triggered control have been proposed as energy-aware control strategies to expand the life-time of battery powered devices in Networked Control Systems (NCSs). In contrast to the previous works in which their control objective is to achieve stability, this paper presents a novel energy-aware control scheme for achieving high level specifications, or more specifically, temporal logic specifications. Inspired by the standard hierarchical strategy that has been proposed in the field of formal control synthesis paradigm, we propose a new abstraction procedure for jointly synthesizing control and communication strategies, such that the communication reduction in NCSs and the satisfaction of the temporal logic specifications are guaranteed. The benefits of the proposal are illustrated through a numerical example.

This paper considers robot motion planning under temporal logic constraints in probabilistic maps obtained by semantic simultaneous localization and mapping (SLAM). The uncertainty in a map distribution presents a great challenge for obtaining correctness guarantees with respect to the linear temporal logic (LTL) specification. We show that the problem can be formulated as an optimal control problem in which both the semantic map and the logic formula evaluation are stochastic. Our first contribution is to reduce the stochastic control problem for a subclass of LTL to a deterministic shortest path problem by introducing a confidence parameter $delta$. A robot trajectory obtained from the deterministic problem is guaranteed to have minimum cost and to satisfy the logic specification in the true environment with probability $delta$. Our second contribution is to design an admissible heuristic function that guides the planning in the deterministic problem towards satisfying the temporal logic specification. This allows us to obtain an optimal and very efficient solution using the A* algorithm. The performance and correctness of our approach are demonstrated in a simulated semantic environment using a differential-drive robot.

Sampling-based Motion Planning with Temporal Logic Missions and Spatial Preferences