Controller Synthesis for Omega-Regular and Steady-State Specifications

Linear time temporal logic (LTL) has become a well established tool for specifying the dynamic behaviour of reactive systems with an interleaving semantics, and the automata-theoretic approach has proven to be a very useful mechanism for performing automatic verification in this setting. Especially alternating automata turned out to be a powerful tool in constructing efficient yet simple to understand decision procedures and directly yield further on-the-fly model checking procedures. In this paper we exhibit a decision procedure for LTL over Mazurkiewicz traces which generalises the classical automata-theoretic approach to a linear time temporal logic interpreted no longer over sequences but certain partial orders. Specifically, we construct a (linear) alternating Buchi automaton accepting the set of linearisations of traces satisfying the formula at hand. The salient point of our technique is to apply a notion of independence-rewriting to formulas of the logic. Furthermore, we show that the class of linear and trace-consistent alternating Buchi automata corresponds exactly to LTL formulas over Mazurkiewicz traces, lifting a similar result from Loding and Thomas formulated in the framework of LTL over words.

With the increase of people’s demand for agents, the activities of agents are no longer limited to simple environment and task format. Facing complex application scenarios, agents need to be able to make decisions and execute them autonomously. This paper studies the probabilistic action planning considering complex task constraints described by linear temporal logic. At the same time, the success rate and cost of task are both considered. The uncertain factors include agent behavior and environment attributes, and the task description is expressed by soft and hard constraints. The strategy of agent is generated here applying model checking in formal method. Single- and multi-agent model is established using Markov decision process, while task model is established using double-layer automata. Then, agent-task network model is designed to describe the constraints and the control strategy is solved through a coupled linear programming. The method above is verified through numerical simulation. The results show that the complex task constraints in the form of soft and hard constraints can be satisfied. The optimal strategy enable the agent to complete the task according to the constraint strength, and the control strategy can be adjusted by controlling the relaxation degree of control network model relevant to the penalty factor.

This article is written in support of the academic discipline “Nonclassical logics”. The objects of study within this discipline are the basic principles and constructive elements used in formal construction of various nonclassical propositional logics. Despite the abstract nature of the theory of nonclassical logics, which is mainly focused on strict mathematical formalization of logical reasoning, there are real practical applications for its theoretical results. In particular, languages of temporal modal logics are widely used for modeling, specification, and verification (correctness analysis) of logic control program systems. This article demonstrates, based on the example of linear temporal logic (LTL), how abstract concepts of nonclassical logics can be applied in practice in the area of information technology and programming. It is shown that the behavior of a software system can be represented as a set of LTL formulas, which can then be used to check the satisfiability of the properties of that software system via the procedure for proving the validity of logical inferences expressed in terms of linear temporal logic (LTL). The approach to using LTL to specify the behavior of software systems is demonstrated based on Minsky counter machines. Minsky machines are one of the ways of formalizing the intuitive concept of an algorithm. Their computational power is equivalent to that of Turing machines. A counter machine is a computer program written in a high-level language, since it contains variables called counters and conditional and unconditional jump operators used for loop construction. It is known that any algorithm can (hypothetically) be implemented as a three-counter Minsky machine.

In this paper, We consider an uncertain Markov decision process (MDP) with a control cost and a linear temporal logic (LTL) control specification. We propose a reinforcement learning (RL) based method for design of an optimal control policy by which the controlled MDP satisfies the control specification with probability 1 and minimizes an expected discounted sum of the control costs. First, we construct a deterministic Rabin automaton (DRA) that accepts all and only infinite words satisfying the LTL control specification. Second, we construct a product MDP of the MDP and the DRA to represent a dynamic control policy that satisfies the control specification. Third, we modify the product MDP in order to apply RL to the design of an optimal control policy. The control action of the modified product MDP is a pair of a pattern and an action, where the pattern is a set of actions. Moreover, we introduce a reward that represents both the satisfaction of the control specification and the minimally restrictiveness of the pattern. Finally, we proposed an algorithm for design of an optimal control policy that consists of a sequential decision making of two steps. At the first decision making, we select a pattern that maximizes a discounted sum of the reward. At the second one, we select an action from the pattern selected at the first one such that it minimizes the expected discounted sum of the costs. Moreover, we consider an illustrative example to show that the proposed algorithm can obtain an optimal control policy.

We present a method to find an optimal policy with respect to a reward function for a discounted Markov decision process under general linear temporal logic (LTL) specifications. Previous work has either focused on maximizing a cumulative reward objective under finite-duration tasks or maximizing an average reward for persistent (e.g., surveillance) tasks. This paper extends and generalizes these results by introducing a pair of occupancy measures to express the LTL satisfaction objective and the expected discounted reward objective, respectively. These occupancy measures are then connected to a single policy via a novel reduction resulting in a mixed integer linear program whose solution provides an optimal policy. Our formulation can also be extended to include additional constraints with respect to secondary reward functions. We illustrate the effectiveness of our approach in the context of robotic motion planning for complex missions under uncertainty and performance objectives.

Temporal logic has been widely used to express complex task specifications for cyber-physical systems (CPSs). One way to synthesize a controller for CPS under temporal logic constraints is to first abstract the CPS as a discrete transition system, and then apply formal methods. This approach, however, is computationally demanding and its scalability suffers due to the curse of dimensionality. In this paper, we propose a control barrier function (CBF) approach to abstraction-free control synthesis under a linear temporal logic (LTL) constraint. We first construct the deterministic Rabin automaton of the specification and compute an accepting run. We then compute a sequence of LTL formulae, each of which must be satisfied during a particular time interval, and prove that satisfying the sequence of formulae is sufficient to satisfy the LTL specification. Finally, we compute a control policy for satisfying each formula by constructing an appropriate CBF. We present a quadratic program to compute the controllers, and show the controllers synthesized using the proposed approach guarantees the system to satisfy the LTL specification, provided the quadratic program is feasible at each time step. A numerical case study is presented to demonstrate the proposed approach.

We investigate logical consequence in temporal logics in terms of logical consecutions, i.e., inference rules. First, we discuss the question: what does it mean for a logical consecution to be ‘correct’ in a propositional logic. We consider both valid and admissible consecutions in linear temporal logics and discuss the distinction between these two notions. The linear temporal logicLDTL, consisting of all formulas valid in the frame 〈L≤, ≥〉 of all integer numbers, is the prime object of our investigation. We describe consecutions admissible inLDTLin a semantic way—via consecutions valid in special temporal Kripke/Hintikka models. Then we state that any temporal inference rule has a reduced normal form which is given in terms of uniform formulas of temporal degree 1. Using these facts and enhanced semantic techniques we construct an algorithm, which recognizes consecutions admissible inLDTL. Also, we note that using the same technique it follows that the linear temporal logicL(N) of all natural numbers is also decidable w.r.t. inference rules. So, we prove that both logicsLDTLandL(N) are decidable w.r.t. admissible consecutions. In particular, as a consequence, they both are decidable (known fact), and the given deciding algorithms are explicit.

In previous work we provided a conflict resolution algorithm allowing agents in stochastic domains (represented by Markov Decision Processes) to maximally satisfy a set of moral or social norms, where such norms are represented by statements in linear temporal logic (LTL). This required the agent designer to provide weights specifying the relative importance of each norm. In this paper, we propose an inverse norm conflict resolution'' algorithm for learning these weights from demonstration. This approach minimizes a cost function based on the relative entropy between a policy encoding the observed behavior and a policy representing optimal norm-following behavior. We demonstrate the effectiveness of the algorithm in a simple GridWorld domain.

: The question of whether branching-time temporal logic or linear-time temporal logic is best for reasoning about concurrent programs is one of the most controversial issues in logics of programs. Concurrent programs are usually modelled by labelled state-transition graphs in which some state is designated as the initial state. For historical reasons such graphs are called Kripke structures. In linear temporal logic operators are provided for describing events along a single time path (i.e., along a single path in a Kripke structure). In a branching-time logic the temporal operators quantify over the futures that are possible from a given state (i.e., over the possible paths that lead from a state). It is well known that the two types of temporal logic have different expressive powers. Linear temporal logic, for example, can express certain fairness properties that cannot be expressed in branching-time temporal logic. On the other hand, certain practical decision problems like model checking are easier for branching-time temporal logic than for linear temporal logic. This paper provides further insight on which type of logic is best. It is shown that if two finite Kripke structures can be distinguished by some formula that contains both branching-time and linear-time operators, then the structures can be distinguished by a formula that contains only branching time operators. Specifically, we show that if two finite Kripke structures can be distinguished by some formula of the logic CTL (i.e., if there is some CTL formula that is true in one but not in the other), then they can be distinguished by some formula of the logic CTL.

This paper considers the control synthesis problem and its solution using symbolic regression. Symbolic regression methods, which were previously called genetic programming methods, allow one to use a computer to find not only the parameters of a given regression function but also its structure. Unlike other works on solving the control synthesis problem using symbolic regression, the novelty of this paper is that for the first time this work employs a training dataset to address the problem of general control synthesis. Initially, the optimal control problem is solved from each point in a given set of initial states, resulting in a collection of control functions expressed as functions of time. A reference model is then integrated into the control object model, which generates optimal motion trajectories using the derived optimal control functions. The control synthesis problem is framed as an approximation task for all optimal trajectories, where the control function is sought as a function of the deviation of the object from the specified terminal state. The optimization criterion for solving the synthesis problem is the accuracy of the object’s movement along the optimal trajectory. The paper includes an example of solving the control synthesis problem for a mobile robot using a supervised machine learning method. A relatively new method of symbolic regression, the method of variational complete binary genetic programming, is studied and proposed for the solution of the control synthesis problem.

This paper proposes an online human-aware behavior planning method to enable Unmanned Aerial Vehicles (UAVs) to dynamically satisfy high-level linear temporal logic (LTL) task descriptions from human collaborators. The proposed method has the potential to be applied to the industrial human–UAV collaboration. The specific process is as follows. Firstly, the global high-level task of the UAV is described using the formal language of LTL, which can usually be issued by human collaborators in natural language. Secondly, the task description is transformed into a deterministic Rabin automaton, and then the state values of the automaton are given by the distance from the accepted states. Thirdly, the Markov decision process is used to construct the probabilistic behavior model of the UAV offline. Based on this model, a finite state automaton in the finite horizon is dynamically constructed online, and the Cartesian product system within the horizon is constructed, in which the expected arrival states within the horizon are set by the minimum state values. Finally, aiming at maximizing the reachability probability of the expected states, the value iterative algorithm is introduced to solve the optimal behavior plan online within the finite horizon. After it, an infinite horizon planning and execution algorithm is formed. Experiments and analysis show that the results of the online behavior planning are consistent with the task logic at the semantic level, indicating the correctness of the proposed method. Moreover, the proposed method can effectively alleviate the state explosion caused by the global probabilistic model checking and improve the efficiency of plan generation.

This paper addresses a control problem for probabilistic models in the setting of Markov decision processes (MDP). We are interested in the steady-state control problem which asks, given an ergodic MDP $\mathcal{M}$ and a distribution δ goal, whether there exists a (history-dependent randomized) policy π ensuring that the steady-state distribution of $\mathcal{M}$ under π is exactly δ goal. We first show that stationary randomized policies suffice to achieve a given steady-state distribution. Then we infer that the steady-state control problem is decidable for MDP, and can be represented as a linear program which is solvable in PTIME. This decidability result extends to labeled MDP (LMDP) where the objective is a steady-state distribution on labels carried by the states, and we provide a PSPACE algorithm. We also show that a related steady-state language inclusion problem is decidable in EXPTIME for LMDP. Finally, we prove that if we consider MDP under partial observation (POMDP), the steady-state control problem becomes undecidable.

Based on the Kripke structure, linear temporal logic and generalized possibility measure, this article studies the model checking problems of generalized possibilistic fuzzy linear temporal logic (GPoFTL). The generalized possibilistic Kripke structure is introduced to describe the system model. The syntax of GPoFTL, which includes fuzzy temporal operators such as ``soon'', ``presently'', ``gradually'', ``within'', ``last'', ``nearly always'', ``almost alwayss'', ``in the long distant future'', ``in the middle of'', ``nearly until'' and ``almost until'', and its language semantics and path semantics under generalized possibility measure are given. Next, we explain the semantics of fuzzy temporal operators through some examples and prove that GPoFTL is an extension of generalized possibilistic linear temporal logic in fuzzy time temporal logic. Furthermore, the GPoFTL model checking algorithm is given by explicit calculation formulas one by one for any GPoFTL formula involving fuzzy temporal operators using fuzzy matrix operations. Last, the algorithm of necessary threshold model checking of GPoFTL is studied using automata theory and its time complexity is discussed.

Controller synthesis for general linear temporal logic (LTL) objectives is a challenging task. The standard approach involves translating the LTL objective into a deterministic parity automaton (DPA) by means of the Safra-Piterman construction. One of the challenges is the size of the DPA, which often grows very fast in practice, and can reach double exponential size in the length of the LTL formula. In this paper we describe a single exponential translation from limit-deterministic B\"uchi automata (LDBA) to DPA, and show that it can be concatenated with a recent efficient translation from LTL to LDBA to yield a double exponential, \enquote{Safraless} LTL-to-DPA construction. We also report on an implementation, a comparison with the SPOT library, and performance on several sets of formulas, including instances from the 2016 SyntComp competition.

We introduce Parametric Linear Dynamic Logic (PLDL), which extends Linear Dynamic Logic (LDL) by temporal operators equipped with parameters that bound their scope. LDL was proposed as an extension of Linear Temporal Logic (LTL) that is able to express all $\omega$-regular specifications while still maintaining many of LTL's desirable properties like an intuitive syntax and a translation into non-deterministic B\"uchi automata of exponential size. But LDL lacks capabilities to express timing constraints. By adding parameterized operators to LDL, we obtain a logic that is able to express all $\omega$-regular properties and that subsumes parameterized extensions of LTL like Parametric LTL and PROMPT-LTL. Our main technical contribution is a translation of PLDL formulas into non-deterministic B\"uchi word automata of exponential size via alternating automata. This yields a PSPACE model checking algorithm and a realizability algorithm with doubly-exponential running time. Furthermore, we give tight upper and lower bounds on optimal parameter values for both problems. These results show that PLDL model checking and realizability are not harder than LTL model checking and realizability.