Using Pairwise Testing to Verify Automatically-Generated Formal Specifications

Bridging the gap between natural language requirements (NLR) and precise formal specifications is a crucial task of knowledge engineering. Software system development has become more complex in recent years, and it includes many requirements in different domains that users need to understand. Many of these requirements are expressed in natural language, which may be incomplete and ambiguous. However, the formal language with its rigorous semantics may accurately represent certain temporal logic properties and allow for automatic validation analysis. It is difficult for software engineers to understand the formal temporal logic from numerous requirements. In this paper, we propose a novel method to automatically mine the linear temporal logic (LTL) from the natural language requirements and check the consistency among different formal properties. We use natural language processing (NLP) to parse requirement sentences and map syntactic dependencies to LTL formulas by using our extraction rules. Also, we apply the automata-based model checking to assess the correctness and consistency of the extracted properties. Through implementation and case studies, we demonstrate that our approach is well suited to deal with the temporal logic requirements upon which the natural language is based.

Ordinary users can build their smart home automation system easily nowadays, but such user-customized systems could be error-prone. Using formal verification to prove the correctness of such systems is necessary. However, to conduct formal proof, formal specifications such as Linear Temporal Logic (LTL) formulas have to be provided, but ordinary users cannot author LTL formulas but only natural language.To address this problem, this paper presents a novel approach that can automatically generate formal LTL specifications from natural language requirements based on domain knowledge and our proposed ambiguity refining techniques. Experimental results show that our approach can achieve a high correctness rate of 95.4% in converting natural language sentences into LTL formulas from 481 requirements of real examples.

Dynamic Bayesian networks (DBNs) are compact graphical representations used to model probabilistic systems where interdependent random variables and their distributions evolve over time. In this paper, we study the verification of the evolution of conditional-independence (CI) propositions against temporal logic specifications. To this end, we consider two specification formalisms over CI propositions: linear temporal logic (LTL), and non-deterministic Büchi automata (NBAs). This problem has two variants. Stochastic CI properties take the given concrete probability distributions into account, while structural CI properties are viewed purely in terms of the graphical structure of the DBN. We show that deciding whether a stochastic CI proposition eventually holds is at least as hard as the Skolem problem for linear recurrence sequences, which is a long-standing open problem in number theory. On the other hand, we show that verifying the evolution of structural CI propositions against LTL and NBA specifications is in PSPACE, and is hard for both NP and coNP. We also identify natural restrictions on the graphical structure of the DBN that make the verification of structural CI properties tractable.

Linear temporal logic (LTL) [1] is a formal language with rich expressibility and provides a formal description of complex task specifications for robots. In a surveillance problem, for example, mobile robots move around the target environment and collect data with attached sensors. In this abstract, we consider a control problem of a mobile robot working for a surveillance mission. We apply supervisory control to enforce the high-level behavior of the robot defined in an abstracted model [2] to satisfy a given syntactically cosafe LTL (scLTL) specification [3] . We extend the method proposed in [4] to the case under partial observation.

Manipulation planning from high-level task specifications, even though highly desirable, is a challenging problem. The large dimensionality of manipulators and complexity of task specifications make the problem computationally intractable. This work introduces a manipulation planning framework with linear temporal logic (LTL) specifications. The use of LTL as the specification language allows the expression of rich and complex manipulation tasks. The framework deals with the state-explosion problem through a novel abstraction technique. Given a robotic system, a workspace consisting of obstacles, manipulable objects, and locations of interest, and a co-safe LTL specification over the objects and locations, the framework computes a motion plan to achieve the task through a synergistic multi-layered planning architecture. The power of the framework is demonstrated through case studies, in which the planner efficiently computes plans for complex tasks. The case studies also illustrate the ability of the framework in intelligently moving away objects that block desired executions without requiring backtracking.

A formal approach for run-time verification of web applications using scope-extended LTL

Linear time temporal logic (LTL) has become a well established tool for specifying the dynamic behaviour of reactive systems with an interleaving semantics, and the automata-theoretic approach has proven to be a very useful mechanism for performing automatic verification in this setting. Especially alternating automata turned out to be a powerful tool in constructing efficient yet simple to understand decision procedures and directly yield further on-the-fly model checking procedures. In this paper we exhibit a decision procedure for LTL over Mazurkiewicz traces which generalises the classical automata-theoretic approach to a linear time temporal logic interpreted no longer over sequences but certain partial orders. Specifically, we construct a (linear) alternating Buchi automaton accepting the set of linearisations of traces satisfying the formula at hand. The salient point of our technique is to apply a notion of independence-rewriting to formulas of the logic. Furthermore, we show that the class of linear and trace-consistent alternating Buchi automata corresponds exactly to LTL formulas over Mazurkiewicz traces, lifting a similar result from Loding and Thomas formulated in the framework of LTL over words.

We investigate logical consequence in temporal logics in terms of logical consecutions, i.e., inference rules. First, we discuss the question: what does it mean for a logical consecution to be ‘correct’ in a propositional logic. We consider both valid and admissible consecutions in linear temporal logics and discuss the distinction between these two notions. The linear temporal logicLDTL, consisting of all formulas valid in the frame 〈L≤, ≥〉 of all integer numbers, is the prime object of our investigation. We describe consecutions admissible inLDTLin a semantic way—via consecutions valid in special temporal Kripke/Hintikka models. Then we state that any temporal inference rule has a reduced normal form which is given in terms of uniform formulas of temporal degree 1. Using these facts and enhanced semantic techniques we construct an algorithm, which recognizes consecutions admissible inLDTL. Also, we note that using the same technique it follows that the linear temporal logicL(N) of all natural numbers is also decidable w.r.t. inference rules. So, we prove that both logicsLDTLandL(N) are decidable w.r.t. admissible consecutions. In particular, as a consequence, they both are decidable (known fact), and the given deciding algorithms are explicit.

This article continues the series of works on the development and verification of control programs based on LTL (linear temporal logic) specifications of a special type. Previously, a declarative LTL specification was proposed, which allows describing the behavior of control programs and building program code based on it in the imperative ST language for programmable logic controllers (PLCs). This LTL specification can be directly verified for compliance with specified temporal properties by the model checking method using the nuXmv symbolic verification tool. In general, it is not required to translate the LTL formulas of the specification into another formalism—an SMV specification (code in the input language of the nuXmv tool). The aim of this study is to explore alternative ways of representing the behavioral model of a program that conforms to a declarative LTL specification during its verification using the nuXmv tool. In this article, we transform the declarative LTL specification into various SMV specifications with accompanying changes of formulation of the verification problem, which leads to a significant reduction in time costs when checking temporal properties by using the nuXmv tool. The acceleration of verification is due to the reduction of the state space of the model being verified. The SMV specifications obtained as a result of the proposed transformations specify identical or bisimulational-equivalent transition systems, ensuring the same verification results when replacing one SMV specification with another.

This article is written in support of the academic discipline “Nonclassical logics”. The objects of study within this discipline are the basic principles and constructive elements used in formal construction of various nonclassical propositional logics. Despite the abstract nature of the theory of nonclassical logics, which is mainly focused on strict mathematical formalization of logical reasoning, there are real practical applications for its theoretical results. In particular, languages of temporal modal logics are widely used for modeling, specification, and verification (correctness analysis) of logic control program systems. This article demonstrates, based on the example of linear temporal logic (LTL), how abstract concepts of nonclassical logics can be applied in practice in the area of information technology and programming. It is shown that the behavior of a software system can be represented as a set of LTL formulas, which can then be used to check the satisfiability of the properties of that software system via the procedure for proving the validity of logical inferences expressed in terms of linear temporal logic (LTL). The approach to using LTL to specify the behavior of software systems is demonstrated based on Minsky counter machines. Minsky machines are one of the ways of formalizing the intuitive concept of an algorithm. Their computational power is equivalent to that of Turing machines. A counter machine is a computer program written in a high-level language, since it contains variables called counters and conditional and unconditional jump operators used for loop construction. It is known that any algorithm can (hypothetically) be implemented as a three-counter Minsky machine.

For human-robot teams that operate in space, safety and robustness are paramount. In situations such as habitat construction, station inspection, or cooperative exploration, incorrect assumptions about the environment or task across the team could lead to mission failure. Thus it is important to resolve any ambiguity about the mission between teammates before embarking on a commanded task. The safeguards guaranteed by formal methods can be used to synthesize correct-by-construction reactive controllers for a robot using Linear Temporal Logic. If a robot fails to synthesize a controller given an instruction, it is clear that there exists a logical inconsistency in the environmental assumptions and/or described interactions. These specifications however are typically crafted in a language unique to the verification framework, requiring the human collaborator to be fluent in the software tool used to construct it. Furthermore, if the controller fails to synthesize, it may prove difficult to easily repair the specification. Language is a natural medium to generate these specifications using modern symbol grounding techniques. Using language empowers non-expert humans to describe tasks to robot teammates while retaining the benefits of formal verification. Additionally, dialogue could be used to inform robots about the environment and/or resolve any ambiguities before mission execution. This paper introduces an architecture for natural language interaction using a symbolic representation that informs the construction of a specification in Linear Temporal Logic. The novel aspect of this approach is that it provides a mechanism for resolving synthesis failure by hypothesizing corrections to the specification that are verified through human-robot dialogue. Experiments involving the proposed architecture are demonstrated using a simulation of an Astrobee robot navigating in the International Space Station.

LTL synthesis is the problem of synthesizing a reactive system from a formal specification in Linear Temporal Logic. The extension of allowing for partial observability, where the system does not have direct access to all relevant information about the environment, allows generalizing this problem to a wider set of real-world applications, but the difficulty of implementing such an extension in practice means that it has remained in the realm of theory. Recently, it has been demonstrated that restricting LTL synthesis to systems with finite executions by using LTL with finite-horizon semantics (LTLf) allows for significantly simpler implementations in practice. With the conceptual simplicity of LTLf, it becomes possible to explore extensions such as partial observability in practice for the first time. Previous work has analyzed the problem of LTLf synthesis under partial observability theoretically and suggested two possible algorithms, one with 3EXPTIME and another with 2EXPTIME complexity. In this work, we first prove a complexity lower bound conjectured in earlier work. Then, we complement the theoretical analysis by showing how the two algorithms can be integrated in practice into an established framework for LTLf synthesis. We furthermore identify a third, MSO-based, approach enabled by this framework. Our experimental evaluation reveals very different results from what the theory seems to suggest, with the 3EXPTIME algorithm often outperforming the 2EXPTIME approach. Furthermore, as long as it is able to overcome an initial memory bottleneck, the MSO-based approach can often outperforms the others.

Despite the current increasing popularity of Behavior Trees (BTs) in the robotics community, there does not currently exist a method to formally verify their correctness without compromising their most valuable traits: modularity, flexibility and reusability. In this letter we present a new mathematical framework in which we formally express Behavior Trees in Linear Temporal Logic (LTL). We show how this framework equivalently represents classical BTs. Then we utilize the proposed framework to construct an algorithm to verify that a given BT satisfies a given LTL specification. We prove that this algorithm is sound. Importantly, we prove that this method does not compromise the flexible design process of BTs, i.e. changes to subtrees can be verified separately and their combination can be assured to be correct. We present an example of the proposed algorithm in use.

: The question of whether branching-time temporal logic or linear-time temporal logic is best for reasoning about concurrent programs is one of the most controversial issues in logics of programs. Concurrent programs are usually modelled by labelled state-transition graphs in which some state is designated as the initial state. For historical reasons such graphs are called Kripke structures. In linear temporal logic operators are provided for describing events along a single time path (i.e., along a single path in a Kripke structure). In a branching-time logic the temporal operators quantify over the futures that are possible from a given state (i.e., over the possible paths that lead from a state). It is well known that the two types of temporal logic have different expressive powers. Linear temporal logic, for example, can express certain fairness properties that cannot be expressed in branching-time temporal logic. On the other hand, certain practical decision problems like model checking are easier for branching-time temporal logic than for linear temporal logic. This paper provides further insight on which type of logic is best. It is shown that if two finite Kripke structures can be distinguished by some formula that contains both branching-time and linear-time operators, then the structures can be distinguished by a formula that contains only branching time operators. Specifically, we show that if two finite Kripke structures can be distinguished by some formula of the logic CTL (i.e., if there is some CTL formula that is true in one but not in the other), then they can be distinguished by some formula of the logic CTL.

The Specification Pattern System (SPS) and the Property Specification (Prospec) tool assist a user in generating formal specifications in Linear Temporal Logic (LTL), as well as other languages, from property patterns and scopes. Patterns are high-level abstractions that provide descriptions of common properties, and scopes describe the extent of program execution over which the property holds. The purpose of the work presented in this paper is to verify that the generated LTL formulas match the natural language descriptions, timelines, and traces of computation that describe the pattern and scope. The LTL formulas were verified using the Spin model checker on test cases developed using boundary value analysis and equivalence class testing strategies. A test case is an LTL formula and a sequence of Boolean valuations. The LTL formulas were those generated from SPS and Prospec. The Boolean valuations of propositions in the LTL formula are generated by a deterministic, single-threaded Promela program that was run using the software model-checker Spin. For each pattern, a suite of test cases was. The experiments uncovered several errors in both the SPS-generated and the Prospec-generated formulas.