Formalization of mathematics through proof assistants

Artificial Intelligence for Theorem Proving (AITP) has given rise to a plethora of benchmarks and methodologies, particularly in Interactive Theorem Proving (ITP). Research in the area is fragmented, with a diverse set of approaches being spread across several ITP systems. This presents a significant challenge to the comparison of methods, which are often complex and difficult to replicate. Addressing this, we present BAIT, a framework for the fair and streamlined comparison of learning approaches in ITP. We demonstrate BAIT’s capabilities with an in-depth comparison, across several ITP benchmarks, of state-of-the-art architectures applied to the problem of formula embedding. We find that Structure Aware Transformers perform particularly well, improving on techniques associated with the original problem sets. BAIT also allows us to assess the end-to-end proving performance of systems built on interactive environments. This unified perspective reveals a novel end-to-end system that improves on prior work. We also provide a qualitative analysis, illustrating that improved performance is associated with more semantically-aware embeddings. By streamlining the implementation and comparison of Machine Learning algorithms in the ITP context, we anticipate BAIT will be a springboard for future research.

Automated and interactive theorem proving are the two main directions in the field of deduction. Most chapters of this book belong to either the one or the other, whether focusing on theory, on methods or on systems. This reflects the fact that, for a long time, research in computer-aided reasoning was divided into these two directions, driven forward by different communities. Both groups offer powerful tools for different kinds of tasks, with different solutions, leading to different performance and application profiles. Some important examples are: ACL2 (Kaufmann and Moore, 1988), HOL (Gordon, 1988), IMPS (Farmer et al., 1996), Isabelle (Paulson, 1994), KIV (Reif et al., 1997) (see also Chapter II.1.1), NQTHM (Boyer and Moore, 1979), and PVS (Owre et al., 1992) for the interactive (or tactical) theorem proving community; and KoMeT (Bibel et al., 1994), Otter (Wos et al., 1992), Protein (Baumgartner and Furbach, 1994), Setheo (Goller et al., 1994), Spass (Weidenbach et al., 1996), and 3 TAP (Beckert et al., 1996) for the automated theorem proving community.KeywordsTheorem ProverSequent CalculusSoftware VerificationAutomate Theorem ProverProof SearchThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Most computer checked proofs are tied to the particular technology of a prover’s software. While sharing results between proof assistants is a recognized and desirable goal, the current organization of theorem proving tools makes such sharing an exception instead of the rule. In this talk, I argue that we need to turn the current architecture of proof assistants and formal proofs inside-out. That is, instead of having a few mature theorem provers include within them their formally checked theorems and proofs, I propose that proof assistants should sit on the edge of a web of formal proofs and that proof assistant should be exporting their proofs so that they can exist independently of any theorem prover. While it is necessary to maintain the dependencies between definitions, theories, and theorems, no explicit library structure should be imposed on this web of formal proofs. Thus a theorem and its proofs should not necessarily be located at a particular URL or within a particular prover’s library. While the world of symbolic logic and proof theory certainly allows for proofs to be seen as global and permanent objects, there is a lot of research and engineering work that is needed to make this possible. I describe some of the required research and development that must be done to achieve this goal.

This volume contains the proceedings of the Workshop on Partiality and Recursion in Interactive Theorem Provers (PAR 2010) which took place on July 15 in Edinburgh, UK. This workshop was held as a satellite workshop of the International Conference on Interactive Theorem Proving (ITP 2010), itself part of the Federated Logic Conference 2010 (FLoC 2010). This workshop is a venue for researchers working on new approaches to cope with partial functions and terminating general (co)recursion in theorem provers.

We introduce a heuristic for automatically checking the validity of first-order formulas of the form forallalpham existbetan middot Psi(alpham,betan) that are encountered in inductive proofs of hardware correctness. The heuristic introduced in this paper is used to automatically check the validity of k-step induction formulas needed to verify hardware designs. The heuristic works on word-level designs that can have data and address buses of arbitrary widths. Our refinement heuristic relies on the idea of predicate instantiation introduced in (H. Abu-Haimed et al., 2003). The heuristic proves quantified formulas by the use of a validity checker, CVC (A. Stump et al., 2002), and a first-order theorem prover, Otter (W.W. McCune, 1994). Our heuristic can be used as a stand-alone technique to verify word-level designs or as a component in an interactive theorem prover. We show the effectiveness of this heuristic for hardware verification by verifying a number of hardware designs completely automatically. The large size of the quantified formulas encountered in these examples shows the effectiveness of our heuristic as a component of a theorem prover

And-Inverter Graphs (AIGs) are a popular way to represent Boolean functions (like circuits). AIG simplification algorithms can dramatically reduce an AIG, and play an important role in modern hardware verification tools like equivalence checkers. In practice, these tricky algorithms are implemented with optimized C or C++ routines with no guarantee of correctness. Meanwhile, many interactive theorem provers can now employ SAT or SMT solvers to automatically solve finite goals, but no theorem prover makes use of these advanced, AIG-based approaches. We have developed two ways to represent AIGs within the ACL2 theorem prover. One representation, Hons-AIGs, is especially convenient to use and reason about. The other, Aignet, is the opposite; it is styled after modern AIG packages and allows for efficient algorithms. We have implemented functions for converting between these representations, random vector simulation, conversion to CNF, etc., and developed reasoning strategies for verifying these algorithms. Aside from these contributions towards verifying AIG algorithms, this work has an immediate, practical benefit for ACL2 users who are using GL to bit-blast finite ACL2 theorems: they can now optionally trust an off-the-shelf SAT solver to carry out the proof, instead of using the built-in BDD package. Looking to the future, it is a first step toward implementing verified AIG simplification algorithms that might further improve GL performance.

This paper presents DAIDALUS (Detect and Avoid Alerting Logic for Unmanned Systems), a reference implementation of a detect and avoid concept intended to support the integration of Unmanned Aircraft Systems into civil airspace. DAIDALUS consists of self-separation and alerting algorithms that provide situational awareness to UAS remote pilots. These algorithms have been formally specified in a mathematical notation and verified for correctness in an interactive theorem prover. The software implementation has been verified against the formal models and validated against multiple stressing cases jointly developed by the US Air Force Research Laboratory, MIT Lincoln Laboratory, and NASA. The DAIDALUS reference implementation is currently under consideration for inclusion in the appendices to the Minimum Operational Performance Standards for Unmanned Aircraft Systems presently being developed by RTCA Special Committee 228.

Intermediate verification languages like Why3 and Boogie have made it much easier to build program verifiers, transforming the process into a logic compilation problem rather than a proof automation one. Why3 in particular implements a rich logic for program specification with polymorphism, algebraic data types, recursive functions and predicates, and inductive predicates; it translates this logic to over a dozen solvers and proof assistants. Accordingly, it serves as a backend for many tools, including Frama-C, EasyCrypt, and GNATProve for Ada SPARK. But how can we be sure that these tools are correct? The alternate foundational approach, taken by tools like VST and CakeML, provides strong guarantees by implementing the entire toolchain in a proof assistant, but these tools are harder to build and cannot directly take advantage of SMT solver automation. As a first step toward enabling automated tools with similar foundational guarantees, we give a formal semantics in Coq for the logic fragment of Why3. We show that our semantics are useful by giving a correct-by-construction natural deduction proof system for this logic, using this proof system to verify parts of Why3’s standard library, and proving sound two of Why3’s transformations used to convert terms and formulas into the simpler logics supported by the backend solvers.

Automated theorem provers for first-order logic usually operate on sets of first-order clauses. It is well-known that the translation of a formula in full first-order logic to a clausal normal form (CNF) can crucially affect performance of a theorem prover. In our recent work we introduced a modification of first-order logic extended by the first class boolean sort and syntactical constructs that mirror features of programming languages. We called this logic FOOL. Formulas in FOOL can be translated to ordinary first-order formulas and checked by first-order theorem provers. While this translation is straightforward, it does not result in a CNF that can be efficiently handled by state-of-the-art theorem provers which use superposition calculus. In this paper we present a new CNF translation algorithm for FOOL that is friendly and efficient for superposition-based first-order provers. We implemented the algorithm in the Vampire theorem prover and evaluated it on a large number of problems coming from formalisation of mathematics and program analysis. Our experimental results show an increase of performance of the prover with our CNF translation compared to the naive translation.

Algorithmic methods can successfully automate the proof, and even the discovery, of a large class of identities involving sums of hypergeometric terms. In particular, the Wilf-Zeilberger (WZ) algorithm is a uniform framework for a substantial class of hypergeometric summation problems. This algorithm can produce a rational function certificate that can, on the face of it, be used to verify the result by routine algebraic manipulations, independently of the working of the algorithm that discovered it. It is therefore very natural to consider using this certificate to produce, by automated means, a rigorous deductive proof in an interactive theorem prover. However, naive presentations of the WZ method tend to gloss over trivial-looking but rather knotty questions about zero denominators, which makes their rigorous formalization tricky and their ultimate logical justification somewhat obscure. We describe how we have handled these difficulties to produce rigorous WZ proofs inside the HOL Light theorem prover.

Proof searching and proof automation are the two most desired properties in interactive theorem provers (ITPs) as they generally require manual user guidance, which can be quite cumbersome. In this paper, we provide an evolutionary proof searching approach for the HOL4 proof assistant, where a genetic algorithm (GA) with different crossover and mutation operators is used to search and optimize the proofs in different HOL theories. Random proof sequences are first generated from a population of frequently occurring HOL4 proof steps that are discovered with sequential pattern mining. Generated proof sequences are then evolved with GA operators (three crossover and two mutation) till their fitness match the fitness of the target proof sequences. Various crossover and mutation operators are used to compare their effect on the performance of GAs in proof searching. Obtained results suggest that integrating GAs with HOL4 allows us to efficiently support proof finding and optimization.

Agda is a dependently typed functional programming language and a proof assistant in which developing programs and proving their correctness is one activity. We show how this process can be enhanced by integrating external automated theorem provers, provide a prototypical integration of the equational theorem prover Waldmeister, and give examples of how this proof automation works in practice.KeywordsProof ObligationProof AssistantAutomate Theorem ProveEquational LogicFunctional Programming LanguageThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Agora system is a prototype for Formal Mathematics, with an aim to support developing and documenting large formalizations of mathematics in a proof assistant. The functions implemented in Agora include in-browser editing, strong AI/ATP proof advice, verification, and HTML rendering. The HTML rendering contains hyperlinks and provides on-demand explanation of the proof state for each proof step. In the present paper we show the prototype Flyspeck Wiki as an instance of Agora for HOL Light formalizations. The wiki can be used for formalizations of mathematics and for writing informal wiki pages about mathematics. Such informal pages may contain islands of formal text, which is used here for providing an initial cross-linking between Hales's informal Flyspeck book, and the formal Flyspeck development. The Agora platform intends to address distributed wiki-style collaboration on large formalization projects, in particular both the aspect of immediate editing, verification and rendering of formal code, and the aspect of gradual and mutual refactoring and correspondence of the initial informal text and its formalization. Here, we highlight these features within the Flyspeck Wiki.

In proof checkers and theorem provers (e.g. Coq [4] and ProPre [13]) recursive definitions of functions are shown to terminate automatically. In standard non-formalised termination proofs of recursive functions, a decreasing measure is sometimes used. Such a decreasing measure is usually difficult to find.By observing the proof trees of the proofs of termination of recursive functions in ProPre (the system used in Coq’s proofs of termination), [14] finds a decreasing measure which could be used to show termination in the standard non-formalised way. This is important because it establishes a method to find decreasing measures that help in showing termination. As the ProPre system made heavy use of structural rather than inductive rules, an extended more powerful version has been built with new proof trees based on new rules.In this article, we show that the ordinal measures found in [14] lose the decreasing property in the extended ProPre system and then, set out to show that the extended ProPre system will still be suitable for finding measures required by other systems (e.g. NQTHM). We do this by showing that exist other measures that can be associated to the proof trees developed in the extended ProPre system that respect the decreasing property. We also show that the new parameterised measure functions preserve the decreasing property up to a simple condition.

Recursive algebraic data types (term algebras, ADTs) are one of the most well-studied theories in logic, and find application in contexts including functional programming, modelling languages, proof assistants, and verification. At this point, several state-of-the-art theorem provers and SMT solvers include tailor-made decision procedures for ADTs, and version~2.6 of the SMT-LIB standard includes support for ADTs. We study an extremely simple approach to decide satisfiability of ADT constraints, the reduction of ADT constraints to equisatisfiable constraints over uninterpreted functions (EUF) and linear integer arithmetic (LIA). We show that the reduction approach gives rise to both decision and Craig interpolation procedures in (extensions of) ADTs.