Academia and industry meet: some experiences of formal methods in practice

There is certainly no doubt that formal methods should be used, as far as possible, in the specification and design of real-time systems [2]. The number and quality of articles published on formal methods grow significantly from year to year. Their use, however, is still not very effective. In this note, I am trying to give a brief summary of such applications, followed by some suggestions which may contribute to a more widespread use of formal methods in practice.

Over recent years the use of formal methods in industry has greatly increased. Formal methods are defined as any method that allows you to reason about a problem, and which provides a logical (mathematical) way of proving the correctness of the proposed solution. Software engineers need competency in managing programming in the large. It is important to know all the methods, when they should be applied and when they should not. The difficult question for the software engineering educator is: when and how should these formal methods be taught in the curriculum?. >

This volume contains the proceedings of the First International Workshop of Formal Techniques for Safety-Critical Systems (FTSCS 2012), held in Kyoto on November 12, 2012, as a satellite event of the ICFEM conference. The aim of this workshop is to bring together researchers and engineers interested in the application of (semi-)formal methods to improve the quality of safety-critical computer systems. FTSCS is particularly interested in industrial applications of formal methods. Topics include: - the use of formal methods for safety-critical and QoS-critical systems, including avionics, automotive, and medical systems; - methods, techniques and tools to support automated analysis, certification, debugging, etc.; - analysis methods that address the limitations of formal methods in industry; - formal analysis support for modeling languages used in industry, such as AADL, Ptolemy, SysML, SCADE, Modelica, etc.; and - code generation from validated models. The workshop received 25 submissions; 21 of these were regular papers and 4 were tool/work-in-progress/position papers. Each submission was reviewed by three referees; based on the reviews and extensive discussions, the program committee selected nine regular papers, which are included in this volume. Our program also included an invited talk by Ralf Huuck.

Formal methods encompass a wide choice of techniques and tools for the specification, development, analysis, and verification of software and hardware systems. Formal methods are widely applied in industry, in activities ranging from the elicitation of requirements and the early design phases all the way to the deployment, configuration, and runtime monitoring of actual systems. Formal methods allow one to precisely specify the environment in which a system operates, the requirements and properties that the system should satisfy, the models of the system used during the various design steps, and the code embedded in the final implementation, as well as to express conformance relations between these specifications. We present a broad scope of successful applications of formal methods in industry, not limited to the well-known success stories from the safety-critical domain, like railways and other transportation systems, but also covering other areas such as lithography manufacturing and cloud security in e-commerce, to name but a few. We also report testimonies from a number of representatives from industry who, either directly or indirectly, use or have used formal methods in their industrial project endeavours. These persons are spread geographically, including Europe, Asia, North and South America, and the involved projects witness the large coverage of applications of formal methods, not limited to the safety-critical domain. We thus make a case for the importance of formal methods, and in particular of the capacity to abstract and mathematical reasoning that are taught as part of any formal methods course. These are fundamental Computer Science skills that graduates should profit from when working as computer scientists in industry, as confirmed by industry representatives.

The title of this paper provides an answer to the question posed by Larry King in his panel paper titled "Do formal methods really belong in the toolbox of the practicing engineer?" Yes, formal methods do indeed belong to the toolbox of the practicing engineer. The capabilities of today's computers are truly impressive and as a result have opened door to the design and development of increasingly ambitious systems. An almost unavoidable by-product of these systems is the growing consequences associated with their failure. Another attribute shared by most of these systems is that their complexity is of an unprecedented nature. Thus, these systems live in the cross section (or shall I say cross-hairs) of high consequence and high complexity. For these reasons, attaining and developing the ability to construct highly reliable software is be coming a significant concern. Furthermore, software manufacturers as well as consumers are getting more and more fed-up with missed deadlines, bugs crashing and freezing hardware, and in the worst case, causing havoc to organizations, as bugs/incompleteness lead to allowing trojan horses for hackers through viruses and attacks. This is leading to attempts to replace ad hoc techniques in system design by more systematic methods. What is the role that formal methods, as a field of study, can play in addressing these issues? If by formal methods, one is exclusively referring to formal verification methods for software, such tools will probably not be part of a practicing engineer's tool in the near future. King's note reflects the prevailing confusion about formal methods, what can be achieved using them, and what cannot be achieved using them, and their overall role in system design and development.

Courses on formal methods are often based on examples and case studies, which are supposed to show students how to apply formal methods in practice. However, examples often fall into one of two categories: First, many are artificial and thus do not relate to practice. Second, other examples are based on projects of industry partners and therefore often are too involved for students to understand them.In this paper, we present a different approach. By formalizing the rules of commonly known games, we achieve examples both engaging and suited for students. Furthermore, we broaden the horizon of formal methods, driving research at the same time: we present extensions such as playable visualizations and explore the relationship between game AIs and model checking heuristics.

This white paper argues that formal methods need to be better rooted in higher education curricula for computer science and software engineering programmes of study. To this end, it advocates improved teaching of formal methods; systematic highlighting of formal methods within existing, ‘classical’ computer science courses; and the inclusion of a compulsory formal methods course in computer science and software engineering curricula. These recommendations are based on the observations that formal methods are an essential and cost-effective means to increase software quality; however computer science and software engineering programmes typically fail to provide adequate training in formal methods; and thus there is a lack of computer science graduates who are qualified to apply formal methods in industry. This white paper is the result of a collective effort by authors and participants of the 1st International Workshop on Formal Methods – Fun for Everybody which was held in Bergen, Norway, 2–3 December 2019. As such, it represents insights based on learning and teaching computer science and software engineering (with or without formal methods) at various universities across Europe.

On Using Large Language Models Pre-trained on Digital Twins as Oracles to Foster the Use of Formal Methods in Practice

Formal methods tools have been shown to be effective at finding defects in and verifying the correctness of safety-critical systems such as avionics systems. The recent release of DO-178C and the accompanying Formal Methods Supplement DO-333 will make it easier for developers of software for commercial aircraft to obtain certification credit for the use of formal methods. However, most developers of avionics systems are unfamiliar with formal methods, and most developers of formal methods tools are unfamiliar with certification requirements and processes. This article provides a brief overview of the certification process for commercial aircraft, as well as some of the issues related to the use of formal methods tools in this context.

In this paper, we outline a possible formalization of the semantics of the XML-based ODX language. ODX is a substantial part of the emerging automotive ASAM MCD standard, which describes a middleware layer between offboard diagnosis applications and onboard diagnosis services of electronic control units (ECUs) of cars. The contribution of our work is threefold: Firstly, a consequent application of our results can contribute to a well-structured development process in the automotive diagnosis domain. We are currently showing this in practice as part of our ongoing cooperation with the diagnosis department of AUDI AG. Secondly, our proposition is the first step towards guaranteed standard-conformity of implementations of the MCD run-time system, also specified by the standard and strongly depending on ODX and its semantics. Last but not least, the paper can serve as an encouraging example of the application of formal methods in practice.

Formal methods for software development receive much attention in research centres, but are rarely used in industry for the development of (large) software systems. One of the reasons is that little is known about the integration of formal methods in the software process, and the exact role of formal methods in the software life-cycle is still unclear. In this paper, a detailed examination is made of the application of, and the benefits resulting from, a generally applicable formal method (VDM) in a standard model for software development (DoD-STD-2167A). Currently, there is no general agreement on how formal methods should be used, but in order to analyse the use of formal methods in the software process, a clear view of such use is essential. Therefore, we show what is meant by ‘using a formal method’. The different activities of DoD-STD-2167A are analysed with regard to their suitability for applying VDM and the benefits that may result from applying VDM for that activity. Based on this analysis, an overall view on the usage of formal methods in the software process is formulated.

Microprocessors increasingly control and monitor our most critical systems, including automobiles, airliners, medical systems, transportation grids, and defense systems. The relentless march of semiconductor process technology has given engineers exponentially increasing transistor budgets at constant recurring cost. This has encouraged increased functional integration onto a single die, as well as increased architectural sophistication of the functional units themselves. Additionally, design cycle times are decreasing, thus putting increased schedule pressure on engineers. Not surprisingly, this environment has led to a number of uncaught design flaws. Traditional simulation-based design verification has not kept up with the scale or pace of modern microprocessor system design. Formal verification methods offer the promise of improved bug-finding capability, as well as the ability to establish functional correctness of a detailed design relative to a high-level specification. However, widespread use of formal methods has had to await breakthroughs in automated reasoning, integration with engineering design languages and processes, scalability, and usability. This book presents several breakthrough design and verification techniques that allow these powerful formal methods to be employed in the real world of high-assurance microprocessor system design.

Recently, formal methods have been used in large industrial organisations (including AWS, Facebook/Meta, and Microsoft) and have proved to be an effective part of a software engineering process finding important bugs. Perhaps because of that, practitioners are interested in using them more often. Nevertheless, formal methods are far less applied than expected, particularly for safety-critical systems where they are strongly recommended and have the most significant potential. We hypothesise that formal methods still seem not applicable enough or ready for their intended use in such areas. In critical software engineering, what do we mean when we speak of a formal method? And what does it mean for such a method to be applicable both from a scientific and practical viewpoint? Based on what the literature tells about the first question, with this manifesto, we identify key challenges and lay out a set of guiding principles that, when followed by a formal method, give rise to its mature applicability in a given scope. Rather than exercising criticism of past developments, this manifesto strives to foster increased use of formal methods in any appropriate context to the maximum benefit.

The main current trend in applied formal methods can be characterized by the term “lightweight”. Historically, formal methods have been viewed as pure alternatives to traditional development methodologies, demanding a revolutionary change in industry to adopt them. With a pragmatic, lightweight approach, the use of formal methods is complementing and improving existing development practices in a company in an evolutionary way, demonstrating more clearly the cost-effectiveness of formal methods. This paper presents our view on lightweight formal methods as a strategy for successful formal methods technology transfer to industry.KeywordsFormal MethodGraphical NotationMetro SystemTechnology Transfer ProcessSoftware Life CycleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper summarises our experience in teaching Formal Methods to Computer Science and Software Engineers students from various universities around the world, including the University of Madeira in Portugal, the Pontificia Universidad Javeriana and the University of The Andes in Colombia, Carnegie Mellon University (CMU) in the USA, and Innopolis University (INNO) in Russia. We report challenges we have faced during the past 10 to 15 years when teaching formal methods using the Event B formalism, and describe how we have evolved the structure of our courses to respond to those challenges. We strive to help students to build skills on Formal Methods that they can employ later on in their future IT jobs in software Industry. Our goal is to promote the wide use of Formal Methods by software Industry. We consider that this goal cannot be achieved without first universities transferring to Industry students with a strong background in Formal Methods and related formal tools. Formal Methods are key to software development because they are based on Discrete Mathematics which can be used to properly reason about properties that the software one develops should have. We have conducted two surveys among our students, the first one at CMU and the second one at INNO, that we use here to document and justify our decisions in terms of the course structure. The first survey is about the use of Event B as main mathematical formalism, and the second one is about the organisation of teams of students within the classroom to work on software projects that use Event B as main mathematical formalism. Our hope is that our work can be reused by other Faculty to make their own decisions on course structure and content in the teaching of their Formal Methods courses.