Gamification of software development, verification and validation

The verification and validation of CFD software is of utmost importance for the reliability and hence for the success of CFD technology in industry. This paper discusses the relative importance, key elements, and some examples of: • Verification of numerical models and computer software; • Validation of solutions against benchmark data; and • Calibration of the models and analysis process. Out of these, the main emphasis is placed on verification. This is in light of the growing generality and complexity of CFD software (in response to continuously increasing expectations in industry). Finally, three key success factors for effective utilization of CFD technology are pointed out. These are: 1) Verifications and Validation of software; 2) Skills of the Users; and 3) Vision, and commitment of the manager (s).

This document is intended to contribute to the area of validation and verification of communicating real time systems, with emphasis put on parametric reachability analysis of systems modeled using timed automata.Reachability analysis is a crucial aspect of validation and verification of software and hardware systems. The reachability analysis for real time systems is area that is studied by many researchers in academic and industrial communities. However, not much work has been done for systems, where time constraints are expressed using parameters. This is serious disproportion with real world, where specifications of most of the communication protocols or embedded software and hardware systems are indeed parameterized.This thesis presents a complete framework for forward and backward parametric reachability analysis. The solution presented here can be used as a base of algorithms for validation and verification of software and hardware real-time systems, modeled as timed automata with parameters. The results of the thesis can be easily applied to model checking or test generation tools and algorithms.The core idea of the thesis is a concept of Extended Difference Bound Matrix (EDBM). This is a data structure that stores relations between all system"s clocks and parameters. In contrast to Parametric DBM, that is the state-of-the-art data structure for parametric analysis, EDBM does not require storing constraints on clocks and constraints on parameters separately. This leads to significant benefits regarding memory consumption and time necessary to perform basic operations for symbolic analysis.The maturity of the solution was proven by implementation of a proof-of-concept tool and by experiments performed with modern communication protocol. The results show that even complex systems can be efficiently handled by the framework.

Chapter 15 - Software Verification and Validation Practice

In this paper we report the outcomes of the 1st and 2nd edition of the International Workshop on Gamification in Software Development, Verification, and Validation (Gamify 2022 and Gamify 2023) which were held as part of the 30th and 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022, in Singapore, November 17, 2022 and ESEC/FSE 2023, online workshop, December 4, 2023).

BackgroundBioinformatics software quality assurance is essential in genomic medicine. Systematic verification and validation of bioinformatics software is difficult because it is often not possible to obtain a realistic "gold standard" for systematic evaluation. Here we apply a technique that originates from the software testing literature, namely Metamorphic Testing (MT), to systematically test three widely used short-read sequence alignment programs.ResultsMT alleviates the problems associated with the lack of gold standard by checking that the results from multiple executions of a program satisfy a set of expected or desirable properties that can be derived from the software specification or user expectations. We tested BWA, Bowtie and Bowtie2 using simulated data and one HapMap dataset. It is interesting to observe that multiple executions of the same aligner using slightly modified input FASTQ sequence file, such as after randomly re-ordering of the reads, may affect alignment results. Furthermore, we found that the list of variant calls can be affected unless strict quality control is applied during variant calling.ConclusionThorough testing of bioinformatics software is important in delivering clinical genomic medicine. This paper demonstrates a different framework to test a program that involves checking its properties, thus greatly expanding the number and repertoire of test cases we can apply in practice.

This INL plan comprehensively describes the software for RELAP-7 and documents the software, interface, and software design requirements for the application. The plan also describes the testing-based software verification and validation (SV&V) process—a set of specially designed software models used to test RELAP-7. The RELAP-7 (Reactor Excursion and Leak Analysis Program) code is a nuclear reactor system safety analysis code being developed at Idaho National Laboratory (INL). The code is based on the INL’s modern scientific software development framework – MOOSE (Multi-Physics Object-Oriented Simulation Environment). The overall design goal of RELAP-7 is to take advantage of the previous thirty years of advancements in computer architecture, software design, numerical integration methods, and physical models. The end result will be a reactor systems analysis capability that retains and improves upon RELAP5’s capability and extends the analysis capability for all reactor system simulation scenarios.

It is tempting just to develop software in the usual way and then try to assure that the finished product is safe. This approach is akin to relying on downstream protection systems-the cost can be exorbitant and the assurance limited. The most practical and effective way to enhance software system safety is to follow a complete safeware program that applies safety enhancing techniques throughout software development and maintenance. Verification testing verifies that requirements for a subsystem have been met. Validation tests a finished device to the original requirements to ensure they have been met. Appropriately applied V and V decreases the cost of software development. Verification and validation can make a significant contribution in making the final product safe and effective. Software changes can be more easily and safely made when the development process is controlled and documented. The template is already in place. Regression testing is easily identified and can reduce the time required to make and test changes. The FDA requires evidence that verification and validation is included in the product development process. The emphasis is on appropriately applied V and V. A software development process that produces, on time, bug free code, meeting all of its requirements, and has intrinsic verification and validation identifying defects as early as possible, is a very cost efficient process, regardless of industry. The earlier a defect is found in the development process, the lower the cost to correct it.

A graduate course on software verification and validation (VV approaches to testing; formal methods of verification; and techniques for ensuring that software is trustworthy (more reliable than can be measured). This makes it challenging to decide what to include in a single course. Justifications are given for some of the choices made. Choices of material for undergraduate curricula are even more difficult to make. Some suggestions are offered in this area. Experience with courses on V&V is relatively thin, and the basis for teaching much of the material is rapidly evolving, so stable widely used curricula are still some time away.

Rapid advancements in Software Verification and Validation have been critical in the wide development of tools and techniques to identify potential Concurrent bugs and hence verify the software correctness. A concurrent program has multiple processes and shared objects. Each process is a sequential program and they use the shared objects for communication for completion of a task. The primary objective of this survey is retrospective review of different tools and methods used for the verification of real-time concurrent software. This paper describes the proposed tool ‘F-JAVA’ for multithreaded Java codebases in contrast with existing ‘FRAMA-C’ platform, which is dedicated to real-time concurrent C software analysis. The proposed system is comprised of three layers, namely Programming rules generation stage, Verification stage with Particle Swarm Optimization (PSO) algorithm, and Performance measurement stage. It aims to address some of the challenges in the verification process such as larger programs, long execution times, and false alarms or bugs, and platform independent code verification

This paper proposes a methodology for the development and its Verification and Validation(V&V) of safety-rated software. The development methodology is based on the structured analysis and structured design methodology, which has been proven in the field of safety software development during long period. The suggested the development and the V&V methodology additionally focuses on the quality attributes 3C+T (completeness, Correctness, Consistency and traceability), which are important criteria in IEC international standards. The software development life cycle is compiled from IEC international standardization, which are of the requirements analysis phase, design phase, implementation phase, testing phase, installation/commissioning phase, and operation & maintenance phases. The activities of the development process method were defined from requirements analysis to implementation, and the activities of the verification and validation were defined to include the scope of the developer's activities and then to include the test phase, installation/commissioning phase, and operation phases additionally. In other words, testing phase performed by the developer was defined as informal testing, and testing performed by the verifier according to test criteria, test plans, and test procedures was defined as formal testing. According to proposed methodology, the requirements phase defines the sequence of Entity Relationship Diagram (ERD)-Context Diagram-Data Flow Diagram (DFD), functional decomposition process and defines operational scenario. In the design phase, module design such as task analysis and task allocation were defined. In the implementation phase, we listed the coding guidelines that must be followed for safety-related system. The test phase defined the test criteria, test plan, test procedure, and test result report activities. In the installation/commissioning and operation phases, recommendations for improving reliability, quality, and safety were proposed for deliverables such as user documentation and technical documentation.

Software is being employed for life-critical, safety-critical, infrastructure-critical and economically critical applications. Our daily lives rely heavily on proper functioning of software in gadgets we directly or indirectly use-airplanes, flight control, high speed trains, cars, cell-phones, medical devices and instruments, banks, and what not. Malfunctioning of a program can have very severe consequences-costing lives (e.g. Therac-25 [13], Patriot missile) and money (e.g. Ariane 5, malfunctioning of economic transactions, problems in stock exchanges) [14]. Validation and verification of software have become even more and more important. Given that full verification of software has been found increasingly difficult to achieve because of lack of rigorous and complete specifications on one hand as well as difficulty of verification systems/theorem provers to address the increasing complexity of software despite considerable advances in automated reasoning techniques, ensuring absence of various types of bugs becomes a critical first step in ensuring reliability.

Nowadays due to the technology development and use of digital computers in various systems, need for development of high performance and robust software is attracting great attentions. Because of increasing complexity in algorithms and implementation hardware for embedded systems, proper simulation tools are required. In sophisticated systems design, hardware in the loop (HIL) simulation is known as a prominent simulation tool before realistic tests of the system and a step after software simulation. Simultaneously it can be used for verification and validation of automation and control software. HIL has had an historical background in aerospace industries. Recently, this tool has spread in different steps of system life cycle such as design, development, implementation and test of various applications including automobile industry, shipbuilding, power lines, robotic systems and etc. Utilizing a suitable hardware in the loop laboratory, in system design stages is a practical way to increase the system reliability and efficiency as well as value of product. Also, by proper investigation in this modelling and simulation method, many errors can be avoided in design procedure of software and hardware as well as their interconnections. In this study, structure and components of an hardware in the loop laboratory for different systems are explored, also it is tried to more evaluate the applications of HIL simulations in dynamics and control engineering. At last, general structure of an hardware in the loop lab for diverse industries is proposed and discussed.

Recent advances in software validation and verification make it possible to widely automate whether a specification is satisfied. This progress is hampered, though, by the persistent difficulty of writing specifications. Are we facing a “specification crisis”? In this paper, I show how to alleviate the burden of writing specifications by reusing and extending specifications as mined from existing software and give an overview on the state of the art in specification mining, its origins, and its potential.

This document provides a record of the verification and Validation of the ANSYS Version 7.0 software that is installed on selected CH2M HILL computers. The issues addressed include: Software verification, installation, validation, configuration management and error reporting. The ANSYS{reg_sign} computer program is a large scale multi-purpose finite element program which may be used for solving several classes of engineering analysis. The analysis capabilities of ANSYS Full Mechanical Version 7.0 installed on selected CH2M Hill Hanford Group (CH2M HILL) Intel processor based computers include the ability to solve static and dynamic structural analyses, steady-state and transient heat transfer problems, mode-frequency and buckling eigenvalue problems, static or time-varying magnetic analyses and various types of field and coupled-field applications. The program contains many special features which allow nonlinearities or secondary effects to be included in the solution, such as plasticity, large strain, hyperelasticity, creep, swelling, large deflections, contact, stress stiffening, temperature dependency, material anisotropy, and thermal radiation. The ANSYS program has been in commercial use since 1970, and has been used extensively in the aerospace, automotive, construction, electronic, energy services, manufacturing, nuclear, plastics, oil and steel industries.

This document provides a record of the verification and Validation of the ANSYS Version 8.0 software that is installed on selected CH2M HILL computers. The issues addressed include: Software verification, installation, validation, configuration management and error reporting.