Generating Test Cases from Requirements: A Case Study in Railway Control System Domain

A description is presented of an expert system called ADES (ATP design expert system) for the design of automatic train-protection (ATP) systems. (An ATP system is a railway signaling system consisting of a set of logic circuits that control the safe movements of trains within a railway station.) Artificial intelligence techniques proved feasible to address this particular design problem, and ADES, which uses artificial intelligence techniques, makes it possible to design good control circuits to meet operational requirements rapidly by using well-structured, explicitly represented knowledge on ATP. ADES' inference engine works as a partial evaluator for the knowledge representation language KARL which is used to write design rules. Since every circuit produced by the expert system is still expressed in KARL, the KARL interpreter has also been used as a simulator for a designed circuit. Prototypes of the inference engine and the KARL interpreter have been implemented as extended Prolog metainterpreters. >

Automatic train protection (ATP) system is a safety critical application for railway signaling with high logic complexity, and it is typically embedded and real-time demanded. As the computational complexity of ATP system, traditional software development method cannot accommodate effectively. In this paper, we propose a novel model-based development (MBD) scheme for ATP system. We argue that high quality software development in ATP system should not only rely on function design, but also on safety verification. A model-based development life cycle and verification approach are described. Furthermore, we present a case study, which is using the tool of SCADE suite. The approach can shorten the development period, enhance system safety, and meet challenges in safety critical software development.

Testing RESTful APIs thoroughly is critical due to their key role in software integration. Existing tools for the automated generation of test cases in this domain have shown great promise, but their applicability is limited as they mostly rely on random inputs, i.e., fuzzing. In this paper, we present RESTest, an open source black-box testing framework for RESTful web APIs. Based on the API specification, RESTest supports the generation of test cases using different testing techniques such as fuzzing and constraint-based testing, among others. RESTest is developed as a framework and can be easily extended with new test case generators and test writers for different programming languages. We evaluate the tool in two scenarios: offline and online testing. In the former, we show how RESTest can efficiently generate realistic test cases (test inputs and test oracles) that uncover bugs in real-world APIs. In the latter, we show RESTest's capabilities as a continuous testing and monitoring framework. Demo video: https://youtu.be/1f_tjdkaCKo.

Automatic train protection (ATP) is an important equipment with a safety requirement of SIL 4 for Chinese Train Control System, whose safety is usually guaranteed by testing. Test cases generation method is an important basis and research focus of ATP. Taking Petri Net as the modelling tool, a test cases generation method of ATP is proposed. First, according to the technical specification, performing a fault mode analysis, ATP Petri Net testing model is established. Then, the test cases are generated automatically by searching the reachability tree of the testing model. Finally, taking the SOM function of ATP as an example, testing model is established and test cases are generated. By comparing the test cases generated by this method and the existing test cases, the correctness and effectiveness of this method is verified.

Safety is a concerned point in today’s railway control system. Historical evidence on railway control system failure makes a matter of concern on safe control system. In this paper, we present a use/misuse case based safety framework for a railway control system with integrated safety analysis. The framework comprises of two major processes i.e. use/misuse case based requirements modelling and application of integrated safety analysis techniques. First, the use case and misuse case specification analyze all the system function and identify possible failures. Second, integrated safety analysis techniques evaluate the specification, identify hazards and propose the mitigations. This paper analyses the railway control system reliability using Unified Modeling Language (UML) and followed by integrated safety analysis techniques such as Hazard and Operability Study (HAZOP) and Fault Tree Analysis (FTA). This approach has been systematically applied to study two different domain in system reliability and system control based on human interaction. We demonstrate how an applicability of UML specification is integrated with safety analysis techniques for the development of safe railway control system.

Railway control systems are based on on-board and trackside subsystems for signaling purposes. Several factors demand for new design and implementation solutions for such railway control systems. These factors are related to the design of interoperable railway networks in Europe, the introduction of new technologies and equipment, and the competition in the market of railway products. The safety of such new design and implementation solutions should still be proved in accordance with the CENELEC recommendations. SFDA, safe message freshness detection algorithm among trackside subsystems, is deeply described. SFDA is included in a new message passing safety protocol stack and it allows the detection of messages and the meeting of real time and safety requirements of trackside railway systems. It is demonstrated that the SFDA can detect all the old messages. Moreover a preliminary analysis of its availability characteristics to check whether it is suitable for railways systems is performed through simulation.

Train Interface Unit (TIU) is the key execution unit of on board Automatic Train Protection (ATP) system. ATP system achieves the control of the train and information exchange with the train via TIU. Developing a TIU which fits for different train platform and is in compliance with the safety integrity level (SIL) 4 can greatly facilitate the system installation and lower operation and maintenance cost. This thesis has designed a TIU for a series of standard train platform. Based on the system design, four key technologies of TIU, include relay and Multifunction Vehicle Bus (MVB) interface, safety communication, emergency brake (EB) interface and service brake (SB) interface, are in-depth in anatomist. Through the thorough analysis and evaluation of the safety and real-time performance, we can get the conclusion that the TIU sub-system meets the requirements of the practical use.

The next few years will see distributed real-time computer systems playing an important role in control systems of high-dependability applications, such as rail transportation. In these applications a failure in the temporal domain can be as critical as a failure in the value domain. In rail transportation, train control system has become more complex and the methods to ensure its correctness of have been considered outmoded. The safety of train control system is becoming increasingly important as computers pervade them on which human life depends and the failure to meet time deadline can have serious or even fatal consequences. This paper proposes a formal modeling and verification approach for real-time system. In the proposed method the real-time system is modeled by timed automata network (TAN) and verified by model checking which explores the state space to determine whether the system satisfies a given specification. The case study of ATP (automatic train protection) shows how the method can assist in designing more efficient and reliable real-time systems. Firstly, automatic train protection (ATP) system and timed automata network (TAN) model is proposed; secondly, the state transitions and multi-tasks ATP onboard model was modeled with TAN model, and then the time sequences of each task are expressed in UML sequence diagrams. Finally, the timing characteristics was verified to meet the requirement by UPPAAL model checker. A major conclusion of the survey is that formal methods, while still immature in some respects, can be used successfully to model and verify real-time systems.

Datalog is a popular and widely-used declarative logic programming language. Datalog engines apply many cross-rule optimizations; bugs in them can cause incorrect results. To detect such optimization bugs, we propose an automated testing approach called Incremental Rule Evaluation (IRE), which synergistically tackles the test oracle and test case generation problem. The core idea behind the test oracle is to compare the results of an optimized program and a program without cross-rule optimization; any difference indicates a bug in the Datalog engine. Our core insight is that, for an optimized, incrementally-generated Datalog program, we can evaluate all rules individually by constructing a reference program to disable the optimizations that are performed among multiple rules. Incrementally generating test cases not only allows us to apply the test oracle for every new rule generated—we also can ensure that every newly added rule generates a non-empty result with a given probability and eschew recomputing already-known facts. We implemented IRE as a tool named Deopt, and evaluated Deopt on four mature Datalog engines, namely Soufflé, CozoDB, μZ, and DDlog, and discovered a total of 30 bugs. Of these, 13 were logic bugs, while the remaining were crash and error bugs. Deopt can detect all bugs found by queryFuzz, a state-of-the-art approach. Out of the bugs identified by Deopt, queryFuzz might be unable to detect 5. Our incremental test case generation approach is efficient; for example, for test cases containing 60 rules, our incremental approach can produce 1.17× (for DDlog) to 31.02× (for Soufflé) as many valid test cases with non-empty results as the naive random method. We believe that the simplicity and the generality of the approach will lead to its wide adoption in practice.

The article presents the issue of digital standard interfaces of railway traffic control systems. The digitization and standardization of interfaces are digital rail elements, an essential feature of integrating various devices, facilities, and processes in one coherent system. The authors described the problems of connecting computer systems, starting from the interface's definition, including technical and functional aspects. Examples of interface types selected problems resulting from connecting incompatible devices and attempts to solve them are presented. It then provides background information on the project to develop a digital interface standard for selected rail traffic control systems. The project described in this article, under the name of "Standardization of selected interfaces of railway traffic control equipment and systems" POIR.04.01.01-00-0005/17, has been created as part of the BRIK (Research and Development in Railway Infrastructure) program and has been cofinanced both by the NCBiR (The National Centre for Research and Development), as well as the PKP Polskie Linie Kolejowe S.A. (PKP PLK - Polish State Railways). The project's goal is to develop, through research, specifications, and requirements for interfaces used in railway traffic control systems, which will become a standard enabling linking of systems from different manufacturers. The project results will be dedicated to using on the railway network managed by PKP PLK (and potentially also by other infrastructure managers) and documentation containing descriptions of standards, guidelines for the application and design of these interfaces.

Services are highly reusable, flexible and loosely coupled, which makes the evolution and the maintenance of composite services more complex. Evolution of BPEL composite service covers changes of processes, bindings and interfaces. In this paper, an approach is proposed to select and generate test cases during the evolution of BPEL composite service. The approach identifies the changes by using control-flow analysis technique and comparing the paths in new composite service version and the old one using extensible BPEL flow graph (or XBFG). Message flow is appended to the control flow so that XBFG can describe the behavior of composite service integrally. The binding and predicate constraint information added in XBFG elements can be used in path selection and test case generation. Theory analysis and case study both show that the approach is effective, and test cases coverage rate is high for the changes of processes, bindings and interfaces.

Runtime verification of train control systems with parameterized modal live sequence charts

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

Test cases are essential parts to perform software testing as they play very significant role defining the test steps to work. Test cases can be generated either manually or automatically. Creating test cases manually has several disadvantages, such as higher probability in producing inaccurate results. Furthermore, generating test cases manually is ineffective since manual methods will take longer and require high human efforts. Therefore, generating test cases automatically is more preferable to limit the previous problems. Automated testing can also provide more accurate results than manual testing because it is less prone to human errors encouraging software developers to generate test cases automatically. Test cases can be generated from various sources, including UML diagrams. In this paper, we propose a method to generate test cases automatically by utilising depth-first search algorithms. The UML diagram used in this study is an activity diagram. This paper also provides a method to parse activity diagram that is created using the visual paradigm application that has been converted into an XML format. Our method is able to parse and convert the XML format into graph or tree form. Using our proposed method, a test case search will be carried out using the depth-first search on the tree form to generate test cases. Lastly, we also evaluate our method by comparing the generated test cases with cyclomatic complexity, the method will be performed manually and automatically to validate the results of many scenarios. Our result showed that our method could generate the same scenario (100% coverage) as the manual validation method.

Extended Finite State Machine (EFSM) is widely- used to represent system specifications. Automated test data generation based on EFSM models is still a challenging task due to the complexity of transition paths. In this paper, we introduce a new approach to generate test cases automatically for given transition paths of an EFSM model. An executable EFSM model is used to provide run-time feedback information as fitness function. And then scatter search algorithm is used to search for test data that can trigger given transition paths. Based on the executable model, the expected outputs associated with test data are also collected for construction of test oracles automatically. Finally, test data (inputs) and test oracles (expected outputs) are combined to be test cases. The experimental results show that our approach can effectively generate test cases to exercise the feasible transition paths.