Airborne Software Testing Technology Analysis Based on Fault Injection

The phenomenon of high availability (HA) is of vital importance in cloud architecture. This paper proposes an HA verification framework, called HAVerifier, for OpenStack, a popular open source cloud platform. Fault injection technology has been adopted to verify the system's reliability by determining its health status after injecting faults. The framework proposed in this paper verifies the availability of services by injecting faults into the different components of OpenStack. Service indicators (for example, downtime) are monitored after the faults are injected. The collected metrics are compared with the provided service level agreement to verify whether the platform's availability meets the requirements. The fault injection steps can be implemented dynamically using the proposed framework, and the faults injected into the platform can be restored without manual intervention. Finally, a prototype for this framework is implemented to prove its applicability to verifying the HA of the OpenStack platform.

Firstly, the paper made a contrast between the current domestic and international research condition, and introduced the basic concept of fault injection and distributed system. secondly, it discussed the classification and requirements of fault injection. There are mainly three types of distributed fault, namely the memory fault, CPU fault and correspondence fault. Besides, it discussed the method of distributed software fault injection about DOCTOR and illustrated the comprehensive structure and its respective parts of DOCTOR in detail. Thirdly, it reached a conclusion about the fault model of distributed system of fault injection and its realization method.

SOFTWARE TESTING is the examination of code and/or execution of a software product/system for purposes that include detecting flaws in the code/logic, measuring the system performance in accordance with certain client-specified benchmarks and criteria, sealing of potential unintended loopholes, and enhancing the degree of user-friendliness. Of its three general strategies - Black-Box (overall skimming of software execution), White-Box (in-depth code analysis), and Grey-Box (combination of the former two) - FAULT INJECTION comes majorly under White-Box Testing, though also referred to by some as a Grey-Box strategy. Most commercial fault injection tools include MODIFI, FERRARI, Xception, and GridFIT. This paper discusses, in brief, two simulation-based fault injection techniques - Chaos Engineering and Virtual-Machine based Fault Injection.

CAFI: A Configurable location-Aware Fault Injection technique for software reliability assessment against soft errors

Bearing condition monitoring is a widely studied field, but applications to the automotive industry have received little attention as the bearing failure rates are typically low in traditional internal combustion engine vehicles with 200 – 300k mile lifespans. The rapid advancement of electric and autonomous vehicles enables vehicles with million-mile lifespans. This implies that the reliable life of existing bearing designs is exceeded throughout the vehicle life, which can potentially lead to vehicle failure. To enable the development of a bearing fault detection and prognostics system, healthy and faulty bearing data must be collected, and the ground-truth states of the health of bearings need to be determined for algorithm refinement and validation. This work explores the fault injecting options, and ground-truthing together with their limitations. Two methods based on precision machining and seeded spalling are developed and used to inject inner race faults in a ball bearing. A non-invasive ground-truthing method is proposed to quantify the state of health of the fault injected bearings in which bench test data is collected under various speed and load conditions. The vibration signals from the bench tests are used to calculate the root-square of the area under the acceleration Power Spectral Density curve (known as GRMS) for each speed and load condition. To remove the dependency of the results on load and speed conditions, a speed-load-GRMS plot is generated, and a plane is fitted to the data for each fault level. Next, the volume under the plot is calculated, yielding a single cumulative GRMS value for each fault level. This value is used as the ground-truth health of bearing for each fault level. For the bearing with the faults injected using precision machining fault injection, the obtained ground-truth values are 1.56, 3.68, and 4.36 times larger than the same figure for the healthy bearing for the faults with the widths of 0.1 mm, 0.5 mm, and 2 mm, respectively. The observed correlation between the fault sizes and the calculated ground-truth values validates the proposed method which can provide a good separation among different health states of a bearing.

Regard to the problem of present testability design develops slowly. The paper makes the fault injection key research projects. It presents a program to test software security by using software fault test. The method solves problems of the fault injection technology aspect of testing software security.

Software fault injection is a process that discovers how badly software can behave after its state gets corrupted. Fault injection results are sometimes viewed suspiciously since state corruption is hypothetically-based. In response to this suspicion, this paper explores the potential return-on-investment when artificial state corruptions are used. We will primarily focus on fault injection's unique ability to reveal hazards that were inadvertently overlooked during software requirements and design. To our knowledge, this application of fault injection has never been exploited.

Electronic systems are essential parts in the everyday life, and these systems have grown continuously in their performance and in their complexity. Therefore, the design process for such a system has also become more and more complex and has become a critical challenge. A popular tool for solving the design process of complex systems is SystemC. SystemC is a C++ language class library, which allows the modeling of hardware behavior in a programming language for software. Advantages by the usage of SystemC are e.g. an integrative concept of the design description in a continuous design flow, combined with a fast execution of compiled code during the simulation process. This chapter describes the simulation of faults in electronic systems by the usage of SystemC. It treats especially faults in hardware, which can occur after a successful validation of the design. The simulation of faults is well known in two operation areas. One field is the fault simulation of test pattern for the detection of fabrication faults. The other field is the simulated fault injection that the effects of faults emerging during the operation of the device. Such procedures are important for analysis of electronic system designs for safety critical applications with respect to their dependability under fault conditions. At first, the chapter explains some issues in the context of fault injection and fault simulation. Furthermore, it relates something about SystemC and the possibility to simulate the design. State of the art applications are also presented for this purpose. A prerequisite for a simulation is an appropriate modeling. The modeling of faults is presented in the context of their design level. It is also shown how a simulation with fault models can be implemented by several injection techniques. In addition, the advantages and disadvantages of the injection techniques are explained. Another challenge for a simulation task is the execution in an acceptable amount of time. Approaches are presented, which help to speed up simulations. Some practical simulation environments are shown at the end of the chapter.

Build-in-test of software is designed in embedded system widely. This paper analyses the general architecture of the embedded system, introduces three types of BIT structure pattern, for each pattern, the arithmetic design is described in details with real practice. This paper presents a new s/w BIT approach based on fault injection. Four important phrases: testing requirement analysis, fault injection, testing design and result analysis are described thoroughly in this approach. The ending part is the summery of the paper.

This paper presents an experimental study on the emulation of software faults by fault injection. In a first experiment, a set of real software faults has been compared with faults injected by a SWIFI tool (Xception) to evaluate the accuracy of the injected faults. Results revealed the limitations of Xception (and other SWIFI tools) in the emulation of different classes of software faults (about 44% of the software faults cannot be emulated). The use of field data about real faults was discussed and software metrics were suggested as an alternative to guide the injection process when field data is nor available. In a second experiment, a set of rules for the injection of errors meant to emulate classes of software faults was evaluated. The fault triggers used seem to be the cause for the observed strong impact of the faults in the target system and in the program results. The results also show the influence in the fault emulation of aspects such as code size, complexity of data structures, and recursive versus sequential execution.

Electromagnetic Fault Injection (EMFI) is a powerful class of fault attacks that can break hardware and software security mechanisms. The main advantage of EMFI over the other fault injection methods is being able to create local faults without depackaging the target chip. Therefore, designing low-cost and efficient detection mechanisms to sense EMFI is a challenging task. In this work, we propose a low-cost, fully-digital, and cycle-accurate mechanism to detect EMFI. We also present a framework to integrate the proposed detector into any logic core. We implemented the proposed detection mechanism on an Altera Cyclone-IV FPGA and integrated it with a round-serial hardware implementation of AES-128. The area overhead of this design is 14.6% and 81.5% in the number of LUTs and registers, respectively. We also experimentally demonstrated the efficiency of the implemented design under actual EM fault injection. In our experiments, we applied 20000 EM pulses with different parameters (i.e, timing, location, pulse width, energy). In total, 7% of the applied pulses causes faults in the output of the AES. The proposed mechanism successfully detected all of the injected faults without any false alarms.

A well-known testing method for the safety evaluation and real-time validation of automotive software systems (ASSs) is Fault Injection (FI). In accordance with the ISO 26262 standard, the faults are introduced artificially for the purpose of analyzing the safety properties and verifying the safety mechanisms during the development phase. However, the current FI method and tools have a significant limitation in that they require manual identification of FI attributes, including fault type, location and time. The more complex the system, the more expensive, time-consuming and labour-intensive the process. To address the aforementioned challenge, a novel Large Language Models (LLMs)-assisted fault test cases (TCs) generation approach for utilization during real-time FI tests is proposed in this paper. To this end, considering the representativeness and coverage criteria, the applicability of various LLMs to create fault TCs from the functional safety requirements (FSRs) has been investigated. Through the validation results of LLMs, the superiority of the proposed approach utilizing gpt-4o in comparison to other state-of-the-art models has been demonstrated. Specifically, the proposed approach exhibits high performance in terms of FSRs classification and fault TCs generation with F1-score of 88% and 97.5 %, respectively. To illustrate the proposed approach, the generated fault TCs were executed in real time on a hardware-in-the-loop system, where a high-fidelity automotive system model served as a case study. This novel approach offers a means of optimizing the real-time testing process, thereby reducing costs while simultaneously enhancing the safety properties of complex safety-critical ASSs.

To verify the Built-in-test (BIT) in the avionics system, a MIL_STD_1553B (1553B for short) bus fault injection device under VxWorks was designed and implemented. The fault injection device was the core of the bus fault injection system. A method for 1553B bus fault injection was proposed by analyzing the characteristics of 1553B message transmission. On the basis of the research on the framework of 1553B bus fault injection system, the detailed configuration of the 1553B bus fault injection device had been described, including the implementation on hardware and the design of software. At last, it's proved that the fault-injection device can inject faults into the target system correctly and reliably by simulation test.

Accelerated assessment of fine-grain AVF in NoC using a Multi-Cell Upsets considered fault injection

Software fault injection (SFI) has been used to intentionally cause "failures" in software components and assess their impacts on the entire software system. A key property that SFI should satisfy is the representative ness of injected failures, the failures caused by SFI should be as close as possible to failures in the wild. If injected failures do not represent realistic failures, the measured resilience or tolerance against failures of the investigated system is not trustworthy. To the best of the authors' knowledge, the representative ness of "faults" has been investigated. However, it is an open problem whether the failures caused by injected faults represent realistic failures. In this paper, we report the preliminary results of the investigation on the representative ness of injected failures. To compare injected failures with real failures, we have collected 43,742 real crash logs of Linux from the Red Hat repository, and conducted a fault injection campaign on Linux, using SAFE, a state-of-theatre injector of software faults. In the fault injection campaign, 50,000 faults are injected to the Linux file system and 71,470 runs of a workload are executed. The crash logs generated by SFI are compared with the real Red Hat logs with respect to crash causes, crashed system calls, and crashed modules. Our preliminary results suggest that failures caused by injected faults do not represent real failures, probably because injected faults are not representative enough or because the selected workload is not realistic.