Software hazard mining

Software fault injection (SFI) has been used to intentionally cause "failures" in software components and assess their impacts on the entire software system. A key property that SFI should satisfy is the representative ness of injected failures, the failures caused by SFI should be as close as possible to failures in the wild. If injected failures do not represent realistic failures, the measured resilience or tolerance against failures of the investigated system is not trustworthy. To the best of the authors' knowledge, the representative ness of "faults" has been investigated. However, it is an open problem whether the failures caused by injected faults represent realistic failures. In this paper, we report the preliminary results of the investigation on the representative ness of injected failures. To compare injected failures with real failures, we have collected 43,742 real crash logs of Linux from the Red Hat repository, and conducted a fault injection campaign on Linux, using SAFE, a state-of-theatre injector of software faults. In the fault injection campaign, 50,000 faults are injected to the Linux file system and 71,470 runs of a workload are executed. The crash logs generated by SFI are compared with the real Red Hat logs with respect to crash causes, crashed system calls, and crashed modules. Our preliminary results suggest that failures caused by injected faults do not represent real failures, probably because injected faults are not representative enough or because the selected workload is not realistic.

This paper presents an experimental study on the emulation of software faults by fault injection. In a first experiment, a set of real software faults has been compared with faults injected by a SWIFI tool (Xception) to evaluate the accuracy of the injected faults. Results revealed the limitations of Xception (and other SWIFI tools) in the emulation of different classes of software faults (about 44% of the software faults cannot be emulated). The use of field data about real faults was discussed and software metrics were suggested as an alternative to guide the injection process when field data is nor available. In a second experiment, a set of rules for the injection of errors meant to emulate classes of software faults was evaluated. The fault triggers used seem to be the cause for the observed strong impact of the faults in the target system and in the program results. The results also show the influence in the fault emulation of aspects such as code size, complexity of data structures, and recursive versus sequential execution.

Firstly, the paper made a contrast between the current domestic and international research condition, and introduced the basic concept of fault injection and distributed system. secondly, it discussed the classification and requirements of fault injection. There are mainly three types of distributed fault, namely the memory fault, CPU fault and correspondence fault. Besides, it discussed the method of distributed software fault injection about DOCTOR and illustrated the comprehensive structure and its respective parts of DOCTOR in detail. Thirdly, it reached a conclusion about the fault model of distributed system of fault injection and its realization method.

Real-world programs require error handling code to handle various kinds of possible errors. However, these errors just infrequently occur due to special conditions, so error handling code is difficult to test. Coverage-guided fuzzing and software fault injection (SFI) are two common techniques that can test error handling code, but they still have major limitations. Specifically, existing fuzzing approaches generate program inputs guided by code coverage, but many occasional errors (such as insufficient memory) are unrelated to inputs, and code coverage cannot effectively reflect the execution contexts of these errors; existing SFI approaches often inject single or random faults, without exploring fault space or using program feedback. In this paper, we propose a new fuzzing framework named EH-Fuzz, to effectively test error handling code. EH-Fuzz uses a context-sensitive SFI-based fuzzing approach to explore fault space and perform fault injection, guided by a new metric named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">error coverage</i> . We evaluate EH-Fuzz on 9 user-level programs and 6 kernel-level modules, and find 45 new real bugs, 31 of which have been confirmed and fixed. We compare EH-Fuzz to existing fuzzing approaches (including AFL, AFL++, Syzkaller, FIZZER and FIFUZZ), and EH-Fuzz finds many real bugs missed by these approaches with higher testing coverage.

CAFI: A Configurable location-Aware Fault Injection technique for software reliability assessment against soft errors

It is now widely believed that FlexRay communication protocol will become the de-facto standard for distributed safety-critical automotive systems. In this paper, the fault sensitivity of the FlexRay communication controller registers are investigated using transient single bit-flip fault injection. To do this, a FlexRay bus network, composed of four nodes, was modeled. A total of 135,600 transient single bit-flip faults were injected to all 408 accessible single-bit and multiple-bit registers of the communication controller in one node. The results showed that among all 408 accessible registers, 30 registers were immediately affected by the injected faults. The results also showed that 26.2% of injected faults caused at least one error. Based on the fault injection results, the TMR and the Hamming code techniques were applied to the most sensitive parts of the FlexRay protocol. These techniques reduced the fault affection to the registers from 26.2% to 10.3% with only 13% hardware overhead.KeywordsSafety-critical applicationsDistributed embedded systemsFlex- Ray protocolFault injection

This paper presents the dynamic robust single-event upset simulator, which is a novel framework for fault injection on hardware (via onchip debugging) and simulation testbeds (via the Simics® full-system simulator from Wind River Systems). Typically, radiation-hardened processers are used for space computing; however, commercial off-the-shelf processors can provide higher performance and lower costs. Although commercial devices are susceptible to radiation-induced faults, fault-injection testing can be used to qualify these devices for use in space. The de facto standard for fault injection is radiation-beam testing, which is often prohibitively expensive and time consuming. The current methodology provides a means to iteratively decrease design vulnerabilities through rapid fault injection before beam testing. Additionally, the methodology can supplement beam-test results by targeting injections at individual components of interest that are difficult to isolate in beam tests. The current fault-injection mechanisms leverage on-chip debuggers and simulation checkpoints, allowing the framework to target a wide range of system components for injection. The injection capabilities and analysis features of the framework are demonstrated by presenting fault-injection results for an image-processing application on two different processor architectures (PowerPC and ARM®) in both hardware and simulation.

This paper presents DrSEUs (Dynamic robust Single-Event Upset simulator), a novel fault injector that uses the Simics full-system simulator. Fault-injection testing enables the use of commercial off-the-shelf (COTS) processors in space, which are susceptible to radiation-induced faults but are desirable due to the lower cost and higher performance of COTS devices. The de facto standard for fault injection is radiation-beam testing, which is often prohibitively expensive and time-consuming. Our methodology provides a means to iteratively decrease design vulnerabilities through rapid fault injection prior to beam testing. Additionally, our methodology can supplement beam-test results by targeting injections at individual components of interest that are difficult to isolate in beam tests. Our fault-injection mechanism uses simulation checkpoints, allowing DrSEUs to target a wide range of system components for injection. The deterministic nature of Simics checkpoints enables the repeatability of injection results and the monitoring of latent faults propagating through the system. We demonstrate the injection capabilities and analysis features of DrSEUs by presenting fault-injection results for an image-processing application.

This paper introduces fault injection technology. This paper discusses the application of software fault injection technology to the testing of airborne aviation software with high security application requirements. The characteristics, classification and application of software fault injection technology are analyzed. This paper focuses on middle-ware based fault injection technology and its application in high security airborne software testing. System level middle-ware fault injection method and module level middle-ware fault injection method are proposed.

With existing virtualized systems, hypervisor failures lead to overall system failure and the loss of all the work in progress of virtual machines (VMs) running on the system. We introduce ReHype, a mechanism for recovery from hypervisor failures by booting a new instance of the hypervisor while preserving the state of running VMs. VMs are stalled during the hypervisor reboot and resume normal execution once the new hypervisor instance is running. Hypervisor failures can lead to arbitrary state corruption and inconsistencies throughout the system. ReHype deals with the challenge of protecting the recovered hypervisor instance from such corrupted state and resolving inconsistencies between different parts of hypervisor state as well as between the hypervisor and VMs and between the hypervisor and the hardware. We have implemented ReHype for the Xen hypervisor. The implementation was done incrementally, using results from fault injection experiments to identify the sources of dangerous state corruption and inconsistencies. The implementation of ReHype involved only 880 LOC added or modified in Xen. The memory space overhead of ReHype is only 2.1MB for a pristine copy of the hypervisor code and static data plus a small reserved memory area. The fault injection campaigns used to evaluate the effectiveness of ReHype involved a system with multiple VMs running I/O and hypercall-intensive benchmarks. Our experimental results show that the ReHype prototype can successfully recover from over 90% of detected hypervisor failures.

To verify the Built-in-test (BIT) in the avionics system, a MIL_STD_1553B (1553B for short) bus fault injection device under VxWorks was designed and implemented. The fault injection device was the core of the bus fault injection system. A method for 1553B bus fault injection was proposed by analyzing the characteristics of 1553B message transmission. On the basis of the research on the framework of 1553B bus fault injection system, the detailed configuration of the 1553B bus fault injection device had been described, including the implementation on hardware and the design of software. At last, it's proved that the fault-injection device can inject faults into the target system correctly and reliably by simulation test.

Critical systems that integrate software components (e.g., from third-parties) need to address the risk of residual software defects in these components. Software fault injection is an experimental solution to gauge such risk. Many error models have been proposed for emulating faulty components, such as by injecting error codes and exceptions, or by corrupting data with bit-flips, boundary values, and random values. Even if these error models have been able to find breaches in fragile systems, it is unclear whether these errors are in fact representative of software faults. To pursue this open question, we propose a methodology to analyze how software faults in C/C++ software components turn into errors at components' interfaces (interface error propagation), and present an experimental analysis on what, where, and when to inject interface errors. The results point out that the traditional error models, as used so far, do not accurately emulate software faults, but that richer interface errors need to be injected, by: injecting both fail-stop behaviors and data corruptions; targeting larger amounts of corrupted data structures; emulating silent data corruptions not signaled by the component; combining bit-flips, boundary values, and data perturbations.

Autonomous vehicles use Electronic Control Units running complex software to improve passenger comfort and safety. To test safety of in-vehicle electronics, the ISO 26262 standard on functional safety recommends using fault injection during component and system-level design. A Fault Injection Framework (FIF) induces hard-to-trigger hardware and software faults at runtime, enabling analysis of fault propagation effects. The growing number and complexity of diverse interacting components in vehicles demands a versatile FIF at the vehicle level. In this paper, we present a novel retargetable FIF based on debugger interfaces available on many target systems. We validated our FIF in three Hardware-In-the-Loop setups for autonomous driving based on the NXP BlueBox prototyping platform. To trigger a fault injection process, we developed an interactive user interface based on Robot Operating System, which also visualized vehicle system health. Our retargetable debugger-based fault injection mechanism confirmed safety properties and identified safety shortcomings of various automotive systems.

In this paper, we present a simulation platform tailored for signal processing circuits that injects bit flips in order to model soft errors. The platform is based on the ESA Data Systems Division's SEE simulation tool upgraded with new functionalities. In order to show the effectiveness of the platform, a digital filter has been tested.

Software fault injection has recently started showing great promise as a means for measuring the safety of software programs. This paper shows the results from applying one method for implementing software fault injection, data state error corruption, to the Departure from Nucleate Boiling Ratio (DNBR) code, a research code from the Halden Reactor Project, located in Halden Norway.