Non-intrusive Fault Injection Techniques for Efficient Soft Error Vulnerability Analysis

Electronic computing systems are integrating modern multicore processors and GPUs aiming to perform complex software stacks in different life-critical systems, including health devices and emerging self-driving cars. Such systems are expected to experience at least one soft error per day in the near future, which may lead to life-threatening failures. To prevent these failures, critical system must be tested and verified while under realistic workloads. This paper presents four novel non-intrusive fault injection techniques that enable full fault injection control and inspection of multicore systems behavior in the presence of faults. Proposed techniques were integrated into a fault injection framework and verified through a real automotive case study with up to 43 billions instructions. Results show that compared to traditional methods, the new techniques can increase the efficiency of fault injection campaigns during early development phase by 32.28%.

Autonomous vehicles use Electronic Control Units running complex software to improve passenger comfort and safety. To test safety of in-vehicle electronics, the ISO 26262 standard on functional safety recommends using fault injection during component and system-level design. A Fault Injection Framework (FIF) induces hard-to-trigger hardware and software faults at runtime, enabling analysis of fault propagation effects. The growing number and complexity of diverse interacting components in vehicles demands a versatile FIF at the vehicle level. In this paper, we present a novel retargetable FIF based on debugger interfaces available on many target systems. We validated our FIF in three Hardware-In-the-Loop setups for autonomous driving based on the NXP BlueBox prototyping platform. To trigger a fault injection process, we developed an interactive user interface based on Robot Operating System, which also visualized vehicle system health. Our retargetable debugger-based fault injection mechanism confirmed safety properties and identified safety shortcomings of various automotive systems.

Aging of integrated circuits is recognized as a major reliability issue and the estimation of its effects is mandatory in the design of safety-critical systems. Fault injection is a valid method to evaluate the effects of faults on systems. This paper presents a fault injection framework to evaluate the effects of aging on the paths of circuits. Among the main aging mechanisms affecting circuits, the focus here is on oxide breakdown. The paper proposes a fault model that combines accuracy for the evaluation of the effects and rapidity for the execution of large fault injection campaigns. The proposed fault model and fault injection framework have been evaluated on two multi-core architectures implemented on two different technologies.

Soft errors are one-time events that corrupt the state of a computing system but not its overall functionality. Large supercomputers are especially susceptible to soft errors because of their large number of components. Soft errors can generally be detected offline through the comparison of the final computation results of two duplicated computations, but this approach often introduces significant overhead. This paper presents Online-ABFT, a simple but efficient online soft error detection technique that can detect soft errors in the widely used Krylov subspace iterative methods in the middle of the program execution so that the computation efficiency can be improved through the termination of the corrupted computation in a timely manner soon after a soft error occurs. Based on a simple verification of orthogonality and residual, Online-ABFT is easy to implement and highly efficient. Experimental results demonstrate that, when this online error detection approach is used together with checkpointing, it improves the time to obtain correct results by up to several orders of magnitude over the traditional offline approach.

CAFI: A Configurable location-Aware Fault Injection technique for software reliability assessment against soft errors

Fault injection in early microarchitecture-level simulation CPU models and beam experiments on the final physical CPU chip are two established methodologies to access the soft error reliability of a microprocessor at different stages of its design flow. Beam experiments, on one hand, estimate the devices expected soft error rate in realistic physical conditions by exposing it to accelerated particles fluxes. Fault injection in microarchitectural models of the processor, on the other hand, provides deep insights on faults propagation through the entire system stack, including the operating system. Combining beam experiments and fault injection data can deliver deep insights about the devices expected reliability when deployed in the field. However, it is yet largely unclear if the fault injection error rates can be compared to those reported by beam experiments and how this comparison can lead to informed soft error protection decisions in early stages of the system design.

Accelerated assessment of fine-grain AVF in NoC using a Multi-Cell Upsets considered fault injection

Due to voltage and structure shrinking, the influence of radiation on a circuit's operation increases, resulting in future hardware designs exhibiting much higher rates of soft errors. Software developers have to cope with these effects to ensure functional safety. However, software-based hardware fault tolerance is a holistic property that is tricky to achieve in practice, potentially impaired by every single design decision. We present FAIL*, an open and versatile architecture-level fault-injection (FI) framework for the continuous assessment and quantification of fault tolerance in an iterative software development process. FAIL* supplies the developer with reusable and composable FI campaigns, advanced pre-and post-processing analyses to easily identify sensitive spots in the software, well-abstracted back-end implementations for several hardware and simulator platforms, and scalability of FI campaigns by providing massive parallelization. We describe FAIL*, its application to the development process of safety-critical software, and the lessons learned from a real-world example.

To ensure resilient neural network processing on even unreliable hardware, comprehensive reliability analysis against various hardware faults is generally required before the neural network models are deployed, and efficient fault injection tools are highly demanded. However, many existing fault injection tools remain limited to basic fault injection and fail to provide fine-grained vulnerability analysis capability. In addition, many of the fault injection tools also need to change the neural network models and make the fault injection closely coupled with normal neural network processing, which complicates the use of these tools and slows down the fault simulation. The various fault injection implementations and error metrics make the comparison between different fault-tolerant studies difficult. To this end, we propose MRFI, a highly configurable multiresolution fault injection tool for deep neural networks. It enables users to modify an independent fault configuration file rather than neural network models for fault injection and vulnerability analysis. Particularly, it integrates extensive fault analysis functionalities from different perspectives and enables multiresolution investigation of the vulnerability of neural networks. In addition, it does not modify the major neural network computing framework of PyTorch. Hence, it allows parallel processing on GPUs naturally and exhibits fast fault simulation according to our experiments. Moreover, we also have the fault injection calibrated with fault simulation with architectural details and validate the accuracy of the proposed fault injection. Finally, MRFI is also open-sourced on GitHub (MRFI https://github.com/fffasttime/MRFI).

Soft error resilience has become an essential design metric in electronic computing systems as advanced technology nodes have become less robust to high‐charged particle effects. Designers, therefore, should be able to assess this metric considering several software stack components running on top of commercial processors, early in the design phase. With this in mind, researchers are using virtual platform (VP) frameworks to assess this metric due to their flexibility and high simulation performance. In this regard, herein, this goal is achieved by analysing the soft error consistency of a just‐in‐time fault injection simulator (OVPsim‐FIM) against fault injection campaigns conducted with event‐driven simulators (i.e. more realistic and accurate platforms) considering single and multicore processor architectures. Reference single‐core fault injection campaigns are performed on RTL descriptions of Arm Cortex‐M0 and M3 processors, while gem5 simulator is used to multicore Arm Cortex‐A9 scenarios. Campaigns consider different open‐source and commercial compilers as well as real software stacks including FreeRTOS/Linux kernels and 52 applications. Results show that OVPsim‐FIM is more than 1000× faster than cycle‐accurate simulators and up to 312× faster than event‐driven simulators, while preserving the soft error analysis accuracy (i.e. mismatch below to 10%) for single and multicore processors.

With the decreasing size and voltage level of internal device components, soft errors are increasing and constitute a major threat on electronic devices. Silent data corruption (SDC) is the most dangerous result type of soft errors as there is no indication that an error occurs during one program execution. Identifying the SDC vulnerability of instructions is the premise for applying selective SDC detection techniques to programs. We propose proPVInsiden to predict the SDC vulnerability of instructions with a lower cost of fault injection and a better adaptability for programs and program inputs. Reducing the number of fault injections will impair the performance of the prediction. Partial fault injection is applied to control the downward slope of the performance of the prediction to maximize the reduction of fault injection. Experimental results show that the number of fault injections is reduced by 55% and 45% fault injections are sufficient to predict the relative SDC vulnerability of an instruction with respect to other instructions. The averaged Spearman's rank correlation coefficient is 0.81. proPVInsiden also shows a better applicability for programs and program inputs.

For accurately assessing the reliability of space application toward soft errors, this paper proposed a method of Single Event Effect simulation method for Large Scale Integrated Circuits (LSI). This simulation method divided into three levels: fault model, fault injection and target system simulation, which consists of a variety of simulation tools, such as TCAD, SPICE circuit simulator and analyzing tools of gate-level netlist. A 4-stage in-order RISC-V soft processor is used as the simulation target system. The simulation method has the following advantages: 1. It is a universal method of testing any large-scale integrated circuit including its peripheral components. 2. By input the gate-level net-list of the circuit and add the simulation stimulus, the method can get the simulation report automatically. 3. The soft error can be traced: The faulty node and the voltage waveform of the node can be reported intuitively.

Soft error susceptibility analysis methodology of HLS designs in SRAM-based FPGAs

Designers of safety-critical VLSI systems are asking for effective tools for evaluating and validating their designs. Fault Injection is commonly adopted for this task, and its effectiveness is therefore a key factor. In this paper we propose to exploit FPGAs to speed-up Fault Injection for fault tolerance evaluation of VLSI circuits. A complete Fault Injection environment is described, relying on FPGA-based emulation of the circuit for fault effect analysis. The proposed approach allows combining the efficiency of hardware-based techniques, and the flexibility of simulation-based techniques. Experimental results are provided to support the feasibility and effectiveness of the approach.

In modern processor based systems, techniques of evaluating the soft error effect on register files (RF) is very important because it makes fine grained protection viable. This paper describes GBMW (Graph Based Mask Window analysis), a framework that can capture soft error masking effects accurately in register files at instruction level. Comparing with previous studies, GBMW not only considers logical masking effects, but also takes error propagation effects and error post-masking effects into account. We verify our method by using fault inject (FI) as a baseline and comparing our method with traditional ones. The results show that GBMW decreases the average error rate to 1/3 of those using traditional methods when predicting soft error masking effects. Furthermore, GBMW still has comparable running time.