Generalization of the method for constructing GL-models of complex fault-tolerant multiprocessor systems with additional failure conditions

The article is focused on the reliability estimation of fault-tolerant multiprocessor systems with huge number of processors and complex behavior of the systems on its processors failures. A universal method for the reliability estimation of such fault-tolerant multi-processor systems for a given time period is proposed. The method is based on conducting statistical experiments (Monte-Carlo) with models that adequately reflect the behavior of the fault-tolerant multiprocessor systems in the flow of failures. For that purpose, it is suggested using GL-model, which is a graph with special way formed Boolean functions assigned to its edges. The questions of synthesis, minimization and transformation of such models are considered. The article addresses the statistical estimation error (Monte-Carlo Error). The upper bound for calculating the error before conducting statistical experiments is suggested. It is shown that the error could be estimated more precisely using the results of conducted statistical experiments. Correspondent statistical estimator is proposed.

The article addresses the problem of evaluating the computational complexity of basic cascade GL-models used for modeling the behavior of fault-tolerant multiprocessor systems in the flow of failures. The purpose of this work is to reduce the complexity of such models by optimal selection of their parameters. It has been demonstrated that a single system usually corresponds to an entire family of cascade GL-models, differing in cascade depth and parameters, with each having its own computational complexity. To simplify the process of modeling the system behavior under a flow of failures, it is advisable to choose the cascade GL-model configuration with the lowest complexity. However, additional constraints on the model, such as cascade depth limitations, must also be considered. This work applies an empirical-analytical research method. An analysis of computational complexity for cascade GL-models was conducted using specially developed software, which automated model construction for various combinations of parameters. Subsequently, a comparative analysis of the complexity of their edge function expressions was performed to identify dependencies on parameter values. Experimental studies were carried out for fault-tolerant multiprocessor systems with varying numbers of processors and different maximum allowable failure multiplicities (but not exceeding half of the total number of processors in the system). It was shown that cascade GL-models typically have significantly lower computational complexity compared to standard basic GL-models, especially for systems with a small maximum number of allowed failures. However, in cases where the allowed number of failures equals or exceeds half of the processor count, standard models may become less complex. Based on the conducted analysis, practical recommendations for selecting the parameters of cascade GL-models were formulated for the first time. In particular, the lowest complexity is achieved when the fault tolerance coefficient of the auxiliary model is minimized at each cascade level; however, this leads to a maximal cascade depth. If cascade depth is limited, the lowest complexity is achieved by evenly or nearly evenly distributing the fault-tolerance coefficients among auxiliary models. If an even distribution is impossible, it is advisable to place higher-value coefficients at deeper cascade levels. Experimental results demonstrate that the application of the proposed recommendations can significantly reduce the overall complexity of edge function expressions in the cascade GL-model compared to the basic GL-model, with the effectiveness of the approach increasing as the system size grows.

The design space exploration (DSE) of fault-tolerant multiprocessor systems is very complex, as it contains three interacting NP-hard problems: 1) task hardening; 2) task mapping; and 3) task scheduling. In addition, replication-based task hardening can introduce new tasks, called replicas, into the system, enlarging the design space further. As a population-based global optimization algorithm, evolutionary algorithms (EAs) have been widely used to explore this huge design space over the last decade. However, as analyzed in this paper, the search space of previous works is highly redundant, resulting in poor efficiency and scalability. This paper proposes an efficient EA-based DSE method for the design of large-scale fault-tolerant multiprocessor systems. The main novelties of this paper include: 1) mapping exploration is explicitly separated, i.e., task mapping is optimized during the evolutionary search, while replica mapping is constructed heuristically according to the current co-synthesis state; 2) the design space of task hardening and task mapping are explored independently by a cooperative co-EA; and 3) as a complement to global search of EA, problem-specific local search operators are designed for both task hardening and task mapping, reducing the number of fitness evaluations required. Compared with the most relevant state-of-the-art method, the superiority of the proposed method is demonstrated using extensive experiments on a large set of benchmarks, e.g., 1.75×~2.50× better results can be obtained on the benchmarks of 300 tasks and 30 processors.

Comparison between failure sequence and failure path for brittle systems

Fault tree analysis of a system can provide a complete characterization of system failure modes, i.e., what combinations of component failures can give rise to system failure. This can be applied to the design process at several levels: (1) confirmatory analysis, in which a fault tree development is used to verify design adequacy, (2) importance analysis, in which fault tree analysis is used to highlight system vulnerabilities, and (3) design optimization, in which fault tree analysis is used to pick the least expensive configuration from a collection of possibilities satisfying a given constraint. Experience shows that the complexity of real systems warrants the systematic and structured development of fault trees for systems whose failure can have severe consequences.

Enhancing the ability to perform diagnostics on a system that has failed can significantly impact maintenance and repair costs. A good diagnostic tool enables a user to analyze a failed system and identify the failed components. While the field of diagnostics is not a modern one, the way in which system diagnostics are performed is continuously changing. The automatic diagnosis based on reliability analysis (ADORA) methodology utilizes reliability information developed during the design phase to build a diagnostic map. Previous work on ADORA demonstrated how a diagnostics procedure can be performed on a system that has been analyzed using a static reliability model, particularly a fault tree (Assaf and Dugan, 2003). In this article, we extend the ADORA methodology to utilize reliability analysis of dynamic fault trees (DFTs), which are reliability models that capture sequences and combinations of component failures that induce system failure. DFTs are particularly well suited for analyzing computer-based systems.As an example the common rail fuel injection system is discussed.

Many emerging embedded real-time applications such as SCADA (Supervisory Control and Data Acquisition), autonomous vehicles and advanced avionics, require a high degree of dependability. Dealing with tasks having both hard real-time requirements and high reliability constraints is a key challenge faced in such systems. This paper addresses the problem of guaranteeing reliability requirements with bounded recovery times on fail-stop processors in fault-tolerant multiprocessor real-time systems. We classify tasks based on their recovery-time requirements into (i) Hard Recovery, (ii) Soft Recovery, and (iii) Best-Effort Recovery tasks. Then, the notion of a Hot Standby for Hard Recovery tasks along with a Cold Standby for Soft Recovery and Best-Effort Recovery tasks is introduced. In order to maximize the benefits of using a Hot Standby, replicas should not be co-located on the same processor. For this purpose, we propose a task allocation algorithm for Hot Standby replicas called R-BFD (Reliable Best-Fit Decreasing) that uses 37% fewer number of processors than BFD-P (Best-Fit Decreasing augmented with placement constraints). For tasks with more relaxed recovery-time constraints, however, additional optimization can be applied by using a Cold Standby that gets activated only when failures occur. Given a system reliability requirement and hence a maximum number of processor failures to tolerate, the required resource overprovisioning for Cold Standby replicas from multiple processors can be consolidated. An algorithm called R-BATCH (Reliable Bin-packing Algorithm for Tasks with Cold standby and Hot standby) reduces the required number of processors by up to 45% compared to R-BFD-based pure Hot Standby replication technique.

At some instances, it is better to terminate operation of a system than to wait for its failure or completion. However, in this article, we are mostly interested in missions that are cost-effective during the whole mission time and, therefore, do not require termination. Moreover, some requirements for parameters of the considered models that guarantee this cost-effectiveness are analyzed. We consider two failure models for systems executing missions of the fixed duration. In the first model, degradation is partially observed via the number of shocks experienced by a system. Shocks act directly on the failure rate forming the shot-noise process. In the second model, degradation is completely observed and is modeled by the Poisson process. Thus, the number of shocks or the number of failed components is the degradation parameters in our models, respectively. The detailed numerical examples illustrate our findings. Specifically, the bounds for the number of events (shocks or component’s failures) observed at each instant of time that guarantee cost-effectiveness of a mission are obtained.

Modern complex systems of responsible purposes are built as fault-tolerant systems, i.e. such systems have the ability to function while some of their separate elements have a fault. This feature is achieved by introduction of different kinds of redundancy (structural, algorithmic, time, etc.), which leads to complications of the algorithm of system internal behavior due to the introduction of the functions of monitoring, fault localization, switching and performance recovery system, resulting in the significant complication of the model of system reliability. The complexity of such fault-tolerant systems makes their reliability of greater dimension (of the order of tens till hundreds of thousand equations), which is almost impossible for their formation and analysis by hand. Therefore, the requires a high level of credibility and an appropriate level of formalization to built models are imposed on these models, which allows to implement the automation of their building and analysis of reliability using modern computer tools. The combination of analytical methods for the research of reliability and computing capability of modern computers is a promising direction for further development of methods of reliability theory. Experience shows that the manual building the state space of fault-tolerant systems even with a small number of elements (up to 10) without the use of software is a time-consuming process. This article describes the procedure of automated development of the reliability models of complex technical systems, implemented in software module ASNA, which developed at the chair Theoretical Radioengineering and Radiomeasurement of Lviv National Polytechnic University

A hybrid design approach of the hierarchical physical implementation design flow is presented and demonstrated on a fault-tolerant low-power multiprocessor system. The proposed flow allows to implement selected submodules in parallel with contrary requirements such as identical placement and individual block implementation. The overall system contains four Leon2 cores and communicates via the Waterbear framework and supports Adaptive Voltage Scaling (AVS) functionality. Three of the processor core variants are derived from the first baseline reference core but implemented individually at block level based on their clock tree specification. The chip is prepared for space applications and designed with triple modular redundancy (TMR) for control parts. The low-power performance is enabled by contemporary power and clock management control. An ASIC is fabricated in a low-power $0.13 \mu \mathrm{m}$ BiCMOS technology process node.

Ensuring the correct behavior of cyber physical systems at run time is of critical importance for their safe deployment. Any malfunctioning of such systems should be detected in a timely manner for further actions. This paper addresses the issue of how quickly a monitor raises an alarm after the occurrence of a failure in cyber physical systems. Towards this end, it introduces a class of systems called exponentially converging monitorable systems. The paper shows that failures in these systems can be detected fast by employing the traditional threshold monitors. It shows that the expected failure detection time for exponentially converging monitorable systems has logarithmic relationship with the inverse of the chosen threshold value. The paper identifies well defined natural classes of these systems. Experimental results are presented that confirm the theoretical results on the relationship between the failure detection time and the chosen threshold values.

Design of finite-time active fault tolerant control system with real-time fault estimation for a remotely operated vehicle

The failure of such systems leads to the catastrophic effects, including injury or death to humans, and harm to the environment. Petri nets (PNs) have been widely used for verification and validation of real-time systems. However, the existing approaches do not consider the critical aspects of reliability and safety that include nonliveness, deadlock, stability, and throughput. In this paper, we introduce these as metrics of reliability and safety for safety critical real-time systems. This paper also proposes an innovative methodology for analysis of nonliveness, deadlock, stability, and throughput metrics by linear programming using PN modeling. The application of the proposed techniques has been validated by applying it on four different safety critical systems, running in six nuclear power plants and shown for reactor protection system.

Model-based Dependability Analysis of Fault-tolerant Inertial Navigation System: A Practical Experience Report

Many dependability analyses are performed using static models, that is, models where time is not an explicit variable. In these models, the system and its components are considered at a fixed point in time, and the word “static” means that the past or future behavior is not relevant for the analysis. Examples of such models are reliability diagrams, or fault trees. The main difficulty when evaluating the dependability of these systems is the combinatorial explosion associated with exact solution techniques. For large and complex models, one may turn to Monte Carlo methods, but these methods have to be modified or adapted in the presence of rare important events, which are commonplace in reliability and dependability systems. This chapter examines a recently proposed method designed to deal with the problem of estimating reliability metrics for highly dependable systems where the failure of the whole system is a rare event. We focus on the robustness properties of estimators. We also propose improvements to the original technique, including its combination with randomized quasi-Monte Carlo, for which we prove that the variance converges at a faster rate (asymptotically) than for standard Monte Carlo.KeywordsPath-based MethodsRandomized quasi-Monte Carlo (RQMC)Reliability DiagramReliability MetricsRQMC MethodsThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.