High availability verification framework for OpenStack based on fault injection

This paper introduces fault injection technology. This paper discusses the application of software fault injection technology to the testing of airborne aviation software with high security application requirements. The characteristics, classification and application of software fault injection technology are analyzed. This paper focuses on middle-ware based fault injection technology and its application in high security airborne software testing. System level middle-ware fault injection method and module level middle-ware fault injection method are proposed.

Large-scale distributed systems are a collection of loosely coupled computers interconnected by a communication network. They are now an integral part of everyday life with the development of large web applications, social networks, peer-to-peer systems, wireless sensor networks and many more. At such a scale, hardware components by themselves are prone to failure. Therefore, one key challenge in designing distributed storage systems is how to tolerate faults. To this end, fault tolerance mechanisms such as replication have been widely used to provide high availability for decades. More recently, many systems start supporting erasure coding for fault tolerance, which is expected to achieve high reliability at a lower storage cost compared to replication. However, the reduced storage overhead comes at the cost of more complicated recovery which hurts performance. In this paper, we study the fault tolerance mechanisms of two representative distributed file systems: HDFS and Ceph. In addition to the traditional replication, both HDFS and Ceph support erasure coding in their latest version. We evaluate the replication and erasure coding implementations in both systems using standard benchmarks and fault injection, and quantitatively measure the performance and storage overhead. Our results demonstrate the trade-offs between replication and erasure coding techniques, and serve as a foundation for building optimal storage systems with high availability as well as high performance.

Today's integrated circuits are increasingly subject to the effects of radiation. To assess the reliability of a digital system and to identify the most critical failure effects, radiation-based experimentation or fault injection campaigns are usually performed, which may be costly and time-expensive. This paper proposes enhancements to current widely employed fault injection techniques, aimed at accelerating the process and lowering the costs of the flow. The employed principles and a new fully automatic fault injection flow are presented. Experimental case studies based on available microprocessor systems demonstrate the effectiveness of the proposed solutions.

Firstly, the paper made a contrast between the current domestic and international research condition, and introduced the basic concept of fault injection and distributed system. secondly, it discussed the classification and requirements of fault injection. There are mainly three types of distributed fault, namely the memory fault, CPU fault and correspondence fault. Besides, it discussed the method of distributed software fault injection about DOCTOR and illustrated the comprehensive structure and its respective parts of DOCTOR in detail. Thirdly, it reached a conclusion about the fault model of distributed system of fault injection and its realization method.

Electronic systems are essential parts in the everyday life, and these systems have grown continuously in their performance and in their complexity. Therefore, the design process for such a system has also become more and more complex and has become a critical challenge. A popular tool for solving the design process of complex systems is SystemC. SystemC is a C++ language class library, which allows the modeling of hardware behavior in a programming language for software. Advantages by the usage of SystemC are e.g. an integrative concept of the design description in a continuous design flow, combined with a fast execution of compiled code during the simulation process. This chapter describes the simulation of faults in electronic systems by the usage of SystemC. It treats especially faults in hardware, which can occur after a successful validation of the design. The simulation of faults is well known in two operation areas. One field is the fault simulation of test pattern for the detection of fabrication faults. The other field is the simulated fault injection that the effects of faults emerging during the operation of the device. Such procedures are important for analysis of electronic system designs for safety critical applications with respect to their dependability under fault conditions. At first, the chapter explains some issues in the context of fault injection and fault simulation. Furthermore, it relates something about SystemC and the possibility to simulate the design. State of the art applications are also presented for this purpose. A prerequisite for a simulation is an appropriate modeling. The modeling of faults is presented in the context of their design level. It is also shown how a simulation with fault models can be implemented by several injection techniques. In addition, the advantages and disadvantages of the injection techniques are explained. Another challenge for a simulation task is the execution in an acceptable amount of time. Approaches are presented, which help to speed up simulations. Some practical simulation environments are shown at the end of the chapter.

Regard to the problem of present testability design develops slowly. The paper makes the fault injection key research projects. It presents a program to test software security by using software fault test. The method solves problems of the fault injection technology aspect of testing software security.

Cloud computing alters the way organizations manage and deploy their IT resource. It provides an organization with scalable, inexpensive, and flexible options. The complexity and dynamic nature of cloud environments pose a challenge to maintaining high availability at all times, especially when the system fails or a disaster arises. The legacy techniques of disaster recovery, fault tolerance, and high availability leave behind much to be desired. These techniques are mostly static, slow to respond, and have a dismal ability to adapt to continuously changing conditions in contemporary cloud systems. Such techniques largely depend on manual configurations and predefined policies; resulting in lots of inefficiencies and increases in the risk of service downtime. This research investigates the way Artificial Intelligence (AI) changes the paradigm on cloud resilience to promote adoption of intelligent systems for guaranteed disaster recovery, better fault-tolerant behavior, or transparent high availability. With machine learning algorithms, AI-based cloud services utilize large data volumes to reveal patterns within system logs, performance metrics, and user behavior data; thereby offering real-time anomaly detection and predictive failure analysis. For example, techniques like predictive analytics help cloud providers predict likely system outages, optimize the resources to be used, and automate failover processes (Xu et al., 2021; Lee & Kumar, 2022). AI-aided disaster recovery techniques employ complex algorithms to produce an adaptive backup mechanism, thereby minimizing loss and reducing restoration time. Fault tolerance in AI cloud systems comes from intelligent error correction, automatic isolation of faults, and self-healing features, i.e. repair of faults without the need for human supervision (Chen et al., 2020). Besides that, AI also contributes to high availability through intelligent load balancing, which ensures that at any given time, resources are optimally distributed throughout the network to sustain continuous service even during peak demand or unanticipated failures (Patel & Zhang, 2023). The approach is a comprehensive review of the various existing literature on the topic, empirical analysis of the current AI-driven cloud solutions available in the market, and case studies for comparison analysis on the different AI systems. The study scenario reveals that AI-driven solutions noticeably reduce downtimes, lead to improved recovery times, and contribute to overall system reliability as compared to traditional methods. However, other areas include model bias, data privacy, and continuous training of AI models. This study expands the trends of AI in the field of cloud computing by documenting the significance of intelligent systems in bridging traditional weaknesses of resilience strategies. It further signifies the need for AI into predictive maintenance, automated disaster response, and proactive fault management of rapidly changing dynamic cloud environments. Future studies will focus on AI integration along with edge computing and blockchain technologies for even more robust and secure services in cloud operations.

Design considerations of local controller for ITER coil power supply with high availability and reliability

In order to improve the reliability of FPGA in space applications, a method of single event upset detection and scrubbing based on internal processor is proposed for Xilinx Zynq device. We propose a method to reduce the sensitive configuration bits and consequently the global scrubbing time based on Xilinx Essential Bits technology. Then the impact on reducing sensitive frames by placing different layout constraints is analyzed. The physical address of one single configuration frame and its corresponding configuration data are obtained based on the configuration memory structure. Simultaneously, Frame_ECC primitive and global CRC circuit is added to the design. Finally, the feasibility of our system is verified by the fault injection, and the time performance is evaluated by Mean Time to Detect (MTTD). The results show that the system can effectively shorten the readback and scrubbing time, single-bit upset and multi-bit upsets can be detected and repaired. Thus, the work is of great significance for improving the reliability of aerospace electronic systems.

Reliability of CTCS-3 train operation and control system is necessary to ensure train running safely and efficiently. Thus, train control system simulation should be used for lots of tests and validations. This paper is focus on the theory and period of fault injection systematically. According to characters of simulation of train control system and advantages of fault injection, application of fault injection in the field of train control system simulation is proposed for reliability test and to obtain system information after fault injection. Main structure and detailed functions, structures of fault injection system is designed and realized. This system could simulate system fault to cause rapid failure in train control system during the experiment. By analyzing the simulation result, the conclusion could be obtained that application of fault injection could help to improve reliability and fault tolerance of C3 simulation.

Safety-critical applications are frequently based on deep learning algorithms. In particular, Convolutional Neural Networks (CNNs) are commonly deployed in autonomous driving applications to fulfil complex tasks such as object recognition and image classification. Ensuring the reliability of CNNs is thus becoming an urgent requirement since they constantly behave in human environments. A common and recent trend is to replace the full-precision CNNs to make way for more optimized models exploiting approximation paradigms such as reduced bit-width data type. If from one hand this is poised to become a sound solution for reducing the memory footprint as well as the computing requirements, it may negatively affect the CNNs resilience. The intent of this work is to assess the reliability of a CNN-based system when reduced bit-widths are used for the network parameters (i.e., synaptic weights). The approach evaluates the impact of permanent faults in CNNs by adopting several bit-width schemes and data types, i.e., floating-point and fixed-point. This determines the trade-off between the CNN accuracy and the bits required to represent network weights. The characterization is performed through a fault injection environment built on the darknet open source framework. Experimental results show the effects of permanent fault injections on the weights of LeNet-5 CNN.

DC–DC converters are important components that affect the reliability of power electronic systems. Predicting the health status of converters helps to obtain performance degradation information in advance before shutdown failures occur in the system, and formulate reasonable maintenance strategies accordingly, improving the reliability and safety of the system. This paper proposes a data-driven method combining Health indicator (HI) extraction and deep learning technology. Firstly, the output voltage of the converter is taken as the health status sensitive information, and the permutation entropy is extracted as the HI. Secondly, the Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise method is adopted to decompose the HI and extract the monotonic degradation trend of the health status. Finally, the bidirectional gated recurrent unit method with the introduced attention mechanism is used to conduct single-step predictions on each intrinsic mode function component and the trend term. The results of each predicted component are reconstructed to realize circuit-level health status prediction. Experiments are carried out using the High-temperature degradation experimental data set with the Boost converter as the research object for verification. The results show that compared with traditional methods, the prediction error of the proposed method is reduced by 53.16%, demonstrating higher accuracy and reliability.

Meso-level Comparison of Mental Health Service Availability and Use in Chile and Spain

Performance and fault tolerance are two major issues that need to be addressed while designing highly available and reliable systems. The network topology or the notion of connectedness among the network nodes defines the system communication architecture and is an important design consideration for fault tolerant systems. A number of fault tolerant designs for specific multi-processor architecture exists in the literature, but none of them discriminates between stateless and stateful failover. In this paper, we propose a reliable network topology and a high availability framework which is tolerant up to a maximum of k node faults in a network and is designed specifically to meet the needs of stateful failover. Assuming the nodes in the network are capable of handling multiple processes, through our design we have been able to prove that in the event of k node failures the load can be uniformly distributed across the network - ensuring load balance. We also provide an useful characterization for the network, which under the proposed framework ensures one hop communication between the required nodes

Setting up High Availability and Disaster Recovery for SAP HANA presents critical strategies for ensuring business continuity in enterprise environments. The implementation incorporates sophisticated architectural components for maintaining system operations through multi-layered redundancy and protection mechanisms. SAP HANA deployments leverage scale-out systems with coordinator and worker nodes, while system replication provides robust disaster recovery capabilities. Pacemaker cluster management enables automated failover and resource orchestration, with comprehensive testing frameworks ensuring system reliability. The architecture supports advanced features including active/active replication and tenant-level management, allowing organizations to achieve optimal resource utilization and granular control over high availability settings. These implementations demonstrate significant improvements in system performance, data consistency, and recovery capabilities, resulting in substantial reductions in downtime costs and data loss risks. The combination of strategic redundancy, proactive monitoring, and sophisticated replication mechanisms enables organizations to maintain continuous operations while meeting stringent service level agreements in mission-critical environments.