A Report on Simulation-Driven Reliability and Failure Analysis of Large-Scale Storage Systems

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

Redundant array of independent disks (RAID) has been widely used to address the reliability and performance issues of storage systems. As the scale of modern storage systems continues growing, disk failure becomes the norm. With the ever-increasing disk capacity, RAID recovery based on disk rebuild becomes more and more costly, which causes significant performance degradation and even unavailability of storage systems. Declustered data layout enables parallel RAID reconstruction by shuffling data and parity blocks among all drives (including spares) in a RAID group. However, the reliability and performance of declustered RAID in real-world storage environments have not been thoroughly studied. With the popularity of ZFS file system and software RAID used in production data centers, in this paper, we extensively evaluate declustered RAID with regard to the RAID recovery time and I/O performance on a high-performance storage platform at Los Alamos National Laboratory. Our empirical study reveals the advantages and disadvantages of declustered RAID technology. We qualitatively characterize the recovery performance of declustered RAID and compare with that of ZFS RAIDZ under various I/O workloads and access patterns. The experimental results show that the speedup of declustered RAID over traditional RAID is sub-linear to the parallelism of recovery I/O. Furthermore, we formally model and analyze the reliability of declustered RAID in terms of the mean-time-to-data-loss (MTTDL) and discover that the improved recovery performance leads to higher storage reliability compared with the traditional RAID.

Today's data storage systems are increasingly adopting low-cost disk drives that have higher capacity but lower reliability, leading to more frequent rebuilds and to a higher risk of unrecoverable media errors. We propose a new XOR-based intra-disk redundancy scheme, called interleaved parity check (IPC), to enhance the reliability of RAID systems that incurs only negligible I/O performance degradation. The proposed scheme introduces an additional level of redundancy inside each disk, on top of the RAID redundancy across multiple disks. The RAID parity provides protection against disk failures, while the proposed scheme aims to protect against media-related unrecoverable errors.We develop a new model capturing the effect of correlated unrecoverable sector errors and subsequently use it to analyze the proposed scheme as well as the traditional redundancy schemes based on Reed-Solomon (RS) codes and single-parity-check (SPC) codes. We derive closed-form expressions for the mean time to data loss (MTTDL) of RAID 5 and RAID 6 systems in the presence of unrecoverable errors and disk failures. We then combine these results for a comprehensive characterization of the reliability of RAID systems that incorporate the proposed IPC redundancy scheme. Our results show that in the practical case of correlated errors, the proposed scheme provides the same reliability as the optimum albeit more complex RS coding scheme. Finally, the throughput performance of incorporating the intra-disk redundancy on various RAID systems is evaluated by means of event-driven simulations. A detailed description of these contributions is given in [1].

With the rapid progress of the capacity and slow pace of the speed/MTTF of hard disks, and increasing size of storage systems, the reliability and availability of storage systems become more and more serious. This paper discusses the method of constructing double- and triple-erasure-correcting codes via combining mirroring and parity approaches in details, and presents a double-erasure code MPDC and a triple-erasure code MPPDC based on one-factorizations of complete graphs. The two codes are simple, easy to implement, and have no disk number limitation. They achieve perfect fault-free load balance and approximately optimal reconstruction load balance. The simulation results show that, compared with other double- and triple-erasure codes, MPDC and MPPDC have comparative light-load and moderate-load performance and better heavy-load performance in fault-free mode. Because parity declustering is used, the two codes are far superior to the other double- and triple-erasure codes in degraded- and reconstruction-mode performance.

The High voltage static converter systems are deployed at different military and DoD assets in which power electronics is used for the isolated and stabilized AC or DC voltage/frequency conversion process and the Insulated Gate Bipolar Transistor (IGBT) triggering system plays crucial role in such systems. Such type of systems and equipments are being operated by the combat deployed personnel and they rely upon in order to complete their critical missions. Therefore, it is evident that reliability and availability during the mission and maintainability of the system before and after the mission is paramount. The objective of this paper is to present the predicted Reliability, Maintainability, Availability and Failure rate analysis of the IGBT triggering system. The analysis is made in the marine environment by defining specific parameters. The method used to calculate the reliability prediction of the IGBT triggering system is based on the calculation of Mean Time Between Failure (MTBF), which is obtained from the calculation of Failure rate of individual components used in the system. The maintainability analysis is done based on the length of the time required by the repairable components, repair through replaceable components or replacing the whole system. Therefore, this analysis will use the Mean Time To Repair (MTTR). MTBF and MTTR are used as key elements in the availability analysis of the system. Pareto analysis is presented in order to show the individual components failure rate contribution used in the system. The presented analysis shows the relationship between reliability, maintainability and availability of a system and also shows how the reliability and maintainability analysis improves the availability of the designed system. The presented analysis can be viewed with reference to system lifetime calculation as it will help in investment plan of designed system in marine environment.

In this paper, we consider two rack-aware storage systems. First, large-scale rack-aware storage system, which is very common in large-scale storage system, is a rack-aware storage system where all sizes of racks are at least the number of redundant nodes. For such storage system, we prove that any Maximum Distance Separable (MDS) codes can have optimal inter-rack repair bandwidth and give a closed-form representation of all repair schemes with optimal inter-rack repair bandwidth. Furthermore, we show that the optimal repair access and optimal inter-rack repair bandwidth can be attained simultaneously for such storage system. Second, we investigate the rack-aware storage system of all racks with the same size, which is called uniform rack-aware storage system. We prove that, except the trivial cases, we cannot attain optimal inter-rack repair bandwidth and optimal repair access for such storage system at the same time. Specifically, we establish the lower bound of repair access for a repair scheme with optimal interrack repair bandwidth, which is tight for some parameters, and also the tight lower bound of inter-rack repair bandwidth for a repair scheme with optimal repair access.

We have developed a distributed parallel storage system that employs the aggregate bandwidth of multiple data servers connected by a high-speed wide-area network to achieve scalability and high data throughput. This paper studies different schemes to enhance the reliability and availability of such network-based distributed storage systems. The general approach of this paper employs erasure error-correcting codes that can be used to reconstruct missing information caused by hardware, software, or human faults. The paper describes the approach and develops optimized algorithms for the encoding and decoding operations. Moreover, the paper presents techniques for reducing the communication and computation overhead incurred while reconstructing missing data from the redundant information. These techniques include clustering, multidimensional coding, and the full two-dimensional parity schemes. The paper considers trade-offs between redundancy, fault tolerance, and complexity of error recovery.

The need for reliable and fast storage systems is increasingly critical in various fields including artificial intelligence and data analytics. This paper proposes a new architecture for large-scale data storage systems, focusing on comparing performance of software and hardware storage technologies that effectively reduce the computational latency and improve performance. The main contributions include: (1) the combination of Singled Magnetic Recording (SMR) for storing data and Solid State Devices (SSD) for storing metadata is a viable solution for implementing large data storage systems; and (2) the combination of Conventional Magnetic Recording (CMR) for storing data and SSD for storing metadata shows the highest performance for high performance computing. Our experiments are carried out in multiple settings, demonstrating that the proposed architecture successfully improves performance for sequential and random reads/writes. The prototypes are evaluated with a set of workloads, showing the superiority of the proposed data storage configurations. This work provides new opportunities for efficiently processing and storing data and metadata in large-scale data analysis systems.

It is critical to reduce the electric energy consumption of information systems to realize green societies. Applications like database and web applications take usage of data in storages systems of servers. In this paper, we consider RAID storage systems which are composed of multiple drives like hard disk drives (HDDs) and solid state drives (SSDs). Types of RAID storage systems, RAID0, RAID10(1+0), and RAID5 are considered in this paper. The performance and reliability of RAID storage systems are so far studied by many researchers. The more number of storage drives are possibly in parallel accessed in the RAID storage systems, the more amount of electric energy is consumed while the higher reliability and availability are supported. The electric energy consumption of the RAID storage systems to read and write data is so far not discussed. In this paper, we measure the power consumption of RAID storage systems and time to read and write data in the storage systems in order to make a power consumption model of a storage system. We make clear how much energy each type of RAID storage system consumes to sequentially and randomly read and write data through experiment in this paper.

Today organizations and business enterprises of all sizes need to deal with unprecedented amounts of digital information, creating challenging demands for mass storage and on-demand storage services. The current trend of clustered scale-out storage systems use symmetric active replication based clustering middleware to provide continuous availability and high throughput. Such architectures provide significant gains in terms of cost, scalability and performance of mass storage and storage services. However, a fundamental limitation of such an architecture is its vulnerability to application-induced massive dependent failures of the clustering middleware. In this paper, we propose hierarchical middleware architectures that improve availability and reliability in scale-out storage systems while continuing to deliver the cost and performance advantages and a single system image (SSI). Hierarchical middleware architectures organize critical cluster management services into an overlay network that provides application fault isolation and eliminates symmetric clustering middleware as a single-point-of-failure. We present an in-depth evaluation of hierarchical middlewares based on an industry-strength storage system. Our results show that hierarchical architectures can significantly improve availability and reliability of scale-out storage clusters.

The development of stationary battery storage systems in Germany – status 2020

The development of stationary battery storage systems in Germany – A market review

With the continuous development of Internet technology, cloud storage has been widely applied and developed. Users can store data through the network anytime, anywhere. However, the frequent occurrence of a series of cloud storage data security events show that cloud storage not only provides data storage services for users, but also poses huge threats and challenges to users' data security. Aiming at the problem of user data security in cloud storage environment, the system introduces multi-cloud storage system on the basis of the traditional single cloud storage system. The system uses Erasure Code to block the original data, and encrypts the data blocks using AES (Advanced Encryption Standard) encryption technology, then stores the encrypted data blocks in different cloud storages terminals. Thus, the hidden danger of data loss or leakage due to relying on a cloud service provider is eliminated, the existing security problems of cloud storage system are improved, and the security and reliability of the multi-cloud secure storage system are improved.

Energy storage systems are an important asset in the future power grid, ensuring the safety and reliability in face of growing power consumption and intermittent supply. Occupying a vital role, the reliability of the storage system itself has to be guaranteed and taken into account for, especially in the design procedure. This study utilizes reliability methods to analyze the storage's system reliability and optimize the system size to an appropriate level. A residential building with a 3.3kW photovoltaic system serves as a case study to analyze different energy storage types and their resulting optimum system structure. The paper also includes an economic evaluation to measure the storage's suitability for the particular case with the photovoltaic system.

Hydrogen produced by electrolysis is one of the promising means by which to store electricity coming from renewables and to address its intermittency. With the growing global electricity share from renewable energy sources, large-scale storage systems are required. Alkaline electrolysis can be a sustainable solution for the large-scale energy storage system. However, high overpotential and slow kinetics particularly of the oxygen evolution reaction (OER) still remains as a hurdle to limit efficiency of the system. To enhance efficiency of alkaline electrolysis cells, catalysts which have high electrocatalytic activity for OER should be developed. Platinum-like behavior of Tungsten carbide in surface catalysis was first reported by Levy and Boudart in 1973[1]. In addition, tungsten carbide was reported to have Pt-like electronic structure by Colton in 1975[2]. After that, tungsten carbide has been studied widely and expected to substitute noble metal catalysts which are used in many electrochemical reactions. Here, we studied tungsten carbide as a support for 3d-transition metals and make bi-metallic surfaces for developing a high activity OER catalyst. First, WC-Co was synthesized by heat treatment. Carbon nitride (g-C3N4) and tungsten chloride were used as a carbide precursor and cobalt acetate was also used for helping carbonization of tungsten. After heat treatment, acidic leaching process was followed to eliminate cobalt. Through additional mixing with Nickel precursor and heat treatment, Ni/WC was synthesized and to compare electrocatalytic activity Ni/C was also prepared. In addition, many different Ni/WC which have different Ni to WC ratios were synthesized and measured in electrochemical way. Finally, the optimal ratio was determined and it show better electrocatalytic activity than precious catalysts and any other non-noble catalysts. Cyclic voltammetry was measured to evaluate OER activity in 1 M KOH in potential range from 1 to 1.8 V. Figure 1 shows that pure WC and Ni/C have 394 mV and 413 mV overpotentials at 10 mA∙cm-2 respectively. In Metal/WC cases, Cobalt/WC shows only 417 mV. But Nickel/WC shows 340 mV overpotential at 10 mA∙cm-2 and higher electrocatalytic activity for OER than RuO2 in alkaline media. η @ 10 mA cm-2 (mV) Ni/C 394 WC 413 Co/WC 417 Ni/WC 340 RuO2 355 Table1. Activities for OER at 10 mA cm-2 References [1] R. B. Levy, M. Boudart, Science, New series, Vol. 181, No. 4099 (1973) 547-579 [2] Richard J. Colton, Jan-Tsyu j. Huang, J.Wayne Rabalais, Chemical Physics Letters, Vol. 34, No. 2 (1975) 337-339 Figure1. Cyclic voltammetry for the oxygen evolution reaction in 1 M KOH of Ni/C, WC, Co/WC, Ni/WC, RuO2 Figure 1