The evolution of storage systems

Redundant array of independent disks (RAID) has been widely used to address the reliability and performance issues of storage systems. As the scale of modern storage systems continues growing, disk failure becomes the norm. With the ever-increasing disk capacity, RAID recovery based on disk rebuild becomes more and more costly, which causes significant performance degradation and even unavailability of storage systems. Declustered data layout enables parallel RAID reconstruction by shuffling data and parity blocks among all drives (including spares) in a RAID group. However, the reliability and performance of declustered RAID in real-world storage environments have not been thoroughly studied. With the popularity of ZFS file system and software RAID used in production data centers, in this paper, we extensively evaluate declustered RAID with regard to the RAID recovery time and I/O performance on a high-performance storage platform at Los Alamos National Laboratory. Our empirical study reveals the advantages and disadvantages of declustered RAID technology. We qualitatively characterize the recovery performance of declustered RAID and compare with that of ZFS RAIDZ under various I/O workloads and access patterns. The experimental results show that the speedup of declustered RAID over traditional RAID is sub-linear to the parallelism of recovery I/O. Furthermore, we formally model and analyze the reliability of declustered RAID in terms of the mean-time-to-data-loss (MTTDL) and discover that the improved recovery performance leads to higher storage reliability compared with the traditional RAID.

Traditionally, network storage systems have mainly been dominated by two IP-based storage technologies, i.e., Network Attached Storage (NAS) and Storage Area Network (SAN). In recent years, cloud based storage (e.g., Amazon S3) has gained growing popularity for its high flexibility and cross-platform compatibility. Many enterprises are considering to replace traditional storage systems with cloud-based systems. Evaluating such a transition demands a systematic study on understanding the performance behaviours of the emerging cloud storage. To fill in this gap, in this paper, we conduct a comprehensive study on the three storage systems with realistic network conditions. Specifically, we select one representative from each category for comparison, i.e., Network File System (NFS) from NAS, Internet Small Computer System Interface (iSCSI) from SAN, and OpenStack Swift from cloud storage. We build a testbed and develop a suite of micro-benchmarks to study the impact of network complexities. Through a set of experiments and detailed analysis, we make several key observations: (1) iSCSI excels under good network conditions, e.g., in local area networks (LANs) where network delay and packet loss are trivial, (2) NFS and Swift are more suitable for complex networks such as wireless networks and Internet environment, (3) Swift is a viable replacement for NFS in all scenarios we investigate, and (4) system configuration on the client side impacts storage performance significantly and deserves adequate attention. We hope our findings can not only shed light on storage service design and optimizations, but also encourage more research on emerging storage technologies.

This paper studies how RAID (redundant array of independent disks) could take full advantage of modern SSDs (solid-state drives) with built-in transparent compression. In current practice, RAID users are forced to choose a specific RAID level (e.g., RAID 10 or RAID 5) with a fixed storage cost vs. speed performance trade-off. The commercial market is witnessing the emergence of a new family of SSDs that can internally perform hardware-based lossless compression on each 4KB LBA (logical block address) block, transparent to host OS and user applications. Beyond straightforwardly reducing the RAID storage cost, such modern SSDs make it possible to relieve RAID users from being locked into a fixed storage cost vs. speed performance trade-off. In particular, RAID systems could opportunistically leverage higher-than-expected runtime user data compressibility to enable dynamic RAID level conversion to improve the speed performance without compromising the effective storage capacity. This paper presents techniques to enable and optimize the practical implementation of such elastic RAID systems. We implemented a Linux software-based elastic RAID prototype that supports dynamic conversion between RAID 5 and RAID 10. Compared with a baseline software-based RAID 5, under sufficient runtime data compressibility that enables the conversion from RAID 5 to RAID 10 over 60% of user data, the elastic RAID could improve the 4KB random write IOPS (I/O per second) by 42% and 4KB random read IOPS in degraded mode by 46%, while maintaining the same effective storage capacity.

Because of the ever increasing amount of information needing to be managed, the development of storage technology has become very important. RAID (redundant array of independent disks) is usually used for administering large storage, although traditional RAIDs have only one or two fault tolerances, which is not sufficient for constructing large storage. Although hierarchical RAID (HiRAID) has three fault tolerances, efficiency in capacity is low. In the previous work, we proposed N-ary RAID, which is flat (not hierarchical) RAID with multiple parities based on a block unit. N-ary RAID is a 3 fault tolerant RAID. In this paper, we propose N-ary RAID MP, which is combined N-ary RAID to RAID MP (multiple parity). It drastically increases the reliability of RAID based storage.

This paper is a summary and evaluation of redundant array of independent disks (also called redundant array of inexpensive disks before) that was proposed years ago. Recently more information is produced every year, and the demand of data storage is increasing exponentially due to Moore's Law. So, it's much more common to group a dozen of hard disk drives together, to achieve better performance or to achieve better data safety. Firstly, the redundant array of independent disks aims to solve the data safety/data in-out performance problems, and multiple ways are proposed including Level 0 to solve the performance problem sacrificing system stability and data safety and Level 1 aims to achieve better stability but using only 50% (or less) of the data capacity. The development of redundant array of independent disks tends to be merged and have more functions in one single system of redundant array of inexpensive disks.

Redundant Array of Independent Disks (RAID) storage architectures provide protection of digital infrastructure against potential disks failures. For example, RAID-5 and RAID-6 architectures provide protection against one and two disk failures, respectively. Recently, the data generation has significantly increased due to the emergence of new technologies. Thus, the size of storage systems is also growing rapidly to accommodate such large data sizes, which increases the probability for disks failures. This necessitates a new RAID architecture that can tolerate up to three disk failures. RAID architectures implement coding techniques. The code specifies how data is stored among multiple disks and how lost data can be recovered from surviving disks. This abstract introduces a novel coding scheme for new RAID-7 architectures that can tolerate up to three disks failures. The code is an improved version of the existing BP-XOR code and is called “Almost BP-XOR”.There are multiple codes that can be used for RAID-7 architectures. However, [5,2] BP-XOR codes have significantly lower encoding and decoding complexities than most common codes [1]. Regardless of this fact, this code does not achieve the fastest data decoding and reconstruction speeds due to its relatively low efficiency of 0.4. Furthermore, the existence of MDS [6,3] bx6 BP-XOR codes, b>2 (which achieves efficiency of 0.5) is still an open research question. This work proposes [6,3] 2 x 6 Almost BP-XOR codes. These codes largely utilize the simple and fast BP-XOR decoder while achieving an efficiency of 0.5, leading to the fastest recovery from disk failures among other state-of-the-art codes. An algorithm to generate a [6, 3] 2 x 6 Almost BP-XOR code has been developed and an example code is provided in Table 1. The [6, 3] 2 x 6 Almost BP-XOR codes are constructed in a way that any three-column-erasure pattern will result in one of the following two main scenarios. First: At least one of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in 70% of three-column erasure cases (i.e. 14 out of the 20 possible cases). The recovery process in such scenario is identical to that of the BP-XOR codes; Knowing any two information symbols in a degree-three encoding symbol is sufficient to know the third information symbol through performing a simple XOR operation. Second: None of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in the remaining 30% of three-column erasure cases (i.e., 6 out if the possible 20). The BP-XOR decoder fails in such a scenario. However, due to the construction of the codes, at least one surviving degree-three encoding symbol contains a known information symbol. Thus, knowing one of the reaming two information symbols in such a degree-three encoding symbol will initiate the BP-XOR decoder again.Table 2 shows these erasure patterns along with an expression for one of the missing information symbols. these expressions can be stored in buffers and used whenever the corresponding erasure pattern occurs. Solutions in Table 2 are derived from the inverse of a 6x6 submatrix that results from a generator matrix G by deleting columns from G corresponding to erased code columns. The read complexity of almost BP-XOR codes is 1. On the other hand. The decoding of almost BP-XOR codes require just 6 XOR operations when for a given three-column-erasure pattern BP-XOR decoding succeeds. However, when the BP- XOR decoder fails, it will require up to 15 XOR operations in total. The normalized repairing complexity is 15/6 = 2.5.Experimentally, Fig.1 shows that the proposed Almost BP-XOR codes require the least amount of time to decode and reconstruct erased columns. Thus, it is concluded that the [6, 3] 2x6 almost BP-XOR codes are best suited for RAID-7 system that requires storage efficiency of 0.5.

A family of codes with a natural 2-D structure is presented, inspired by an application of redundant arrays of independent disks (RAID) type of architectures whose units are solid-state drives (SSDs). Arrays of SSDs behave differently from arrays of hard disk drives, since hard errors in sectors are common and traditional RAID approaches (like RAID 5 or RAID 6) may be either insufficient or excessive. An efficient solution to this problem is given by the new codes presented, called partial maximum distance separable (PMDS) codes.

To satisfy the explosive growth of data in large-scale data centers, where redundant arrays of independent disks (RAIDs), especially RAID-5, are widely deployed, effective storage scaling and disk expansion methods are desired. However, a way to reduce the data migration overhead and maintain the reliability of the original RAID are major concerns of storage scaling. To address these problems, we propose a new RAID scaling scheme, H-Scale, to achieve fast RAID scaling via hybrid stripe layouts. H-Scale takes advantage of the loose restriction of stripe structures to choose migrated data and to create hybrid stripe structures. The main advantages of our scheme include: (1) dramatically reducing the data migration overhead and thus speeding up the scaling process, (2) maintaining the original RAID’s reliability, (3) balancing the workload among disks after scaling, and (4) providing a general scaling approach for different RAID levels. Our theoretical analysis show that H-Scale outperforms existing scaling solutions in terms of data migration, I/O overheads, and parity update operations. Evaluation results on a prototype implementation demonstrate that H-Scale speeds up the online scaling process by up to 60% under SPC traces, and similar improvements on scaling time and user response time are also achieved by evaluations using standard benchmarks.

본 논문에서는 고비용의 엔트프라이즈급 RAID 저장장치를 저비용의 RAID 저장장치로 대체하고 인공위성 연구센터에서 개발한 데이터 수신카드를 이용하여 고속 데이터 수신저장 시스템을 구축하였다. 350Mbps로 수신되는 데이터를 처리하기 위해서 S-ATA2 방식의 하드디스크 3개와 자체 제작한 성능 시험 소프트웨어를 이용하였다. 성능 시험 결과 79MB의 속도로 RAID에 저장할 수 있음을 확인하였다. This paper describes the establishment result of high speed data receiving and archiving system(HDRAS) using data receiving card(DRC) developped by SaTReC and low cost RAID(Redundant Array of Independent Disks) storage instead of expensive enterprise RAID storage. We used three S-ATA2 type hard disks and performance test program developped by SaTReC to receive and save data at 350Mbps. Finally, we verified that the HDRAS using low cost RAID storage has a capacity to process received data at 79MB/s(632Mbps).

RAIDs (Redundant Arrays of Independent Disks) are a set of disks organized to achieve parallel I/O to multiple disks and to provide tolerance of disk failures. RAIDs offer these advantages at the cost of additional space and additional disk I/O for writes. Previous methods of reducing this I/O overhead suffered from such problems as requiring periods during which data is reorganized and not available, destroying the physical locality of data, or weakening the RAID's fault-tolerance properties. We propose a new method called data logging which reduces the I/O overhead without requiring periodic downtime for reorganization. Instead, incremental maintenance can be performed concurrently with routine processing. This is particularly advantageous in applications requiring 24/spl times/7 uptime. Data logging preserves both physical locality of data and RAID fault tolerance. The major cost of our method is a moderate amount of nonvolatile RAM. This paper describes our method, as well as two schemes for efficient encoding of the information that must be stored in nonvolatile RAM.

Disk reliability is a serious problem in the big data foundation environment. Although the reliability of disk drives has greatly improved over the past few years, they are still the most vulnerable core components in the server. If they fail, the result can be catastrophic: it can take some days to recover data, sometimes data lost forever. These are unacceptable for some important data. XOR parity is a typical method to generate reliability syndrome, thus improving the reliability of the data. In practice, we find that the data is still likely to be lost. In most storage systems reliability improvements are achieved through the allocation of additional disks in Redundant Arrays of Independent Disks (RAID), which will increase the hardware costs, thus it will be very difficult for cost-constrained environments. Therefore, how to improve the data integrity without raising the hardware cost has aroused much interest of big data researchers. This challenge is when creating non-traditional RAID geometries, care must be taken to respect data dependence relationships to ensure that the new RAID strategy improves reliability, which is a NP-hard problem. In this paper, we present an approach for characterizing these challenges using high-dimension variants of the n-queens problem that enables performable solutions via the SAT solver MiniSAT, and use the greedy algorithm to analyze the queen's attack domain, as a basis for reliability syndrome generation. A large number of experiments show that the approach proposed in this paper is feasible in software-defined data centers and the performance of the algorithm can meet the current requirements of the big data environment.

The computing industry is one of the fastest growing industries and it is caused by the rapid progression of computer hardware and software technological developments. The high-capacity storage systems become the important element of the network architectures. The network architectures of high-capacity storage systems can be developed by using various, different techniques. The present paper aims to give a review about the storage systems architecture, systematize the various types of connection, and present the significance of virtualization. Based on industry specific literature and on own research, the present paper sets up a clear systematization of these techniques by providing a brief summary of their essence, while also providing examples of their application in different network storages. The present article also provides an insight into the topic of the different connecting types of network systems, while highlighting the importance of virtualization and application of the iSCSI technology. After the explanation of the challenge of redundancies in storage systems, the author outlines the importance of using RAID (Redundant Array of Independent Disks) technology for handling the issue of redundancy

In this paper, a multistate cloud‐RAID (redundant array of independent disks) storage system subject to fault‐level coverage (FLC) is modeled and analyzed. Most of the existing works on reliability analysis of cloud‐RAID systems have either assumed binary‐state for storage disks or failed to consider imperfect fault coverage, an inherent behavior of fault‐tolerant systems. This work advances the state of the art by proposing a combinatorial method based on multivalued decision diagrams for analyzing reliability of a multistate cloud‐RAID system with FLC. The FLC is one common type of imperfect fault coverage behaviors, where the system fault recovery capability is dependent on the number of disk faults happening within a certain recovery window. Effects of the functional dependence behavior between the RAID controller and disks are addressed. The method is illustrated through a detailed analysis of an example cloud‐RAID 5 storage system. Numerical results are provided to show the impact of different design parameters on system performance. These results also demonstrate that failure to consider FLC leads to inaccurate system state probabilities, further misleading system design activities based on these probabilities such as maintenance and optimization.

The high dependence on information technology in modern society leads to the ever-increasing demand for data reliability and availability. However, the increasing size of systems and the use of cheap but less reliable components have made component failures such as disk failures more common. As a new algorithm Row-Diagonal Parity (RDP) for the propose of protecting against double disk failures. It stores all unencoded data and uses only the entire operation during construction and reconstruction to computational complexity. RDP works in individual stripe blocks commonly used in file systems, disk arrays, and databases. In the information column, use parity lines of a different slope, or slopes, to obtain a check column. The code word of an EVENODD code is placed in an array of (m-1)*(m+2), where m is a prime number, where the information is set in an array of (m-1)*m, and the last two columns are the parity information characters. Two columns of parity bits are XOR by the information bits in the same row or on the diagonal of a given slope. The layout strategy using EVENODD coding algorithm can allow two data blocks to error at the same time, which can ensure the stability of the system. It has been widely used in technologies such as RAID (Redundant Arrays of Independent Disks).A fast repair algorithm is proposed the check disk failure repair problem in distributed storage systems based on RAID 6 encoding. This paper proposed a fast repair algorithm. Through the theoretical analysis of RDP and EVENODD encoding, the node's computational encoding capability is used to transfer encoded data blocks to repair the checksum disk, which reduces the data transfer during the repair process and shortens the repair time. The theoretical analysis shows that this algorithm can significantly reduce the bandwidth resources consumed during the check disk failure repair process and improve the repair efficiency compared to traditional repair algorithms.

The complexity of programmable logic design has increased ever since its conception because of the increased focus on seamless integration of design creation. Interconnections, which occupy 60 to 90% of the chip area, play a vital role in deciding the power and delay in such designs. Hence there is an increased focus on Multi-Valued Logic, because of its inherent ability to reduce the number of interconnections. In this paper, we present the design of (n=15, k=13) Reed Solomon code algorithm for advanced Redundant Array of Independent Disks (RAID) system in quaternary domain, to tolerate multiple disk failures. We present two designs to implement the algorithm on a FPGA- a heterogeneous design consisting of quaternary and binary circuits, a complete quaternary design. The first design performs all computations using only binary EX-OR gates. This design requires more EX-OR operations than the binary counterpart. This number is a function of the generator polynomial used for encoding. The second design based on quaternary look-up tables is efficient and can be easily implemented. The look-up tables are in turn based on quaternary multiplexers that can be further optimized by reducing the feature size.