H-Scale

This paper studies how RAID (redundant array of independent disks) could take full advantage of modern SSDs (solid-state drives) with built-in transparent compression. In current practice, RAID users are forced to choose a specific RAID level (e.g., RAID 10 or RAID 5) with a fixed storage cost vs. speed performance trade-off. The commercial market is witnessing the emergence of a new family of SSDs that can internally perform hardware-based lossless compression on each 4KB LBA (logical block address) block, transparent to host OS and user applications. Beyond straightforwardly reducing the RAID storage cost, such modern SSDs make it possible to relieve RAID users from being locked into a fixed storage cost vs. speed performance trade-off. In particular, RAID systems could opportunistically leverage higher-than-expected runtime user data compressibility to enable dynamic RAID level conversion to improve the speed performance without compromising the effective storage capacity. This paper presents techniques to enable and optimize the practical implementation of such elastic RAID systems. We implemented a Linux software-based elastic RAID prototype that supports dynamic conversion between RAID 5 and RAID 10. Compared with a baseline software-based RAID 5, under sufficient runtime data compressibility that enables the conversion from RAID 5 to RAID 10 over 60% of user data, the elastic RAID could improve the 4KB random write IOPS (I/O per second) by 42% and 4KB random read IOPS in degraded mode by 46%, while maintaining the same effective storage capacity.

Redundant array of independent disks (RAID) has been widely used to address the reliability and performance issues of storage systems. As the scale of modern storage systems continues growing, disk failure becomes the norm. With the ever-increasing disk capacity, RAID recovery based on disk rebuild becomes more and more costly, which causes significant performance degradation and even unavailability of storage systems. Declustered data layout enables parallel RAID reconstruction by shuffling data and parity blocks among all drives (including spares) in a RAID group. However, the reliability and performance of declustered RAID in real-world storage environments have not been thoroughly studied. With the popularity of ZFS file system and software RAID used in production data centers, in this paper, we extensively evaluate declustered RAID with regard to the RAID recovery time and I/O performance on a high-performance storage platform at Los Alamos National Laboratory. Our empirical study reveals the advantages and disadvantages of declustered RAID technology. We qualitatively characterize the recovery performance of declustered RAID and compare with that of ZFS RAIDZ under various I/O workloads and access patterns. The experimental results show that the speedup of declustered RAID over traditional RAID is sub-linear to the parallelism of recovery I/O. Furthermore, we formally model and analyze the reliability of declustered RAID in terms of the mean-time-to-data-loss (MTTDL) and discover that the improved recovery performance leads to higher storage reliability compared with the traditional RAID.

Because of the ever increasing amount of information needing to be managed, the development of storage technology has become very important. RAID (redundant array of independent disks) is usually used for administering large storage, although traditional RAIDs have only one or two fault tolerances, which is not sufficient for constructing large storage. Although hierarchical RAID (HiRAID) has three fault tolerances, efficiency in capacity is low. In the previous work, we proposed N-ary RAID, which is flat (not hierarchical) RAID with multiple parities based on a block unit. N-ary RAID is a 3 fault tolerant RAID. In this paper, we propose N-ary RAID MP, which is combined N-ary RAID to RAID MP (multiple parity). It drastically increases the reliability of RAID based storage.

This paper is a summary and evaluation of redundant array of independent disks (also called redundant array of inexpensive disks before) that was proposed years ago. Recently more information is produced every year, and the demand of data storage is increasing exponentially due to Moore's Law. So, it's much more common to group a dozen of hard disk drives together, to achieve better performance or to achieve better data safety. Firstly, the redundant array of independent disks aims to solve the data safety/data in-out performance problems, and multiple ways are proposed including Level 0 to solve the performance problem sacrificing system stability and data safety and Level 1 aims to achieve better stability but using only 50% (or less) of the data capacity. The development of redundant array of independent disks tends to be merged and have more functions in one single system of redundant array of inexpensive disks.

A family of codes with a natural 2-D structure is presented, inspired by an application of redundant arrays of independent disks (RAID) type of architectures whose units are solid-state drives (SSDs). Arrays of SSDs behave differently from arrays of hard disk drives, since hard errors in sectors are common and traditional RAID approaches (like RAID 5 or RAID 6) may be either insufficient or excessive. An efficient solution to this problem is given by the new codes presented, called partial maximum distance separable (PMDS) codes.

본 논문에서는 고비용의 엔트프라이즈급 RAID 저장장치를 저비용의 RAID 저장장치로 대체하고 인공위성 연구센터에서 개발한 데이터 수신카드를 이용하여 고속 데이터 수신저장 시스템을 구축하였다. 350Mbps로 수신되는 데이터를 처리하기 위해서 S-ATA2 방식의 하드디스크 3개와 자체 제작한 성능 시험 소프트웨어를 이용하였다. 성능 시험 결과 79MB의 속도로 RAID에 저장할 수 있음을 확인하였다. This paper describes the establishment result of high speed data receiving and archiving system(HDRAS) using data receiving card(DRC) developped by SaTReC and low cost RAID(Redundant Array of Independent Disks) storage instead of expensive enterprise RAID storage. We used three S-ATA2 type hard disks and performance test program developped by SaTReC to receive and save data at 350Mbps. Finally, we verified that the HDRAS using low cost RAID storage has a capacity to process received data at 79MB/s(632Mbps).

RAIDs (Redundant Arrays of Independent Disks) are a set of disks organized to achieve parallel I/O to multiple disks and to provide tolerance of disk failures. RAIDs offer these advantages at the cost of additional space and additional disk I/O for writes. Previous methods of reducing this I/O overhead suffered from such problems as requiring periods during which data is reorganized and not available, destroying the physical locality of data, or weakening the RAID's fault-tolerance properties. We propose a new method called data logging which reduces the I/O overhead without requiring periodic downtime for reorganization. Instead, incremental maintenance can be performed concurrently with routine processing. This is particularly advantageous in applications requiring 24/spl times/7 uptime. Data logging preserves both physical locality of data and RAID fault tolerance. The major cost of our method is a moderate amount of nonvolatile RAM. This paper describes our method, as well as two schemes for efficient encoding of the information that must be stored in nonvolatile RAM.

Redundant Array of Independent Disks (RAID) storage architectures provide protection of digital infrastructure against potential disks failures. For example, RAID-5 and RAID-6 architectures provide protection against one and two disk failures, respectively. Recently, the data generation has significantly increased due to the emergence of new technologies. Thus, the size of storage systems is also growing rapidly to accommodate such large data sizes, which increases the probability for disks failures. This necessitates a new RAID architecture that can tolerate up to three disk failures. RAID architectures implement coding techniques. The code specifies how data is stored among multiple disks and how lost data can be recovered from surviving disks. This abstract introduces a novel coding scheme for new RAID-7 architectures that can tolerate up to three disks failures. The code is an improved version of the existing BP-XOR code and is called “Almost BP-XOR”.There are multiple codes that can be used for RAID-7 architectures. However, [5,2] BP-XOR codes have significantly lower encoding and decoding complexities than most common codes [1]. Regardless of this fact, this code does not achieve the fastest data decoding and reconstruction speeds due to its relatively low efficiency of 0.4. Furthermore, the existence of MDS [6,3] bx6 BP-XOR codes, b>2 (which achieves efficiency of 0.5) is still an open research question. This work proposes [6,3] 2 x 6 Almost BP-XOR codes. These codes largely utilize the simple and fast BP-XOR decoder while achieving an efficiency of 0.5, leading to the fastest recovery from disk failures among other state-of-the-art codes. An algorithm to generate a [6, 3] 2 x 6 Almost BP-XOR code has been developed and an example code is provided in Table 1. The [6, 3] 2 x 6 Almost BP-XOR codes are constructed in a way that any three-column-erasure pattern will result in one of the following two main scenarios. First: At least one of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in 70% of three-column erasure cases (i.e. 14 out of the 20 possible cases). The recovery process in such scenario is identical to that of the BP-XOR codes; Knowing any two information symbols in a degree-three encoding symbol is sufficient to know the third information symbol through performing a simple XOR operation. Second: None of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in the remaining 30% of three-column erasure cases (i.e., 6 out if the possible 20). The BP-XOR decoder fails in such a scenario. However, due to the construction of the codes, at least one surviving degree-three encoding symbol contains a known information symbol. Thus, knowing one of the reaming two information symbols in such a degree-three encoding symbol will initiate the BP-XOR decoder again.Table 2 shows these erasure patterns along with an expression for one of the missing information symbols. these expressions can be stored in buffers and used whenever the corresponding erasure pattern occurs. Solutions in Table 2 are derived from the inverse of a 6x6 submatrix that results from a generator matrix G by deleting columns from G corresponding to erased code columns. The read complexity of almost BP-XOR codes is 1. On the other hand. The decoding of almost BP-XOR codes require just 6 XOR operations when for a given three-column-erasure pattern BP-XOR decoding succeeds. However, when the BP- XOR decoder fails, it will require up to 15 XOR operations in total. The normalized repairing complexity is 15/6 = 2.5.Experimentally, Fig.1 shows that the proposed Almost BP-XOR codes require the least amount of time to decode and reconstruct erased columns. Thus, it is concluded that the [6, 3] 2x6 almost BP-XOR codes are best suited for RAID-7 system that requires storage efficiency of 0.5.

Disk reliability is a serious problem in the big data foundation environment. Although the reliability of disk drives has greatly improved over the past few years, they are still the most vulnerable core components in the server. If they fail, the result can be catastrophic: it can take some days to recover data, sometimes data lost forever. These are unacceptable for some important data. XOR parity is a typical method to generate reliability syndrome, thus improving the reliability of the data. In practice, we find that the data is still likely to be lost. In most storage systems reliability improvements are achieved through the allocation of additional disks in Redundant Arrays of Independent Disks (RAID), which will increase the hardware costs, thus it will be very difficult for cost-constrained environments. Therefore, how to improve the data integrity without raising the hardware cost has aroused much interest of big data researchers. This challenge is when creating non-traditional RAID geometries, care must be taken to respect data dependence relationships to ensure that the new RAID strategy improves reliability, which is a NP-hard problem. In this paper, we present an approach for characterizing these challenges using high-dimension variants of the n-queens problem that enables performable solutions via the SAT solver MiniSAT, and use the greedy algorithm to analyze the queen's attack domain, as a basis for reliability syndrome generation. A large number of experiments show that the approach proposed in this paper is feasible in software-defined data centers and the performance of the algorithm can meet the current requirements of the big data environment.

The high dependence on information technology in modern society leads to the ever-increasing demand for data reliability and availability. However, the increasing size of systems and the use of cheap but less reliable components have made component failures such as disk failures more common. As a new algorithm Row-Diagonal Parity (RDP) for the propose of protecting against double disk failures. It stores all unencoded data and uses only the entire operation during construction and reconstruction to computational complexity. RDP works in individual stripe blocks commonly used in file systems, disk arrays, and databases. In the information column, use parity lines of a different slope, or slopes, to obtain a check column. The code word of an EVENODD code is placed in an array of (m-1)*(m+2), where m is a prime number, where the information is set in an array of (m-1)*m, and the last two columns are the parity information characters. Two columns of parity bits are XOR by the information bits in the same row or on the diagonal of a given slope. The layout strategy using EVENODD coding algorithm can allow two data blocks to error at the same time, which can ensure the stability of the system. It has been widely used in technologies such as RAID (Redundant Arrays of Independent Disks).A fast repair algorithm is proposed the check disk failure repair problem in distributed storage systems based on RAID 6 encoding. This paper proposed a fast repair algorithm. Through the theoretical analysis of RDP and EVENODD encoding, the node's computational encoding capability is used to transfer encoded data blocks to repair the checksum disk, which reduces the data transfer during the repair process and shortens the repair time. The theoretical analysis shows that this algorithm can significantly reduce the bandwidth resources consumed during the check disk failure repair process and improve the repair efficiency compared to traditional repair algorithms.

The complexity of programmable logic design has increased ever since its conception because of the increased focus on seamless integration of design creation. Interconnections, which occupy 60 to 90% of the chip area, play a vital role in deciding the power and delay in such designs. Hence there is an increased focus on Multi-Valued Logic, because of its inherent ability to reduce the number of interconnections. In this paper, we present the design of (n=15, k=13) Reed Solomon code algorithm for advanced Redundant Array of Independent Disks (RAID) system in quaternary domain, to tolerate multiple disk failures. We present two designs to implement the algorithm on a FPGA- a heterogeneous design consisting of quaternary and binary circuits, a complete quaternary design. The first design performs all computations using only binary EX-OR gates. This design requires more EX-OR operations than the binary counterpart. This number is a function of the generator polynomial used for encoding. The second design based on quaternary look-up tables is efficient and can be easily implemented. The look-up tables are in turn based on quaternary multiplexers that can be further optimized by reducing the feature size.

정전 방지 장치가 없는 중소형 레이드 (RAID: redundant arrays of independent disks)는 갑작스런 정전 또는 오류로 인한 종료 이후에 수시간의 긴 재동기화 시간을 요구한다. 레이드에서는 데이터 블록과 패리티 블록이 일관성 있게 갱신되어야 하는데, 데이터를 기록하다가 정전이 되면 데이터 블록은 갱신되었는데 패리티 블록은 갱신되지 않거나 반대인 경우가 발생할 수 있다. 이렇게 부분적으로 갱신된 스트라이프를 반드시 올바른 패리티로 갱신해야 하나 어떤 스트라이프에 이런 문제가 발생하였는지 찾기가 매우 어려웠다. 기존에는 전 저장공간을 검색하고 오류 있는 스트라이프를 수정하는, 수 시간을 요구하는, 패리티 재동기화 방법이 사용되어 왔다. 본 논문은 중소형 레이드에서 낮은 오버헤드를 갖는 고속의 재동기화 기술을 제안한다. 제안하는 기술은 리눅스의 소프트웨어 레이드에서 구현되었다. 성능 실험 결과에 따르면, 제안하는 기법은 재동기화 과정을 200분에서 5초로 단축시키고, 일반 입출력에서 22%의 오버헤드를 2%로 낮추었다. Redundant arrays of independent disks (RAID) without a power-fail-safe component in small and mid-sized business suffers from intolerably long resynchronization time after a unclean power-failure. Data blocks and a parity block in a stripe must be updated in a consistent manner, however a data block may be updated but the corresponding parity block may not be updated when a power goes off. Such a partially modified stripe must be updated with a correct parity block. However, it is difficult to find which stripe is partially updated (inconsistent). The widely-used traditional parity resynchronization manner is a intolerably long process that scans the entire volume to find and fix inconsistent stripes. This paper presents a fast resynchronization scheme with a negligible overhead for small and mid-sized RAIDs. The proposed scheme is integrated into a software RAID driver in a Linux system. According to the performance evaluation, the proposed scheme shortens the resynchronization process from 200 minutes to 5 seconds with 2% overhead for normal I/Os.

In this paper, a multistate cloud‐RAID (redundant array of independent disks) storage system subject to fault‐level coverage (FLC) is modeled and analyzed. Most of the existing works on reliability analysis of cloud‐RAID systems have either assumed binary‐state for storage disks or failed to consider imperfect fault coverage, an inherent behavior of fault‐tolerant systems. This work advances the state of the art by proposing a combinatorial method based on multivalued decision diagrams for analyzing reliability of a multistate cloud‐RAID system with FLC. The FLC is one common type of imperfect fault coverage behaviors, where the system fault recovery capability is dependent on the number of disk faults happening within a certain recovery window. Effects of the functional dependence behavior between the RAID controller and disks are addressed. The method is illustrated through a detailed analysis of an example cloud‐RAID 5 storage system. Numerical results are provided to show the impact of different design parameters on system performance. These results also demonstrate that failure to consider FLC leads to inaccurate system state probabilities, further misleading system design activities based on these probabilities such as maintenance and optimization.

Adaptive control of pre-fetching

Storage systems are built by taking the basic capability of a storage device, such as the hard disk drive, and adding layers of hardware and software to obtain a highly reliable, high-performance, and easily managed system. We explain in this paper how storage systems have evolved over five decades to meet changing customer needs. First, we briefly trace the development of the control unit, RAID (redundant array of independent disks) technologies, copy services, and basic storage management technologies. Then, we describe how the emergence of low-cost local area data networking has allowed the development of network-attached storage (NAS) and storage area network (SAN) technologies, and we explain how block virtualization and SAN file systems are necessary to fully reap the benefits of these technologies. We also discuss how the recent trend in storage systems toward managing complexity, ease-of-use, and lowering the total cost of ownership has led to the development of autonomic storage. We conclude with our assessment of the current state-of-the-art by presenting a set of challenges driving research and development efforts in storage systems.