Data logging: a method for efficient data updates in constantly active RAIDs

This paper studies how RAID (redundant array of independent disks) could take full advantage of modern SSDs (solid-state drives) with built-in transparent compression. In current practice, RAID users are forced to choose a specific RAID level (e.g., RAID 10 or RAID 5) with a fixed storage cost vs. speed performance trade-off. The commercial market is witnessing the emergence of a new family of SSDs that can internally perform hardware-based lossless compression on each 4KB LBA (logical block address) block, transparent to host OS and user applications. Beyond straightforwardly reducing the RAID storage cost, such modern SSDs make it possible to relieve RAID users from being locked into a fixed storage cost vs. speed performance trade-off. In particular, RAID systems could opportunistically leverage higher-than-expected runtime user data compressibility to enable dynamic RAID level conversion to improve the speed performance without compromising the effective storage capacity. This paper presents techniques to enable and optimize the practical implementation of such elastic RAID systems. We implemented a Linux software-based elastic RAID prototype that supports dynamic conversion between RAID 5 and RAID 10. Compared with a baseline software-based RAID 5, under sufficient runtime data compressibility that enables the conversion from RAID 5 to RAID 10 over 60% of user data, the elastic RAID could improve the 4KB random write IOPS (I/O per second) by 42% and 4KB random read IOPS in degraded mode by 46%, while maintaining the same effective storage capacity.

Because of the ever increasing amount of information needing to be managed, the development of storage technology has become very important. RAID (redundant array of independent disks) is usually used for administering large storage, although traditional RAIDs have only one or two fault tolerances, which is not sufficient for constructing large storage. Although hierarchical RAID (HiRAID) has three fault tolerances, efficiency in capacity is low. In the previous work, we proposed N-ary RAID, which is flat (not hierarchical) RAID with multiple parities based on a block unit. N-ary RAID is a 3 fault tolerant RAID. In this paper, we propose N-ary RAID MP, which is combined N-ary RAID to RAID MP (multiple parity). It drastically increases the reliability of RAID based storage.

Redundant array of independent disks (RAID) has been widely used to address the reliability and performance issues of storage systems. As the scale of modern storage systems continues growing, disk failure becomes the norm. With the ever-increasing disk capacity, RAID recovery based on disk rebuild becomes more and more costly, which causes significant performance degradation and even unavailability of storage systems. Declustered data layout enables parallel RAID reconstruction by shuffling data and parity blocks among all drives (including spares) in a RAID group. However, the reliability and performance of declustered RAID in real-world storage environments have not been thoroughly studied. With the popularity of ZFS file system and software RAID used in production data centers, in this paper, we extensively evaluate declustered RAID with regard to the RAID recovery time and I/O performance on a high-performance storage platform at Los Alamos National Laboratory. Our empirical study reveals the advantages and disadvantages of declustered RAID technology. We qualitatively characterize the recovery performance of declustered RAID and compare with that of ZFS RAIDZ under various I/O workloads and access patterns. The experimental results show that the speedup of declustered RAID over traditional RAID is sub-linear to the parallelism of recovery I/O. Furthermore, we formally model and analyze the reliability of declustered RAID in terms of the mean-time-to-data-loss (MTTDL) and discover that the improved recovery performance leads to higher storage reliability compared with the traditional RAID.

This paper is a summary and evaluation of redundant array of independent disks (also called redundant array of inexpensive disks before) that was proposed years ago. Recently more information is produced every year, and the demand of data storage is increasing exponentially due to Moore's Law. So, it's much more common to group a dozen of hard disk drives together, to achieve better performance or to achieve better data safety. Firstly, the redundant array of independent disks aims to solve the data safety/data in-out performance problems, and multiple ways are proposed including Level 0 to solve the performance problem sacrificing system stability and data safety and Level 1 aims to achieve better stability but using only 50% (or less) of the data capacity. The development of redundant array of independent disks tends to be merged and have more functions in one single system of redundant array of inexpensive disks.

Redundant Array of Independent Disks (RAID) storage architectures provide protection of digital infrastructure against potential disks failures. For example, RAID-5 and RAID-6 architectures provide protection against one and two disk failures, respectively. Recently, the data generation has significantly increased due to the emergence of new technologies. Thus, the size of storage systems is also growing rapidly to accommodate such large data sizes, which increases the probability for disks failures. This necessitates a new RAID architecture that can tolerate up to three disk failures. RAID architectures implement coding techniques. The code specifies how data is stored among multiple disks and how lost data can be recovered from surviving disks. This abstract introduces a novel coding scheme for new RAID-7 architectures that can tolerate up to three disks failures. The code is an improved version of the existing BP-XOR code and is called “Almost BP-XOR”.There are multiple codes that can be used for RAID-7 architectures. However, [5,2] BP-XOR codes have significantly lower encoding and decoding complexities than most common codes [1]. Regardless of this fact, this code does not achieve the fastest data decoding and reconstruction speeds due to its relatively low efficiency of 0.4. Furthermore, the existence of MDS [6,3] bx6 BP-XOR codes, b>2 (which achieves efficiency of 0.5) is still an open research question. This work proposes [6,3] 2 x 6 Almost BP-XOR codes. These codes largely utilize the simple and fast BP-XOR decoder while achieving an efficiency of 0.5, leading to the fastest recovery from disk failures among other state-of-the-art codes. An algorithm to generate a [6, 3] 2 x 6 Almost BP-XOR code has been developed and an example code is provided in Table 1. The [6, 3] 2 x 6 Almost BP-XOR codes are constructed in a way that any three-column-erasure pattern will result in one of the following two main scenarios. First: At least one of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in 70% of three-column erasure cases (i.e. 14 out of the 20 possible cases). The recovery process in such scenario is identical to that of the BP-XOR codes; Knowing any two information symbols in a degree-three encoding symbol is sufficient to know the third information symbol through performing a simple XOR operation. Second: None of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in the remaining 30% of three-column erasure cases (i.e., 6 out if the possible 20). The BP-XOR decoder fails in such a scenario. However, due to the construction of the codes, at least one surviving degree-three encoding symbol contains a known information symbol. Thus, knowing one of the reaming two information symbols in such a degree-three encoding symbol will initiate the BP-XOR decoder again.Table 2 shows these erasure patterns along with an expression for one of the missing information symbols. these expressions can be stored in buffers and used whenever the corresponding erasure pattern occurs. Solutions in Table 2 are derived from the inverse of a 6x6 submatrix that results from a generator matrix G by deleting columns from G corresponding to erased code columns. The read complexity of almost BP-XOR codes is 1. On the other hand. The decoding of almost BP-XOR codes require just 6 XOR operations when for a given three-column-erasure pattern BP-XOR decoding succeeds. However, when the BP- XOR decoder fails, it will require up to 15 XOR operations in total. The normalized repairing complexity is 15/6 = 2.5.Experimentally, Fig.1 shows that the proposed Almost BP-XOR codes require the least amount of time to decode and reconstruct erased columns. Thus, it is concluded that the [6, 3] 2x6 almost BP-XOR codes are best suited for RAID-7 system that requires storage efficiency of 0.5.

A family of codes with a natural 2-D structure is presented, inspired by an application of redundant arrays of independent disks (RAID) type of architectures whose units are solid-state drives (SSDs). Arrays of SSDs behave differently from arrays of hard disk drives, since hard errors in sectors are common and traditional RAID approaches (like RAID 5 or RAID 6) may be either insufficient or excessive. An efficient solution to this problem is given by the new codes presented, called partial maximum distance separable (PMDS) codes.

To satisfy the explosive growth of data in large-scale data centers, where redundant arrays of independent disks (RAIDs), especially RAID-5, are widely deployed, effective storage scaling and disk expansion methods are desired. However, a way to reduce the data migration overhead and maintain the reliability of the original RAID are major concerns of storage scaling. To address these problems, we propose a new RAID scaling scheme, H-Scale, to achieve fast RAID scaling via hybrid stripe layouts. H-Scale takes advantage of the loose restriction of stripe structures to choose migrated data and to create hybrid stripe structures. The main advantages of our scheme include: (1) dramatically reducing the data migration overhead and thus speeding up the scaling process, (2) maintaining the original RAID’s reliability, (3) balancing the workload among disks after scaling, and (4) providing a general scaling approach for different RAID levels. Our theoretical analysis show that H-Scale outperforms existing scaling solutions in terms of data migration, I/O overheads, and parity update operations. Evaluation results on a prototype implementation demonstrate that H-Scale speeds up the online scaling process by up to 60% under SPC traces, and similar improvements on scaling time and user response time are also achieved by evaluations using standard benchmarks.

본 논문에서는 고비용의 엔트프라이즈급 RAID 저장장치를 저비용의 RAID 저장장치로 대체하고 인공위성 연구센터에서 개발한 데이터 수신카드를 이용하여 고속 데이터 수신저장 시스템을 구축하였다. 350Mbps로 수신되는 데이터를 처리하기 위해서 S-ATA2 방식의 하드디스크 3개와 자체 제작한 성능 시험 소프트웨어를 이용하였다. 성능 시험 결과 79MB의 속도로 RAID에 저장할 수 있음을 확인하였다. This paper describes the establishment result of high speed data receiving and archiving system(HDRAS) using data receiving card(DRC) developped by SaTReC and low cost RAID(Redundant Array of Independent Disks) storage instead of expensive enterprise RAID storage. We used three S-ATA2 type hard disks and performance test program developped by SaTReC to receive and save data at 350Mbps. Finally, we verified that the HDRAS using low cost RAID storage has a capacity to process received data at 79MB/s(632Mbps).

Disk reliability is a serious problem in the big data foundation environment. Although the reliability of disk drives has greatly improved over the past few years, they are still the most vulnerable core components in the server. If they fail, the result can be catastrophic: it can take some days to recover data, sometimes data lost forever. These are unacceptable for some important data. XOR parity is a typical method to generate reliability syndrome, thus improving the reliability of the data. In practice, we find that the data is still likely to be lost. In most storage systems reliability improvements are achieved through the allocation of additional disks in Redundant Arrays of Independent Disks (RAID), which will increase the hardware costs, thus it will be very difficult for cost-constrained environments. Therefore, how to improve the data integrity without raising the hardware cost has aroused much interest of big data researchers. This challenge is when creating non-traditional RAID geometries, care must be taken to respect data dependence relationships to ensure that the new RAID strategy improves reliability, which is a NP-hard problem. In this paper, we present an approach for characterizing these challenges using high-dimension variants of the n-queens problem that enables performable solutions via the SAT solver MiniSAT, and use the greedy algorithm to analyze the queen's attack domain, as a basis for reliability syndrome generation. A large number of experiments show that the approach proposed in this paper is feasible in software-defined data centers and the performance of the algorithm can meet the current requirements of the big data environment.

The high dependence on information technology in modern society leads to the ever-increasing demand for data reliability and availability. However, the increasing size of systems and the use of cheap but less reliable components have made component failures such as disk failures more common. As a new algorithm Row-Diagonal Parity (RDP) for the propose of protecting against double disk failures. It stores all unencoded data and uses only the entire operation during construction and reconstruction to computational complexity. RDP works in individual stripe blocks commonly used in file systems, disk arrays, and databases. In the information column, use parity lines of a different slope, or slopes, to obtain a check column. The code word of an EVENODD code is placed in an array of (m-1)*(m+2), where m is a prime number, where the information is set in an array of (m-1)*m, and the last two columns are the parity information characters. Two columns of parity bits are XOR by the information bits in the same row or on the diagonal of a given slope. The layout strategy using EVENODD coding algorithm can allow two data blocks to error at the same time, which can ensure the stability of the system. It has been widely used in technologies such as RAID (Redundant Arrays of Independent Disks).A fast repair algorithm is proposed the check disk failure repair problem in distributed storage systems based on RAID 6 encoding. This paper proposed a fast repair algorithm. Through the theoretical analysis of RDP and EVENODD encoding, the node's computational encoding capability is used to transfer encoded data blocks to repair the checksum disk, which reduces the data transfer during the repair process and shortens the repair time. The theoretical analysis shows that this algorithm can significantly reduce the bandwidth resources consumed during the check disk failure repair process and improve the repair efficiency compared to traditional repair algorithms.

The complexity of programmable logic design has increased ever since its conception because of the increased focus on seamless integration of design creation. Interconnections, which occupy 60 to 90% of the chip area, play a vital role in deciding the power and delay in such designs. Hence there is an increased focus on Multi-Valued Logic, because of its inherent ability to reduce the number of interconnections. In this paper, we present the design of (n=15, k=13) Reed Solomon code algorithm for advanced Redundant Array of Independent Disks (RAID) system in quaternary domain, to tolerate multiple disk failures. We present two designs to implement the algorithm on a FPGA- a heterogeneous design consisting of quaternary and binary circuits, a complete quaternary design. The first design performs all computations using only binary EX-OR gates. This design requires more EX-OR operations than the binary counterpart. This number is a function of the generator polynomial used for encoding. The second design based on quaternary look-up tables is efficient and can be easily implemented. The look-up tables are in turn based on quaternary multiplexers that can be further optimized by reducing the feature size.

정전 방지 장치가 없는 중소형 레이드 (RAID: redundant arrays of independent disks)는 갑작스런 정전 또는 오류로 인한 종료 이후에 수시간의 긴 재동기화 시간을 요구한다. 레이드에서는 데이터 블록과 패리티 블록이 일관성 있게 갱신되어야 하는데, 데이터를 기록하다가 정전이 되면 데이터 블록은 갱신되었는데 패리티 블록은 갱신되지 않거나 반대인 경우가 발생할 수 있다. 이렇게 부분적으로 갱신된 스트라이프를 반드시 올바른 패리티로 갱신해야 하나 어떤 스트라이프에 이런 문제가 발생하였는지 찾기가 매우 어려웠다. 기존에는 전 저장공간을 검색하고 오류 있는 스트라이프를 수정하는, 수 시간을 요구하는, 패리티 재동기화 방법이 사용되어 왔다. 본 논문은 중소형 레이드에서 낮은 오버헤드를 갖는 고속의 재동기화 기술을 제안한다. 제안하는 기술은 리눅스의 소프트웨어 레이드에서 구현되었다. 성능 실험 결과에 따르면, 제안하는 기법은 재동기화 과정을 200분에서 5초로 단축시키고, 일반 입출력에서 22%의 오버헤드를 2%로 낮추었다. Redundant arrays of independent disks (RAID) without a power-fail-safe component in small and mid-sized business suffers from intolerably long resynchronization time after a unclean power-failure. Data blocks and a parity block in a stripe must be updated in a consistent manner, however a data block may be updated but the corresponding parity block may not be updated when a power goes off. Such a partially modified stripe must be updated with a correct parity block. However, it is difficult to find which stripe is partially updated (inconsistent). The widely-used traditional parity resynchronization manner is a intolerably long process that scans the entire volume to find and fix inconsistent stripes. This paper presents a fast resynchronization scheme with a negligible overhead for small and mid-sized RAIDs. The proposed scheme is integrated into a software RAID driver in a Linux system. According to the performance evaluation, the proposed scheme shortens the resynchronization process from 200 minutes to 5 seconds with 2% overhead for normal I/Os.

Providing data availability in a high performance computing environment is very important, especially in this data-intensive world. Most clusters either equip with RAID (Redundant Array of Independent Disks) devices or use redundant nodes to protect data from loss. However, neither of these can really solve the reliability problem incurred in a striped file system. Striping provides an efficient way to increase I/O throughput both in the distributed and parallel paradigms. But it also reduces the overall reliability of a disk system by N fold, where N is the number of independent disks in the system. Parallel Virtual File System (PVFS) is an open source parallel file system which has been widely used in the Linux environment. Its striping structure is good for performance but provides no fault tolerance. We implement Reliable Parallel File System (RPFS) based on PVFS but with reliability support. Our quantitative analysis shows that MTTF (Mean Time To Failure) of our RPFS is better than that of PVFS. Besides, we propose a parity cache table (PCT) to alleviate the penalty of parity updating. The evaluation of our RPFS shows that its read performance is almost the same as that of PVFS (2% to 13% degradation). As to the write performance, 28% to 45% improvement can be achieved depending on the behavior of the operations.

While the performance of individual disks is well understood, the performance of disk arrays configured as redundant arrays of independent disks - RAID requires further investigation. We report on a tool that can be used for the performance evaluation of k disk failure tolerant (kDFT) disk arrays. RAID0 is a 0DFT. RAID5 is a 1DFT utilizing a single check disk. RAID6, EVENODD, and RDP are 2DFTs with two check disks and a minimum level of redundancy, while RM2 is a 2DFT with a slightly higher redundancy level. The capacity overhead is less important for high capacity modern disks than the disk access overhead to update parities, especially the small write penalty. Degraded mode operation and rebuild processing incur extra overhead on surviving disks. The RAID level, configuration, operating mode (normal, degraded with 1 or 2 failures, and rebuild mode with various options), and workload characteristics have a significant effect on RAID performance, which can be evaluated using our tool. We consider here only discrete requests and derive cost functions to determine: (a) the volume of data to be transmitted; (b) the maximum attainable throughput for given disk characteristics; (c) the mean response time given the execution plans of RAID operations specified as dags (directed acyclic graphs). Analytic solutions are possible when arrivals are Poisson with the FCFS policy. Some interesting results obtained by the tool are given to illustrate its capabilities.

Adaptive control of pre-fetching