Fault Tolerance Performance Evaluation of Large-Scale Distributed Storage Systems HDFS and Ceph Case Study

One challenge in applying erasure codes (or error-correcting codes) to distributed storage systems is to maintain consistency between data and redundancy blocks in the face of crashing servers. We present two access protocols that provide sequential consistency and maximum distance separable fault tolerance at the same time. The protocols use sequence numbers to recover a consistent version in the presence of failures or partial writes. The first (pessimistic) PSW protocol uses a master per stripe to execute updates in sequence. The second (optimistic) OCW protocol allows concurrent writes to blocks in the same stripe to happen in parallel at the cost of additional buffer space.We present empirical performance results for PSW and OCW and compare them to other protocols. Our results show that OCW is as fast as simple replication while providing better fault tolerance and/or reduced storage overhead. This demonstrates that erasure coding can be used as a space-efficient alternative to replication in distributed storage systems.

Sun, J., 2020. Fault tolerance mechanism of distributed system for marine communication network. In: Gong, D.; Zhang, M., and Liu, R. (eds.), Advances in Coastal Research: Engineering, Industry, Economy, and Sustainable Development. Journal of Coastal Research, Special Issue No. 106, pp. 605–608. Coconut Creek (Florida), ISSN 0749-0208.When the communication network fails or the signal is interrupted, the distributed fault-tolerant system can continue to complete the communication task, which is needed to improve the fault-tolerant performance of the distributed system. Therefore, the key technologies of fault tolerance need to be studied, including the distributed fault-tolerant communication network simulation platform, the communication-network information-storage data structure, the communication protocol between communication network nodes, and the fault-tolerant algorithm of distributed communication network system. In the marine environment, the fleet exchanges data through wireless networks to understand the situation of the fleet-detection area, which can share the processing and calculation tasks. However, the marine environment is complex and changeable, which can cause great interference to the wireless signal, such as sea and cloud-rain clutter. These marine environments will cause the network communication signals to be greatly disturbed, which makes the distributed system unable to guarantee the communication interconnection between each node at any time. Therefore, a data-sharing and distributed-system platform needs to be built, which will design a reasonable system fault-tolerance mechanism. Through the fault-tolerance mechanism of the communication network system, the problem of frequent-node exits can be solved. By optimizing the fault-tolerant strategy of distributed architecture, the system's disaster resistance can be improved. In this research, a fault-tolerance mechanism for a distributed system was designed, which is more suitable for node election.

Chunk-based deduplication has been widely used in storage systems to save storage space. However, deduplication impairs data reliability due to the inter-file chunk sharing. The loss of shared chunks will make these referenced files inaccessible. Meanwhile, we find that inter-file and highly-referenced chunks are important that need higher reliability assurance, but occupy a small fraction of physical storage. Traditional deduplication systems utilize erasure coding or replication techniques to ensure data reliability. With the growth of shared chunks, promoting the reliability of erasure-coded systems incurs large I/O cost because of the weakness of coding scalability. Although replication is easy to scale, it incurs larger storage overhead. In this paper, we present DARM, a Deduplication-Aware Redundancy Management approach via exploiting deduplication semantics (e.g., inter-/intra-file duplicates, chunk size and reference count) to improve data reliability with low overhead. DARM leverages erasure coding for storing unique and low-referenced chunks to improve both storage reliability and space efficiency, and employs Selective and Dynamic Chunk-based Replication (SDCR) for maintaining inter-file and highly-referenced chunks to enhance storage reliability. Experimental results based on real-world datasets show that DARM reduces storage overhead by up to 43.4% and achieves at most 12.7% reliability improvements over the state-of-the-art schemes.

Coping with failures in modern distributed storage systems that handle massive volumes of heterogeneous and potentially rapidly changing data, has become a very important challenge. A common practice is to utilize fault tolerance methods like Replication and Erasure Coding for maximizing data availability. However, while erasure codes provide better fault tolerance compared to replication with a more affordable storage overhead, they frequently suffer from high reconstruction cost as they require to access all available nodes when a data block needs to be repaired, and also can repair up to a limited number of unavailable data blocks, depending on the number of the code’s parity block capabilities. Furthermore, storing and placing the encoded data in the federated storage system also remains a challenge. In this paper we present Fed-DIC, a framework which combines Diagonally Interleaved Coding on client devices at the edge of the network with organized storage of encoded data in a federated cloud system comprised of multiple independent storage clusters. The erasure coding operations are performed on client devices at the edge while they interact with the federated cloud to store the encoded data. We describe how our solution integrates the functionality of federated clouds alongside erasure coding implemented on edge devices for maximizing data availability and we evaluate the working and benefits of our approach in terms of read access cost, data availability, storage overhead, load balancing and network bandwidth rate compared to popular Replication and Erasure Coding schemes.

In the era of big data, ensuring both data confidentiality and reliability has become a critical concern for data owners. While encryption algorithms and erasure coding techniques can independently guarantee confidentiality and reliability, respectively, traditional methods that apply encryption before or after erasure coding often lead to significant performance degradation and increased storage overhead. To address these limitations, this paper proposes DERAID, a novel RAID-based technology that integrates encryption directly into the coding process. DERAID is designed to improve the efficiency of encoding, decoding, updating and reconstruction operations, while simultaneously ensuring data confidentiality and reliability and minimizing the data expansion rate. The scheme employs a unified coding structure based on the ECB encryption mode, where conventional parity blocks are replaced by a combination of cryptographic keys and verification blocks. Experimental results demonstrate that, compared to traditional methods and QS-code, DERAID improves encoding performance by 44.7–95.3%, updating performance by 48.2–99.6% and reconstruction performance by 63.0–99.9%, while reducing the data expansion rate by 3.3–36.9%. These results indicate that DERAID achieves a practical balance between security, reliability and performance.

Distributed storage systems are increasingly transitioning to the use of erasure codes since they offer higher reliability at significantly lower storage costs than data replication. However, these codes tradeoff recovery performance as they require multiple disk reads and network transfers for reconstructing an unavailable data block. As a result, most existing systems use an erasure code either optimized for storage overhead or recovery performance.In this paper, we present HACFS, a new erasure-coded storage system that instead uses two different erasure codes and dynamically adapts to workload changes. It uses a fast code to optimize for recovery performance and a compact code to reduce the storage overhead. A novel conversion mechanism is used to efficiently upcode and downcode data blocks between fast and compact codes. We show that HACFS design techniques are generic and successfully apply it to two different code families: Product and LRC codes.We have implemented HACFS as an extension to the Hadoop Distributed File System (HDFS) and experimentally evaluate it with five different workloads from production clusters. The HACFS system always maintains a low storage overhead and significantly improves the recovery performance as compared to three popular single-code storage systems. It reduces the degraded read latency by up to 46%, and the reconstruction time and disk/network traffic by up to 45%.

Erasure coding has been increasingly replacing replication in distributed storage systems, thanks to its lower storage overhead with the same level of failure tolerance. However, with lower storage overhead, the reconstruction overhead of erasure codes can increase significantly as well. Under the ever-changing workload, in which the data access can be highly skewed, it is difficult to achieve a well trade-off between the storage overhead and the reconstruction overhead. In this paper, we propose Zebra, a framework that encodes data into multiple tiers by their demand. Given the overall storage overhead and the number of failures to tolerate, Zebra determines the parameters of erasure coding in each tier by solving a geometric programming problem. Based on the demand of data, Zebra can dynamically assign data into the corresponding tiers to minimize the overall reconstruction overhead, and achieve a flexible tradeoff between the storage overhead and the reconstruction overhead in multiple tiers, such that hot data can enjoy less overhead of reconstruction and cold data can be stored with lower storage overhead. When demand changes, Zebra can adjust itself accordingly with a marginal amount of network transfer.

Distributed storage systems use erasure codes to reliably store data with a small storage overhead. To further improve system performance, some novel erasure codes introduce new features such as the regenerating property or symbol locality, enabling these codes to have optimal repair times and optimal degraded read performance. Unfortunately, the introduction of these new features often exacerbates the performance of other system metrics such as encoding throughput, data reliability, and storage overhead, among others. In this paper we describe the intricate relationships between erasure code properties and system-level performance metrics, showing the different tradeoffs distributed storage designers need to face. We also present Spider Codes, a new erasure code achieving a practical trade-off between the different system-level performance metrics.

In this research paper, the positions of sensor nodes are used for generating their public keys. In a wireless sensor networks (WSN), the nodes observe the environment in their vicinity and send the position specific data to the sink. Since the position and sensed data of a sensor are linked up to each other, the sensor position can be used as its unique public key. Multivariate cryptography offers a lightweight alternative to public key cryptography for sensor networks and position as public key can enhance its viability for WSN. In this study, we proposed a position-based multivariate cryptosystem for WSN that verifies the position of the sensor node and also prevented from the collusion attack. The performance and security analysis of the proposed scheme demonstrates that reduce storage overhead, communication overhead and prevent collusion attack in various contexts are possible within lightweight computational constraints.

With billions of devices adding up to the internet, broadly termed as Internet of Things (IoT), the need for reliable communication, distributed storage and computation has seamlessly increased. At this juncture, need for reliable communication, distributed storage and computation, adoption of error correcting codes and erasure codes plays a significant role. In this chapter, we give an overview of construction of erasure codes required for reliable communication with emphasis on Internet of Things (IoT) communication which have wireless sensors or sensor networks as its core. Wireless sensors form an integral part of Internet of Things (IoT) devices bridging the virtual world and the real world. Achieving reliability in such networks is highly desirable due to their broad range of applications. The discussed erasure codes in this chapter can be directly employed or with little modification in the context of reliable communication in Internet of Things. In this chapter, the two methods of information transmission, end-to-end transmission and hop-by-hop transmission, prevalent in digital communication scenario is discussed in detail with emphasis on with and without erasure coding. Also, the erasure codes used extensively in the context of achieving reliability in wireless sensor communication namely Reed-Solomon codes, Fountain codes and Decentralized Erasure codes are discussed and compared. This chapter serves as a starting point for researchers interested in working on reliability aspects of communication in Internet of Things embedded with wireless sensors and cyber physical systems.

The size and complexity of today's multiprocessor systems require the development of new techniques to measure their dependability. An effective technique allowing one to inject faults in message passing multiprocessor systems is presented. Interrupt messages are used to trigger fault injection routines in the targeted processors. Any fault that can be emulated by a modification of the memory content of processors can be injected. That includes faults that could occur within the processors, memories and even in the communication network. The proposed technique allows one to control the time and location of faults as well as other characteristics. It has been used in a prototype multiprocessor system running real applications in order to compare the efficiency of various error detection and correction mechanisms.

Data centers store a massive amount of data in a large number of servers built by commodity hardware. To maintain data integrity against server failures, erasure codes have been extensively deployed in modern data centers to provide a higher level of failure tolerance with less storage overhead than replication. Yet, compared to replication, disseminating erasure-coded data from a source server into multiple servers will also take significantly more time. In this paper, we design and implement Mist, a new mechanism for disseminating erasure-coded data efficiently to multiple receiving servers (receivers) in data centers. Mist speeds up the dissemination process by building an efficient topology among the receivers with heterogeneous performance, so that coded data can be received from other receivers in a pipelined fashion, rather than directly from the source. Mist flexibly supports a wide range of erasure codes, without imposing constraints to the range of system parameters, and can be extended for specific erasure codes with better performance by taking advantage of the corresponding erasure code. We have implemented Mist in Python, and our experimental results in Amazon EC2 have demonstrated that the dissemination time can be reduced by up to 96.3 percent with different kinds of erasure codes. © 2013 IEEE.

High availability in cloud computing services is essential for maintaining customer confidence and avoiding revenue losses due to SLA violation penalties. Since the software and hardware components of cloud infrastructures may have limited reliability, fault tolerance mechanisms are a means of achieving the necessary dependability requirements. This paper investigates the benefits of a warm-standy replication mechanism in a Eucalyptus cloud computing environment. A hierarchical heterogeneous modeling approach is used to represent a redundant architecture and compare its availability to that of a non-redundant architecture. Both hardware and software failures are considered in the proposed analytical models. The results show an enhanced dependability for the proposed redundant system, as well as a decrease in the annual downtime. The results also demonstrate that the simple replacement of hardware by more reliable machines would not produce improvements in system availability to the same extent as would the fault tolerant approach.

An adaptive cellular automata scheme for diagnosis of fault tolerance and connectivity preserving in wireless sensor networks

Distributed connectivity restoration in multichannel wireless sensor networks