DS-MOSD: A Dynamic Selectable Master OSD Model for Large-Scale Object-Based Storage System

In this paper, we consider two rack-aware storage systems. First, large-scale rack-aware storage system, which is very common in large-scale storage system, is a rack-aware storage system where all sizes of racks are at least the number of redundant nodes. For such storage system, we prove that any Maximum Distance Separable (MDS) codes can have optimal inter-rack repair bandwidth and give a closed-form representation of all repair schemes with optimal inter-rack repair bandwidth. Furthermore, we show that the optimal repair access and optimal inter-rack repair bandwidth can be attained simultaneously for such storage system. Second, we investigate the rack-aware storage system of all racks with the same size, which is called uniform rack-aware storage system. We prove that, except the trivial cases, we cannot attain optimal inter-rack repair bandwidth and optimal repair access for such storage system at the same time. Specifically, we establish the lower bound of repair access for a repair scheme with optimal interrack repair bandwidth, which is tight for some parameters, and also the tight lower bound of inter-rack repair bandwidth for a repair scheme with optimal repair access.

Large-scale storage systems are crucial components in data-intensive applications such as search engine clusters, video-on-demand servers, sensor networks and grid computing. A storage server typically consists of a set of storage devices. In such systems, data layouts may need to be reconfigured over time for load balancing or in the event of system failure/upgrades. It is critical to migrate data to their target locations as quickly as possible to obtain the best performance. Most of the previous results on data migration assume that each storage node can perform only one data transfer at a time. A storage node, however, can typically handle multiple transfers simultaneously and this can reduce the total migration time significantly. Moreover, storage devices tend to have heterogeneous capabilities as devices may be added over time due to storage demand increase. In this paper, we consider the heterogeneous data migration problem, where we assume that each storage node v has different transfer constraint cv, which represents how many simultaneous transfers v can handle. We develop algorithms to minimize the data migration time. We show that it is possible to find an optimal migration schedule when all cvs are even. Furthermore, though the problem is NP-hard in general, we give an efficient algorithm that offers a rigorous (1 + o(1))-approximation guarantee.

Distributed storage systems provide large-scale data storage services, yet they are confronted with frequent node failures. To ensure data availability, a storage system often introduces data redundancy via replication or erasure coding. As erasure coding incurs significantly less redundancy overhead than replication under the same fault tolerance, it has been increasingly adopted in large-scale storage systems. In erasure-coded storage systems, degraded reads to temporarily unavailable data are very common, and hence boosting the performance of degraded reads becomes important. One challenge is that storage nodes tend to be heterogeneous with different storage capacities and I/O bandwidths. To this end, we propose FastDR, a system that addresses node heterogeneity and exploits I/O parallelism, so as to boost the performance of degraded reads to temporarily unavailable data. FastDR incorporates a greedy algorithm that seeks to reduce the data transfer cost of reading surviving data for degraded reads, while allowing the search of the efficient degraded read solution to be completed in a timely manner. We implement a FastDR prototype, and conduct extensive evaluation through simulation studies as well as testbed experiments on a Hadoop cluster with 10 storage nodes. We demonstrate that our FastDR achieves efficient degraded reads compared to existing approaches.

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

System management is the collection of functions that are primarily concerned with the control, performance, and utilization of the mass storage system (MSS) defined by the IEEE mass storage system reference model. These functions are often very site-dependent, involve human decision making, and span multiple servers of the mass storage system. The functions may be implemented as standalone programs, may be integrated with the other mass storage system software, or may just be policy. A description is given of these functions, which consist of storage management, operations, systems maintenance, software support, hardware support, administrative control, and file management. >

Hydrogen produced by electrolysis is one of the promising means by which to store electricity coming from renewables and to address its intermittency. With the growing global electricity share from renewable energy sources, large-scale storage systems are required. Alkaline electrolysis can be a sustainable solution for the large-scale energy storage system. However, high overpotential and slow kinetics particularly of the oxygen evolution reaction (OER) still remains as a hurdle to limit efficiency of the system. To enhance efficiency of alkaline electrolysis cells, catalysts which have high electrocatalytic activity for OER should be developed. Platinum-like behavior of Tungsten carbide in surface catalysis was first reported by Levy and Boudart in 1973[1]. In addition, tungsten carbide was reported to have Pt-like electronic structure by Colton in 1975[2]. After that, tungsten carbide has been studied widely and expected to substitute noble metal catalysts which are used in many electrochemical reactions. Here, we studied tungsten carbide as a support for 3d-transition metals and make bi-metallic surfaces for developing a high activity OER catalyst. First, WC-Co was synthesized by heat treatment. Carbon nitride (g-C3N4) and tungsten chloride were used as a carbide precursor and cobalt acetate was also used for helping carbonization of tungsten. After heat treatment, acidic leaching process was followed to eliminate cobalt. Through additional mixing with Nickel precursor and heat treatment, Ni/WC was synthesized and to compare electrocatalytic activity Ni/C was also prepared. In addition, many different Ni/WC which have different Ni to WC ratios were synthesized and measured in electrochemical way. Finally, the optimal ratio was determined and it show better electrocatalytic activity than precious catalysts and any other non-noble catalysts. Cyclic voltammetry was measured to evaluate OER activity in 1 M KOH in potential range from 1 to 1.8 V. Figure 1 shows that pure WC and Ni/C have 394 mV and 413 mV overpotentials at 10 mA∙cm-2 respectively. In Metal/WC cases, Cobalt/WC shows only 417 mV. But Nickel/WC shows 340 mV overpotential at 10 mA∙cm-2 and higher electrocatalytic activity for OER than RuO2 in alkaline media. η @ 10 mA cm-2 (mV) Ni/C 394 WC 413 Co/WC 417 Ni/WC 340 RuO2 355 Table1. Activities for OER at 10 mA cm-2 References [1] R. B. Levy, M. Boudart, Science, New series, Vol. 181, No. 4099 (1973) 547-579 [2] Richard J. Colton, Jan-Tsyu j. Huang, J.Wayne Rabalais, Chemical Physics Letters, Vol. 34, No. 2 (1975) 337-339 Figure1. Cyclic voltammetry for the oxygen evolution reaction in 1 M KOH of Ni/C, WC, Co/WC, Ni/WC, RuO2 Figure 1

The development of stationary battery storage systems in Germany – status 2020

The development of stationary battery storage systems in Germany – A market review

The failure of a single unreliable commodity component is very common in large-scale distributed storage systems. In order to ensure the reliability of data in large-scale distributed storage systems, many studies have emerged one after another. Among them, Erasure Codes are widely used in actual storage systems, such as Hadoop Distributed File System (HDFS), which provides high fault-tolerance with lower storage overhead. However, usually the recovery of erasure-coded storage system when encountering node failure results in severe cross-node and cross-rack bandwidth consumption, which affects the efficiency of failure recovery and wastes additional resources. In this paper, we improve the erasure coding storage strategy in distributed storage systems, and propose a low-overhead data recovery method based on cross-checking, namely HV-SNSP. In HV-SNSP, horizontal and vertical cross parity checking is realized by adding RS parity inside the data node, that is, <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', \textrm {}k')}}$ </tex-math></inline-formula> storage architecture. Based on <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, \textrm {}k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', k')}}$ </tex-math></inline-formula>, a low-cost supply node selection strategy, namely SNSP, is designed, and nodes with shorter network distance and lower load are selected to participate in recovery. This strategy can effectively reduce the amount of data transmission, shorten the recovery time, and improve the recovery efficiency. The experimental results show that compared with traditional RS, HV-SNSP can reduce the amount of cross-rack data transmission by 62.5&#x0025; during data recovery, and can shorten the recovery time by up to 42.41&#x0025;; Compared with D3, HV-SNSP can reduce the occupation of cross-rack bandwidth by 25&#x0025; and shorten the recovery time by 36.58&#x0025;.

This paper is motivated by a global online satellite images distribution system operated at the Earth Resources Observation and Science (EROS) center of the U.S Geological Survey. Fundamental objectives of EROS include, but are not limited to, building high-speed and cost-effective massive data processing and storage systems to support online satellite images distribution. Hybrid storage systems -- containing solid-state drives (SSD), hard disks (HDD), and tapes -- can provide an ideal data storage solution for a wide variety of data processing centers like EROS. Large-scale hybrid storage systems will become increasingly popular in the next few years for the following two reasons. First, highly accessed storage objects in a hybrid storage system can be prefetched and cached to high-speed storage components such as solid-state drives. SSD-based hybrid storage system can provide large storage capacity, high I/O performance and data reliability. Second, hybrid storage systems are cost-effective, because inexpensive tapes help in increasing storage capacities at very low cost. Transferring data back and forth among SSDs, HDDs, and tapes plays a critical role in achieving high I/O performance. Thus, we proposed data mining algorithms that can judiciously prefetch data. Our analytical model and the experimental results reveals that our data mining prefetching algorithm increase the performance of the hybrid storage systems.

In this paper, an extensible system prototype of cloud storage is designed. Considering the load balance of cloud storage system, an improved data storage strategy based on consistent hashing algorithm is proposed in this paper. The strategy adopts virtual nodes to storage data and real–time monitoring of load rate of each storage node to adjust load balancing of the whole storage system. In the improved strategy, the priority is introduced into the storage system to rapidly improve the utilization rate of the new storage nodes. The strategy can effectively optimize the performance of the whole storage system, and improve the overall effect of the load balancing.

Large-scale storage systems may service millions of clients. Effectively identifying and authenticating these clients can not only prevent illegal accesses but also eliminate unnecessary redundancies of access control. However, existing security solutions have largely ignored the identification and authentication issues or separately consider them from access control mechanisms. As a result, these solutions either have incurred a high cost of access control or led to new insurmountable problems. In this paper, we discuss the important consideration in designing an appropriate authentication scheme for large-scale storage systems. First, we discuss potential authentication techniques for large-scale storage systems. The looming crisis of PKI, a widely used technology for authentication in today's information security area is discussed and potential alternative technologies to PKI are suggested. Second, by merging authentication into access control we develop a decentralized security architecture for large-scale storage systems, designed to improve the scalability of authentication and reduce the overhead of access control. Finally, we experimentally evaluate how much overhead can be caused when various cryptographic algorithms used in an authentication protocol are applied to a large-scale storage system.

The Babel File System is a massive storage system, which is made up from three main components: clients, proxy and storage nodes. Each client perceives a single computer, called proxy, which dispatches all service requests randomly to an arbitrary storage node and manages the metadata. However, availability can be compromised if the proxy suddenly halts. To avoid this problem we propose the utilization of a redundant set of proxies. Nevertheless, this solution implies dealing with a consensus problem in order to guarantee the consistency of the metadata copies recorded at each and every proxy. In this paper we propose the use of the Paxos algorithm in order to overcome this problem. Our proposal can be understood as a middleware through which the proxies communicate among themselves to enforce metadata consistency. The experiments showed that, under certain random conditions, the progress of the protocol could be threatened. We propose the mechanisms to address these issues to set the system back to its stable state. However, stability is strongly limited by the correlation between failure and recovery rates.

High-performance computing (HPC) storage systems provide data availability and reliability using various hardware and software fault tolerance techniques. Usually, reliability and availability are calculated at the subsystem or component level using limited metrics such as, mean time to failure (MTTF) or mean time to data loss (MTTDL). This often means settling on simple and disconnected failure models (such as exponential failure rate) to achieve tractable and close-formed solutions. However, such models have been shown to be insufficient in assessing end-to-end storage system reliability and availability. We propose a generic simulation framework aimed at analyzing the reliability and availability of storage systems at scale, and investigating what-if scenarios. The framework is designed for an end-to-end storage system, accommodating the various components and subsystems, their interconnections, failure patterns and propagation, and performs dependency analysis to capture a wide-range of failure cases. We evaluate the framework against a large-scale storage system that is in production and analyze its failure projections toward and beyond the end of lifecycle. We also examine the potential operational impact by studying how different types of components affect the overall system reliability and availability, and present the preliminary results

Applications require storage systems to provide more and more storage space, at the same time; the performance of storage systems is also more and more demanding. In order to solve the contradiction between the cost and performance of storage system, hierarchical storage system composed of solid state disk and mechanical hard disk is widely used, which aims to obtain the performance of close solid state disk at the cost of approaching mechanical hard disk. Aiming at the problem of space utilization efficiency of solid-state disk, an adaptive hierarchical storage system is proposed ATS, ATS which can dynamically adjust the migration threshold according to the current access data scale of the load and the utilization of the storage device. The load can make full use of the storage space of solid-state disk without frequent data exchange at different levels. The experimental results show that compared with the static threshold mode and the MOD (Migrate on demand, access is migration) mode, ATS can obtain a better average response time regardless of whether the current load is in a case of a very large amount of data or a very small amount of data. ATS can reduce the average response time by up to 53% compared with the static threshold mode, and ATS can reduce the average response time by up to 50% compared with the MOD mode.