Using ZeroMQ as communication/synchronization mechanisms for IO requests simulation

In this paper, we consider two rack-aware storage systems. First, large-scale rack-aware storage system, which is very common in large-scale storage system, is a rack-aware storage system where all sizes of racks are at least the number of redundant nodes. For such storage system, we prove that any Maximum Distance Separable (MDS) codes can have optimal inter-rack repair bandwidth and give a closed-form representation of all repair schemes with optimal inter-rack repair bandwidth. Furthermore, we show that the optimal repair access and optimal inter-rack repair bandwidth can be attained simultaneously for such storage system. Second, we investigate the rack-aware storage system of all racks with the same size, which is called uniform rack-aware storage system. We prove that, except the trivial cases, we cannot attain optimal inter-rack repair bandwidth and optimal repair access for such storage system at the same time. Specifically, we establish the lower bound of repair access for a repair scheme with optimal interrack repair bandwidth, which is tight for some parameters, and also the tight lower bound of inter-rack repair bandwidth for a repair scheme with optimal repair access.

Large-scale storage systems are crucial components in data-intensive applications such as search engine clusters, video-on-demand servers, sensor networks and grid computing. A storage server typically consists of a set of storage devices. In such systems, data layouts may need to be reconfigured over time for load balancing or in the event of system failure/upgrades. It is critical to migrate data to their target locations as quickly as possible to obtain the best performance. Most of the previous results on data migration assume that each storage node can perform only one data transfer at a time. A storage node, however, can typically handle multiple transfers simultaneously and this can reduce the total migration time significantly. Moreover, storage devices tend to have heterogeneous capabilities as devices may be added over time due to storage demand increase. In this paper, we consider the heterogeneous data migration problem, where we assume that each storage node v has different transfer constraint cv, which represents how many simultaneous transfers v can handle. We develop algorithms to minimize the data migration time. We show that it is possible to find an optimal migration schedule when all cvs are even. Furthermore, though the problem is NP-hard in general, we give an efficient algorithm that offers a rigorous (1 + o(1))-approximation guarantee.

Variable-size data item placement for load and storage balancing

The development of stationary battery storage systems in Germany – status 2020

The development of stationary battery storage systems in Germany – A market review

Hydrogen produced by electrolysis is one of the promising means by which to store electricity coming from renewables and to address its intermittency. With the growing global electricity share from renewable energy sources, large-scale storage systems are required. Alkaline electrolysis can be a sustainable solution for the large-scale energy storage system. However, high overpotential and slow kinetics particularly of the oxygen evolution reaction (OER) still remains as a hurdle to limit efficiency of the system. To enhance efficiency of alkaline electrolysis cells, catalysts which have high electrocatalytic activity for OER should be developed. Platinum-like behavior of Tungsten carbide in surface catalysis was first reported by Levy and Boudart in 1973[1]. In addition, tungsten carbide was reported to have Pt-like electronic structure by Colton in 1975[2]. After that, tungsten carbide has been studied widely and expected to substitute noble metal catalysts which are used in many electrochemical reactions. Here, we studied tungsten carbide as a support for 3d-transition metals and make bi-metallic surfaces for developing a high activity OER catalyst. First, WC-Co was synthesized by heat treatment. Carbon nitride (g-C3N4) and tungsten chloride were used as a carbide precursor and cobalt acetate was also used for helping carbonization of tungsten. After heat treatment, acidic leaching process was followed to eliminate cobalt. Through additional mixing with Nickel precursor and heat treatment, Ni/WC was synthesized and to compare electrocatalytic activity Ni/C was also prepared. In addition, many different Ni/WC which have different Ni to WC ratios were synthesized and measured in electrochemical way. Finally, the optimal ratio was determined and it show better electrocatalytic activity than precious catalysts and any other non-noble catalysts. Cyclic voltammetry was measured to evaluate OER activity in 1 M KOH in potential range from 1 to 1.8 V. Figure 1 shows that pure WC and Ni/C have 394 mV and 413 mV overpotentials at 10 mA∙cm-2 respectively. In Metal/WC cases, Cobalt/WC shows only 417 mV. But Nickel/WC shows 340 mV overpotential at 10 mA∙cm-2 and higher electrocatalytic activity for OER than RuO2 in alkaline media. η @ 10 mA cm-2 (mV) Ni/C 394 WC 413 Co/WC 417 Ni/WC 340 RuO2 355 Table1. Activities for OER at 10 mA cm-2 References [1] R. B. Levy, M. Boudart, Science, New series, Vol. 181, No. 4099 (1973) 547-579 [2] Richard J. Colton, Jan-Tsyu j. Huang, J.Wayne Rabalais, Chemical Physics Letters, Vol. 34, No. 2 (1975) 337-339 Figure1. Cyclic voltammetry for the oxygen evolution reaction in 1 M KOH of Ni/C, WC, Co/WC, Ni/WC, RuO2 Figure 1

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

The ever-growing creation of data requires large-scale network storage systems. One of the key issues related such systems is how to place several petabytes of data among large number of devices. Itpsilas necessary to design a reliable, fair, adaptable data placement algorithm. All proposed approaches are oblivious to the reliability-based requirements of data in such systems. In this paper, we present a reliability-aware data placement algorithm for large-scale network storage systems. With considering the reliability-aware differences of storage devices and the reliability-based requirement of data, we formulate the problem as an integer programming to minimize the reliability cost and propose a polynomial-time algorithm to solve the problem. For each reliability level, we use a fair and adaptive data placement to distribute data. It can support arbitrary heterogeneous storage systems, distribute data in a fair way and allow an efficient adaptation to a changing set of devices. The theoretical analysis as well as the experimental study show that the approach can meet the reliability-based requirements of data, distribute data evenly among devices, and adapt well to the changes of devices.

The eleven papers in this special section focus on both system- and circuit-level aspects of emerging memory and storage technologies. The overwhelmingly increasing demand for both storage and computation necessitates revisiting the traditional memory subsystems used in processors and storage systems to take advantage of emerging memory technologies. Dynamic Random Access Memory (DRAM) and Static Random Access Memory (SRAM) have been ubiquitously used for decades as main memory and as on-chip cache, respectively. Scaling and power issues of these traditional memory technologies have led to significant investment in emerging memory technologies. On the other hand, fundamental limitations of mechanical disk drives have brought great attention to further explore the design space of solid-state drives (SSDs). The promising features of emerging memory technologies such as low power consumption, increased performance, lower susceptibility to particle strikes, and higher bit density have prompted researchers to seek new organizations in the different memory-hierarchy levels, to propose new circuitry and algorithms to improve performance and reduce power consumption, and to develop new schemes to enhance system reliability.

Distributed storage systems provide large-scale data storage services, yet they are confronted with frequent node failures. To ensure data availability, a storage system often introduces data redundancy via replication or erasure coding. As erasure coding incurs significantly less redundancy overhead than replication under the same fault tolerance, it has been increasingly adopted in large-scale storage systems. In erasure-coded storage systems, degraded reads to temporarily unavailable data are very common, and hence boosting the performance of degraded reads becomes important. One challenge is that storage nodes tend to be heterogeneous with different storage capacities and I/O bandwidths. To this end, we propose FastDR, a system that addresses node heterogeneity and exploits I/O parallelism, so as to boost the performance of degraded reads to temporarily unavailable data. FastDR incorporates a greedy algorithm that seeks to reduce the data transfer cost of reading surviving data for degraded reads, while allowing the search of the efficient degraded read solution to be completed in a timely manner. We implement a FastDR prototype, and conduct extensive evaluation through simulation studies as well as testbed experiments on a Hadoop cluster with 10 storage nodes. We demonstrate that our FastDR achieves efficient degraded reads compared to existing approaches.

The need for reliable and fast storage systems is increasingly critical in various fields including artificial intelligence and data analytics. This paper proposes a new architecture for large-scale data storage systems, focusing on comparing performance of software and hardware storage technologies that effectively reduce the computational latency and improve performance. The main contributions include: (1) the combination of Singled Magnetic Recording (SMR) for storing data and Solid State Devices (SSD) for storing metadata is a viable solution for implementing large data storage systems; and (2) the combination of Conventional Magnetic Recording (CMR) for storing data and SSD for storing metadata shows the highest performance for high performance computing. Our experiments are carried out in multiple settings, demonstrating that the proposed architecture successfully improves performance for sequential and random reads/writes. The prototypes are evaluated with a set of workloads, showing the superiority of the proposed data storage configurations. This work provides new opportunities for efficiently processing and storing data and metadata in large-scale data analysis systems.

This paper is motivated by a global online satellite images distribution system operated at the Earth Resources Observation and Science (EROS) center of the U.S Geological Survey. Fundamental objectives of EROS include, but are not limited to, building high-speed and cost-effective massive data processing and storage systems to support online satellite images distribution. Hybrid storage systems -- containing solid-state drives (SSD), hard disks (HDD), and tapes -- can provide an ideal data storage solution for a wide variety of data processing centers like EROS. Large-scale hybrid storage systems will become increasingly popular in the next few years for the following two reasons. First, highly accessed storage objects in a hybrid storage system can be prefetched and cached to high-speed storage components such as solid-state drives. SSD-based hybrid storage system can provide large storage capacity, high I/O performance and data reliability. Second, hybrid storage systems are cost-effective, because inexpensive tapes help in increasing storage capacities at very low cost. Transferring data back and forth among SSDs, HDDs, and tapes plays a critical role in achieving high I/O performance. Thus, we proposed data mining algorithms that can judiciously prefetch data. Our analytical model and the experimental results reveals that our data mining prefetching algorithm increase the performance of the hybrid storage systems.

The failure of a single unreliable commodity component is very common in large-scale distributed storage systems. In order to ensure the reliability of data in large-scale distributed storage systems, many studies have emerged one after another. Among them, Erasure Codes are widely used in actual storage systems, such as Hadoop Distributed File System (HDFS), which provides high fault-tolerance with lower storage overhead. However, usually the recovery of erasure-coded storage system when encountering node failure results in severe cross-node and cross-rack bandwidth consumption, which affects the efficiency of failure recovery and wastes additional resources. In this paper, we improve the erasure coding storage strategy in distributed storage systems, and propose a low-overhead data recovery method based on cross-checking, namely HV-SNSP. In HV-SNSP, horizontal and vertical cross parity checking is realized by adding RS parity inside the data node, that is, <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', \textrm {}k')}}$ </tex-math></inline-formula> storage architecture. Based on <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, \textrm {}k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', k')}}$ </tex-math></inline-formula>, a low-cost supply node selection strategy, namely SNSP, is designed, and nodes with shorter network distance and lower load are selected to participate in recovery. This strategy can effectively reduce the amount of data transmission, shorten the recovery time, and improve the recovery efficiency. The experimental results show that compared with traditional RS, HV-SNSP can reduce the amount of cross-rack data transmission by 62.5&#x0025; during data recovery, and can shorten the recovery time by up to 42.41&#x0025;; Compared with D3, HV-SNSP can reduce the occupation of cross-rack bandwidth by 25&#x0025; and shorten the recovery time by 36.58&#x0025;.

At present, global climate change is intensifying and climate extreme events are occurring frequently, so the new energy system represented by the combined wind and storage system will play an important role in the future decarbonization process. Based on the life cycle analysis method, this study constructs a carbon footprint accounting model for large-scale combined wind and storage systems. The results show that the emission hotspot of the onshore wind power system is the component manufacturing stage, and the emission hotspot of the energy storage system is the operation and maintenance stage. This study provides a basis for the accurate calculation of the carbon emissions of the combined wind and storage system. It provides technical support for realizing the goals of “carbon peaking” and “carbon neutrality”.

Stabilizing zinc anodes for dual-membrane Zn-Ce redox flow battery via constructing zincophilic protective layer