Dynamic grid quorum: a reconfigurable grid quorum and its power optimization algorithm

In this paper, we present the dynamic grid quorum, a technique for reducing the power consumption of large-scale storage systems. The key idea is to skew the workload towards a small number of quorums by means of a novel optimization algorithm. Moreover, this system allows reconfiguration by exchanging nodes without any data migration so as to reallocate high-capacity nodes to busier quorums. We demonstrate that the dynamic grid quorum saves, on average, 6–12% energy when compared with the static configurations.

The failure of a single unreliable commodity component is very common in large-scale distributed storage systems. In order to ensure the reliability of data in large-scale distributed storage systems, many studies have emerged one after another. Among them, Erasure Codes are widely used in actual storage systems, such as Hadoop Distributed File System (HDFS), which provides high fault-tolerance with lower storage overhead. However, usually the recovery of erasure-coded storage system when encountering node failure results in severe cross-node and cross-rack bandwidth consumption, which affects the efficiency of failure recovery and wastes additional resources. In this paper, we improve the erasure coding storage strategy in distributed storage systems, and propose a low-overhead data recovery method based on cross-checking, namely HV-SNSP. In HV-SNSP, horizontal and vertical cross parity checking is realized by adding RS parity inside the data node, that is, <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', \textrm {}k')}}$ </tex-math></inline-formula> storage architecture. Based on <inline-formula> <tex-math notation="LaTeX">$\text{H}^{\mathrm {RS(n, \textrm {}k)}}$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\text{V}^{\mathrm {RS(n', k')}}$ </tex-math></inline-formula>, a low-cost supply node selection strategy, namely SNSP, is designed, and nodes with shorter network distance and lower load are selected to participate in recovery. This strategy can effectively reduce the amount of data transmission, shorten the recovery time, and improve the recovery efficiency. The experimental results show that compared with traditional RS, HV-SNSP can reduce the amount of cross-rack data transmission by 62.5&#x0025; during data recovery, and can shorten the recovery time by up to 42.41&#x0025;; Compared with D3, HV-SNSP can reduce the occupation of cross-rack bandwidth by 25&#x0025; and shorten the recovery time by 36.58&#x0025;.

In this paper, we consider two rack-aware storage systems. First, large-scale rack-aware storage system, which is very common in large-scale storage system, is a rack-aware storage system where all sizes of racks are at least the number of redundant nodes. For such storage system, we prove that any Maximum Distance Separable (MDS) codes can have optimal inter-rack repair bandwidth and give a closed-form representation of all repair schemes with optimal inter-rack repair bandwidth. Furthermore, we show that the optimal repair access and optimal inter-rack repair bandwidth can be attained simultaneously for such storage system. Second, we investigate the rack-aware storage system of all racks with the same size, which is called uniform rack-aware storage system. We prove that, except the trivial cases, we cannot attain optimal inter-rack repair bandwidth and optimal repair access for such storage system at the same time. Specifically, we establish the lower bound of repair access for a repair scheme with optimal interrack repair bandwidth, which is tight for some parameters, and also the tight lower bound of inter-rack repair bandwidth for a repair scheme with optimal repair access.

Large-scale storage systems are crucial components in data-intensive applications such as search engine clusters, video-on-demand servers, sensor networks and grid computing. A storage server typically consists of a set of storage devices. In such systems, data layouts may need to be reconfigured over time for load balancing or in the event of system failure/upgrades. It is critical to migrate data to their target locations as quickly as possible to obtain the best performance. Most of the previous results on data migration assume that each storage node can perform only one data transfer at a time. A storage node, however, can typically handle multiple transfers simultaneously and this can reduce the total migration time significantly. Moreover, storage devices tend to have heterogeneous capabilities as devices may be added over time due to storage demand increase. In this paper, we consider the heterogeneous data migration problem, where we assume that each storage node v has different transfer constraint cv, which represents how many simultaneous transfers v can handle. We develop algorithms to minimize the data migration time. We show that it is possible to find an optimal migration schedule when all cvs are even. Furthermore, though the problem is NP-hard in general, we give an efficient algorithm that offers a rigorous (1 + o(1))-approximation guarantee.

An effective decentralized metadata management schema is critical to a reliable and scalable large-scale distributed storage system.With regard to the defects of huge expansion cost and high sensitivity to cluster alteration borne in both the hash-based and subtree-based metadata management schemas, a metadata server (MDS) clustering schema based on consistent hash structure, named DSVL, is proposed.By introducing virtual MDS into the consistent MDS cluster, it can effectively balance MDS cluster load.Besides, the standby mechanism and the lazy update policy are merged and applied to the MDS cluster to accomplish rapid MDS failover as well as zero data migration in case of cluster alteration.The results of prototype system and simulation test prove that DSVL, with the features of balanced metadata distribution, rapid failover, flexible scalability and zero data migration amount in case of node alteration, can meet the demands for flexible and effective management of large-scale data storage systems with ever increasing data amount.

Hydrogen produced by electrolysis is one of the promising means by which to store electricity coming from renewables and to address its intermittency. With the growing global electricity share from renewable energy sources, large-scale storage systems are required. Alkaline electrolysis can be a sustainable solution for the large-scale energy storage system. However, high overpotential and slow kinetics particularly of the oxygen evolution reaction (OER) still remains as a hurdle to limit efficiency of the system. To enhance efficiency of alkaline electrolysis cells, catalysts which have high electrocatalytic activity for OER should be developed. Platinum-like behavior of Tungsten carbide in surface catalysis was first reported by Levy and Boudart in 1973[1]. In addition, tungsten carbide was reported to have Pt-like electronic structure by Colton in 1975[2]. After that, tungsten carbide has been studied widely and expected to substitute noble metal catalysts which are used in many electrochemical reactions. Here, we studied tungsten carbide as a support for 3d-transition metals and make bi-metallic surfaces for developing a high activity OER catalyst. First, WC-Co was synthesized by heat treatment. Carbon nitride (g-C3N4) and tungsten chloride were used as a carbide precursor and cobalt acetate was also used for helping carbonization of tungsten. After heat treatment, acidic leaching process was followed to eliminate cobalt. Through additional mixing with Nickel precursor and heat treatment, Ni/WC was synthesized and to compare electrocatalytic activity Ni/C was also prepared. In addition, many different Ni/WC which have different Ni to WC ratios were synthesized and measured in electrochemical way. Finally, the optimal ratio was determined and it show better electrocatalytic activity than precious catalysts and any other non-noble catalysts. Cyclic voltammetry was measured to evaluate OER activity in 1 M KOH in potential range from 1 to 1.8 V. Figure 1 shows that pure WC and Ni/C have 394 mV and 413 mV overpotentials at 10 mA∙cm-2 respectively. In Metal/WC cases, Cobalt/WC shows only 417 mV. But Nickel/WC shows 340 mV overpotential at 10 mA∙cm-2 and higher electrocatalytic activity for OER than RuO2 in alkaline media. η @ 10 mA cm-2 (mV) Ni/C 394 WC 413 Co/WC 417 Ni/WC 340 RuO2 355 Table1. Activities for OER at 10 mA cm-2 References [1] R. B. Levy, M. Boudart, Science, New series, Vol. 181, No. 4099 (1973) 547-579 [2] Richard J. Colton, Jan-Tsyu j. Huang, J.Wayne Rabalais, Chemical Physics Letters, Vol. 34, No. 2 (1975) 337-339 Figure1. Cyclic voltammetry for the oxygen evolution reaction in 1 M KOH of Ni/C, WC, Co/WC, Ni/WC, RuO2 Figure 1

Cloud computing systems require a huge number of storage servers, with growing implications for power bills, carbon emissions and the logistics of data centers. These considerations have motivated researchers to improve the energy efficiency of storage servers. Most servers use a lot of power irrespective of the amount of computing that they are doing, and one important goal is to redesign servers to be power-proportional, meaning that the power consumption is in step with the workload. Research on large-scale storage systems is hampered by their cost (Wang et al. 2014). It is therefore desirable to develop a scalable test-bed for evaluating the power consumption of large-scale storage systems. We are building on two open-source projects to construct a test-bed which will contribute to the assessment of power consumption in tiered storage systems.

Distributed storage systems provide large-scale data storage services, yet they are confronted with frequent node failures. To ensure data availability, a storage system often introduces data redundancy via replication or erasure coding. As erasure coding incurs significantly less redundancy overhead than replication under the same fault tolerance, it has been increasingly adopted in large-scale storage systems. In erasure-coded storage systems, degraded reads to temporarily unavailable data are very common, and hence boosting the performance of degraded reads becomes important. One challenge is that storage nodes tend to be heterogeneous with different storage capacities and I/O bandwidths. To this end, we propose FastDR, a system that addresses node heterogeneity and exploits I/O parallelism, so as to boost the performance of degraded reads to temporarily unavailable data. FastDR incorporates a greedy algorithm that seeks to reduce the data transfer cost of reading surviving data for degraded reads, while allowing the search of the efficient degraded read solution to be completed in a timely manner. We implement a FastDR prototype, and conduct extensive evaluation through simulation studies as well as testbed experiments on a Hadoop cluster with 10 storage nodes. We demonstrate that our FastDR achieves efficient degraded reads compared to existing approaches.

The development of stationary battery storage systems in Germany – status 2020

The development of stationary battery storage systems in Germany – A market review

The paper reviews and introduces the state-of-the-art nature-inspired metaheuristic algorithms in optimization, including Firefly algorithm, PSO algorithms and ABC algorithm. By implementing these algorithms in Matlab, we will use worked examples to show how each algorithm works. Firefly algorithm is one of the evolutionary optimization algorithms, and is inspired by the flashing behaviour of fireflies in nature. There are many noisy non-linear mathematical optimization problems that can be effectively solved by Metaheuristic Algorithms. Mathematical optimization or programming is the study of such planning and design problems using mathematical tools. Nowadays, computer simulations become an indispensable tool for solving such optimization problems with various efficient search algorithms. Nature-inspired algorithms are among the most powerful algorithms for optimization. Firefly algorithm is one of the new metaheuristic algorithms for optimization problems. The algorithm is inspired by the flashing behaviour of fireflies. A Firefly Algorithm (FA) is a recent nature inspired optimization algorithm, which simulates the flash pattern and characteristics of fireflies. It is a powerful swarm intelligence algorithm inspired by the flash phenomenon of the fireflies. In this context, three types of meta-heuristics called Artificial bee Colony algorithm, Particle Swarm Optimization (PSO) and Firefly algorithms were devised to find optimal solutions of noisy non-linear continuous mathematical models. A series of computational experiments using each algorithm were conducted. The stimulation result of this experiment were analyzed and compared to the best solutions found so The Firefly algorithm in each noisy non linear optimization function seems to perform better and efficient.

Due to consistent improvements in memory and processor technology, object storage devices (OSDs) have greater memory space and more powerful processing power, which allow the OSDs to execute user-defined programs. Shifting part of an application's processing to the disk drives drops the amount of data transferred across the network and explores the parallelism of large-scale distributed storage systems, reducing the execution time for many basic data analytics tasks. In this paper, we propose a large-scale object-based active storage platform, named Gem, for data analytics in the internet of things (IoT). All data from the IoT that resides in disk drives form objects with attributes, methods and policies. For some applications such as data analytics, application-specific operations are executed by the drive processors. In this way, only the results are returned to clients, rather than data files being read by the clients. Therefore, the platform Gem is able to greatly reduce the overhead of data analytics applications in the Internet of Things. By conducting performance evaluation, experimental results demonstrate the effectiveness and scalability of Gem.

At present, global climate change is intensifying and climate extreme events are occurring frequently, so the new energy system represented by the combined wind and storage system will play an important role in the future decarbonization process. Based on the life cycle analysis method, this study constructs a carbon footprint accounting model for large-scale combined wind and storage systems. The results show that the emission hotspot of the onshore wind power system is the component manufacturing stage, and the emission hotspot of the energy storage system is the operation and maintenance stage. This study provides a basis for the accurate calculation of the carbon emissions of the combined wind and storage system. It provides technical support for realizing the goals of “carbon peaking” and “carbon neutrality”.

Data migration schemes are critical to balance the load in storage clusters for performance improvement. However, as NAND flash based SSDs are widely deployed in storage systems, extending the lifespan of SSD storage clusters becomes a new challenge for data migration. Prior approaches designed for HDD storage clusters, however, are inefficient due to excessive write amplification during data migration, which significantly decrease the lifespan of SSD storage clusters. To overcome this problem, we propose EDM, an endurance aware data migration scheme with careful data placement and movement to minimize the data migrated, so as to limit the worn-out of SSDs while improving the performance. Based on the observation that performance degradation is dominated by the wear speed of an SSD, which is affected by both the storage utilization and the write intensity, two complementary data migration policies are designed to explore the trade-offs among throughput, response time during migration, and lifetime of SSD storage clusters. Moreover, we design an SSD wear model and quantitatively calculate the amount of data migrated as well as the sources and destinations of the migration, so as to reduce the write amplification caused by migration. Results on a real storage cluster using real-world traces show that EDM performs favorably versus existing HDD based migration techniques, reducing cluster-wide aggregate erase count by up to 40%. In the meantime, it improves the performance by 25% on average compared to the baseline system which achieves almost the same effectiveness of performance improvement as previous migration techniques.

Optimal expansion of medium-voltage power networks is a common issue in electrical distribution planning. Minimizing the total cost of the objective function with technical constraints make it a combinatorial problem which should be solved by powerful optimization algorithms. In this paper, a new improved hybrid Tabu search/particle swarm optimization algorithm is proposed to optimize the electric expansion planning. The proposed method is analyzed both mathematically and experimentally and it is applied to three different electric distribution networks as case studies. Numerical results and comparisons are presented and show the efficiency of the proposed algorithm. As a result, the proposed algorithm is more powerful than the other algorithms, especially in larger dimension networks.