EDM: An Endurance-Aware Data Migration Scheme for Load Balancing in SSD Storage Clusters

This paper analyzes the problems of load balance caused by data access in parallel file systems and gives an accurate way to estimate the load of Data Servers. On the base of this evaluation, the paper proposes an adaptive loading data migration (ALDM) strategy to balance the load of data servers. Experiment results show ALDM has little impact on system performance during data migration, and the performance improves about 10% after data migration completed. Furthermore, ALDM also has good adaptability to various loads in different storage systems.

SideIO: A Side I/O system framework for hybrid scientific workflow

In large scale data storage systems, RAID-6 has received more attention due to its capability to tolerate concurrent failures of any two disks, providing a higher level of reliability. However, a challenging issue is its scalability, or how to efficiently expand the disks. The main reason causing this problem is the typical fault tolerant scheme of most RAID-6 systems known as Maximum Distance Separable (MDS) codes, which offer data protection against disk failures with optimal storage efficiency but they are difficult to scale. To address this issue, we propose a novel Stripe-based Data Migration (SDM) scheme for large scale storage systems based on RAID-6 to achieve higher scalability. SDM is a stripe-level scheme, and the basic idea of SDM is optimizing data movements according to the future parity layout, which minimizes the overhead of data migration and parity modification. SDM scheme also provides uniform data distribution, fast data addressing and migration. We have conducted extensive mathematical analysis of applying SDM to various popular RAID-6 coding methods such as RDP, P-Code, H-Code, HDP, X-Code, and EVENODD. The results show that, compared to existing scaling approaches, SDM decreases more than 72.7% migration I/O operations and saves the migration time by up to 96.9%, which speeds up the scaling process by a factor of up to 32.

We have proposed a distributed storage system which aims to utilize storage fragments distributed in network nodes without administrators' maintenance cost. Even the performance of storage devices are not uniform, autonomously data block migration helps to achieve enhancing the whole I/O performance of the storage system. However, we have considered all the storage devices to be reliable equally. On this assumption, migration of an important data block to unstable storage might cause degradation of the availability of the data block. In this paper, we propose a scheme to choose the destination of data block migration by evaluating the nodes' reliability along with the performance of storage devices. This scheme aims to improve system stability by preventing to locate a critical data onto an unstable storage device. The storage nodes which lack stability are also utilized for storing a data block which can be easily regenerated or duplicated. The results of the preliminary investigation show that a large number of warning logs from Machine Check Architecture may suggest a sign of unexpected system shutdown.

Large-scale storage systems are crucial components in data-intensive applications such as search engine clusters, video-on-demand servers, sensor networks and grid computing. A storage server typically consists of a set of storage devices. In such systems, data layouts may need to be reconfigured over time for load balancing or in the event of system failure/upgrades. It is critical to migrate data to their target locations as quickly as possible to obtain the best performance. Most of the previous results on data migration assume that each storage node can perform only one data transfer at a time. A storage node, however, can typically handle multiple transfers simultaneously and this can reduce the total migration time significantly. Moreover, storage devices tend to have heterogeneous capabilities as devices may be added over time due to storage demand increase. In this paper, we consider the heterogeneous data migration problem, where we assume that each storage node v has different transfer constraint cv, which represents how many simultaneous transfers v can handle. We develop algorithms to minimize the data migration time. We show that it is possible to find an optimal migration schedule when all cvs are even. Furthermore, though the problem is NP-hard in general, we give an efficient algorithm that offers a rigorous (1 + o(1))-approximation guarantee.

Multi-tiered storage systems today are integrating Solid State Disks (SSD) on top of traditional rotational hard disks for performance enhancement due to the significant IO improvements in SSD technology. It is widely recognized that automated data migration between SSD and HDD plays a critical role in effective integration of SSD into multi-tiered storage systems. Furthermore, effective data migration has to take into account of application specific workload characteristics, deadlines, and IO profiles. An important and interesting challenge for automated data migration in multi-tiered storage systems is how to fully release the power of data migration while guaranteeing the migration deadline is critical to maximizing the performance of SSD-enabled multi-tiered storage system. In this paper, we present an adaptive look ahead data migration model that can incorporate application specific characteristics and I/O profiles as well as workload deadlines. Our adaptive data migration model has three unique features. First, it incorporates a set of key factors that may impact on the performance of look ahead migration efficiency in our formal model develop. Second, our data migration model can adaptively determine the optimal look ahead window size, based on several parameters, to optimize the effectiveness of look ahead migration. Third, we formally and experimentally show that the adaptive data migration model can improve overall system performance and resource utilization while meeting workload deadlines. Through our trace driven experimental study, we compare the adaptive look ahead migration approach with the basic migration model and show that the adaptive migration model is effective and efficient for continuously improving and tuning of the performance and scalability of multi-tier storage systems.

Using phase change memory (PCM) as main memory for embedded systems, such as Beidou navigation terminals, has many advantages, such as non-volatile, energy saving, anti-crack and so on. However, the problem of write operations needs to be solved to prevent premature failure of the PCM chips. In this paper, the data allocation problem of a hybrid PCM and DRAM main memory in Beidou navigation terminals is studied. A data allocation and migration strategy between PCM and DRAM is proposed to effectively reduce the write operations on PCM, thus extending the lift endurance of the terminals without affecting the real-time performance.

Cloud computing and big data: Technologies and applications

As distributed storage clusters have been used more and more widely in recent years, data replication management has become a hot research topic. In storage clusters, internal network bandwidth is usually a scarce resource. Misplaced replicas may take up too much network bandwidth and greatly deteriorate the overall performance of the cluster. Based on multi-objective evolutionary algorithm(MOEA), we developed a replication management scheme to improve performance by reducing internal network traffic and balancing load of storage clusters. The replica placement problem is formulated as some multi-objective programming optimization problems and solved by MOEA to get a Pareto solution set. Based on the average access time, a method is proposed to pick out the suitable solution from the set. Those suitable solutions are gathered to form the final replica location. Then we propose a method to make adjustments step by step according to the replica location. A method to reduce problem size is also proposed. MRMS is evaluated by the access history from a distributed storage cluster of Xunlei Inc. The experimental results show that MRMS can effectively improve the overall performance of the storage cluster.

An algorithmic mapping of storage locations brings high storage efficiency to the storage system, but the loss of efficient scheduling makes systems prone to crashing at low usage. This paper uses the Ceph storage system as a research sample to analyze these issues and proposes a grading mapping adaptive storage resource collaborative optimization mechanism. This approach grading both the storage device and the storage content, and introduced random factors and influence factors as two-factors to quantify the grading mapping relationship between the two of them. This relation coordinates the storage systems’ performance and reliability. In addition, a collaborative storage algorithm is proposed to realize balanced storage efficiency and control data migration. The experimental results show that in comparison with the inherent mechanism in the traditional Ceph system, the proposed cooperative storage adaptation mechanism for data balancing has increased the average system usage by 17% and reduces data migration by 50% compared to the traditional research approach.

A modern service-oriented distributed storage solution

In this paper, an incremental data migration technique is evaluated in the multi-database system based on MySQL with SPIDER storage engine and the improvement of the turn-around times of other operations is proved in this systems. In this method, a large data migration is divided into small data migrations and other operations are inserted between these small data migrations. This technique is easy to implement in the multi-database system.

At present, there are multiple sets of information systems in substation, which are relatively independent, and exist some problems of repeated collection, low utilization rate and difficult data sharing. These systems contain not only “four remote” structured data, but also unstructured data such as waveform, model and graphics. There is no unified storage and access interface established which hinders the improvement of intelligent level of advanced applications. To solve the above problems, a unified storage technology of multidimensional historical data of substation based on Redis and SSDB is proposed in this paper. The system incldes the Redis proxy server and the SSDB storage cluster. In the proxy server, the SCADA historical data and the unstructured historical data of all substations in a region are stored in the key-value network cache library by the unified import tool supporting multi vendor and multi service history data. Then, through the proxy service program, the cache data is transferred to the distributed storage cluster. For each storage machine of the storage cluster, the cache data is stored in the disk file through the LevelDB storage engine, and the distributed permanent storage of the historical data is realized. The laboratory tests have proved that the functional requirements and performance index of the unified data storage system have reached the expected goal.

Data migration and batch processing remain rudimentary processes in database systems while dealing with enormous volumes of data from multiple sources. Even before running Extract, Transform, Load (ETL) on parallel architectures, extensive querying and performance in a way challenges are required. Real-time data analysis, together with data aggregation and transformation, remains a problem for decision-making since data warehouses retain historical data and update it on a regular basis. However, optimization lessens resource consumption as well as ensures parallel processing efficiency while reducing the time window. The ultimate purpose of this paper is to instantly improve the performance of parallel data loading through concurrency. The conducted analysis shows concurrency on the Oracle database significantly improves the performance gain along with data loading time.

Considerable effort has been devoted to the migration of multicomponent data in elastic media with wavefield separation techniques being the most successful. Most of this work has been carried out in the time domain. In this paper, we formulate a multicomponent migration technique in the frequency domain. Reverse-time migration can be viewed as the zero-lag cross-correlation between virtual source and back-propagated wavefields. Cross-correlating the Helmholtz decomposed wavefields rather than directly correlating the vector displacement fields results in sharper, more interpretable images, contaminated by fewer crosstalk artefacts. The end products are separate P and S wave (and if desired, PS and SP) migration images. We test our migration algorithm on synthetic seismic data generated using the SEG/EAGE salt-dome, Overthrust and Marmousi-2 models. We correctly image the location and shape of the target zone for oil exploration using these data sets. Furthermore, we demonstrate that our new migration technique provides good images even when the initial velocity model is only approximate.