QoS-aware replica placement for data intensive applications

In data grids, data replication on variant nodes can change some problems such as response time and availability. Also, in data replication, there are some challenges to finding the best replica efficiently in relation to performance and location of physical storage systems. In this paper, various replica placement strategies are discussed. These replica placement strategies are available in the works. Replica placement contains recognizing the best possible node to duplicate data based on network latency and user request. These strategies measure and analyze different parameters such as access cost, bandwidth consumption, scalability, execution time and storage consumption. This paper also analyses the performance of various strategies with respect to the parameters mentioned above in data grid.

Data grid integrates geographically distributed resources for solving data-sensitive scientific applications. As tasks are sensitive to data, dealing with large amount of data makes the requirement for efficiency in data access more critical. The goal of replica placement is to shorten data access time for enhancing the task execution performance. Therefore, replica placement strategies are often integral to task scheduling algorithms. However, all existing integration strategies make an assumption that the disk space of resources in data grid is unlimited. In this paper, we extended MinMin heuristic to cater to the situation where the disk space of a computational resource is limited. In addition, a heuristic replica placement algorithm is proposed, in which the limited disk space of a storage resource is considered as well. Another character of this heuristic replica placement algorithm is that it can map more than one hot file to several storage resources. We study our approach and evaluate it through simulation. The result shows that the integration of the two algorithms has improved the performance of data grid especially when the whole disk space of storage resources is relatively smaller than the amount of all data files.

Data grids support distributed data-intensive applications that need to access massive datasets stored around the world. Ensuring efficient access to such datasets is hindered by the high latencies of wide-area networks. To speed up access, files can be replicated so a user can access a nearby replica. Replication also provides improved availability, decreased bandwidth use, increased fault tolerance, and improved scalability. Since a grid environment is dynamic, resource availability, network latency, and user requests may change. To address these issues a dynamic replica placement strategy that adapts to changing behaviour is needed. In this paper, we introduce a highly distributed replica placement algorithm for hierarchical data grids. Our algorithm exploits data access histories to identify popular files and determines optimal replication locations to improve access performance by minimizing replication overhead (access and update) assuming a given traffic pattern. The problem is formulated using dynamic programming. We evaluate our algorithm using the OptorSim simulator and find that it offers shorter execution time and reduced bandwidth consumption compared to other dynamic replica placement methods.

Data replication is an excellent technique to move and cache data close to users. By replication, data access performance can be improved dramatically. One of the challenges in data replication is to select the candidate sites where replicas should be placed. We use a multi-objective model to address the replica placement problem. The multi-objective model considers the objectives of p-median and p-center models simultaneously to select the candidate sites that will host replicas. The objective of the p-median model is to find the locations of p possible candidate replication sites by optimizing total (or average) response time; where the p-center model finds p candidate sites by optimizing maximum response time. A grid environment is highly dynamic so user requests and network latency vary constantly. Therefore, candidate sites currently holding replicas may not be the best sites to fetch replica on subsequent requests. We propose a dynamic replica maintenance algorithm that re-allocates to new candidate sites if a performance metric degrades significantly over last K time periods. Simulation results demonstrate that the dynamic maintenance algorithm with multi-objective static placement decision performs best in dynamic environments like data grids.

Optimizing the use of grid resources is critical for users to effectively exploit a Data Grid (DG). Data replication is considered as a major technique for increasing access performance and data availability in DG systems. Current works on data replications in Grid systems focuses on infrastructure for replication mechanism for creating or deleting replicas. One of the challenges in data replication is determining Optimal Number of Replicas (ONR) with Quality of Service (QoS) assurance, as well as their Optimal Location of Replicas (OLR) in DG. In this paper, we propose an algorithm that finds ONR of an object over DG systems, such that the overall communication and storage cost is minimized. This algorithm ensures that QoS required from the users are satisfied. In addition, we proposed a sketch of the proof for our algorithm and its integrity.

Replica placement in ring based content delivery networks

A data replication algorithm for groups of files in data grids

In distributed mobile systems, communication cost and disconnections are major concerns. In this paper, we address replica placement issues to achieve improved performance for systems supporting mobile transactions. We focus on handling correlated data objects and disconnections. Frequently, requests and/or transactions issued by mobile clients may access multiple data objects and should be considered together in terms of replica allocation. We discuss the replication cost model for correlated data objects and show that the problem of finding an optimal solution is NP. We further adjust the replication cost model for disconnections. A heuristic "expansion-shrinking" algorithm is developed to efficiently make replica placement decisions. The algorithm obtains near optimal solutions for the correlated data model and yields significant performance gains when disconnection is considered. Experimental studies show that the heuristic expansion-shrinking algorithm significantly outperforms the general frequency-based replication schemes.

Enhanced Dynamic Hierarchical Replication and Weighted Scheduling Strategy in Data Grid

In recent years data grids have been deployed and grown in many scientific experiments and data centers. The deployment of such environments has allowed grid users to gain access to a large number of distributed data. Data replication is a key issue in a data grid and should be applied intelligently because it reduces data access time and bandwidth consumption for each grid site. Therefore this area will be very challenging as well as providing much scope for improvement. In this paper, we introduce a new dynamic data replication algorithm named Popular File Group Replication, PFGR which is based on three assumptions: first, users in a grid site (Virtual Organization) have similar interests in files and second, they have the temporal locality of file accesses and third, all files are read-only. Based on file access history and first assumption, PFGR builds a connectivity graph for a group of dependent files in each grid site and replicates the most popular group files to the requester grid site. After that, when a user of that grid site needs some files, they are available locally. The simulation results show that our algorithm increases performance by minimizing the mean job execution time and bandwidth consumption and avoids unnecessary replication.

The data grid is one of the infrastructures for data and storage resource management. Data replication can improve data access performance, fault tolerance, and reduce network transmission bandwidth. As the site storage space is limited in the grid, how to effectively use grid resources become an important challenge. This paper introduces a dynamic grid replication algorithm based on popularity support and confidence (BPSC). Through the algorithm, data and its associated copy can be placed on a suitable site, with reduced access latency. Through Optorsim, simulation results show that the algorithm can provide better performance compared with other algorithms Grid computing is an important branch of distributed computing. Grid computing constructs a virtual computer system through using a different location of the large number of heterogeneous resources (CPU and storage), provides the model to solve the problem of large scale computing and storage. The data grid is mainly the grid for data storage management. The data grid is defined as an effective combination of data and computing resources of distributed systems by Data Grid Group. The large-scale data- intensive applications, such as bioinformatics, high energy physics, space information generated vast amounts of data, these data may be TB scale, even up to the scale of PB. It is not practical to maintain large scale data at each site, and provide computing resources for visitors. At the same time, it will lead to increased access latency through a centralized way to storing and accessing these data from each site. Data replication technology is an effective method to reduce access latency in the data grid system. Data replication is one of the basic methods of distributed systems to improve the efficiency of system access, maintenance, system high availability. Data replication is to store data in the appropriate site for users to effective access to massive data. Data replication can reduce data access latency, reduce network bandwidth consumption, avoid congestion, adjust the load balancing in server-side, improve system reliability. Many data replication algorithm can reduce data access latency, improve system reliability, but which data will be copied, when the data will be copied at, where the data will be copied are the basic means of determining data replication algorithm. In this paper, we introduced a real-time dynamic data replication algorithm. The algorithm is based on the popularity of file dynamic to replicate data, and consider the storage space limitations of the site. Through optorsim simulation verified the algorithm. This paper is organized as follows: Section 2 is about related works on replication in data grids. In Section 3, system model and component in system are presented. The file

Cloud computing environment is getting more interesting as a new trend of data management. Data replication has been widely applied to improve data access in distributed systems such as Grid and Cloud. However, due to the finite storage capacity of each site, copies that are useful for future jobs can be wastefully deleted and replaced with less valuable ones. Therefore, it is considerable to have appropriate replication strategy that can dynamically store the replicas while satisfying quality of service (QoS) requirements and storage capacity constraints. In this paper, we present a dynamic replication algorithm, named hierarchical data replication strategy (HDRS). HDRS consists of the replica creation that can adaptively increase replicas based on exponential growth or decay rate, the replica placement according to the access load and labeling technique, and finally the replica replacement based on the value of file in the future. We evaluate different dynamic data replication methods using CloudSim simulation. Experiments demonstrate that HDRS can reduce response time and bandwidth usage compared with other algorithms. It means that the HDRS can determine a popular file and replicates it to the best site. This method avoids useless replications and decreases access latency by balancing the load of sites.

We take an availability-centric view on quality of service (QoS) and propose a model and mechanisms for studying the effectiveness of realistic replication schemes on availability QoS for peer-to-peer (P2P) systems. We especially tackle the dynamic replica placement (RP) problem where our focus is on choosing dynamically the number and location of replicas while (1) meeting different availability QoS requirements for all individual peers and (2) taking the intermittent connectivity of peers explicitly into account. We model P2P systems as a dynamic stochastic graph in which the nodes go up and down depending on their assigned up probability. We develop some simple heuristic algorithms for solving the RP problem, which are fully distributed and adaptive. Through an event-driven simulation study we compare and evaluate the achieved availability QoS of the proposed RP algorithms. Simulation results show that (1) even simple heuristics can achieve reasonably high availability QoS, and (2) satisfying availability QoS requires more replicas than for only increasing the hit rate.

Optimized QoS-aware replica placement heuristics and applications in astronomy data grid

Distributed storage systems rely heavily on redundancy to ensure data availability as well as durability. In networked systems subject to intermittent node unavailability, the level of redundancy introduced in the system should be minimized and maintained upon failures. Repairs are well-known to be extremely bandwidth-consuming and it has been shown that, without care, they may significantly congest the system. In this paper, we propose an approach to redundancy management accounting for nodes heterogeneity with respect to availability. We show that by using the availability history of nodes, the performance of two important faces of distributed storage (replica placement and repair) can be significantly improved. Replica placement is achieved based on complementary nodes with respect to nodes availability, improving the overall data availability. Repairs can be scheduled thanks to an adaptive per-node timeout according to node availability, so as to decrease the number of repairs while reaching comparable availability. We propose practical heuristics for those two issues. We evaluate our approach through extensive simulations based on real and well-known availability traces. Results clearly show the benefits of our approach with regards to the critical trade-off between data availability, load-balancing and bandwidth consumption.