Combining Data Replication Algorithms and Job Scheduling Heuristics in the Data Grid

The data grid is one of the infrastructures for data and storage resource management. Data replication can improve data access performance, fault tolerance, and reduce network transmission bandwidth. As the site storage space is limited in the grid, how to effectively use grid resources become an important challenge. This paper introduces a dynamic grid replication algorithm based on popularity support and confidence (BPSC). Through the algorithm, data and its associated copy can be placed on a suitable site, with reduced access latency. Through Optorsim, simulation results show that the algorithm can provide better performance compared with other algorithms Grid computing is an important branch of distributed computing. Grid computing constructs a virtual computer system through using a different location of the large number of heterogeneous resources (CPU and storage), provides the model to solve the problem of large scale computing and storage. The data grid is mainly the grid for data storage management. The data grid is defined as an effective combination of data and computing resources of distributed systems by Data Grid Group. The large-scale data- intensive applications, such as bioinformatics, high energy physics, space information generated vast amounts of data, these data may be TB scale, even up to the scale of PB. It is not practical to maintain large scale data at each site, and provide computing resources for visitors. At the same time, it will lead to increased access latency through a centralized way to storing and accessing these data from each site. Data replication technology is an effective method to reduce access latency in the data grid system. Data replication is one of the basic methods of distributed systems to improve the efficiency of system access, maintenance, system high availability. Data replication is to store data in the appropriate site for users to effective access to massive data. Data replication can reduce data access latency, reduce network bandwidth consumption, avoid congestion, adjust the load balancing in server-side, improve system reliability. Many data replication algorithm can reduce data access latency, improve system reliability, but which data will be copied, when the data will be copied at, where the data will be copied are the basic means of determining data replication algorithm. In this paper, we introduced a real-time dynamic data replication algorithm. The algorithm is based on the popularity of file dynamic to replicate data, and consider the storage space limitations of the site. Through optorsim simulation verified the algorithm. This paper is organized as follows: Section 2 is about related works on replication in data grids. In Section 3, system model and component in system are presented. The file

More and more large amounts of data shared with researchers and workstations worldwide are generated by scientific experiments and engineering applications. High performance grids aim at facilitating the distribution of such data to geographically remote places. For high performance data grids where users require remote access to large files, dynamic replication is an efficient method to improve performance. In this paper, Fast Cascading, a new dynamic replication algorithm, is proposed for the multi-tier Data Grid. The simulation results obtained by Simulator OptorSim which has been modified show that this algorithm can reduce the average response time of data access greatly for random access pattern compared with other algorithms,such as Fast Spread, LRU and Economy-Zipf dynamic replication strategy, and the gain increases with the number of jobs submitted.

Replication is a widely used method for achieving high availability in database systems. The data replication approach can be used for solutions such as sharing data among remote offices, sharing data among dispersed users, make server data more accessible, distribute solution updates, back up data, provide internet or intranet replication. The main aim of this research is to provide a better technique for solving the drawbacks that currently exist in the literary works of data replica environment. Here, we intend to propose data replication system based on data mining techniques. That is in proposed method FDDRA (data mining based dynamic replication algorithm), we will use a data mining with combination of replication algorithm and data grid policy for data replication in the data mining environment through monitoring all data process. The data replication will be done by identifying the frequently used data patterns in the large database of a node. This will be done by Apriorialgorithm. The system availability and replica will be measured and identifying the location in which the replicated data will be stored. The result shows the efficiency of our algorithm in terms latency time and shortest path of nodes.

We present two distributed algorithms for dynamic replication of a data-item in communication networks. The algorithms are adaptive in the sense that they change the replication scheme of the item (i.e. the set of processors at which the data-item is replicated), as the read-write pattern of the processors in the network changes. Each algorithm continuously moves the replication scheme towards an optimal one, where optimality is defined with respect to different objective functions. One algorithm optimizes the communication cost objective function, and the other optimizes the communication time. We also provide a lower bound on the performance of any dynamic replication algorithm.

A data replication algorithm for groups of files in data grids

In recent years data grids have been deployed and grown in many scientific experiments and data centers. The deployment of such environments has allowed grid users to gain access to a large number of distributed data. Data replication is a key issue in a data grid and should be applied intelligently because it reduces data access time and bandwidth consumption for each grid site. Therefore this area will be very challenging as well as providing much scope for improvement. In this paper, we introduce a new dynamic data replication algorithm named Popular File Group Replication, PFGR which is based on three assumptions: first, users in a grid site (Virtual Organization) have similar interests in files and second, they have the temporal locality of file accesses and third, all files are read-only. Based on file access history and first assumption, PFGR builds a connectivity graph for a group of dependent files in each grid site and replicates the most popular group files to the requester grid site. After that, when a user of that grid site needs some files, they are available locally. The simulation results show that our algorithm increases performance by minimizing the mean job execution time and bandwidth consumption and avoids unnecessary replication.

This paper makes two contributions. First, we introduce a model for evaluating the performance of data allocation and replication algorithms in distributed databases. The model is comprehensive in the sense that it accounts for I/O cost, for communication cost, and, because of reliability considerations, for limits on the minimum number of copies of the object. The model captures existing replica-management algorithms, such as read-one-write-all, quorum-consensus, etc. These algorithms are static in the sense that, in the absence of failures, the copies of each object are allocated to a fixed set of processors. In modern distributed databases, particularly in mobile computing environments, processors will dynamically store objects in their local database and will relinquish them. Therefore, as a second contribution of this paper, we introduce an algorithm for automatic dynamic allocation of replicas to processors. Then, using the new model, we compare the performance of the traditional read-one-write-all static allocation algorithm to the performance of the dynamic allocation algorithm. As a result, we obtain the relationship between the communication cost and I/O cost for which static allocation is superior to dynamic allocation, and the relationships for which dynamic allocation is superior.

Optimizing the use of grid resources is critical for users to effectively exploit a Data Grid (DG). Data replication is considered as a major technique for increasing access performance and data availability in DG systems. Current works on data replications in Grid systems focuses on infrastructure for replication mechanism for creating or deleting replicas. One of the challenges in data replication is determining Optimal Number of Replicas (ONR) with Quality of Service (QoS) assurance, as well as their Optimal Location of Replicas (OLR) in DG. In this paper, we propose an algorithm that finds ONR of an object over DG systems, such that the overall communication and storage cost is minimized. This algorithm ensures that QoS required from the users are satisfied. In addition, we proposed a sketch of the proof for our algorithm and its integrity.

Secret distribution and erasure convention - based approaches have been used in distributed storage systems to provide the confidentiality, integrity, and availability of critical information. To achieve performance goals in data accesses, these data fragmentation approaches can be combined with dynamic replication. In this paper, we consider data partitioning (both secret distribution and erasure convention) and dynamic replication in data grids, in which security and data access performance are critical issues. More specifically, we investigate the problem of optimal allocation of sensitive data objects that are divided by using secret distribution scheme or erasure convention scheme and/or replicated. The grid topology we consider consists of two layers. In the upper layer, multiple clusters form a network topology that can be represented by a general graph. The topology within each cluster is represented by a tree graph. We decompose the share replica allocation problem into two sub problems: the Optimal Inter cluster Resident Set Problem (OIRSP) that determines which clusters share replicas and the Optimal Intra need cluster Share Allocation Problem (OISAP) that determines the number of share replicas needed in a cluster and their placements. We develop two heuristic algorithms for the two sub problems.

In this paper, we propose an adaptive data replication algorithm, called the Popular File Replicate First algorithm (PFRF for short), which is developed on a star-topology data grid with limited storage space based on aggregated information on previous file accesses. The PFRF periodically calculates file access popularity to track the variation of usersi¦ access behaviour behaviors, and then replicates popular files to appropriate sites to adapt to the variation. We employ several types of file access behaviors, including Zipf-like, geometric, and uniform distributions, to evaluate PFRF. The simulation results show that PFRF can effectively improve average job turnaround time and data availability as compared with those of the tested algorithms.

Data Grid environment is a geographically distributed that deal with date-intensive application in scientific and enterprise computing. In data-intensive applications data transfer is a primary cause of job execution delay. Data access time depends on bandwidth, especially when hierarchy of bandwidth appears in network. Effective job scheduling can reduce data transfer time by considering hierarchy of bandwidth and also dispatching a job to where the needed data are present. Additionally, replication of data from primary repositories to other locations can be an important optimization step to reduce the frequency of remote data access. Objective of dynamic replica strategies is reducing file access time which leads to reducing job runtime. In this paper we develop a job scheduling policy, called TLSS (Two level Scheduling strategy), and a dynamic data replication strategy, called TLRS (Two level Replication Strategy), to improve the data access efficiencies in a cluster grid. We study our approach and evaluate it through simulation. The results show that combination of TLSS and TLRS has improved 17% over other combinations.

The phenomenal growth of the object has brought huge increase in the traffic on World Wide Web. Long read latency of service experienced by the end-users, especially during the peak hours, continues to be the common problem to the popular web servers while retrieving popular objects. A replication and deletion algorithm is presented to solve object replication problem, which is a NP-hard problem and can be formulated as a 0-1 constraint optimization problem. The algorithm defines an appropriate replica distribution with the objective to decrease the number of object access operations over excessive distances, balance the load of popular web servers, and then minimize the network traffic of the object. Object deletion in a distributed web-server environment is one of the possible solutions to extend available memory and reduce the load of the server. During the adaptive time period due to different object update frequency, objects are dynamically duplicated and deleted in a distributed web server system with limited storage capacity. Simulation results demonstrate the performance of the proposed algorithm.

In data grids, data replication on variant nodes can change some problems such as response time and availability. Also, in data replication, there are some challenges to finding the best replica efficiently in relation to performance and location of physical storage systems. In this paper, various replica placement strategies are discussed. These replica placement strategies are available in the works. Replica placement contains recognizing the best possible node to duplicate data based on network latency and user request. These strategies measure and analyze different parameters such as access cost, bandwidth consumption, scalability, execution time and storage consumption. This paper also analyses the performance of various strategies with respect to the parameters mentioned above in data grid.

The network technologies and storage devices are developing quickly and the technical devices are inexpensive. Therefore, users have a large amount of storage space, Data Grid can collect distributed storage device to share to other users. In this paper, we discuss several file systems that consist of NFS, SRB, iRODS, Lustre, Gfarm and GPFS-WAN. In Data Grid, we select the specific file system to administrate the storage devices. We must consider the file system that obtains scalibilty, security and stability. Finally, we choose two file systems that consist of Gfarm and GPFS-WAN to be our evaluative target. Gfarm and GPFS-WAN are used to build up large-scale dataset storage system that consists of PRAGMA and TeraGrid, respectively. In this paper, we use Gfarm and GPFS-WAN to create a Data Grid environment and evaluate the performance of data transmission, respectively. As results, the GPFS-WAN's performance of data transmission is better than Gfarm. The data transmission is one of the factors in Data Grid environment. For instance, the system obtains replication, open source software, easy to install and maintain that is important to be a great Data Grid environment, these important information will be discussed in this paper.

Cloud computing environment is getting more interesting as a new trend of data management. Data replication has been widely applied to improve data access in distributed systems such as Grid and Cloud. However, due to the finite storage capacity of each site, copies that are useful for future jobs can be wastefully deleted and replaced with less valuable ones. Therefore, it is considerable to have appropriate replication strategy that can dynamically store the replicas while satisfying quality of service (QoS) requirements and storage capacity constraints. In this paper, we present a dynamic replication algorithm, named hierarchical data replication strategy (HDRS). HDRS consists of the replica creation that can adaptively increase replicas based on exponential growth or decay rate, the replica placement according to the access load and labeling technique, and finally the replica replacement based on the value of file in the future. We evaluate different dynamic data replication methods using CloudSim simulation. Experiments demonstrate that HDRS can reduce response time and bandwidth usage compared with other algorithms. It means that the HDRS can determine a popular file and replicates it to the best site. This method avoids useless replications and decreases access latency by balancing the load of sites.