Enhanced Dynamic Hierarchical Replication and Weighted Scheduling Strategy in Data Grid

Data Grid environment is a geographically distributed that deal with date-intensive application in scientific and enterprise computing. In data-intensive applications data transfer is a primary cause of job execution delay. Data access time depends on bandwidth, especially when hierarchy of bandwidth appears in network. Effective job scheduling can reduce data transfer time by considering hierarchy of bandwidth and also dispatching a job to where the needed data are present. Additionally, replication of data from primary repositories to other locations can be an important optimization step to reduce the frequency of remote data access. Objective of dynamic replica strategies is reducing file access time which leads to reducing job runtime. In this paper we develop a job scheduling policy, called TLSS (Two level Scheduling strategy), and a dynamic data replication strategy, called TLRS (Two level Replication Strategy), to improve the data access efficiencies in a cluster grid. We study our approach and evaluate it through simulation. The results show that combination of TLSS and TLRS has improved 17% over other combinations.

Data grids deal with a huge amount of data regularly. It is a fundamental challenge to ensure efficient accesses to such widely distributed data sets. Creating replicas to a suitable site by data replication strategy can increase the system performance. It shortens the data access time and reduces bandwidth consumption. In this paper, a dynamic data replication mechanism called Latest Access Largest Weight (LALW) is proposed. LALW selects a popular file for replication and calculates a suitable number of copies and grid sites for replication. By associating a different weight to each historical data access record, the importance of each record is differentiated. A more recent data access record has a larger weight. It indicates that the record is more pertinent to the current situation of data access. A Grid simulator, OptorSim, is used to evaluate the performance of this dynamic replication strategy. The simulation results show that LALW successfully increases the effective network usage. It means that the LALW replication strategy can find out a popular file and replicates it to a suitable site without increasing the network burden too much.

The data grid is one of the infrastructures for data and storage resource management. Data replication can improve data access performance, fault tolerance, and reduce network transmission bandwidth. As the site storage space is limited in the grid, how to effectively use grid resources become an important challenge. This paper introduces a dynamic grid replication algorithm based on popularity support and confidence (BPSC). Through the algorithm, data and its associated copy can be placed on a suitable site, with reduced access latency. Through Optorsim, simulation results show that the algorithm can provide better performance compared with other algorithms Grid computing is an important branch of distributed computing. Grid computing constructs a virtual computer system through using a different location of the large number of heterogeneous resources (CPU and storage), provides the model to solve the problem of large scale computing and storage. The data grid is mainly the grid for data storage management. The data grid is defined as an effective combination of data and computing resources of distributed systems by Data Grid Group. The large-scale data- intensive applications, such as bioinformatics, high energy physics, space information generated vast amounts of data, these data may be TB scale, even up to the scale of PB. It is not practical to maintain large scale data at each site, and provide computing resources for visitors. At the same time, it will lead to increased access latency through a centralized way to storing and accessing these data from each site. Data replication technology is an effective method to reduce access latency in the data grid system. Data replication is one of the basic methods of distributed systems to improve the efficiency of system access, maintenance, system high availability. Data replication is to store data in the appropriate site for users to effective access to massive data. Data replication can reduce data access latency, reduce network bandwidth consumption, avoid congestion, adjust the load balancing in server-side, improve system reliability. Many data replication algorithm can reduce data access latency, improve system reliability, but which data will be copied, when the data will be copied at, where the data will be copied are the basic means of determining data replication algorithm. In this paper, we introduced a real-time dynamic data replication algorithm. The algorithm is based on the popularity of file dynamic to replicate data, and consider the storage space limitations of the site. Through optorsim simulation verified the algorithm. This paper is organized as follows: Section 2 is about related works on replication in data grids. In Section 3, system model and component in system are presented. The file

Computational grid is a good solution to large scale data processing and management problems including efficient checkpoint transfer and replication. Due to heterogeneous nature of grids most of time grids more prone to failure or latency delay. Subsequently checkpointing and replication is indispensable to tolerate such faults efficiently. Dynamic checkpoint data replication in computational grid aims to improve data access time and to utilize network and storage resources efficiently. Since the data checkpoints are very large and grid storages are limited, managing replicas in storage for the purpose of more effective utilization of them require more attention. In this work, a dynamic checkpoint data replication mechanism is proposed, which is called checkpoint based optimal replication (CBOR). CBOR selects a checkpoint for replication and calculates a suitable number of copies and grid sites for replication by setting different weight for each data access record. The data access records in the near past have higher weights. A grid simulator Optorsim is used to evaluate the performance of CBOR dynamic replication strategy. The experimental results show that CBOR successfully increases the effective network usage by finding out a popular checkpoint and replicates it to a suitable site.

A data replication algorithm for groups of files in data grids

Grid computing is the most popular infrastructure in many emerging field of science and engineering where extensive data driven experiments are conducted by thousands of scientists all over the world. Efficient transfer and replication of these peta-byte scale data sets are the fundamental challenges in Scientific Grid. Data grid technology is developed to permit data sharing across many organizations in geographically disperse locations. Replication of data helps thousands of researchers all over the world to access those data sets more efficiently. Data replication is essential to ensure data reliability and availability across the grid. Replication ensures above mentioned criteria by creating more copies of same data sets across the grid. In this paper, we proposed a data mining based replication to accelerate the data access time. Our proposed algorithm mines the hidden rules of association among different files for replica optimization which proves highly efficient for different access patterns. The algorithm is simulated using data grid simulator, OptorSim, developed by European Data Grid project. Then our algorithm is compared with the existing approaches where it outperforms others.

Creating many replicas in the processing of data-intensive jobs in data grid is an efficient strategy. Replica replacement is the crucial step to this strategy. Economic model, popularity model and hybrid model etc. have been proposed to solve this issue of replica replacement with analysis and prediction based on each data file, however, these models neglect association relationships among different data files. To find out these association relationships hidden in data-intensive jobs, Apriori algorithm in data mining field is adopted to analyze behaviors of each data-intensive job. An associated replica replacement algorithm based on Apriori approach in data grid is proposed in this paper. This algorithm has two major steps: 1) associated behavior analysis and classification of data files in each node; 2) generation and application of replica replacement rules. Our proposed algorithm is simulated in Optorsim to be compared with LFU algorithm. The experiment shows that there is a relative advantage compared with LFU in mean job times of all jobs, number of remote file access and effective network usage perspectives.

QoS-aware replica placement for data intensive applications

Data access latency is an important metric of system performance in data grid. By means of efficient replication strategy, the amount of data transferred in a wide area network will decrease, and the average access latency of data will decrease ultimately. The motivation of our research is to solve the optimized replica distribution problem in a data grid; that is, the system should utilize many replicas for every data with storage constraints to minimize the average access latency of data. This paper proposes a model of replication strategy in federated data grid and gives the optimized solution. The analysis results and simulation results show that the optimized replication strategy proposed in this paper is superior to LRU caching strategy, uniform replication strategy, proportional replication strategy and square root replication strategy in terms of wide area network bandwidth requirement and in the average access latency of data.

The data availability in a data grid system is complicated by node failure, data catalog error and an unreliable network. To improve the job response time and data availability, data is typically replicated in large scale data-massive applications. However, the dynamic behavior of a Grid user makes it difficult to determine where and how to make data replications to meet the system availability goal. Some strategies for data replication have previously been proposed, but they assumed unlimited storage for replicas. In this paper, we present two new metrics to measure the system data availability. We then model the system availability problem assuming limited replica storage and transfer this to a classic optimal problem. We present four strategies for limited replica storage that maximize the data availability by minimizing the data missed rate (MinDmr), based on a file weight and prediction function. Our simulation on the OptorSim shows our MinDmr algorithm achieves better performance overall than others in term of data availability. Results indicate the performance of MinDmr is always better than others with varying prediction functions, job schedulers and file access patterns, as far as the data missing rate is concerned.

Data-intensive applications produce huge amounts of data that need to be stored, analyzed, and interpreted. A data grid serves as a cost-effective infrastructure for solving these data-intensive applications. Existing scheduling strategies are best suited for handling compute-intensive applications, although they lack in performance while handling data-intensive applications. In this work, a novel mechanism of incorporating cognitive science in a data grid is proposed for scheduling data-intensive workflows. A unique model is derived in which a cognitive engine (CE) is built into the middleware of the data grid. The intelligent agents present in the CE handle the request for data sets and use the LTP algorithm (learning, thinking, and perception) to effectively schedule the tasks using three phases. The CE also~finds a unique solution for placing data sets dynamically nearer to the execution site based on network resource considerations by reducing the~waiting time and data availability time for I/O-intensive jobs. The performance of the CE is validated by simulation and compared with that of existing scheduling strategies. The results of the simulation show that CE optimizes the data availability time, waiting time, data transfer time, and makespan.

Effective data management is a crucial problem in distributed systems such as data grid and cloud. This can be achieved by replicating file in a wise manner, which reduces data access time, increases data availability, reliability and system load balancing. Determining a reasonable number and appropriate location of replicas is essential decision in cloud computing. In this paper, a new dynamic replication strategy called Data Mining-based Data Replication (DMDR) is proposed, which determines the correlation of the data files accessed using the file access history. We focus particularly on how extracted knowledge with maximal frequent correlated pattern mining improves data replication. We can group files with high dependency in the same replica set. Through the DMDR strategy, replicas can be stored in the suitable locations, with reduced access latency according to the centrality factor. In addition, due to the finite storage space of each node, replicas that are useful for future tasks can be wastefully deleted and replaced with less beneficial ones. Results of simulation using CloudSim indicate that DMDR strategy has a relative advantage in effective network usage, average response time, hit ratio in comparison with current methods. It can be concluded from this investigation that data mining technique is effective and helpful in the finding of users’ future access behavior in cloud environment.

Data Grid deals with the movement of massive amount of data among storage elements. One of the challenges in Data Grids is how to reduce the average data access time by replicating data. In this article, a method for dynamic data replication called RBAAC (Replica Based Average Access Cost) is proposed which, removes files with least cost of average access time when encountering insufficient space for replication. Simulation results show that the proposed method, improved the average access time to files and network bandwidth use.

In recent years data grids have been deployed and grown in many scientific experiments and data centers. The deployment of such environments has allowed grid users to gain access to a large number of distributed data. Data replication is a key issue in a data grid and should be applied intelligently because it reduces data access time and bandwidth consumption for each grid site. Therefore this area will be very challenging as well as providing much scope for improvement. In this paper, we introduce a new dynamic data replication algorithm named Popular File Group Replication, PFGR which is based on three assumptions: first, users in a grid site (Virtual Organization) have similar interests in files and second, they have the temporal locality of file accesses and third, all files are read-only. Based on file access history and first assumption, PFGR builds a connectivity graph for a group of dependent files in each grid site and replicates the most popular group files to the requester grid site. After that, when a user of that grid site needs some files, they are available locally. The simulation results show that our algorithm increases performance by minimizing the mean job execution time and bandwidth consumption and avoids unnecessary replication.

From Rio to Iragua--sustainability versus efficiency and equity for preventive health interventions.