Distributed Popularity Based Replica Placement in Data Grid Environments

In data grids, data replication on variant nodes can change some problems such as response time and availability. Also, in data replication, there are some challenges to finding the best replica efficiently in relation to performance and location of physical storage systems. In this paper, various replica placement strategies are discussed. These replica placement strategies are available in the works. Replica placement contains recognizing the best possible node to duplicate data based on network latency and user request. These strategies measure and analyze different parameters such as access cost, bandwidth consumption, scalability, execution time and storage consumption. This paper also analyses the performance of various strategies with respect to the parameters mentioned above in data grid.

Data Grid Environments consist of geographically distributed resources to solve scientific problems and tasks of researchers, scientists and engineers, which are difficult to accomplish by traditional methods based on computer networks. Scheduling and replication are considered some of the most important techniques used in data grid environments, which are used to improve performance and availability to get the best throughput in the shortest possible time. Thus, some algorithms are used for these purposes. Effective scheduling working to reduce the time of implementation of tasks (makespan) of the available resources in the data grid, while replication is working to provide appropriate places or replace similar data to accelerate job execution time. On the other hand, there is another technique, which is important as scheduling and replication, which can be used to reduce the time of implementation for a user request. This technique called Access Pattern, defines the order in which the files are requested for each job to accelerate the completion of the task. Most researchers are focusing on the scheduling, replication, or Access Pattern separately, which leads to variation in the results and gives them unsatisfactory results. The contribution of this paper is present the impact and effect of the triple integration of the three techniques to completing tasks in data grid environments by comparing the results of different algorithms available in the OptorSim simulator.

Data replication is an excellent technique to move and cache data close to users. By replication, data access performance can be improved dramatically. One of the challenges in data replication is to select the candidate sites where replicas should be placed. We use a multi-objective model to address the replica placement problem. The multi-objective model considers the objectives of p-median and p-center models simultaneously to select the candidate sites that will host replicas. The objective of the p-median model is to find the locations of p possible candidate replication sites by optimizing total (or average) response time; where the p-center model finds p candidate sites by optimizing maximum response time. A grid environment is highly dynamic so user requests and network latency vary constantly. Therefore, candidate sites currently holding replicas may not be the best sites to fetch replica on subsequent requests. We propose a dynamic replica maintenance algorithm that re-allocates to new candidate sites if a performance metric degrades significantly over last K time periods. Simulation results demonstrate that the dynamic maintenance algorithm with multi-objective static placement decision performs best in dynamic environments like data grids.

Data grid integrates geographically distributed resources for solving data-sensitive scientific applications. As tasks are sensitive to data, dealing with large amount of data makes the requirement for efficiency in data access more critical. The goal of replica placement is to shorten data access time for enhancing the task execution performance. Therefore, replica placement strategies are often integral to task scheduling algorithms. However, all existing integration strategies make an assumption that the disk space of resources in data grid is unlimited. In this paper, we extended MinMin heuristic to cater to the situation where the disk space of a computational resource is limited. In addition, a heuristic replica placement algorithm is proposed, in which the limited disk space of a storage resource is considered as well. Another character of this heuristic replica placement algorithm is that it can map more than one hot file to several storage resources. We study our approach and evaluate it through simulation. The result shows that the integration of the two algorithms has improved the performance of data grid especially when the whole disk space of storage resources is relatively smaller than the amount of all data files.

QoS-aware replica placement for data intensive applications

The network technologies and storage devices are developing quickly and the technical devices are inexpensive. Therefore, users have a large amount of storage space, Data Grid can collect distributed storage device to share to other users. In this paper, we discuss several file systems that consist of NFS, SRB, iRODS, Lustre, Gfarm and GPFS-WAN. In Data Grid, we select the specific file system to administrate the storage devices. We must consider the file system that obtains scalibilty, security and stability. Finally, we choose two file systems that consist of Gfarm and GPFS-WAN to be our evaluative target. Gfarm and GPFS-WAN are used to build up large-scale dataset storage system that consists of PRAGMA and TeraGrid, respectively. In this paper, we use Gfarm and GPFS-WAN to create a Data Grid environment and evaluate the performance of data transmission, respectively. As results, the GPFS-WAN's performance of data transmission is better than Gfarm. The data transmission is one of the factors in Data Grid environment. For instance, the system obtains replication, open source software, easy to install and maintain that is important to be a great Data Grid environment, these important information will be discussed in this paper.

In this paper, we propose a dynamic file transfer scheme with co-allocation architecture, called Dynamic Adjustment Strategy, a dynamic file transfer scheme with co-allocation architecture that reduce the file transfer times and improves the performance in Data Grid environments. Our approach reduces the idle time faster servers spend waiting for the slowest server, and decreases file transfer completion time. We also present a new toolkit, called Cyber-Transformer, with a friendly graphical user interface interface running on the client side that makes it easy for inexperienced users to manage replicas and download the files in Data Grid environments. We also provide an effective scheme for reducing the cost of reassembling data blocks.

This paper presents a Web based portal, which enables intelligent processing of biological data in Grid environments. The deployed software aims at creation of tools for processing data from microarray experiments over the Hellenic Grid infrastructure. Emphasis is given on user interface and access issues, while the paper describes also the data parsing and parallelization of the microarray data processing. The description of the system is oriented to Grid developers and users, since it focuses on the customization and use of the microarray applications over the Grid. Apart from supporting the high performance and economical execution of microarray experiments, the proposed system endeavors to provide access to a distributed repository of experiments information and results.

Grid data sharing services provide a unified platform for dynamic discovery, access and sharing of distributed data in Grid environments. A common authorization system is needed to provide access control for both Grid data sharing services as well as the data resources that are being shared through these services, accommodating different security requirements from the service providers and the data providers. In this paper, we present a flexible policy-driven authorization system, called RamarsAuthZ, for secure data sharing services in Grid environments. RamarsAuthZ adopts a flexible role-based approach with trust-aware feature to advocate originator control and provide unified access control both at the service level and at the data level.KeywordsAccess ControlGrid EnvironmentGrid ServiceAuthorization PolicyIdentity ProviderThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Co-allocation architecture was developed to enable parallel transferring of files from multiple replicas stored in the different servers. Several co-allocation strategies have been coupled and used to exploit the different transfer rates among various client-server links and to address dynamic rate fluctuations by dividing files into multiple blocks of equal sizes. The paper presents a dynamic file transfer scheme, called dynamic adjustment strategy (DAS), for co-allocation architecture in concurrently transferring a file from multiple replicas stored in multiple servers within a data grid. The scheme overcomes the obstacle of transfer performance due to idle waiting time of faster servers in co-allocation based file transfers and, therefore, provides reduced file transfer time. A tool with user friendly interface that can be used to manage replicas and downloading in a data grid environment is also described. Experimental results show that our DAS can obtain high-performance file transfer speed and reduce the time cost of reassembling data blocks.

In the Data Grid environment, the primary goal of data replication is to shorten the data access time that is experienced by the job and reduce the job turnaround time as a consequence. After introducing a Data Grid architecture that supports efficient data access for the Grid job, two dynamic data replication algorithms are put forward. Combined with different Grid scheduling heuristics, the performances of the data replication algorithms are evaluated with various simulations. The simulation results demonstrate that the dynamic replication algorithms can reduce the job turnaround time remarkably. Especially the combination of Shortest Turnaround Time (STT) scheduling heuristic and Centralized Dynamic Replication (CDR) algorithm exhibits prominent performance in diverse conditions of workload and system environment.

The data grid has provided the emerging technology for constructing the ultra large-scale data storage and management in the grid computing. This paper presents a new kind of data grid middleware for data storage resources discovery and dynamic management in gird environment. The data grid middleware has the characteristics of self-adaptive software system structure. The architecture of grid storage resources discovery and dynamic management is presented for discovering data storage resources from the different computer organizational structure, the different grid, and different store medium in the grid. This middleware can realize the necessary functions for the ultra large-scale application in data grid environment. It could be applied to the ultra large-scale data storage management in grid computing in next generation internet.

Optimizing the use of grid resources is critical for users to effectively exploit a Data Grid (DG). Data replication is considered as a major technique for increasing access performance and data availability in DG systems. Current works on data replications in Grid systems focuses on infrastructure for replication mechanism for creating or deleting replicas. One of the challenges in data replication is determining Optimal Number of Replicas (ONR) with Quality of Service (QoS) assurance, as well as their Optimal Location of Replicas (OLR) in DG. In this paper, we propose an algorithm that finds ONR of an object over DG systems, such that the overall communication and storage cost is minimized. This algorithm ensures that QoS required from the users are satisfied. In addition, we proposed a sketch of the proof for our algorithm and its integrity.

Replicating of objects at multi nodes has been considered a promising technique for decreasing access cost and network traffic for a large distributed systems like Data Grids. In this paper we propose Optimal Placement of Replicas (OPR) algorithm with the objective of minimizing the total replication cost including communication and storage cost, given that a maximum of k replicas are allowed in a tree data grid with n nodes and height h. OPR has time complexity of O(nhk), which is the lowest time complexity compared with the other algorithms with the same objective function in the past literature. In addition, two greedy algorithms, namely, Greedy Algorithm (GA) and Simple Greedy Algorithm (SGA) are proposed for near optimal placement of k replicas with the lower time complexity compared with that of OPR. We evaluate the proposed algorithms through an extensive simulation experiments in terms of access cost, network traffic and locality access to objects.

Much work has been done in the area of data access and integration using various data mapping, matching, and loading techniques. One of the main concerns when integrating data from heterogeneous data sources is data redundancy. The concern is mainly due to the different business contexts and purposes from which the data systems were originally built. A common process for accessing data from integrated databases involves the use of each data source’s own catalogue or metadata schema. In this article, the authors take the view that there is a greater chance of data inconsistencies, such as data redundancies when integrating them within a grid environment as compared to traditional distributed paradigms. The importance of improving the data search and matching process is briefly discussed, and a partial service oriented generic strategy is adopted to consolidate distinct catalogue schemas of federated databases to access information seamlessly. To this end, a proposed matching strategy between structure objects and data values across federated databases in a grid environment is presented.