Triple integration optimization techniques in data grid environment using OptorSim simulator

Data grids support distributed data-intensive applications that need to access massive datasets stored around the world. Ensuring efficient access to such datasets is hindered by the high latencies of wide-area networks. To speed up access, files can be replicated so a user can access a nearby replica. Replication also provides improved availability, decreased bandwidth use, increased fault tolerance, and improved scalability. Since a grid environment is dynamic, resource availability, network latency, and user requests may change. To address these issues a dynamic replica placement strategy that adapts to changing behaviour is needed. In this paper, we introduce a highly distributed replica placement algorithm for hierarchical data grids. Our algorithm exploits data access histories to identify popular files and determines optimal replication locations to improve access performance by minimizing replication overhead (access and update) assuming a given traffic pattern. The problem is formulated using dynamic programming. We evaluate our algorithm using the OptorSim simulator and find that it offers shorter execution time and reduced bandwidth consumption compared to other dynamic replica placement methods.

The network technologies and storage devices are developing quickly and the technical devices are inexpensive. Therefore, users have a large amount of storage space, Data Grid can collect distributed storage device to share to other users. In this paper, we discuss several file systems that consist of NFS, SRB, iRODS, Lustre, Gfarm and GPFS-WAN. In Data Grid, we select the specific file system to administrate the storage devices. We must consider the file system that obtains scalibilty, security and stability. Finally, we choose two file systems that consist of Gfarm and GPFS-WAN to be our evaluative target. Gfarm and GPFS-WAN are used to build up large-scale dataset storage system that consists of PRAGMA and TeraGrid, respectively. In this paper, we use Gfarm and GPFS-WAN to create a Data Grid environment and evaluate the performance of data transmission, respectively. As results, the GPFS-WAN's performance of data transmission is better than Gfarm. The data transmission is one of the factors in Data Grid environment. For instance, the system obtains replication, open source software, easy to install and maintain that is important to be a great Data Grid environment, these important information will be discussed in this paper.

In this paper, we propose a dynamic file transfer scheme with co-allocation architecture, called Dynamic Adjustment Strategy, a dynamic file transfer scheme with co-allocation architecture that reduce the file transfer times and improves the performance in Data Grid environments. Our approach reduces the idle time faster servers spend waiting for the slowest server, and decreases file transfer completion time. We also present a new toolkit, called Cyber-Transformer, with a friendly graphical user interface interface running on the client side that makes it easy for inexperienced users to manage replicas and download the files in Data Grid environments. We also provide an effective scheme for reducing the cost of reassembling data blocks.

Optimizing the use of grid resources is critical for users to effectively exploit a Data Grid (DG). Data replication is considered as a major technique for increasing access performance and data availability in DG systems. Current works on data replications in Grid systems focuses on infrastructure for replication mechanism for creating or deleting replicas. One of the challenges in data replication is determining Optimal Number of Replicas (ONR) with Quality of Service (QoS) assurance, as well as their Optimal Location of Replicas (OLR) in DG. In this paper, we propose an algorithm that finds ONR of an object over DG systems, such that the overall communication and storage cost is minimized. This algorithm ensures that QoS required from the users are satisfied. In addition, we proposed a sketch of the proof for our algorithm and its integrity.

In data grids, data replication on variant nodes can change some problems such as response time and availability. Also, in data replication, there are some challenges to finding the best replica efficiently in relation to performance and location of physical storage systems. In this paper, various replica placement strategies are discussed. These replica placement strategies are available in the works. Replica placement contains recognizing the best possible node to duplicate data based on network latency and user request. These strategies measure and analyze different parameters such as access cost, bandwidth consumption, scalability, execution time and storage consumption. This paper also analyses the performance of various strategies with respect to the parameters mentioned above in data grid.

Co-allocation architecture was developed to enable parallel transferring of files from multiple replicas stored in the different servers. Several co-allocation strategies have been coupled and used to exploit the different transfer rates among various client-server links and to address dynamic rate fluctuations by dividing files into multiple blocks of equal sizes. The paper presents a dynamic file transfer scheme, called dynamic adjustment strategy (DAS), for co-allocation architecture in concurrently transferring a file from multiple replicas stored in multiple servers within a data grid. The scheme overcomes the obstacle of transfer performance due to idle waiting time of faster servers in co-allocation based file transfers and, therefore, provides reduced file transfer time. A tool with user friendly interface that can be used to manage replicas and downloading in a data grid environment is also described. Experimental results show that our DAS can obtain high-performance file transfer speed and reduce the time cost of reassembling data blocks.

In the Data Grid environment, the primary goal of data replication is to shorten the data access time that is experienced by the job and reduce the job turnaround time as a consequence. After introducing a Data Grid architecture that supports efficient data access for the Grid job, two dynamic data replication algorithms are put forward. Combined with different Grid scheduling heuristics, the performances of the data replication algorithms are evaluated with various simulations. The simulation results demonstrate that the dynamic replication algorithms can reduce the job turnaround time remarkably. Especially the combination of Shortest Turnaround Time (STT) scheduling heuristic and Centralized Dynamic Replication (CDR) algorithm exhibits prominent performance in diverse conditions of workload and system environment.

The data grid has provided the emerging technology for constructing the ultra large-scale data storage and management in the grid computing. This paper presents a new kind of data grid middleware for data storage resources discovery and dynamic management in gird environment. The data grid middleware has the characteristics of self-adaptive software system structure. The architecture of grid storage resources discovery and dynamic management is presented for discovering data storage resources from the different computer organizational structure, the different grid, and different store medium in the grid. This middleware can realize the necessary functions for the ultra large-scale application in data grid environment. It could be applied to the ultra large-scale data storage management in grid computing in next generation internet.

SUMMARYAggregate queries are useful tools in the context of sensor network‐based systems as they retrieve knowledge from huge amounts of summarized readings to be exploited for knowledge discovery purposes. Actually, data representation and query models are problematic issues for managing sensor network data, because streams produced by sensors are theoretically unbounded. In this paper, we present a Grid framework, called SensorGrid, on the basis of data compression and approximation paradigms, which allows us to provide approximate answers to aggregate queries on summarized sensor network data. These queries are the basis for achieving Online Analytical Processing (OLAP) over sensor network readings in Data Grid environments, with both effectiveness and efficiency. We also present our experience in the context of a real‐life system focused on the management of environmental sensor network data. Another contribution of our research is represented by the extensive experimental evaluation and analysis of SensorGrid, which, in more details, focuses on two main classes of aggregate range queries over sensor readings, namely, (i) the window queries, which apply an SQL aggregation operator over a fixed window over the reading stream produced by the sensor network, and (ii) the continuous queries, which instead consider a ‘moving’ window and produce as output a stream of answers. Both classes of queries are extremely useful to extract summarized knowledge to be exploited by OLAP‐like analysis tools over sensor network data. The experimental results, conducted on both synthetic and real‐life data sets, clearly confirm the benefits deriving from embedding data compression and approximation paradigms into Grid‐based sensor network data‐intensive management systems.Copyright © 2013 John Wiley & Sons, Ltd.

Data Grid environment is a geographically distributed that deal with date-intensive application in scientific and enterprise computing. In data-intensive applications data transfer is a primary cause of job execution delay. Data access time depends on bandwidth, especially when hierarchy of bandwidth appears in network. Effective job scheduling can reduce data transfer time by considering hierarchy of bandwidth and also dispatching a job to where the needed data are present. Additionally, replication of data from primary repositories to other locations can be an important optimization step to reduce the frequency of remote data access. Objective of dynamic replica strategies is reducing file access time which leads to reducing job runtime. In this paper we develop a job scheduling policy, called TLSS (Two level Scheduling strategy), and a dynamic data replication strategy, called TLRS (Two level Replication Strategy), to improve the data access efficiencies in a cluster grid. We study our approach and evaluate it through simulation. The results show that combination of TLSS and TLRS has improved 17% over other combinations.

In this paper we report on architectural design carried out the data grid environment for managing and preserving large scientific data volume. Our aim is to provide design of infrastructure in research data access, share, and collaboration. We describe data grid architecture by explain the network project design, work flow project management, and data management components. By this design, we believe that many large-scale scientific data experiments, simulations, and management could be generated.

One of the principal motivations to use the grids computing and data grids comes from the applications using of large sets from data, for example, in high-energy physics or life science to improve the total output of the software environments used to carry these applications on the grids, data replication are deposited on various selected sites. In the field of the grids the majority of the strategies of replication of the data and scheduling of the jobs were tested by simulation. Several simulators of grids were born. One of the most simulators interesting for our study is the OptorSim tool. In this paper, we present an extension of the OptorSim simulator by a consistency management module of the replicas in the Data Grids. This extension corresponds to a hybrid approach of consistency, it inspired by the pessimistic and optimistic approaches of consistency. This suggested approach has two vocations, in the first time, it makes it possible to reduce the response times compared with the completely pessimistic approach, in the second time, it gives a good quality of service compared with the optimistic approach.

Grid data sharing services provide a unified platform for dynamic discovery, access and sharing of distributed data in Grid environments. A common authorization system is needed to provide access control for both Grid data sharing services as well as the data resources that are being shared through these services, accommodating different security requirements from the service providers and the data providers. In this paper, we present a flexible policy-driven authorization system, called RamarsAuthZ, for secure data sharing services in Grid environments. RamarsAuthZ adopts a flexible role-based approach with trust-aware feature to advocate originator control and provide unified access control both at the service level and at the data level.KeywordsAccess ControlGrid EnvironmentGrid ServiceAuthorization PolicyIdentity ProviderThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper outlines the research, development, and implementation plans for the Cheshire system as part of an overall digital library framework. Our plans are based on developing a high-level component framework extensible and flexible enough to accommodate radically different architectural models; one which supports large-scale preservation environments characteristic of content management systems such as DSpace and Fedora; and the support for semantic retrieval and natural language processes involving large-scale distributed datasets, served in a highly parallel environment. Rather than prototyping a single solution, we intend to follow a modular approach with a variety of text and data mining, ontological, document rendering, and text retrieval tools which can be used in various combinations according to need, in essence, the Cheshire digital library framework is designed to address the twin aspects of access and preservation in new and sustainable ways.

Data allocation comprises data placement or migration into or between the data servers in a distributed data sharing system. In the process of data placement, static equations would mostly select only the best data servers. This would overload some of the data servers. An algorithm for optimizing file-placement in data grid environment (DGE) is presented. The algorithm adapts game theory and fictitious play to provide a mechanism for load balancing. The algorithm considers the overall placement behaviours and dependencies. A discreet-time based simulator has been developed to test proposed algorithm. Initial simulation result showed that game theory and fictitious play can improve the overall performance of the system.