Peer-to-Peer resource discovery in Grids: Models and systems

Grid computing originated as paradigm for high performance computing and massive parallel processing. Currently, Grid systems have become an important paradigm for efficiently solving large scale complex problems from many fields. On the other hand, P2P paradigm originated from file sharing but each time more is being used for the development of large scale distributed platforms. Grid and P2P systems have thus followed different trajectories pushed by different motivations, needs and research communities. Yet, both paradigms are evolving in a way that each time more they are sharing common characteristics and are mutually benefiting from their best features. Among these characteristics, we could distinguish the cooperative model for solving complex problems by exploiting the large computing capacity offered by the nodes of the system altogether. As such, Grid and P2P systems have achieved notable success, in particular, for e-Science applications, a family of complex applications arising in science and engineering that need considerable computation power.

Since ten years, the Grid systems are hot research topics. Recently, the Grid systems open towards the management of heterogeneous and distributed data on a large-scale environment. The Grid data management raises new problems and presents real challenges: (i) resource discovery and allocation, (ii) query processing and optimization, (iii) monitoring services, (iv) replication and caching, (v) cost models, (vi) autonomic data management, and (vii) security. The main characteristics offered by these systems are: large scale (e.g. high numbers of data sources, and computing resources) and dynamicity of nodes (unstable system). The synergy and convergence of interests between Grid systems and agent systems have been clearly pointed out. To address some of the above problems, a complementary approach (with respect to web services and P2P techniques) based on mobile agents deserves to be explored. In this talk, we draw up a synthetic state of the art of using mobile agents to solve some fundamental problems of data management in grid systems (e.g. resource discovery and allocation, distributed query and optimization, monitoring services for query optimization, cost models). We point out the advantages of mobile agents and how they can help for decentralized control, and scaling.

Grid computing is the next generation of distributed system. Its goal is to create a huge, independent and powerful virtual machine, and it has been created by gathering different machines with the aim of sharing them. Resource discovery is the most fundamental phase of resource management which has been considered in our work. Considering the advantages and disadvantages of P2P and grid and their architectural similarity; using some of P2P approaches as a solution in Grid can improve the performance of Grid. Semantic technology and fuzzy logic are the two intelligent approaches, which recently apply on P2P network and Grid environment. In P2P networks, nodes are grouping based on their interests. Moreover semantic can be very useful for grouping nodes. This paper follows the identification of resource discovery in grid environment with the help of semantic and fuzzy theory and seeking to present a resource discovery algorithm with respect to exiting obstacles. The proposed approach is assessed in simulated grid environment and the results are compared with other approaches in same conditions. The goals of our approach are to obtain precious matching by clustering nodes which have services semantically related, improving search expressiveness by considering the distances between nodes (delay) in clustering phase, and decreasing response time and processing time by considering the nodes computational capabilities and free space of each node. We evaluate the performance of our approach with some distributed semantic based grid resource discovery models. The results of the experiments show the efficiency of the proposed approach in term of scalability, precision, search expressiveness and response time.

Grid computing emerged as a paradigm for high‐performance computing and massive parallel processing. Currently, Grid systems have become an important paradigm for efficiently solving large‐scale complex problems from many fields. On the other hand, P2P paradigm originated from file‐sharing, but each time more is being used for the development of large‐scale distributed platforms. Grid and P2P systems have thus followed different trajectories pushed by different motivations, needs and research communities. In fact, both paradigms are evolving in a way that each time they are sharing more common characteristics and are mutually benefiting from their best features. Among these characteristics, we could distinguish the cooperative model for solving complex problems by exploiting their large computing capacity. As such, Grid and P2P systems have achieved notable success, in particular, for e‐Science applications, a family of complex applications arising in science and engineering that need considerable computation power. Despite important advances in the design and use of Grid and P2P systems, they remain still difficult to implement and apply to real‐life problems. The main difficulties reside in the lack of easy‐to‐use middleware for Grid and P2P, in the complexities of setting up and in the tedious task of deploying real‐world Grid/P2P platforms as well as in experimental studies which are often complex and not easy to repeat. In this paper we survey and analyze the advances in communication libraries and middleware for both Grid and P2P systems as well as their limitations when used in real Grid and P2P infrastructures. We also bring examples of real‐life applications of massive data processing that can be efficiently handled through Grid and P2P approaches. Copyright © 2010 John Wiley & Sons, Ltd.

Grids enable users to share and access large collections and various types of resources in wide areas, and how to locate resources in such dynamic, heterogeneous and autonomous distributed environments is a key and challenging issue. In this paper, a three-level decentralized and dynamic VEGA Infrastructure for Resource Discovery (VIRD) is proposed. In this architecture, every Border Grid Resource Name Server (BGRNS) or Grid Resource Name Server (GRNS) has its own local policies, governing information organization, management and searching. Changes in resource information are propagated dynamically among GRNS servers according to a link-state-like algorithm. A client can query its designated GRNS either recursively or iteratively. Optimizing techniques, such as shortcut, are adopted to make the dynamic framework more flexible and efficient. A simulator called SimVIRD is developed to verify the proposed architecture and algorithms. Experiment results indicate that this architecture could deliver good scalability and performance for grid resource discovery.

The aim of this thesis is to design and implement a new Grid Resource Management methodology, where non-massive resources owners would be able to share their resources and integrate human collaboration across multiple domains regardless of network technology, operative platform or administrative domain.<br/>This thesis proposes a distributed and heuristic policy-based resource management architecture for large-scale Grids. The resource management architecture proposed herein is composed of four main building blocs: services management, resource discovery and monitoring, resource scheduling and jobs allocation and activation. The Grid Services Management (GSM) and Jobs Allocation and Activation (JAA) are supported by means of a Policy-based Grid Resource Management Architecture (PbGRMA). This architecture is able to identify service needs arising from diverse sources during the deployment and management of Grid Services, such as requirements demanded by customers, applications and network conditions. Afterwards, the PbGRMA merges these requirements into deployment policies for the corresponding Grid Services. The Grid Resource Discovery and Monitoring (GRDM) is supported by the introduction of the SNMP-based Balanced Load Monitoring Agents for Resource Scheduling (SBLOMARS), in which network and computational resources are monitored by distributed agents. This allows for a flexible, heterogeneous and scalable monitoring system. The Grid Resource Scheduling (GRS) is based on the Balanced Load Multi-Constrained Resource Scheduler (BLOMERS). This heuristic scheduler represents an alternate way of solving the inherent NP-hard problem for resource scheduling in large-scale distributed networks by means of the implementation of a Genetic Algorithm.<br/>Finally, based on the outcome of both the GRDM and GRS, the PbGRMA allocates the corresponding Grid Services by means of its interfaces with Globus ToolKit Middleware and Unix-based CLI commands along of any large-scale Grid Infrastructure. <br/>The synergy obtained by these components allows Grid administrators to exploit the available resources with predetermined levels of Quality of Service (QoS), reducing computational costs and makespan in resource scheduling while ensuring that the resource load is balanced throughout the Grid. The makespan of a schedule is the time required for all jobs to be processed when no one job could be interrupted during its execution and each node can perform at most one operation at any time.<br/>This new approach has been successfully tested in a real large-scale scenario such as Grid5000. The results presented along this Thesis show that our general solution is a reliable, flexible and scalable architecture to deploy and manage Grid Services in large-scale Grid Infrastructures. Moreover, the substitution of the heuristic algorithm approach used into the Grid Resource Scheduling (GRS) phase by other non-heuristics selection algorithms could make our solution useful in smaller Grid Infrastructures.

P2P-based resource discovery in dynamic grids allowing multi-attribute and range queries

A hybrid mechanism for resource/service discovery in ad-hoc grids

In grid environment where resources are generally owned by different people, communities or organizations with varied administration policies and capabilities, managing the grid resources is not a simple task. Resource brokers simplify this process by providing an abstraction layer for users to access heterogeneous resources transparently. However, discovery of grid resources that suits to the user's job requirements is always a difficult job as the probability of grid resources satisfying the user's requirements is very less. Conventionally, in order to run a job on the grid a user has to identify a set of platforms capable of running that job by the virtue of having the required installation of operating system, libraries, tools, and the configuration of environment variables, etc. In practice, the availability of such software environments will either be limited to a very narrow set, or the job has to be made compatible with an environment supported by a large resource provider, such as TeraGrid. Further, if we could identify such an environment, it is hard to guarantee that the resource will be available when needed, for as long as needed, and that user will get his or her fair share of that resource. This difficulty can be overcome by incorporating the concept of Virtualization in Grid environment that enables the creation of dynamic user defined execution environment in a grid resource.

Integration of mobile resources in grid environment increases the complexity of resource discovery due to heterogeneity, dynamic nature of resources, mobility pattern and uncertainty in the availability. The mobile grid has geographically distributed autonomous resources and is composed of hardware and software resources that need to be located. The existing resource discovery approaches in mobile grid computing treat resources equally. These approaches face problems in terms of response times and the need to generate complex queries. This can be reduced by grouping of resources based on its characteristics. The dynamic characteristics of mobile resources increase complexities in resource grouping and filtering process, which in turn increases resource discovery time. In this paper, ontology based resource discovery mechanism is proposed for the mobile grid environment. This mechanism uses ontology to group the resources and a map per to filter the resources based on resource information which reduces resource discovery time.

The hardware and software design of a new measure and control system of power grid (MCPG) is introduced in this paper. This new system is based on communication technology. It serves as a platform for the utilities of many measure and control applications. By the integration of the hardware and software resources in this platform, further developments of measure and control systems for a particular application could be much more effective and economically.

The current grid computing practices and techniques built on static resource management technique associate with several drawbacks. These drawbacks forced the current grid systems of taking the direction of remote viewing and zooming techniques of already processed images. There is an urgent need for algorithms and techniques to support automatic resources discovery and selection of grid resources. This research investigates how the grid services could be used to support remote visualization. The implemented grid enabled visualization architecture is supported with an automatic resources selection mechanism to enable the visualization pipeline components to be automatically orchestrated without human interference. The presented findings are supported with practical implementation of grid enabled visualization prototype.

A Grid system is essentially an infrastructure that allows location independent access to the resources and services that are provided by geographically distributed machines and networks. One of the fundamental operations needed to support location-independent computing is resource discovery. Generally, resource discovery schemes maintain and query a resource status database. Dissemination of the resource status information is one of the key operations required to keep the resource status databases consistent. This paper examines several approaches for resource status dissemination. A new concept called the Grid potential is introduced in this paper. This concept is used to control the extent of data dissemination in Grid systems.

Grid technologies enable the sharing and collaborating of a wide variety of resources. To fully utilize these resources, effective resource discovery mechanisms are necessities. However, the complicated and dynamic characteristics of the grid resources make sharing and discovering them a challenging issue. In this paper, we propose a semantic community approach to assisting the efficient resource discovery and query in grids. Semantic community is a natural arrangement of distributed systems. It prunes the searching space and reduces the cost of searching. Moreover, it exhibits many desirable properties, including being fully decentralized and supporting complex queries. Our simulation results show how searching the girds can take advantage of semantic communities to reduce the searching cost and improve the quality of search results.

Ad hoc network is an infra structure less network, which is formed by mobile devices like laptops, PDAs, cell phones etc. Each device has different computational capability, power, hardware and software, which forms a heterogeneous network. These devices can be integrated to form an infrastructure known as grid. A grid integrates and coordinates resources and users that are within the same network with different capabilities. Hence we can visualize a grid over an ad hoc network that effectively utilizes the heterogeneity in the mobile devices. The major challenge in forming a grid over an ad hoc network is the mobility of the nodes. In this paper, we address the challenges due to mobility by considering a trace model for the movement of the nodes. Next, we demonstrate the feasibility of forming a grid over a mobile ad hoc network by proposing lightweight algorithms for grid formation, resource discovery, negotiation, job scheduling, and resource sharing. We have analyzed the performance of mobile ad hoc grid both by using a theoretical model and by simulation. The results point to a promising approach to form a mobile ad hoc grid.