Flexible I/O Support for Reconfigurable Grid Environments

QoS-based storage resources provisioning for grid applications

Research on transparent access method for multiple types of data acquisition device in EAST PCS

In this paper, we have described a non-intrusive, scalable administration and monitoring system for high performance grid environment. It is a fundamental building block for achieving and analyzing the performance of the storage virtualization system in a huge and heterogeneous grid storage environment. It offers a very flexible and simple model that collects nodes state information and requirements needed by the other services of our virtualization storage system improving storage distributed data performance. It is based on a multi-tiered hierarchical architecture the start-up of the monitoring and administration system.

Grid computing is a loosely couple distributed system, and it can solve complex problem with large-scale computing and storage resources. Middleware plays important role to integrate heterogeneous computing nodes. Globus Toolkit (GT) is a popular open source middleware to build grid environment. However, a job submission has lots of complicate operations in GT especially in a large scale gird. Moreover, the information discovery component of Globus Toolkit can only provide the summarized information from Grid Head instead of each computing node. Furthermore, job scheduling is another important issue in the high performance Grid computing. An appropriate scheduling algorithm can efficiently reduce the response time, turnaround time and increase the throughput. In this paper, we develop a resource broker module for GT infrastructure, which can dynamically describe and discover the resource information of computing nodes. Moreover, we design an adaptive fuzzy logic scheduler, which utilizes the fuzzy logic control technology to select the most suitable computing node in the Grid environment. For verifying the performance of the proposed scheduling algorithm, we also implement a resource broker as well as fuzzy logic scheduler based on Globus Toolkit 4. The experimental results show our algorithm can reduce the turnaround time compared with round-robin and random dispatching methods. The experiments also show that our algorithm has better speed-up ratio than round-robin and random dispatching when number of computing nodes increasing.

This work is part of the ViSaGe project. This project concerns the storage virtualization applied in the grid environment. Its goal is to create and manage virtual storage resources by aggregating geographically distributed physical storage resources shared on a Grid. To these shared resources, a quality of service will be associated and related to the data storage performance. In a Grid environment, sharing resources may improve performances if these resources are well managed and if the management software obtains sufficient knowledge about the grid resources workload (computing nodes, storage nodes and links). The grid resources workload is mainly perceived by a monitoring system. Several existing monitoring systems are available for monitoring the grid resources and applications. Each one provides informations according to its aim. For example, Network Weather Service [6] is designed to monitor grid computing nodes and links. On the other hand, Netlogger [8] monitors grid resources and applications in order to detect application's bottleneck. These systems are useful for a post mortem analysis. However, in ViSaGe, we need a system that analyzes the necessary nodes state during execution time. This system is represented by the ViSaGe Administration and monitoring service "Admon". In this paper, we present our scalable distributed system: Admon. Admon traces applications, and calculates the system resources consumption (CPU, Disks, Links). Its originality consists in providing a new opportunity to improve virtual data storage performance by using workload's constraints (e.g., maximum CPU usage percentage). Many constraints will be discussed (such as time's constraint). These constraints allow performance improvement by assigning nodes to the ViSaGe's jobs in an effective manner.

Grid and cloud computing systems have been extensively used to solve large and complex problems in science and engineering areas. These systems include powerful computing resources connected through high-speed networks. Due to recent advances in mobile computing and networking technologies, it has become feasible to integrate various mobile devices such as robots, aerial vehicles, sensors, and smartphones with grid and cloud computing systems. This integration enables design and development of next generation of applications through sharing of resources in mobile environments and also introduces several challenges due to dynamic and unpredictable network. This paper discusses applications, research challenges involved in design and development of mobile grid and cloud computing systems, and recent advances in the field.

In the high-speed backbone network, with the increasing speed of network link, the number of network flows increase rapidly. Meanwhile, with restrictions on hardware computing and storage resources, so, how to identify and measure large flows timely and accurately in massive data become a hot issue in high speed network flow measurement area. In this paper, we propose a new algorithm based on double hash algorithm to realize large flow frequent items identification, according to the defect of MF algorithm which produces false positive easily and frequent updates to bring the huge pressure to the system. The complexity and false positive rate of the algorithm was analyzed. The effect of large flow frequent items statistical accuracy and discard rate for parameter configuration was analyzed through simulation. The theoretical analysis and the simulation result indicate that compare to MF algorithm, our algorithm can identify large flow frequent items more accurately, and satisfies the need of actual measurement.

Distributed computing will be viewed from three levels. First, from possible hardware configurations, where emphasis will be placed on loosely coupled systems interconnected with High Speed Networks (HSN). Second, from the Operating System (OS) level where five fundamental additions to existing uniprocessor OSs are required. They include the creation of a message-based interprocess communication facility, a communication manager, an extended file manager, a resource manager, and an extended user interface. The third and final view of truly distributed OSs will be from the applications level. How the user views the logical system, and automatic versus manual control of computational granularity will be discussed.

Grid computing comprises of distributed computer systems which are geographically dispersed to share the combination of resources in a heterogeneous environment. Grid computing is one of the latest technologies to solve very complex problems in scientific applications, using the resource sharing concept in the dynamic virtual organizations. The main problem occurs in most of the organization is that the resources are become idle and underutilized. In some organizations, even the server machines can often be relatively idle. Grid computing provides an approach for exploiting these underutilized resources and thus has the possibility of substantially increasing the efficiency of resource usage. Effective and Efficient Task scheduling is an important aspect of Grid computing. The objective of this paper is that a grid scheduler first allocates the important(high priority) jobs to the resources and then it allocates the low prioritize job so as to achieve the maximum resource utilization rate, minimize the makespan and avoid the load balancing level problem. In this paper, we propose a new framework and QoS(Quality of Service) Priority Based Scheduling Algorithm for effective task scheduling to the resources in the grid environment. The algorithm is simulated using Java. The results show that our proposed QoS priority based scheduling algorithm gives better results in makespan and resource utilization rate than other algorithms such as QoS guided weighted mean time min (QWMTM), Min-Min and Max-Min heuristic algorithms.

Service location based on greedy search is studied. A model is built to analyze the influence of the network topologies on this location method, and different network topologies are constructed to validate the model by way of simulation. The results show that, though the model assumes a uniform degree distribution, it agrees with the simulation result well if the average degree of the network is reasonably big. Hops and relative QoS index of the node found in a service location process are used to evaluate the effectiveness of the location method as well as the probability of locating the first 5% nodes with higher QoS level. Both model and simulation result show that the performance of greedy search based service location improves significantly with the increase of the average degree of the network. If changes of both topology and the QoS level of nodes can be ignored during a location process, it can be guaranteed that the location has high probability of finding the nodes with relatively high QoS level in small number of hops in a big service community. Model extension under arbitrary network degree distribution is further studied. A "two-phase" topology construction and service location based on greedy-search is also put forward.

Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.

With improvements in computation and storage resources, data access through the network becomes the bottleneck for several cloud applications. Even with high-speed networks, the high latency of the cloud access makes it unfeasible or un-favourable for latency-sensitive applications such as autonomous driving, smart factories, and video streaming. Edge computing provides a solution by utilizing the network edge resources that are closer to the end users. Nevertheless, it is a non-trivial task to design a large-scale edge-capable system that is stable, fault-tolerant, and efficient [1]. In this paper, present the design of EdgeKV: a novel general-purpose distributed key-value store for the network edge. We demonstrate the features of EdgeKV for achieving high efficiency and scalability while providing flexibility, ease of use, and data privacy. We evaluated our prototype on the Grid'5000 framework with multiple realistic Yahoo! Cloud Serving Benchmark (YCSB) workloads. Our initial results show that EdgeKV achieves 72% higher throughput and 47% lower latency on average than centralized cloud storage, for read-dominated workloads.

Usually data warehouses store a large volume of data, keeping in their structures all information decision-makers need on their daily activities. Usually, a significant amount of this information is processed and materialized in data cubes to be explored latter using the most flexible ways that on-line analytical processing platforms dispose to do that. However, this materialization process requires a lot of computational resources (processing time and storage), provoking as well serious performance bottlenecks in analytical systems. The distribution of such analytical structures across an organization seems to be quite appellative to attenuate such annoying effects, being in most of the cases a good alternative for improving system performance without great expense. Still, a distributed computational platform use to be less expensive than a centralized one, what give it some good advantages in any solution related to data cube distribution. Following this slope appears Grid environments. As we know these can use computational resources geographically dispersed, heterogeneously and non-dedicated as it was a single a unique environment. However, in order to reach higher performance levels, Grids need proper scheduling and data so they can be able to respond effectively to the jobs that were sent to them. In this paper, we integrate a performance prediction method in a data cube distribution strategy to define a more effective allocation of data cube views in a conventional Grid environment with the ability to receive them.

Data transfers between Supercomputing centers and Grid sites may be complicated since, in general, Supercomputing centers are not completely incorporated in the Grid environment. In particular, the Port d'Informació Científica (PIC) had to transfer data of cosmological simulations from the Barcelona Supercomputing Center (BSC) to Grid storage resources in order to post-process those data at PIC. The data connection is limited to the Secure Shell (SSH) protocol using the scp command. Besides a limitof bandwidth, we had to deal with a lack of automation and reliability of the transfers. PIC managed to control those scp transfers through gLite's File Transfer Service (FTS). Therefore, the discs containing the data at the BSC are mounted via SSH to a local Storage Resource Management (SRM) server at PIC. Once established the access to the data through the SRM server, the data can be transfered to any other SRM server, including the storage system at PIC, using grid protocols. Although we cannot improve the bandwidth, we reach a level of automation and reliability comparable to transfers from the Tier-0 to Tier-1 sites of the Large Hadron Collider (LHC) experiments.

Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and/or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread computing and storage resources are made available to scientists belonging to virtual organizations sharing resources across multiple administrative domains through established service‐level agreements. Grids provide an unprecedented opportunity for distributed workflow execution; indeed, many applications are well beyond the capabilities of a single computer, and partitioning the overall computation on different components whose execution may benefit from runs on different architectures could provide better performances. In this paper we describe the design and implementation of the Grid Resource Broker (GRB) workflow engine. Copyright © 2008 John Wiley & Sons, Ltd.