Allocating Data Cubes on a Grid Environment Endorsing an Alternative Approach

The pre-computation of data cubes is critical to improving the response time of on-line analytical processing (OLAP) systems and can be instrumental in accelerating data mining tasks in large data warehouses. In order to meet the need for improved performance created by growing data sizes, parallel solutions for generating the data cube are becoming increasingly important. The paper presents a parallel method for generating data cubes on a shared-nothing multiprocessor. Since no (expensive) shared disk is required, our method can be used on low cost Beowulf style clusters consisting of standard PCs with local disks connected via a data switch. Our approach uses a ROLAP representation of the data cube where views are stored as relational tables. This allows for tight integration with current relational database technology. We have implemented our parallel shared-nothing data cube generation method and evaluated it on a PC cluster, exploring relative speedup, local vs. global schedule trees, data skew, cardinality of dimensions, data dimensionality, and balance tradeoffs. For an input data set of 2000000 rows (72 Megabytes), our parallel data cube generation method achieves close to optimal speedup; generating a full data cube of /spl ap/227 million rows (5.6 Gigabytes) on a 16 processors cluster in under 6 minutes. For an input data set of 10,000,000 rows (360 Megabytes), our parallel method, running on a 16 processor PC cluster, created a data cube consisting of /spl ap/846 million rows (21.7 Gigabytes) in under 47 minutes.

Grid computing provides a high performance computing platform to solve larger scale applications by coordinating and sharing computational power, data storage and network resources across dynamic and geographically dispersed organizations. Scheduling onto the Grid is NP-complete, so there is no best scheduling algorithm for all grid computing systems. An alternative is to select an appropriate scheduling algorithm to use in a given grid environment because of the characteristics of the tasks, machines and network connectivity. Job scheduling is one of the key research area in grid computing. The goal of scheduling is to achieve highest possible system throughput and to match the application need with the available computing resources. Motivation of this study is to encourage and help the amateur researcher in the field of grid computing, so that they can understand easily the concept of scheduling and can contribute in developing more efficient and practical scheduling algorithm. This will benefit interested researchers to carry out further work in this thrust area of research.

The pre-computation of data cubes is critical to improving the response time of on-line analytical processing (OLAP) systems and can be instrumental in accelerating data mining tasks in large data warehouses. However, as the size of data warehouses grows, the time it takes to perform this pre-computation becomes a significant performance bottleneck. This work presents a fast parallel method for generating ROLAP data cubes on a shared-nothing multiprocessor based on a novel optimized data partitioning technique. Since no shared disk is required, this method can be applied on highly scalable processor clusters consisting of standard PCs with local disks, connected via a data switch. The approach taken, which uses a ROLAP representation of the data cube, is well suited to large data warehouses on high dimensional data, and supports the generation of both fully materialized and partially materialized cubes. In comparison with previous approaches, our new method does significantly improve the scalability with respect to both, the number of processors and the I/O bandwidth (number of parallel disks). We have implemented our new parallel shared-nothing data cube generation method and evaluated it on a PC cluster, exploring relative speedup, scaleup, sizeup, output sizes and data skew. For a fact table with 16 million rows and 8 attributes, our parallel data cube generation method achieves close to optimal speedup for as many as 32 processors, generating a full data cube in under 7 minutes. For a fact table with 256 million rows and 8 attributes, our parallel method achieves optimal speedup for 32 processors, generating a full data cube consisting of /spl ap/7 billion rows (200 Gigabytes) in under 88 minutes.

QoS-based storage resources provisioning for grid applications

Grid computing is a loosely couple distributed system, and it can solve complex problem with large-scale computing and storage resources. Middleware plays important role to integrate heterogeneous computing nodes. Globus Toolkit (GT) is a popular open source middleware to build grid environment. However, a job submission has lots of complicate operations in GT especially in a large scale gird. Moreover, the information discovery component of Globus Toolkit can only provide the summarized information from Grid Head instead of each computing node. Furthermore, job scheduling is another important issue in the high performance Grid computing. An appropriate scheduling algorithm can efficiently reduce the response time, turnaround time and increase the throughput. In this paper, we develop a resource broker module for GT infrastructure, which can dynamically describe and discover the resource information of computing nodes. Moreover, we design an adaptive fuzzy logic scheduler, which utilizes the fuzzy logic control technology to select the most suitable computing node in the Grid environment. For verifying the performance of the proposed scheduling algorithm, we also implement a resource broker as well as fuzzy logic scheduler based on Globus Toolkit 4. The experimental results show our algorithm can reduce the turnaround time compared with round-robin and random dispatching methods. The experiments also show that our algorithm has better speed-up ratio than round-robin and random dispatching when number of computing nodes increasing.

The cost of privacy on blockchain: A study on sealed-bid auctions

Mobile edge computing (MEC) is envisioned as a prospective technology that supports latency-critical and computation-intensive applications by using storage and computation resources in network edges. The advantages of this technology are trapped in limited edge cloud resources, and one of the prime challenges is how to allocate available edge cloud resources to satisfy user requests. However, previous works usually optimize service (data& code) placement and request routing simultaneously within the same timescale, ignoring the fact that frequent service replacement will incur expensive operational expenses. In this paper, we jointly optimize service placement and request routing in the MEC network for data analysis applications, under the constraints of computation and storage resource. In particular, the Cloud Radio Access Network (C-RAN) architecture is applied to pool available resources and realize load balancing among edge clouds. In addition, we adopt a two timescale framework to reduce higher operating expenses caused by frequent cross-cloud service migration. Then, we develop a greedy-based approximation algorithm for service placement subproblem and a linear programming (LP) relaxation-based heuristic algorithm for request routing subproblem, respectively. Finally, the numerical results demonstrate that our proposed solution reaches 90% of the optimal performance in services homogeneous case and 76% in services heterogeneous case.

G‐Lite is a Grid middleware, currently the main middleware installed on all clusters in Bulgaria. The middleware is used by scientists for solving problems, which require a large amount of storage and computational resources. On the other hand, the scientists work with complex processes, where job execution in Grid is just a step of the process. That is why, it is strategically important g‐Lite to provide a mechanism for service compositions and business process management. Such mechanism is not specified yet. In this article we propose a framework for service composition in g‐Lite. We discuss business process modeling, deployment and execution in this Grid environment. The examples used to demonstrate the concept are based on some IBM products.

With growing computational power of current supercomputers, scientific computing applications can work on larger problems. The corresponding increase in dataset size is often correlated to an increase in needed storage for the results. Current storage area networks (sans) balance i/o load on multiple disks using high speed networks, but are integrated on the operating system level, demanding administrative intervention if the usage topology changes. While this is practical for single sites or fairly static grid environments, it is hard to extend to a user defined per-job basis. Reconfigurable grid environments, where computing and storage resources are coupled on a per-job basis, need a more flexible approach for parallel i/o on remote locations.This paper gives a detailed overview of the abilities of the transparent remote access provided by tunnelfs, a part of the viola parallel i/o project. We show how tunnelfs manages flexible and transparent access to remote i/o resources in a reconfigurable grid environment, supporting the definition of the amount and location of persistent storage services on a per-job basis.

The Grid environment contains a large and growing number of widely distributed sites with heterogeneous resources. It is a great challenge to dynamically manage and account for usage data of Grid resources, such as computational, network, and storage resources. A distributed Resource Usage Data management and Accounting system (RUDA) is designed to perform accounting in the Grid environment. RUDA utilizes fully decentralized design to enhance scalability and supports heterogeneous resources with no significant impact on local systems. It can easily be integrated into Grid infrastructures and maintains the integrity of the Grid security features.

In this paper, we have described a non-intrusive, scalable administration and monitoring system for high performance grid environment. It is a fundamental building block for achieving and analyzing the performance of the storage virtualization system in a huge and heterogeneous grid storage environment. It offers a very flexible and simple model that collects nodes state information and requirements needed by the other services of our virtualization storage system improving storage distributed data performance. It is based on a multi-tiered hierarchical architecture the start-up of the monitoring and administration system.

As Grid technology evolves it is becoming evident that the inclusion of mobile devices in a Grid environment will benefit both the Grid and mobile network communities. Many mobile devices, however, have limits on their computational resources that make it difficult to achieve this goal. The Personal Distributed Environment (PDE) is a new concept in mobile networks which, by constraining the problem, overcomes the resource limitation. This paper introduces the concept of the PDE and compares it with the Grid environment. The paper focuses on the specific problem of how to secure the mobile device and in particular how Grid security may be applied to this new environment. The PDE allows simple Grid security mechanisms to be adapted and applied within the PDE boundaries, while allowing a gateway device to support open standards as they emerge. In adopting a hybrid approach, a migration path to open standards is provided, as the computational resources of mobile devices increase.KeywordsMobile DeviceMutual AuthenticationGrid EnvironmentDigital Right ManagementGrid TechnologyThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

New technological advancements in synchrotron light sources enable data acquisitions at unprecedented levels. This emergent trend affects not only the size of the generated data but also the need for larger computational resources. Although beamline scientists and users have access to local computational resources, these are typically limited and can result in extended execution times. Applications that are based on iterative processing as in tomographic reconstruction methods require high-performance compute clusters for timely analysis of data. Here, time-sensitive analysis and processing of Advanced Photon Source data on geographically distributed resources are focused on. Two main challenges are considered: (i) modeling of the performance of tomographic reconstruction workflows and (ii) transparent execution of these workflows on distributed resources. For the former, three main stages are considered: (i) data transfer between storage and computational resources, (i) wait/queue time of reconstruction jobs at compute resources, and (iii) computation of reconstruction tasks. These performance models allow evaluation and estimation of the execution time of any given iterative tomographic reconstruction workflow that runs on geographically distributed resources. For the latter challenge, a workflow management system is built, which can automate the execution of workflows and minimize the user interaction with the underlying infrastructure. The system utilizes Globus to perform secure and efficient data transfer operations. The proposed models and the workflow management system are evaluated by using three high-performance computing and two storage resources, all of which are geographically distributed. Workflows were created with different computational requirements using two compute-intensive tomographic reconstruction algorithms. Experimental evaluation shows that the proposed models and system can be used for selecting the optimum resources, which in turn can provide up to 3.13× speedup (on experimented resources). Moreover, the error rates of the models range between 2.1 and 23.3% (considering workflow execution times), where the accuracy of the model estimations increases with higher computational demands in reconstruction tasks.

Grid computing is the next generation of distributed system. Its goal is to create a huge, independent and powerful virtual machine, and it has been created by gathering different machines with the aim of sharing them. Resource discovery is the most fundamental phase of resource management which has been considered in our work. Considering the advantages and disadvantages of P2P and grid and their architectural similarity; using some of P2P approaches as a solution in Grid can improve the performance of Grid. Semantic technology and fuzzy logic are the two intelligent approaches, which recently apply on P2P network and Grid environment. In P2P networks, nodes are grouping based on their interests. Moreover semantic can be very useful for grouping nodes. This paper follows the identification of resource discovery in grid environment with the help of semantic and fuzzy theory and seeking to present a resource discovery algorithm with respect to exiting obstacles. The proposed approach is assessed in simulated grid environment and the results are compared with other approaches in same conditions. The goals of our approach are to obtain precious matching by clustering nodes which have services semantically related, improving search expressiveness by considering the distances between nodes (delay) in clustering phase, and decreasing response time and processing time by considering the nodes computational capabilities and free space of each node. We evaluate the performance of our approach with some distributed semantic based grid resource discovery models. The results of the experiments show the efficiency of the proposed approach in term of scalability, precision, search expressiveness and response time.

This work is part of the ViSaGe project. This project concerns the storage virtualization applied in the grid environment. Its goal is to create and manage virtual storage resources by aggregating geographically distributed physical storage resources shared on a Grid. To these shared resources, a quality of service will be associated and related to the data storage performance. In a Grid environment, sharing resources may improve performances if these resources are well managed and if the management software obtains sufficient knowledge about the grid resources workload (computing nodes, storage nodes and links). The grid resources workload is mainly perceived by a monitoring system. Several existing monitoring systems are available for monitoring the grid resources and applications. Each one provides informations according to its aim. For example, Network Weather Service [6] is designed to monitor grid computing nodes and links. On the other hand, Netlogger [8] monitors grid resources and applications in order to detect application's bottleneck. These systems are useful for a post mortem analysis. However, in ViSaGe, we need a system that analyzes the necessary nodes state during execution time. This system is represented by the ViSaGe Administration and monitoring service "Admon". In this paper, we present our scalable distributed system: Admon. Admon traces applications, and calculates the system resources consumption (CPU, Disks, Links). Its originality consists in providing a new opportunity to improve virtual data storage performance by using workload's constraints (e.g., maximum CPU usage percentage). Many constraints will be discussed (such as time's constraint). These constraints allow performance improvement by assigning nodes to the ViSaGe's jobs in an effective manner.