A Data Broker for Distributed Computing Environments

In this chapter, we give more detailed review of on-going research activities in virtual computing environment. The projects to be reviewed are not by no means a comprehensive list, but can be considered as a representative of the main project in this area.

Dynamic responses considering fluid structure interaction (FSI) is important in many engineering fields and some of the FSI phenomena are treated as an acoustic fluid and structure interaction (AFSI) problem. The dynamic interactions between the fluid and structure can change dynamic characteristics of structures and their responses to external excitation such as seismic loading. The authors have developed a coupled simulation system for the large scale AFSI problems using an iterative partitioned coupling technique. In the system, the authors employed ADVENTURE system which adopted an efficient preconditioned iterative linear algebraic solver, and ADVENTURE Coupler is used to handle interface variable efficiently on various parallel computational environments. The authors employ Broyden method for updating interface accelerations to obtain the robust and fast convergence property of fixed point iterations. This paper presents the overview of the coupled analysis system and the results of its application to several AFSI problems are shown. The system runs efficiently in a parallel environment and it is capable for analyses of complex shaped three dimensional structures with more than 20 million degrees of freedom model.

We propose a methodology to determine the suitability of different data representations in terms of their error-tolerance for a given application with accelerator-based computing. This methodology helps match the characteristics of a representation to the data access patterns in an application. For this, we first identify a benchmark of key kernels from linear algebra that can be used to construct applications of interest using any of several widely used data representations. This is then used in an experimental framework for studying the error tolerance of a specific data format for an application. As case studies, we evaluate the error-tolerance of seven dataformats on sparse matrix to vector multiplication, diagonal add, and two machine learning applications i) principal component analysis (PCA), which is a statistical technique widely used in data analysis and ii) movie recommendation system with Restricted Boltzmann Machine (RBM) as the core. We observe that the Dense format behaves well for complicated data accesses such as diagonal accessing but is poor in utilizing local memory. Sparse formats with simpler addressing methods and a careful selection of stored information, e.g., CRS and ELLPACK, demonstrate a better error-tolerance for most of our target applications.

Many data grid applications manage and process huge datasets distributed across multiple grid nodes and stored into heterogeneous databases; e-Science projects need to access widespread databases within a computational grid environment, through a set of secure, interoperable and efficient data grid services. In the data grid management area several projects aims at addressing these issues providing different solutions and proposing a set of data access and integration/federation services.In this paper we present the GRelC Data Access, a WS-I based data grid access service developed by SPACI Consortium and the University of Salento. Its architectural design is discussed and experimental results related to an European testbed are also reported.

Virtual screening applications are highly parameterized to optimize the balance between quality and execution performance. While output quality is critical, the entire screening process must be completed within a reasonable time. In fact, a slight reduction in output accuracy may be acceptable when dealing with large datasets. Finding the optimal quality-throughput trade-off depends on the specific HPC system used and should be re-evaluated with each new deployment or significant code update. This paper presents two parallel autotuning techniques for constrained optimization in distributed High-Performance Computing (HPC) environments. These techniques extend sequential Bayesian Optimization (BO) with two parallel asynchronous approaches, and they integrate predictions from Machine Learning (ML) models to help comply with constraints. Our target application is LiGen, a real-world virtual screening software for drug discovery. The proposed methods address two relevant challenges: efficient exploration of the parameter space and performance measurement using domain-specific metrics and procedures. We conduct an experimental campaign comparing the two methods with a popular state-of-the-art autotuner. Results show that our methods find configurations that are, on average, up to 35–42% better than the ones found by the autotuner and the default expert-picked LiGen configuration. • We propose two parallel algorithms for black-box constrained optimization. • We integrate Bayesian Optimization and Machine Learning for constraint estimation. • We use a meta-scheduler to hinge on the resources available in HPC settings. • We evaluate the benefits of the proposed approach using a relevant case study.

PAC is a parallel environment, based on a MIMD distributed computing model, which is intended to aid in the development of computer algebra algorithms. It uses parallelism as a tool for processing large problems. This paper discusses the general relationship between computer algebra and parallelism. The general features of the PAC project are described and some of the results obtained with PAC are presented. One of the crucial elements of symbolic computation on parallel architectures is efficient implementation of fast arbitrary precision arithmetic. This paper presents a nodal (arbitrary precision) integer arithmetic package and discusses the fast division algorithm which we have implemented. The representation used is designed to take advantage of a vectorized floating point unit. Our experiences with this approach are also discussed.PAC has been implemented on the FPS T series hypercube (32 processors) and an implementation on the TELMAT Meganode (128 processors) is in progress.

This paper proposes a distributed computing environment based on data-driven mechanism, which can be deployed on different levels of Industrial Internet, designs and implements an Industrial Internet architecture with high reusability and rapid deployment. This architecture has the consistency of the computing environment in the cloud and terminal control layer, which is beneficial to the realization of remote management and collaborative control of Industrial Internet. Data driven technology enables heterogeneous industrial equipment to achieve interconnection and interworking at a low cost. The multi-engine computing environment constructed by it is not only an effective method to solve the application design and development of complex systems, but also a key technology to realize edge computing and fog computing. This paper designed a corresponding test bed platform, test results show that the technology can not only provide connectivity of heterogeneous data source and efficient technical support, and to ensure that the distributed industrial equipment real-time and synchronization of data transmission, but also has the strong edge computing capabilities to cope with the constantly generated data stream, and reducing the pressure of the cloud server storage, computing.

In this paper, the general design methodology for development of a Performance Prediction System for the Grid computing and Distributed Computing has been discussed. Thereafter, its adaptation in the Cloud computing has been discussed. The design analysis begins by comparing the appropriateness of the Explicit prediction approach and Implicit prediction approach to the Grid/ Distributed computing/ Cloud computing environment. The Explicit Approach has two main divisions namely the Resource-oriented prediction and Application oriented prediction. Thereafter the two branches of the Explicit prediction approach have been discussed against Grid/ Distributed Computing/ Cloud computing environments. Finally, in an appropriate case study, the application of the aforementioned design methodology in development of an actual Performance Prediction Engine for a real-time Grid computing environment has been presented.

The paper presents a distributed computing environment for embedded control systems with time-triggered and event-triggered distributed processing. We have already presented a time-triggered distributed object model and a time-triggered distributed computing environment for embedded control systems. However, there are many embedded control systems with time-triggered and event-triggered processing. In this paper, we present two kinds of event-triggered distributed object models, a pure event-triggered distributed object model and a data-triggered distributed object model, in addition to the time-triggered distributed object model. We also present a distributed object computing environment based on a time-division scheduling for the mixed architecture with time-triggered and event-triggered distributed processing. The time division scheduling divides an execution cycle into a time-triggered processing segment and a non-time-triggered processing segment. The time-triggered distributed processing is executed in the former segment, and the event-triggered distributed processing is executed in the latter segment. The distributed object computing environment consists of a real-time operating system with the time division scheduling and distributed computing middleware to support the three kinds of distributed object models. We provide a development environment that generates stubs and configuration data to build distributed control systems.

As part of their e-business activities, enterprises create, share, recombine, and process an ever increasing amount of data and documents that range from simple transactional data to complex business process definitions. In many respective processes, multiple spheres overlap, e.g. (1) inter- with intra-organizational data sources in collaborative processes, (2) master data with transactional data, (3) regular data with behavioral aspects of processes, or (4) facts with normative assertions. Since business entities are members of multiple value chains, same as data sources are often used in multiple contexts and processes, there are often conflicting requirements on the representation of such data. All this in combination makes e-business data engineering a very challenging task, already at the level of a single company but even more in value chains. Also, respective modeling choices may have long-lasting and far reaching impact, and affect (1) operational efficiency, (2) business process agility, (3) the range of analytical tasks for decision support, and (4) strategic options. Eventually, those choices determine the current and future degree of automation in content and process integration. While e-business standards in general help, standardization alone does not solve the problem. This is because yielding a consensual representation takes time, consumes resources, and constrains an entity's ability to capture individual details. Moreover, the cost of implementing and enforcing the standards is sometimes prohibitive for small businesses. Proprietary representations, on the other hand, hamper interoperability, and complicate B2B integration. In service-oriented architectures (SOAs), this complexity is further increased by the behavioral dimension of e-business interactions, e.g. services choreography and orchestration. This workshop aims at providing a venue for discussion and the exchange of ideas on data and knowledge engineering issues in the dynamic environment of e-business, enterprise computing, and business services and transformation.

This paper shows that graph traversal techniques have fundamental differences between serial and distributed computations in their behaviors, computational complexities, and effects on the design of graph algorithms. It has three major parts. Section I describes the computational environment for the design and description of distributed graph algorithms in terms of an architectural model for message exchanges. The computational complexity is measured in terms of the number of messages transmitted. Section II presents several distributed algorithms for the pure traversal, depth-first search, and breadth-first search techniques. Their complexities are also given. Through these descriptions are brought out some of the intrinsic differences in the behaviors and complexities of the fundamental traversal techniques between a serial and a distributed computation environment. Section III gives the distributed version of the Ford and Fulkerson algorithm for the maximum flow problem by means of depth-first search, the largest-augmentation search and breadth-first search. The complexities of these methods are found to be 0(f*|A|), 0((l + logM/(M-1)f*|V||A|) and O(|V|6), respectively, where f* is the maximum flow value of the problem, M is the maximum number of ucs in a cut, |V| is the number of vertices, and |A| is the number of arcs. Lastly, it is shown that the largest augmentation search may be a better method than the other two. This is contrary to the known results in serial computation.

In theDistributed Computing Environment, many processors/nodes remain active in system at any moment, coordinate or operate without interfering. Nodes have certain degree of freedom like own hardware, own code, sometimes ownindependent task, but at the same time there will still be sharing of some common resources and information. One of the main themes of recent research in distributed computing is "locality" (as utilized by decentralized/peer-to-peer systems). Networks grow fast, thus locality and scalability becomes a major issues. Computing security is, at its core, more than a technical issue. It is a fundamental business challenge. Managers have plenty of security alternatives, but have very little real guidance on making intelligent decisions about them. Today's distributed, multi-vendor, Internet-connected environments encompass more insecure systems and networks than ever before. Security in Distributed Computing offers the manager of distributed systems a thorough, common-sense framework for cost-effective computer security. The first step to understand security is to know what the potential risks are, or more specifically, to determine the type and level of security risks for the organizations. Security risks are unique to each organization because they are dependent on the nature of the business and the environment in which the organization operates. Security risk is determined by identifying the assets that need to be protected.

Adaptive Sequential Posterior Simulators for Massively Parallel Computing Environments

In Geophysics measured seismic data sets need to be analyzed using numerical operators. Such operators consist of Fourier Transforms, integral transforms, convolutions and other mathematical concepts. The result of such inversion methods is a realistic description of the subsurface of the earth. In this project we port a seismic processing package, called SEPLIB, which has been developed at Stanford University, to a parallel environment such as the Cray T3E. This package is a cornerstone of current software development for seismic algorithms in the Geophysical Institute at the University of Karlsruhe and as such is used as a basis for other computationally intensive seismic projects. We concentrate in this article on the end user aspect of utilizing High Performance Fortran to implement geophysical seismic algorithms. We compare strategies in which the computational domain is distributed over various processors with ensembles of computational grids aligned using the HPF standard language.

This article presents a review of various hardware and software parallel computing environments so far developed for the implementation of parallel algorithms to solve complex and computationally dominated optimization problems. These parallel environments are composed of parallel programming tools, performance evaluation tools, debuggers and optimization libraries. The environments can be classified based on the ease of implementation, hardware independence, ease to understand, and guarantee of performance.