A parallel method for large sparse generalized eigenvalue problems using a GridRPC system

Supporting fault-tolerant and open distributed processing using RPC

We present four numerical methods for computing the singular value decomposition (SVD) of large sparse matrices on a multiprocessor architecture. We emphasize Lanczos and subspace iteration-based methods for determining several of the largest singular triplets (singular values and corresponding left- and right-singular vectors) for sparse matrices arising from two practical applications: information retrieval and seismic reflection tomography. The target architectures for our implementations are the CRAY-2S/4–128 and Alliant FX/80. The sparse SVD problem is well motivated by recent information-retrieval techniques in which dominant singular values and their corresponding singular vectors of large sparse term-document matrices are desired, and by nonlinear inverse problems from seismic tomography applications which require approximate pseudo-inverses of large sparse Jacobian matrices. This research may help advance the development of future out-of-core sparse SVD methods, which can be used, for example, to handle extremely large sparse matrices 0 × (106) rows or columns associated with extremely large databases in query-based information-retrieval applications.

Although high performance computing has been achieved over computational Grids using various techniques, the support for high performance computing on the Grids using Remote Procedure Call (RPC) mechanisms is fairly limited. In this paper, we discuss a RPC system called GrADSolve that supports execution of parallel applications over Grid resources. GrADSolve employs powerful scheduling techniques for dynamically choosing the resources used for the execution of parallel applications and also uses robust data staging mechanisms based on the data distribution used by the end application. Experiments and results are presented to prove that GrADSolve’s data staging mechanisms can significantly reduce the overhead associated with data movement in current RPC systems.

Supernode pivoting for unsymmetric matrices coupled with supernode partitioning and asynchronous computation can achieve high gigaflop rates for parallel sparse LU factorization on shared memory parallel computers. The progress in weighted graph matching algorithms helps to extend these concepts further and prepermutation of rows is used to place large matrix entries on the diagonal. Supernode pivoting allows dynamical interchanges of columns and rows during the factorization process. The BLAS-3 level efficiency is retained. An enhanced left-right looking scheduling scheme is uneffected and results in good speedup on SMP machines without increasing the operation count. These algorithms have been integrated into the recent unsymmetric version of the PARDISO solver. Experiments demonstrate that a wide set of unsymmetric linear systems can be solved and high performance is consistently achieved for large sparse unsymmetric matrices from real world applications.

We present a class of new preconditioners based on the incomplete Givens orthogonalization (IGO) methods for solving large sparse systems of linear equations. In the new methods, instead of dropping entries and accepting fill-ins according to the magnitudes of values and the sparsity patterns, we adopt a diagonal compensation strategy, in which the dropped entries are re-used by adding to the main diagonal entries of the same rows of the incomplete upper-triangular factors, possibly after suitable relaxation treatments, so that certain constraints on the preconditioning matrices are further satisfied. This strategy can make the computed preconditioning matrices possess certain desired properties, e.g., having the same weighted row sums as the target matrices. Theoretical analysis shows that these modified incomplete Givens orthogonalization (MIGO) methods can preserve certain useful properties of the original matrix, and numerical results are used to verify the stability, the accuracy, and the efficiency of the MIGO methods employed to precondition the Krylov subspace iteration methods such as GMRES. Both theoretical and numerical studies show that the MIGO methods may have the potential to present high-quality preconditioners for large sparse nonsymmetric matrices.

The Computational Grid[1] is a promising platform for running large-scale scientific applications. It provides a base software infrastructure that allows for the development of middleware aimed at deploying applications on Grid resources. The question is, how do you program it---in this regard, Network-Enabled Server (NES) paradigm, which enables Grid-based RPC, or GridRPC for short is a good candidate as a viable Grid middleware that offers a simple yet powerful programming paradigm for programming on the Grid. Several systems that facilitate whole or parts of the paradigm are already in existence, such as Neos[7], Netsolve[3], Nimrod/G[4], Ninf[2], and RCS[6], and we feel that pursuit of a common design in GridRPC, as had been done for MPI for message passing, will bring benefits of standardized programming model to the Grid world. This talk will introduce the NES/Grid RPC features, discuss early user experiences, and touch upon the Grid Data Farm project, based on Grid RPC, which involving processing Petabytes of collider accelerator data streaming over the Euro-Japanese link with thousands-node scale cluster possibly spread over several Japanese institutions. Compared to traditional RPC systems, such as CORBA, designed for applications that facilitate nonscientific applications, GridRPC systems offer features and capabilities that make it easy to program mediumto coarse-grained, task parallel applications that involve hundreds to thousands or more high-performance nodes, either concentrated as a tightly coupled cluster, or a set of them spread over a wide-area network. Such applications will often require handling of shipping megabytes of multidimensional array data in a user-transparent and efficient way, as well as requiring the support of RPC calls that range anywhere from 100s of milliseconds up to several days or even weeks. There are other necessary features of Grid RPC systems such as dynamic resource discovery, dynamic load balancing, fault tolerance, security (multisite authentication, delegation of authentication, adapting to multiple security policies, etc.), easy-to-use client/server management, firewall and private address considerations, remote large file and I/O support etc. These features are essentially what is needed for the Grid RPC systems to execute well on the Grid---features either missing or incomplete in traditional `closed world’ RPC systems ---and in fact are what are provided by lower level Grid substrates such as Condor[10], Globus[8], and Legion[9]. As such GridRPC systems either provide these features themselves, or builds upon the features provided by such substrates.

Two state‐of‐the‐art parallel software packages for the direct solution of sparse linear systems based on LU‐decomposition, MUMPS and SuperLU_DIST have been tested as black‐box solvers on problems derived from finite difference discretizations of the Helmholtz equation. The target architecture has been Linux clusters, for which no consistent set of tests of the algorithms implemented in these packages has been published. The investigation consists of series of memory and time scalability checks and has focused on examining the applicability of the algorithms when processing very large sparse matrices on Linux cluster platforms. Special emphasis has been put on monitoring the behaviour of the packages when the equation systems need to be solved for multiple right‐hand sides, which is the case, for instance, when modelling a seismic survey. The outcome of the tests points at poor efficiency of the tested algorithms during application of the LU‐factors in the solution phase on this type of architecture, where the communication acts as an impasse. Copyright © 2005 John Wiley & Sons, Ltd.

The computation of high-precision mathematical constants in a combined cluster and grid environment is presented. Mathematical constants (e.g., π and e) are computed from their series expansions. The binary splitting method recursively reduces the calculation of the sum of the series by effectively splitting the problem into two halves and performing the same calculation on each half. By using grid computing for part of the binary splitting process, the supercomputer computation time, which is very expensive, can be reduced. We implemented the independent binary splitting process in a grid environment using a grid RPC system called OmniRPC. We successfully achieved nearly linear speedup for larger digits on an 8-node PC cluster used over a wide-area network.

An important aspect of designing a distributed application is the communication model we use to connect its various components. Some middleware solutions, such as Web services, offer two of the most important communication paradigms: asynchronous messaging and remote procedure call (RPC). Message-oriented middleware (MOM) lets a service's consumers physically and temporally decouple from the service providers. Communication between service providers and the consumers is asynchronous, and it don't need to be available at the same time because they communicate by sending and receiving messages from designated message queues. In contrast, RPC is a synchronous method of requesting remote service execution. Consumers must suspend service execution until they receive a reply from the provider. MOM and RPC have advantages and disadvantages. MOM solutions tend to be more robust to failures than RPC, and it allows service requesters to continue to process while service provider's work on their requests. However, programming MOM-based applications is more cumbersome because distribution isn't as transparent to the programmer as with RPCs. In this column, we provide a quantitative framework, which we can use to compare MOM- and RPC-based solutions.

In this chapter we present several useful libraries and API which we could not categorize with any of the other libraries we presented in this part of the book. We first present the RPC (remote procedure call) library for remote invocation of predefined functions.We present a real-life example motivated from physical chemistry to use RPC for optimizing runtime using remote servers. In any discussion of cluster computing, data endianess issues always crop up, and a part of the RPC library is XDR (extensible data representation). We continue to present APIs for checksum, and hash signature computation to check and prevent transmission errors. An alternative to XDR is to use XML files, and library APIs for XML processing is presented in this chapter. For persistent storage we show examples of using Berkeley DB, and present examples using C++. A network based caching library (Memcache) is presented with examples in C++ and Python. To seamlessly use Python and other interactive languages, SWIG (the Simplified Wrapper Interface Generator) is presented.

Remote Procedure Calls (RPCs) realize client-server interactions via a request-response message-passing protocol. They simplify distributed application programming by eliminating the need for explicitly having to code the details of a remote interaction. However, none of the existing RPC implementations are designed to work properly for Delay/Disruption-Tolerant Networking (DTN) where network connectivity is periodic, intermittent, and prone to disruptions. In this paper, we present DTN- RPC, a new approach to provide RPCs for DTN environments. DTN-RPC relies on (a) control and data channels to cope with potentially short contact durations in DTN where large amounts of data cannot be transmitted, (b) explicit and implicit modes to address remote servers, (c) Non-DTN and DTN transport protocols for issuing calls and receiving results, and (d) predicates that servers check to decide whether a procedure should be executed. The implementation of DTN-RPC is based on Serval, an open-source, disruption-tolerant wireless ad-hoc networking system. Our experimental results indicate that the measured CPU and network overheads for DTN-RPC are reasonably low so that it can be executed on smartphones or routers, and that the round-trip times and the number of successful RPCs are highly satisfactory in dynamic networks with unstable links.

In the era of Internet, service-oriented development becomes a popular software development paradigm for developing large-scale systems and cloud apps. Followed by this new emerging paradigm, functions would be designed and developed as components, deployed separately and provided as services for integration. In generally, experienced developers tend to introduce some reusable distributed service frameworks in their projects. These frameworks are usually developed based on Service-Oriented Architecture (SOA), and take charge of Remote Procedure Call (RPC), distributed service collaboration, remote service communication and other common duties. It has been proved that these distributed service frameworks are a kind of effective software infrastructures to facilitate development. However, in large-scale cloud ecosystems development, chaos might occur if interdependent remote services composed together to enable cloud services and complex systems. Towards this problem, this paper proposes a novel framework developed by leveraging a platform as a unified access gateway of remote services for different cloud services in the ecosystem. Experimental results show that the framework could effectively reduce the development difficulty of the large-scale cloud ecosystems, and improve the performance of the developed systems.

A scheme is presented that makes a remote procedure call (RPC) mechanism fault-tolerant to hardware failures. Fault tolerance is provided by replicating the procedure at a group of nodes, called a cluster. The copies in a cluster are linearly ordered. A call to a procedure is sent to the first copy in the cluster and is propagated internally to all other copies. In the event of failures, the first copy in the cluster that has not failed returns the result to the caller. The scheme is transparent to the user and supports nested procedure calls. It has been implemented on a network of Sun workstations making use of Sun's existing RPC mechanism. >

Singular Value Decomposition (SVD) is a well studied research topic in many fields and applications from data mining to image processing. Data arising from these applications can be represented as a matrix where this matrix is large and sparse. Most existing algorithms are used to calculate singular values, left and right singular vectors of a largedense matrix but not large-sparse matrix. Even if they can find SVD of a large matrix, calculation of large-dense matrix has high time complexity due to sequential algorithms. Distributed approaches are proposed for computing SVD of large matrices. However, rank of the matrix is still being a problem when solving SVD with these distributed algorithms. In this paper we propose Ranky, set of methods to solve rank problem on large-sparse matrices in a distributed manner. Experimental results show that the Ranky approach recovers singular values, singular left and right vectors of a given large-sparse matrix with negligible error.

Distributed scene graphs are important in virtual reality, both in collaborative virtual environments and in cluster rendering. Modern scalable visualization systems have high local throughput, but collaborative virtual environments (VEs) over a wide‐area network (WAN) share data at much lower rates. This complicates the use of one scene graph across the whole application. Myriad is an extension of the Syzygy VR toolkit in which individual scene graphs form a peer‐to‐peer network. Myriad connections filter scene graph updates and create flexible relationships between nodes of the scene graph. Myriad's sharing is fine‐grained: the properties of individual scene graph nodes to share are dynamically specified (in C++ or Python). Myriad permits transient inconsistency, relaxing resource requirements in collaborative VEs. A test application, WorldWideCrowd, demonstrates collaborative prototyping of a 300‐avatar crowd animation viewed on two PC‐cluster displays and edited on low‐powered laptops, desktops, and over a WAN. We have further used our framework to facilitate collaborative educational experiences and as a vehicle for undergraduates to experiment with shared virtual worlds. Copyright © 2006 John Wiley & Sons, Ltd.