A structuring technique for compute-aggregate-broadcast algorithms on distributed memory computers

Winograd’s algorithm to multiply two n × n matrices reduces the asymptotic operation count from O(n 3) of the traditional algorithm to O(n 2.81), hence on distributed memory computers, the combination of Winograd’s algorithm and the parallel matrix multiplication algorithms always gives remarkable results. Within this combination, the application of Winograd’s algorithm at the inter-processor level requires us to solve more difficult problems but it leads to more effective algorithms. In this paper, a general formulation of these algorithms will be presented. We also introduce a scalable method to implement these algorithms on distributed memory computers. This work also opens a new direction to parallelize Winograd’s algorithm based on the generalization of Winograd’s formula for the case where the matrices are partitioned into 2 k parts (the case k = 2 gives us the original formula).

We introduce the use of the task migration paradigm for compiling and executing fine grain parallel applications on distributed memory computers. With this model, the execution of each iteration of a parallel loop is a task. When a task needs to access non local data, it migrates on the processing element which owns this data, and its execution resumes on this processor. The parallel data are never moved unlike the owner compute rule paradigm where the data is explicitly sent to the processor using it or in SVM systems where the data is implicitly requested. Our experiments show that good performance on irregular codes can be obtained on a distributed memory computer even when the application exhibits poor memory locality.

This paper presents a computational performance analysis of a parallel implementation of a conjugate gradient (CG) solver using domain decomposition and distributed memory computers, applied to a 3-D finite element method problem. The results show a superlinear speedup, which is not usually expected. The analysis shows why and how it can happen.

The efficient solution of many large-scale scientific calculations depends on unstructured mesh strategies. For example, problems where the solution changes rapidly in small regions of the domain require an adaptive mesh strategy. In this paper we discuss the main algorithmic issues to be addressed with an integrated approach to solving these problems on massively parallel architectures. We review new parallel algorithms to solve two significant problems that arise in this context: the refinement mesh and the linear solver. A procedure to support parallel refinement and redistribution of two dimensional unstructured finite element meshes on distributed memory computers is presented. The parallelization of the solver is based on a parallel conjugate gradient method using domain decomposition. The error indicator and the resulting refinement parameters are computed in parallel.

Algebraic multigrid methods are well suited as preconditioners for iterative solvers. We consider linear systems of equations which are sparse and symmetric positive definite and which stem from a finite element discretization of a second order self-adjoint elliptic partial differential equation or a system of them. Since preconditioners based on algebraic multigrid are very efficient, additional speedup can only be achieved by parallelization. In this paper, we propose a general parallel algebraic multigrid algorithm for finite element discretizations based on domain decomposition ideas which is well suited for distributed memory computers. This paper pays special attention to the coarsening strategy which has to be adapted in the parallel case. Moreover, a general framework of data distribution gives rise to a construction scheme for the prolongation operators. Results of numerical studies on parallel computers with distributed memory are presented which show the high efficiency of the approach.

This paper considers the problem of distributing data and code among the processors of a distributed memory supercomputer. Provided that the source program is amenable to detailed dataflow analysis, one may determine a placement function by an algorithm analogous to Gaussian elimination. Such a function completely characterizes the distribution by giving the identity of the virtual processor on which each elementary calculation is executed. One has then to “realize” the virtual processors on the PE. The resulting structure satisfies the “owner computes” rule and is reminiscent of two-level distribution schemes, like HPF's ALIGN and DISTRIBUTE directives, or the CM-2 virtual processor system.

There exist a large number of scientific and industrial applications with inherently unstructured parallelism which cannot be solved effectively using a data parallel approach. In this paper we present TGEX, a generic tool for the parallel execution of unstructured problems on distributed memory computers. A general computational problem is first expressed in terms of a computation task graph which serves as a machine-independent representation of the parallel program. TGEX then executes the task graph in a macro data flow style on the target distributed-memory machine. Thus TGEX simplifies parallel programming by freeing the application programmer from the burden of low level task synchronisation and communication. TGEX is implemented using PVM and is therefore portable on a large number of distributed memory systems.KeywordsParallel ProgramParallel ExecutionTask GraphCentral SchedulerUnstructured ProblemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The gradual evolution of language features and approaches used for the programming of distributed memory machines underwent substantial advances in the 1990s. One of the most promising and widely praised approaches was based on data parallelism and resulted in High Performance Fortran. This paper reports on an experiment using that approach based on a commercial distributed memory machine, available compilers and simple test programs. The results are disappointing and not encouraging. The variety of components involved and the lack of detailed knowledge available for the compilers compound the difficulties of obtaining results and doing comparisons. The results show great variation and question the premise that communication is the decisive factor in performance determination. The results are also a contribution towards the difficult tasks of predicating performance on a distributed memory computer.

Design and implementation of Java bindings in Open MPI

Many parallel applications require periodic redistribution of workloads and associated data. In a distributed memory computer, this redistribution can be difficult if limited memory is available for receiving messages. We propose a model for optimizing the exchange of messages under such circumstances which we call the minimum phase remapping problem. We first show that the problem is NP-complete, and then analyze several methodologies for addressing it. First, we show how the problem can be phrased as an instance of multicommodity flow. Next, we study a continuous approximation to the problem. We show that this continuous approximation has a solution which requires at most two more phases than the optimal discrete solution, but the question of how to consistently obtain a good discrete solution from the continuous problem remains open. We also devise a simple and practical approximation algorithm for the problem with a bound of 1.5 times the optimal number of phases. We also present an empirical study of variations of our algorithms which indicate that our approaches are quite practical.

Scalable parallel suffix array construction

The three-dimensional nonlinear evolution of moderate wavelength pressure-driven edge-localized instabilities is calculated in divertor tokamaks. The evolution consists of several stages: linear growth, nonlinear saturation and turbulence, shear flow generation, pressure outflow to the outboard divertor, and finally, outflow to the inboard divertor. The shear flow generation appears to be an important factor in transporting pressure perturbations to the inboard side, and in causing pressure loss to the inboard divertor. The physical model consists of dissipative compressional reduced magnetohydrodynamics which includes the important effect of sound wave propagation. A novel numerical discretization, using a poloidal unstructured mesh and a staggered toroidal mesh, has been implemented on a parallel, distributed memory computer. Nonlinear, three-dimensional numerical computations include a separatrix X-point in the computational domain.

Exploiting multiple levels of parallelism in Molecular Dynamics based calculations via modern techniques and software paradigms on distributed memory computers

An unstructured multi-block local grid refinement solver for the 3-D Euler equations and its implementation on distributed memory computers

The implementation of a large finite element program used for nonlinear industrial applications on an iPSC/2 Hypercube is described in this paper. Two strategies for the solution of the linearized system of equations are outlined and tested. A new method for the automatic spatial decomposition of the discretized finite element domain is presented and compared with other approaches. Finally, strategies for adaptive mesh refinement are discussed.