Analytical Models of Probability Distributions for MPI Point-to-Point Communication Times on Distributed Memory Parallel Computers

We present fast and highly scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include matrix multiplication, matrix chain product, and computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. Our highly scalable parallel computations for these problems are based on a highly scalable implementation of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known parallel time complexities on parallel random access machines (PRAM), the most powerful but unrealistic shared memory model of parallel computing, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast (in terms of parallel time complexity) and highly scalable (in terms of our definition of scalability) parallel matrix computations were rarely seen before on any distributed memory systems.

We present fast and scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. These parallel computations are based on efficient implementations of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known time complexities on PRAM, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast and scalable parallel matrix computations were not seen before on any distributed memory systems.

The complex nature of the ocean environment requires advanced computational acoustic models to gain insights into the dominating physical factors controlling the underwater sound propagation and scattering in the ocean. In this study, we explore the “ab-initio” approach to solve wave equations with numerical algorithms that can be implemented in distributed memory parallel computers. The goal is to improve the calculation speed so long-range global scale hydroacoustic wave propagation can be studied more efficiently and effectively. Two major algorithms: the finite difference time domain (FDTD) method and the Parabolic Equation (PE) method are investigated. Two PE-based numerical models are considered. One is the Split-Step-Fourier Parabolic Equation (SSFPE) model using split-step Fourier schemes in three-dimensional (3D) environments. The second PE model is a multi-frequency implementation of the Range-dependent Acoustic Model (RAM) that computes two-dimensional (2D) sound pressure fields. One of the primary challenges in global-scale hydroacoustic propagation modeling with the “ab-initio” approach is the demand for significant computational resources on both memories and computational speeds. To address this, we employed parallel computing techniques using directive-based programming languages, such as OpenACC and XscalableACC, to leverage the power of multiple Graphics Processing Unit (GPU) systems and Central Processing Unit (CPU) cluster computers. Our performance evaluation revealed substantial speedup gains. For the FDTD method, we achieved approximately 3.5-fold and 4-fold speedups with four GPUs and four CPU cluster nodes, respectively. With the 3D SSFPE model, we obtained a speedup of 3.7-fold using four CPU cluster nodes. A significant result was observed with the 2D broadband multi-frequency RAM model, where we achieved a 110-fold speedup compared to one core original implementation. These results demonstrate the promising potential of massively parallel computers for ocean acoustic propagation modeling and highlight the significant performance benefits of utilizing parallel computing techniques. Our findings emphasize the importance of efficient computational strategies.

An efficient parallel-computing method for modeling nonisothermal multiphase flow and multicomponent transport in porous and fractured media

Ray tracing is a well known technique to generate life-like images. Unfortunately, ray tracing complex scenes can require large amounts of CPU time and memory storage. Distributed memory parallel computers with large memory capacities and high processing speeds are ideal candidates to perform ray tracing. However, the computational cost of rendering pixels and patterns of data access cannot be predicted until runtime. To parallelize such an application efficiently on distributed memory parallel computers, the issues of database distribution, dynamic data management and dynamic load balancing must be addressed. In this paper, we present a parallel implementation of a ray tracing algorithm on the Intel Delta parallel computer. In our database distribution, a small fraction of database is duplicated on each processor, while the remaining part is evenly distributed among groups of processors. In the system, there are multiple copies of the entire database in the memory of groups of processors. Dynamic data management is acheived by an ALRU cache scheme which can exploit image coherence to reduce data movements in ray tracing consecutive pixels. We balance load among processors by distributing subimages to processors in a global fashion based on previous workload requests. The success of our implementation depends crucially on a number of parameters which are experimentally evaluated. © 1997 John Wiley & Sons, Ltd.

PurposeA computational fluid dynamics code for the calculation of laminar hypersonic multi‐species gas flows in chemical non‐equilibrium in axisymmetric or two‐dimensional configuration on shared and distributed memory parallel computers is presented and validated. The code is designed to work efficiently in combination with an automatic domain decompositioning method developed to facilitate efficient parallel computations of various flow problems.Design/methodology/approachThe baseline implicit numerical method developed is the lower‐upper symmetric Gauss‐Seidel scheme, which is combined with a sub‐iteration scheme to achieve time‐accuracy up to third‐order. The spatial discretisation is based on Roe's flux‐difference splitting and various non‐linear flux limiters maintaining total‐variation diminishing properties and up to third‐order spatial accuracy in continuous regions of flow. The domain subdivision procedure is designed to work for single‐ and multi‐block domains without being constrained by the block boundaries, and an arbitrary number of processors used for the computation.FindingsThe code developed reproduces accurately various types of flows, e.g. flow over a flat plate, diffusive mixing and oscillating shock induced combustion around a projectile fired into premixed gas, and demonstrates close to linear scalability within limits of load imbalance.Research limitations/implicationsThe cases considered are axisymmetric or two‐dimensional, and assume laminar flow. An extension to three‐dimensional turbulent flows is left for future work.Originality/valueResults of a parallel computation, utilising a newly developed automatic domain subdivision procedure, for oscillating shock‐induced combustion around a projectile and various other cases are presented. The influence of entropy correction in Roe's flux‐difference splitting algorithm on diffusive mixing of multi‐species flows was examined.

The production of realistic image generated by computer requires a huge amount of computation and a large memory capacity. The use of highly parallel computers allows this process to be performed faster. Distributed memory parallel computers (DMPCs), such as hypercubes or transputer-based machines, offer an attractive performance/cost ratio when the load balancing has been balance and the partition of the data domain has been performed. This paper presents a parallel ray tracing algorithm for DMPC using a Shared Virtual Memory (SVM) which solves these two classical problems. This algorithm has been implemented on a hypercube iPSC/2 and results are given.

Currently, there are two approaches to implementing large-scale ice sheet simulations. One approach, which is based on the use of an evenly-spaced structured grid, is to simulate the entire ice sheet at the highest resolution required. The other approach is to model the ice sheet at different levels of resolution using unstructured meshes. Structured grids lend themselves quite well to partitioning on multi-core, distributed memory parallel computers, leading to good computational efficiency, load-balancing and reduced communication costs. The disadvantage is the significant computational resources required to model a system of this size at a high enough resolution to correctly model regions of the ice sheet undergoing rapid change. Unstructured meshes can reduce such costs by allocating more computational resources to areas of the ice sheet whose dynamics are evolving quickly and fewer resources to areas evolving more slowly. However, unstructured meshes do not easily lend themselves to effective decomposition on distributed memory parallel computers, and can experience poor cache utilization, significant load imbalance, high communication costs, and increased synchronization times. In this paper, we describe our alternative approach based on embedded simulation, which provides a degree of multi-resolution capabilities to structured grids while maintaining many of the computational efficiencies that come with the structured approach. We provide experimental results focusing on the scalability characteristics and cache efficiency of each approach as a function of increasing problem size and core counts. We show that the embedded approach scales very well across all problem sizes and core counts, and that the unstructured approach scales well at smaller problem sizes and core counts but degrades significantly as the problem size becomes large.

A parallel algorithm for the simulation of steady‐state, two‐dimensional flows of integral viscoelastic fluids on Multiple Instruction Multiple DATA (MIMD) computers with distributed memory is proposed. The algorithm has been implemented within the commercial finite element package polyflow developed at Louvain‐la‐Neuve and widely used in polymer processing applications. Parallel performance results obtained on the Intel iPSC/860 hypercube are discussed for the flow of a K.B.K.Z. fluid through an abrupt contraction. It is found that satisfactory levels of parallel speed‐up can be obtained by using a rather simple allocation scheme to schedule workload to the available processors. The optimal use of the parallel computer requires, however, a dynamic load allocation scheme. An adaptive strategy that reallocates the workload to the processors has been proposed on the basis of the history of the nonlinear iterative process. The proposed adaptive allocation scheme is found to carry only marginal compute and communication overheads.

A mesh-vertex finite volume scheme for solving the Euler equations on triangular unstructured meshes is implemented on a multiple-instruction/multiple-data stream parallel computer. An explicit four-stage Runge-Kutta scheme is used to solve two-dimensional flow problems. A family of implicit schemes is also developed to solve these problems, where the linear system that arises at each time step is solved by a preconditioned GMRES algorithm. Two partitioning strategies are employed: one that partitions triangles and the other that partitions vertices. The choice of the preconditioner in a distributed memory setting is discussed. All of the methods are compared both in terms of elapsed times and convergence rates. It is shown that the implicit schemes offer adequate parallelism at the expense of minimal sequential overhead. The use of a global coarse grid to further minimize this overhead is also investigated. The schemes are implemented on a distributed memory parallel computer, the Intel iPSC/860.

Min–max-boundary domain decomposition

Parallel computation of three-dimensional flows using overlapping grids with adaptive mesh refinement

Efficient solid state NMR powder simulations using SMP and MPP parallel computation

with the development of parallel technology and application, the performance analysis and visualization of parallel computing is one of the most important parts. This paper presents a parallel computing performance visualization system which is developed to visualize and analyze the speedup and efficiency of parallel computing. We visualize and analyze the results of magnetotelluric parallel forward modeling program based on MPI which used finite element method. It succeeds in collecting the data of program execution time and communication time, visualizing and analyzing the speedup and efficiency.

The scalability of distributed-memory parallel computers makes them attractive candidates for solving large-scale problems. New languages, such as HPF, FortranD, and VPP Fortran, have been developed to enable existing software to be easily ported to such machines. Many distributed-memory parallel computers have been built, but none of them support the mechanisms required by such languages. We studied the mechanisms required by parallelizing compilers and proposed a new architecture to support them. Based on this proposed architecture, we developed a new distributed-memory parallel computer, the AP1000+, which is an enhanced version of the AP1000. Using scientific applications in VPP Fortran and C, such as NAS parallel benchmarks, we simulated the performance of the AP1000+.