An Automatic Computation and Data Decomposition Algorithm of Prioritized Dominant Array

The success of large-scale parallel architectures is limited by the difficulty of developing machine-independent parallel programs. We have developed Fortran D, a version of Fortran extended with data decomposition specifications, to provide a portable data-parallel programming model. This paper presents the design of two key components of the Fortran D programming system: a prototype compiler and an environment to assist automatic data decomposition. The Fortran D compiler addresses program partitioning, communication generation and optimization, data decomposition analysis, run-time support for unstructured computations, and storage management. The Fortran D programming environment provides a static performance estimator and an automatic data partitioner. We believe that the Fortran D programming system will significantly ease the task of writing machine-independent data-parallel programs.

A critical issue in achieving the performance of data parallel programs is how to efficiently decompose data across processors. On distributed-memory machines, a good data decomposition should increase processor workload balance and reduce interprocessor communication. Data decomposition consists of data distribution and data alignment. In this paper, we propose a trapezoid data distribution pattern and new data alignment algorithms using alignment graph (AG). Our AG-based alignment framework is unique from other related work because it takes advantage of the effect of optimal expression evaluation with regard to multiple assignment statements.

Structural safety analysis of fixed offshore platforms

Efficient solid state NMR powder simulations using SMP and MPP parallel computation

Currently, there are two approaches to implementing large-scale ice sheet simulations. One approach, which is based on the use of an evenly-spaced structured grid, is to simulate the entire ice sheet at the highest resolution required. The other approach is to model the ice sheet at different levels of resolution using unstructured meshes. Structured grids lend themselves quite well to partitioning on multi-core, distributed memory parallel computers, leading to good computational efficiency, load-balancing and reduced communication costs. The disadvantage is the significant computational resources required to model a system of this size at a high enough resolution to correctly model regions of the ice sheet undergoing rapid change. Unstructured meshes can reduce such costs by allocating more computational resources to areas of the ice sheet whose dynamics are evolving quickly and fewer resources to areas evolving more slowly. However, unstructured meshes do not easily lend themselves to effective decomposition on distributed memory parallel computers, and can experience poor cache utilization, significant load imbalance, high communication costs, and increased synchronization times. In this paper, we describe our alternative approach based on embedded simulation, which provides a degree of multi-resolution capabilities to structured grids while maintaining many of the computational efficiencies that come with the structured approach. We provide experimental results focusing on the scalability characteristics and cache efficiency of each approach as a function of increasing problem size and core counts. We show that the embedded approach scales very well across all problem sizes and core counts, and that the unstructured approach scales well at smaller problem sizes and core counts but degrades significantly as the problem size becomes large.

Efficient and scalable performance modeling is essential to high-performance cluster computing. The critical path based performance analysis has been widely used as it provides valuable insights into the performance of parallel programs, but it is also expensive, inefficient, and inflexible due to its strong reliance on trace-driven simulation. This paper presents an innovative performance modeling framework based on a novel concept of critical-path candidates. The critical-path candidates refer to a group of paths that could potentially be the critical path. Using the instruction and communication counts as the metrics, the critical-path candidate captures the intrinsic computation and communication dependencies, and hence can be reused for exploring multiple design options. Using real-world MPI workloads, we show that the proposed framework achieves a modeling accuracy within 10% compared with the measured runtime for up to 16K MPI ranks. This framework provides an efficient and scalable platform for performance analysis as well as load imbalance analysis.

The complex nature of the ocean environment requires advanced computational acoustic models to gain insights into the dominating physical factors controlling the underwater sound propagation and scattering in the ocean. In this study, we explore the “ab-initio” approach to solve wave equations with numerical algorithms that can be implemented in distributed memory parallel computers. The goal is to improve the calculation speed so long-range global scale hydroacoustic wave propagation can be studied more efficiently and effectively. Two major algorithms: the finite difference time domain (FDTD) method and the Parabolic Equation (PE) method are investigated. Two PE-based numerical models are considered. One is the Split-Step-Fourier Parabolic Equation (SSFPE) model using split-step Fourier schemes in three-dimensional (3D) environments. The second PE model is a multi-frequency implementation of the Range-dependent Acoustic Model (RAM) that computes two-dimensional (2D) sound pressure fields. One of the primary challenges in global-scale hydroacoustic propagation modeling with the “ab-initio” approach is the demand for significant computational resources on both memories and computational speeds. To address this, we employed parallel computing techniques using directive-based programming languages, such as OpenACC and XscalableACC, to leverage the power of multiple Graphics Processing Unit (GPU) systems and Central Processing Unit (CPU) cluster computers. Our performance evaluation revealed substantial speedup gains. For the FDTD method, we achieved approximately 3.5-fold and 4-fold speedups with four GPUs and four CPU cluster nodes, respectively. With the 3D SSFPE model, we obtained a speedup of 3.7-fold using four CPU cluster nodes. A significant result was observed with the 2D broadband multi-frequency RAM model, where we achieved a 110-fold speedup compared to one core original implementation. These results demonstrate the promising potential of massively parallel computers for ocean acoustic propagation modeling and highlight the significant performance benefits of utilizing parallel computing techniques. Our findings emphasize the importance of efficient computational strategies.

Ray tracing is a well known technique to generate life-like images. Unfortunately, ray tracing complex scenes can require large amounts of CPU time and memory storage. Distributed memory parallel computers with large memory capacities and high processing speeds are ideal candidates to perform ray tracing. However, the computational cost of rendering pixels and patterns of data access cannot be predicted until runtime. To parallelize such an application efficiently on distributed memory parallel computers, the issues of database distribution, dynamic data management and dynamic load balancing must be addressed. In this paper, we present a parallel implementation of a ray tracing algorithm on the Intel Delta parallel computer. In our database distribution, a small fraction of database is duplicated on each processor, while the remaining part is evenly distributed among groups of processors. In the system, there are multiple copies of the entire database in the memory of groups of processors. Dynamic data management is acheived by an ALRU cache scheme which can exploit image coherence to reduce data movements in ray tracing consecutive pixels. We balance load among processors by distributing subimages to processors in a global fashion based on previous workload requests. The success of our implementation depends crucially on a number of parameters which are experimentally evaluated. © 1997 John Wiley & Sons, Ltd.

We present fast and highly scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include matrix multiplication, matrix chain product, and computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. Our highly scalable parallel computations for these problems are based on a highly scalable implementation of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known parallel time complexities on parallel random access machines (PRAM), the most powerful but unrealistic shared memory model of parallel computing, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast (in terms of parallel time complexity) and highly scalable (in terms of our definition of scalability) parallel matrix computations were rarely seen before on any distributed memory systems.

We present fast and scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. These parallel computations are based on efficient implementations of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known time complexities on PRAM, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast and scalable parallel matrix computations were not seen before on any distributed memory systems.

The scalability of distributed-memory parallel computers makes them attractive candidates for solving large-scale problems. New languages, such as HPF, FortranD, and VPP Fortran, have been developed to enable existing software to be easily ported to such machines. Many distributed-memory parallel computers have been built, but none of them support the mechanisms required by such languages. We studied the mechanisms required by parallelizing compilers and proposed a new architecture to support them. Based on this proposed architecture, we developed a new distributed-memory parallel computer, the AP1000+, which is an enhanced version of the AP1000. Using scientific applications in VPP Fortran and C, such as NAS parallel benchmarks, we simulated the performance of the AP1000+.

An efficient parallel-computing method for modeling nonisothermal multiphase flow and multicomponent transport in porous and fractured media

Parallel symbolic programs usually exhibit certain characteristics: dynamic, data-dependent behavior patterns; complex run-time interactions that cross modularity boundaries; and control structures characterized by recursion and hierarchy. Studying the performance of such programs requires flexible tools that can adapt to the many different execution patterns that can occur.

Our experiments with Qlisp have led to two ways of improving the performance of parallel Lisp programs. One is to reduce the cost of process creation and scheduling; the other is to use a dynamic partitioning and scheduling method. We describe these techniques and present the results of several experiments that use them. We also present an analysis of the dynamic partitioning method to explain the reasons for its success.

The performance of SPMD parallel programs is strongly affected by dynamic load imbalancing factors. The use of a suitable load balancing strategy is essential in overcoming the effects of these imbalancing factors. This chapter deals with concepts and experiments related to load balancing in SPMD applications. Initially, we discuss a set of classification criteria for load balancing algorithms designed for SPMD applications. In addition, we define a load imbalancing index in order to measure the load imbalance of a parallel application execution. In the experimental part of this chapter, we describe the development of an SPMD parallel application which computes the macroscopic thermal dispersion in porous media. Nine versions of this scientific application were developed, each one adopting a different load balancing strategy. We evaluate and compare the performance of these nine versions and show the importance of using an appropriate load balancing strategy for the characteristics of a specific SPMD parallel application.