Compiling dynamic mappings with array copies

Recently, the first commercial High Performance Fortran (HPF) subset compilers have appeared. This article reports on our experiences with the xHPF compiler of Applied Parallel Research, version 1.2, for the Intel Paragon. At this stage, we do not expect very High Performance from our HPF programs, even though performance will eventually be of paramount importance for the acceptance of HPF. Instead, our primary objective is to study how to convert large Fortran 77 (F77) programs to HPF such that the compiler generates reasonably efficient parallel code. We report on a case study that identifies several problems when parallelizing code with HPF; most of these problems affect current HPF compiler technology in general, although some are specific for the xHPF compiler. We discuss our solutions from the perspective of the scientific programmer, and presenttiming results on the Intel Paragon. The case study comprises three programs of different complexity with respect to parallelization. We use the dense matrix‐matrix product to show that the distribution of arrays and the order of nested loops significantly influence the performance of the parallel program. We use Gaussian elimination with partial pivoting to study the parallelization strategy of the compiler. There are various ways to structure this algorithm for a particular data distribution. This example shows how much effort may be demanded from the programmer to support the compiler in generating an efficient parallel implementation. Finally, we use a small application to show that the more complicated structure of a larger program may introduce problems for the parallelization, even though all subroutines of the application are easy to parallelize by themselves. The application consists of a finite volume discretization on a structured grid and a nested iterative solver. Our case study shows that it is possible to obtain reasonably efficient parallel programs with xHPF, although the compiler needs substantial support from the programmer.

This paper introduces the ideas that underly the data-parallel language High Performance Fortran (HPF) and the new ideas in version 2 of HPF. It first reviews HPF's key language elements. It discusses the meaning of data parallelism and the limitations of HPF version 1 as a data-parallel programming language. The second part of the paper is a review of the development of version 2 of HPF. The extended language, under development in 1996, includes a richer data mapping capability; an extension to the independent loop that allows reduction operations in the loop range; a means for directing the mapping of computation as well as data; and a way to specify concurrent execution of several parallel tasks on disjoint subsets of processors.

Particle-in-Cell (PIC) plasma simulation codes model the interaction of charged particles with surrounding electrostatic and magnetic fields. Its computational requirements made it to be classified as one of the grand-challenge problems facing the high performance community. In this paper we present the implementation of 1-D and 2-D electrostatic PIC codes in High Performance Fortran(HPF) on a IBM SP-2. HPF expands Fortran 90 with data distribution and alignment directives and data parallel statements. It is a powerful language for writing portable and high performance programs across many platforms. We used one of the most successful commerical HPF compilers currently available in the market and augmented the compiler's missing HPF functions with extrinsic routines when necessary. We obtained near linear speed-up in all of our test cases. The performance of the HPF programs is comparable to the native message passing implementations of the same codes on the SP-2.

Distributed memory parallel machines are increasingly being used by the scientific community to provide computational power for solving problems. Building a distributed memory parallel machine offers a tremendous advantage in terms of cost and scalability over building a shared memory parallel machine. Unfortunately, distributed memory machines are much more difficult to program, mainly due to the lack of a single global address space. Communication between processors in a distributed memory machine is accomplished by message passing. The time cost for acquiring data from another processor's memory is significantly greater than the time cost of acquiring data from local memory. To assist the programmer in using a distributed memory machine, compilers have been developed which alleviate the need for the programmer to manage the lack of a global address space. These compilers will generate semantically correct code containing communication routines to produce a logically shared memory machine on a physically distributed memory machine. However, the runtime performance of the code emitted from some of these compilers can be further improved. This thesis is primarily concerned with advanced compilation techniques which will provide better runtime performance of programs written in imperative languages for regular scientific programs. The optimization of communication generation for distributed memory machines is used to produce this result. We look at two "state of the art" compilation systems and find that improvement in the communication code generated by these compilers is still possible. We have simplified existing descriptions of the various formalisms found in these compilers, while still providing enough formalism to study the problems in depth. We have extended some existing optimization techniques by enhancing the elimination of masks, by extending the notion of relaxing the owner computes rule, by extending the notion of replicating computation, and by extending the notion of replicating data. This thesis has discussed a novel technique to split loops so that they do not contain any guarded statements. Finally, we have suggested avenues for future research, as it is clear from this thesis that more work is required before compilation systems for distributed memory parallel machines will be widely used for general purpose computing.

The High Performance Fortran (HPF) programming language provides the data parallel programming paradigm for high performance architectures with different hierarchies of memory. HPF programs are much easier to write and to read than conventional message passing programs.

The ScaLAPACK numerical library for MIMD distributed-memory parallel computers comprises highly efficient and robust parallel dense linear algebra routines, implemented using explicit message passing. High Performance Fortran (HPF) was developed as an alternative to the message-passing paradigm. It extends Fortran 90 with directives to automatically distribute data and to parallelize loops, such that all required inter-processor communication is generated by the compiler. While HPF can ease parallelization of many applications, it still does not make sense to re-program existing libraries like ScaLAPACK. Rather, programmers should have the opportunity to use them from within HPF programs.

Structured Adaptive Mesh Refinement (SAMR) techniques use a hierarchy of dynamically refined grids to solve PDEs with variable resolution. Exploiting the parallelism in such codes is a complex problem and requires the grids to be efficiently distributed across the target parallel system. High Performance Fortran (HPF) is a data-parallel language that was designed with the goal of providing the user with a high-level interface for programming scientific applications, while delegating the task of generating an explicitly parallel message-passing program to the compiler. In this paper we discuss the issues that arise in using HPF for parallelizing SAMR codes.KeywordsDistribution StrategyAdaptive Mesh RefinementMapping ArrayNASA Langley ResearchTotal Variation Diminish SchemeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Parallel iterative solvers for irregular sparse matrices in High Performance Fortran

DAQV: Distributed array query and visualization framework

High Performance Fortran (HPF) is a data-parallel programming language that allows the programmer to specify the data decomposition onto the processors while the compiler takes care of the tedious tasks of communication generation and computation partitioning. Shifting some of the complex tasks from the user to the compiler should encourage programmers to write and port code to parallel machines especially if the compiler implements these tasks efficiently. In this paper, performance results and analysis of a subset of the SPEC92 is presented for the XL HPF compiler on IBM SP2 machines. In addition to obtaining good performance from the compiler, one of the the main concerns of HPF users is portability. Experimental results and analysis are presented in this paper to investigate performance portability (consistency) first across multiprocessor architectures and then across compilers. For performance portability across multiprocessor machines, the same XL HPF compiler used for the IBM SP2 distributed memory machine experiment is also used to compile and execute the same applications but on IBM SMP machines. The comparable speedup and behaviour obtained for both machines indicates that HPF compilers can be portable across different architectures. For performance portability across compilers, various HPF programming techniques and recommendations are introduced to increase the chances of obtaining performance consistency with different HPF compilers.

Efficient Computation of Address Sequences in Data Parallel Programs Using Closed Forms for Basis Vectors

High Performance Fortran is a dataparallel language that allows the user to specify the parallelism in his program. It is not always easy to extract the parallelism in a given program. To help the user, an automatic loop parallelizer has been developed: Bouclettes.Bouclettes has been written to validate some scheduling and mapping techniques that are mentioned in this paper. A Fortran 77 loop nest is rewritten into a HPF loop nest with explicit parallel loops (INDEPENDENT loops) and explicit data distribution (DISTRIBUTE and ALIGN directives).The different stages of parallelization (analysis, dependence analysis, scheduling, mapping and code generation) and their implementation issues are described here.KeywordsData DependenceLoop NestIteration DomainIndependent LoopAlign DirectiveThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Loops are the main source of parallelism in scientific programs. Hence, several techniques were developed to detect parallelism in these loops and transform them into parallel forms. In this dissertation, compile time transformations and efficient parallel execution of loops with various type of dependencies are investigated. First, Doacross loops with uniform dependencies are considered for execution on distributed memory parallel machines (multicomputers). Most known Doacross loop execution techniques can be applied efficiently only to shared memory parallel machines. In this thesis, code reordering technique, improvements to partitioning strategies, and finding a balance between communication and parallelism are presented to reduce the execution time of Doacross loops on multicomputers. As with most parallelizing transformation techniques, only single loopnests are considered in the first part of this dissertation. However, parallelizing each loopnest in a program separately, even if an optimal execution can be obtained for each loopnest, may not result in an efficient execution of all the code in the program because of communication overhead across the loops in a multicomputer environment. Hence, across loop data dependence representation and analysis are introduced in this work to improve the parallel execution of the whole code. Our contribution consists of finding and representing data dependencies whose sources and destinations are subspaces of the iteration space mainly common across the loops. This type of dependence information is used in this thesis to improve global iteration space partitioning, automatic generation of communication statements across loops, and index alignment. The final part of this dissertation presents new parallelizing techniques for loops with irregular and complex dependencies. Various data dependence analysis algorithms can be found in the literature even for loops with complex array indexing. However, the improvement in data dependence testing has not been followed by similar amelioration in restructuring transformations for loops with complex dependencies. Such loops are mostly executed in serial mode. Our parallelizing techniques for these loops consists of identifying regions of the iteration space where all iterations can be executed in parallel. The advantages of all the transformations presented in this dissertation are: (1) they significantly reduce the execution time of loops with various types of dependencies as shown in this work using the MasPar machine (2) they can be implemented at compile, time which makes the task of parallel programming easier.

Over the last year, a considerable portion of FEFLO, a general-purpose legacy CFD code operating on unstructured grids, was ported to run on GPUs. Like so many other legacy codes, FEFLO is an adaptive, edge-based nite element code for the solution of compressible and incompressible ow, which was primarily written in Fortran 77 and has previously been ported to vector, shared memory parallel and distributed memory parallel machines. Due to the large size of FEFLO and the likelihood of human error in porting, as well as the desire for continued development within a single codebase, a specialized Python script, based on FParser (Peterson, 2009), was written to perform automated translation from the OpenMP-parallelized loops to GPU kernels implemented in CUDA, along with GPU memory management, while integrating with the existing framework for distributed memory parallelism via MPI. The present paper describes extensions of the script and algorithmic techniques that enable the ecient running on GPUs of the modules that are not straightforward to port. In particular, we consider LU-SGS algorithms, linelet preconditioning and particle-mesh algorithms.

Our goal is to ease the parallelization of applications on distributed-memory parallel processors. Part of our team is implementing parallel kernels common to industrially significant applications using High Performance Fortran (HPF) and the Message Passing Interface (MPI). They are assisted in this activity by a second group developing an integrated tool environment, Annai, consisting of a parallelization support tool, a debugger, and a performance monitor and analyzer. These two groups interact closely, with application developers defining requirements and evaluating prototypes of the tool environment. The Annai environment is unique in providing a suite of general-purpose tools for scientific application development, encompassing comprehensive support for programming irregular problems and the debugging and tuning of HPF and/or MPI-based parallel programs. Addressing application developer requirements, while making leading-edge technology available in a genuinely-usable form, has resulted in the timely provision of powerful parallel programming tools. The paper describes goals, achievements and perspectives of the project, illustrating with specific case studies of three application kernels how the tool environment assists in the parallelization process: development effort and resulting performance are discussed.