Communication generation for aligned and cyclic(k) distributions using integer lattice

This paper introduces the ideas that underly the data-parallel language High Performance Fortran (HPF) and the new ideas in version 2 of HPF. It first reviews HPF's key language elements. It discusses the meaning of data parallelism and the limitations of HPF version 1 as a data-parallel programming language. The second part of the paper is a review of the development of version 2 of HPF. The extended language, under development in 1996, includes a richer data mapping capability; an extension to the independent loop that allows reduction operations in the loop range; a means for directing the mapping of computation as well as data; and a way to specify concurrent execution of several parallel tasks on disjoint subsets of processors.

Notice of Violation of IEEE Publication Principles<br><br> "Communication Generation for Irregular Parallel Applications,"<br>by Changjun Hu, Jing Li, Jue Wang, Yonghong Li, Liang Ding, and Jianjiang Li,<br>in the Proceedings of the International Symposium on Parallel Computing in Electrical Engineering, 2006 (PAR ELEC 2006) pp.263-270<br><br> After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE's Publication Principles.<br><br>This paper contains portions of original text from the papers cited below. The original text was copied without appropriate attribution.<br><br> "Communication Generation for Aligned and Cyclic (K) Distributions Using Integer Lattice," <br>by Eric Hung-Yu Tseng and Jean-Luc Gaudiot<br>in IEEE Transactions on Parallel and Distributed Systems, 10(2), February 1999 <br/> Irregular computing significantly influences the performance of large scale parallel applications. How to generate local memory access sequence and communication set efficiently for irregular parallel application is an important issue in compiling a data parallel language into a single program multiple data (SPMD) code for distributed-memory machines. In this paper, we propose a hybrid approach, which combines the advantages of the algebra method and the integer lattice method. Our approach derives an algebraic solution of communication set enumeration at compile time for the situation of irregular array references in nested loops. Based on the integer lattice, we develop our method for global-to-local and local-to-global index translations in the case of alignment and cyclic (k) distribution. Then, we present our algorithm for the corresponding SPMD code generation, which adopts some communication optimization techniques. In our method, when parameters are known, the communication set generation, the global-to-local and local-to-global index translations, and the SPMD code generation can be completed at compile time without inspector phase of run time

Structured Adaptive Mesh Refinement (SAMR) techniques use a hierarchy of dynamically refined grids to solve PDEs with variable resolution. Exploiting the parallelism in such codes is a complex problem and requires the grids to be efficiently distributed across the target parallel system. High Performance Fortran (HPF) is a data-parallel language that was designed with the goal of providing the user with a high-level interface for programming scientific applications, while delegating the task of generating an explicitly parallel message-passing program to the compiler. In this paper we discuss the issues that arise in using HPF for parallelizing SAMR codes.KeywordsDistribution StrategyAdaptive Mesh RefinementMapping ArrayNASA Langley ResearchTotal Variation Diminish SchemeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Efficient Computation of Address Sequences in Data Parallel Programs Using Closed Forms for Basis Vectors

Recently, the first commercial High Performance Fortran (HPF) subset compilers have appeared. This article reports on our experiences with the xHPF compiler of Applied Parallel Research, version 1.2, for the Intel Paragon. At this stage, we do not expect very High Performance from our HPF programs, even though performance will eventually be of paramount importance for the acceptance of HPF. Instead, our primary objective is to study how to convert large Fortran 77 (F77) programs to HPF such that the compiler generates reasonably efficient parallel code. We report on a case study that identifies several problems when parallelizing code with HPF; most of these problems affect current HPF compiler technology in general, although some are specific for the xHPF compiler. We discuss our solutions from the perspective of the scientific programmer, and presenttiming results on the Intel Paragon. The case study comprises three programs of different complexity with respect to parallelization. We use the dense matrix‐matrix product to show that the distribution of arrays and the order of nested loops significantly influence the performance of the parallel program. We use Gaussian elimination with partial pivoting to study the parallelization strategy of the compiler. There are various ways to structure this algorithm for a particular data distribution. This example shows how much effort may be demanded from the programmer to support the compiler in generating an efficient parallel implementation. Finally, we use a small application to show that the more complicated structure of a larger program may introduce problems for the parallelization, even though all subroutines of the application are easy to parallelize by themselves. The application consists of a finite volume discretization on a structured grid and a nested iterative solver. Our case study shows that it is possible to obtain reasonably efficient parallel programs with xHPF, although the compiler needs substantial support from the programmer.

Data parallel languages designed for distributed memory computing environments provide a single global address space to the programmer. The mapping from this global address space to the distributed local address space is performed by a compiler, which does this mapping based on the array distribution format. Thus, each array in a data parallel language program has its own distribution format. The Reshape function changes the array distribution format as well as the array shape. However, the changed distribution format cannot be represented by any distribution format supported in current languages. Because there is no suitable distribution format, it is necessary to change the reshaped distribution format to an existing distribution format, with heavy overhead due to the redistribution function. To eliminate the redistribution step in Reshape function, we have proposed a new distribution format, HIER-CYCLIC, which can represent the reshaped distribution format. We have also proposed a language syntax to use HIER-CYCLIC and a compiling mechanism. Finally, we performed an experiment on an IBM-SP2 machine using a shift function.

High Performance Fortran is a dataparallel language that allows the user to specify the parallelism in his program. It is not always easy to extract the parallelism in a given program. To help the user, an automatic loop parallelizer has been developed: Bouclettes.Bouclettes has been written to validate some scheduling and mapping techniques that are mentioned in this paper. A Fortran 77 loop nest is rewritten into a HPF loop nest with explicit parallel loops (INDEPENDENT loops) and explicit data distribution (DISTRIBUTE and ALIGN directives).The different stages of parallelization (analysis, dependence analysis, scheduling, mapping and code generation) and their implementation issues are described here.KeywordsData DependenceLoop NestIteration DomainIndependent LoopAlign DirectiveThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

DAQV: Distributed array query and visualization framework

Array remapping are useful to many applications on distributed memory parallel machines. They are available in High Performance Fortran, a Fortran-based data-parallel language. This paper describes techniques to handle dynamic mappings through simple array copies: array remapping are translated into copies between statically mapped distinct versions of the array. It discusses the language restrictions required to do so. The remapping graph which captures all remapping and liveness information is presented, as well as additional data-flow optimizations that can be performed on this graph, so as to avoid useless remapping at run time. Such useless remapping appear for arrays that are not used after a remapping. Live array copies are also kept to avoid other flow-dependent useless remappings. Finally the code generation and runtime required by our scheme are discussed. These techniques are implemented in our prototype HPF compiler.

Object constraint language for code generation from activity models

This paper describes extensions to OpenMP that implement data placement features needed for NUMA architectures. OpenMP is a collection of compiler directives and library routines used to write portable parallel programs for shared‐memory architectures. Writing efficient parallel programs for NUMA architectures, which have characteristics of both shared‐memory and distributed‐memory architectures, requires that a programmer control the placement of data in memory and the placement of computations that operate on that data. Optimal performance is obtained when computations occur on processors that have fast access to the data needed by those computations. OpenMP ‐‐ designed for shared‐memory architectures ‐‐ does not by itself address these issues. The extensions to OpenMP Fortran presented here have been mainly taken from High Performance Fortran. The paper describes some of the techniques that the Compaq Fortran compiler uses to generate efficient code based on these extensions. It also describes some additional compiler optimizations, and concludes with some preliminary results.

The college credit system allows students to take their own courses according to their own interests and development plans, but since the time for course-choosing is limited and the huge corresponding data requires a large amount of human resources, it is necessary to use computer software for course-choosing. The proposed system follows the complete design flow of computer software. System requirement analysis was done around the core function of the system which is course selection. Based on the system requirement analysis, the system function and the data structure were determined. And then, the system information flow, the code generation algorithm, the back-end database, the course setting algorithm, and the course-choosing algorithm were designed. In addition, a new algorithm for student ID generation is proposed. The student ID generated by this algorithm has simple structure, and also follows the principle of unique, reasonableness and extensibility. Therefore, this course-choosing system could be applied in every college and university.

We present a higher-order programmer-level technique for compiling particular kinds of irregular data-parallel problems to parallel hardware. The technique, which we have named ``flattening-by-expansion'' builds on a number of segmented data-parallel operations but is itself implemented as a higher-order generic function, which makes it useful for many irregular problems. Concretely, the implementation is given in Futhark and we demonstrate the usefulness of the functionality for a number of irregular problems and show that, in practice, the irregular problems are compiled to efficient parallel code that can be executed on GPUs. The technique is useful in any data-parallel language that provides a key set of primitives.

In high-performance systems execution time is of crucial importance justifying advanced optimization techniques. Traditionally, optimization is based on static program analysis. The quality of program optimizations, however, can be substantially improved by utilizing runtime information. Probabilistic data-flow frameworks compute the probability with what data-flow facts may hold at some program point based on representative profile runs. Advanced optimizations can use this information in order to produce highly efficient code. In this paper we introduce a novel optimization technique in the context of High Performance Fortran (HPF) that is based on probabilistic data-flow information. We consider statically undefined attributes which play an important role for parallelization and compute for those attributes the probabilities to hold some specific value during runtime. For the most probable attribute values highly-optimized, specialized code is generated. In this way significantly better performance results can be achieved. The implementation of our optimization is done in the context of VFC, a source-to-source parallelizing compiler for HPF/F90.

This paper presents extensions of High Performance Fortran (HPF) for specifying non-local access patterns of distributed arrays, called halos, and for controlling communication associated with these non-local accesses. Using these features, crucial optimization techniques required for an efficient parallelization of irregular applications may be applied. The information provided by halos is utilized by the compiler and the runtime system for optimizing the management of distributed arrays and the computation of communication schedules. High-level communication primitives for halos enable the programmer to avoid redundant communication, to reuse communication schedules, and to hide communication overheads by overlapping communication with computation. Performance results of a kernel from a crash simulation application on the NEC Cenju-4, the IBM SP2, and on the NEC SX-4 demonstrate that by using the proposed extensions, a performance close to handwritten message-passing codes can be achieved for irregular problems. Copyright © 2000 John Wiley & Sons, Ltd.