Interprocedural Array Region Analyses

Distributed memory parallel machines are increasingly being used by the scientific community to provide computational power for solving problems. Building a distributed memory parallel machine offers a tremendous advantage in terms of cost and scalability over building a shared memory parallel machine. Unfortunately, distributed memory machines are much more difficult to program, mainly due to the lack of a single global address space. Communication between processors in a distributed memory machine is accomplished by message passing. The time cost for acquiring data from another processor's memory is significantly greater than the time cost of acquiring data from local memory. To assist the programmer in using a distributed memory machine, compilers have been developed which alleviate the need for the programmer to manage the lack of a global address space. These compilers will generate semantically correct code containing communication routines to produce a logically shared memory machine on a physically distributed memory machine. However, the runtime performance of the code emitted from some of these compilers can be further improved. This thesis is primarily concerned with advanced compilation techniques which will provide better runtime performance of programs written in imperative languages for regular scientific programs. The optimization of communication generation for distributed memory machines is used to produce this result. We look at two "state of the art" compilation systems and find that improvement in the communication code generated by these compilers is still possible. We have simplified existing descriptions of the various formalisms found in these compilers, while still providing enough formalism to study the problems in depth. We have extended some existing optimization techniques by enhancing the elimination of masks, by extending the notion of relaxing the owner computes rule, by extending the notion of replicating computation, and by extending the notion of replicating data. This thesis has discussed a novel technique to split loops so that they do not contain any guarded statements. Finally, we have suggested avenues for future research, as it is clear from this thesis that more work is required before compilation systems for distributed memory parallel machines will be widely used for general purpose computing.

Recently, use of parallel machines in reservoir simulation has received considerable attention from the petroleum industry. This paper presents parallelization of a 3D compositional, equation-of-state reservoir simulator on the CM-5 and KSR-1. To the best of our knowledge, this is the first time that the parallelization of a compositional reservoir simulator has been performed on both the CM-5 and KSR-1. For new users of the CM-5 machines, the software and hardware of CM-5 architecture is presented, as well as details of the parallelization techniques. For example, domain decomposition, I/O's, phase equilibrium computations, and well model are described. The parallelism techniques on the KSR-1 are presented with the emphasis on the porting of the phase equilibrium calculation. The performance of each machine is evaluated by showing the speedup on different sets of processing nodes. Two test problems were used to explore the capability of the parallelized version of the code; one is a waterflood problem and the other is a CO2 multiple contact miscible flood, both in a West Texas oil field. These field problems were run on 1, 2, 4, 8, 16, and 32 processors to get insight into the locations of communication bottlenecks, generally occurring in the programming with distributed memory machines. The problems of latency and bandwidth which are associated with communication efficiency of the CM-5 are also addressed.

This paper presents a performance estimator, a prototype of which is implemented within the parallel programming environment PCASE. The estimation is based on static performance prediction, using not only the design information of target machines but also benchmarking results. Additionally, communication costs are estimated based on a hierarchical memory machine model, which enables users to understand all the underlying communication costs on distributed memory machines for each parallel loop. With this performance suggestion, it is possible to interactively optimize data distribution and appropriately select vectorized or parallelized loops. Moreover, the skeleton profiling method is presented. It makes high speed trace (execution count of each statement) generation possible by deleting statements which do not affect the execution path of a program.KeywordsExecution PathExecution CostTarget MachineMemory MachineVector InstructionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Chapter 6 - Multiprocessor Machines

A number of computer scientists have contended that Linda cannot possibly be implemented efficiently on distributed memory machines because there is simply too much overhead. They believe that Linda will not be able to compete with message passing on such machines even for solving computationally intensive problems. The authors address this claim by discussing C-Linda's performance in solving a particular scientific computing problem, the shallow water equations. They have implemented and evaluated the performance of the Linda program on a variety of machines. They present results for shared memory machines (Sequent Symmetry and the Encore Multimax), for distributed memory machines (iPSC/2 and iPSC/860 hypercubes), and for a network of Sparcstations connected by an Ethernet. The same Linda program was executed on all these machines and its performance was evaluated and compared to that of implementations using alternative methods available on all machines. In the authors' experience, the Linda program has generally been easier and more convenient to write than the native versions for each machine. >

Graph algorithms are notorious for not getting good speedup on parallel architectures. These algorithms tend to suffer from irregular dependencies and a high synchronization cost that prevent an efficient execution on distributed memory machines. Hence such algorithms are mostly parallelized on shared memory machines. However, current commodity shared memory machines do not typically offer enough parallelism to process these problems. In this paper, we are presenting an early investigation of the scalability of such algorithms on Intel's upcoming Many Integrated Core (Intel MIC) architecture which, when it will be released in 2012, is expected to provide more than 50 physical cores with SMT capability. The Intel MIC architecture can be programmed through many programming models, here we investigate the three most popular of these models namely OpenMP, Cilk Plus and Intel's TBB. We present scalability results of a parallel graph coloring algorithm, three variations of a breadth-first search algorithm and a micro benchmark for irregular computations using these three programming models. Our results on a prototype board show that the multi-threaded architecture of Intel MIC can be effectively used for hiding latencies in irregular applications to achieve almost perfect speedup.

We present a system for functional parallel computing on distributed memory machines, which does dynamic load balancing and is totally symmetric. This means that every node can perform any computation and is client and server at the same time. Furthermore, the calling convention is similar to threads on shared memory machines to allow a uniform usage of parallel and distributed procedure calls. Finally the communication and computation is strictly separated for easy maintenance of the user program.

Algorithmic skeletons encapsulate typical parallel programming patterns such that they can be easily applied by users. Existing skeleton libraries usually work on distributed memory machines. We present an extension of our skeleton library Muesli which now allows to use the same application without modifications on a variety of parallel machines ranging from multi-processor distributed memory to many-core shared memory machines and combinations of those such as clusters of multi-core nodes. Internally, the skeletons are based on MPI and Open MP. We demonstrate the efficiency of our approach by providing experimental results.

The results produced by five different MPI benchmark programs on an SGI Altix 3700 are analyzed and compared. There are significant differences in the results for some MPI operations. We investigate the reasons for these discrepancies, which are due to differences in the measurement techniques, implementation details and default configurations of the different benchmarks. The variation in results on the Altix are generally much greater than on a distributed memory machine, due primarily to the ccNUMA architecture and the importance of cache effects, as well as some implementation details of the SGI MPI libraries.

Parallel graph generation algorithms for shared and distributed memory machines

Among elliptic boundary value problems, the class of problems with separable variables can be solved fast and directly. Elliptic problems with separable variables usually assume that the computational domains are simple e.g., rectangle or circle, and constant coefficients [1]. This kind of problems can serve to generate preconditioners in iterative procedures for more complex methods. They can also be used in low-accuracy models similar to the ones used in Numerical Weather Simulations. A straightforward implementation of solvers for such problems makes them considered as too simple to pay much attention to them. For instance, NETLIB* contains the codes of various solvers for problems with separable variables. We are not aware if such solvers are provided as separate solvers in other software packages except NETLIB Fishpack* and Intel$\textcircled{\tiny{\,R}}$ Math Kernel Library (Intel$\textcircled{\tiny{\,R}}$ MKL) together with Intel$\textcircled{\tiny{\,R}}$ Cluster Poisson Solver Library (Intel$\textcircled{\tiny{\,R}}$ CPSL). The problems with the separable variables can be suboptimal in the sense that they require slightly more arithmetic operations (up to logarithmic factor) than the number of unknowns to compute the solution to the problem. This statement is true if, for example, the sizes of the discretized problems are powers of 2. Computational Mathematics suggests that we take into consideration not only arithmetic operations, but also the cost of memory operations as well. Modern computers can perform computations at a very high speed, while lacking the ability to deliver data to the computational units. Keeping in mind that a memory operation can easily be dozen to hundred times slower than an arithmetic one, a computationally optimal algorithm could compute the solution slower than memory optimal algorithm. The recent developments in processor industry resulted in multicore processors become standard processors not only for powerful clusters, but also for home computers and laptops. Therefore, the algorithm can also be non-optimal from the parallelization point of view. Optimality here can be understood in terms of the number of synchronization points and/or in terms of the amount of data that needs to be transferred between different cores/processors. In summary, the modern computational algorithm should focus on 3 key aspects, namely, parallelization, memory operations, and arithmetic costs. The purpose of this paper is to demonstrate on a simple 2D problem with separable variables that taking into account modern model the solution can be computed efficiently and fast. This would also help developers to compute solution to the problem with separable variables really negligible with respect to other computations. To complete our goal, we will evaluate software provided by Intel Corporation. In particular, we focus on the comparison (where possible) of NETLIB Fishpack* and Intel$\textcircled{\tiny{\,R}}$ CPSL provided at [2] as well as Intel$\textcircled{\tiny{\,R}}$ MKL provided at [3]. This paper is organized as follows. First we are solving a 2D Helmholtz problem using single precision on a shared memory machine with two 4-Core Intel$\textcircled{\tiny{\,R}}$ Xeon$\textcircled{\tiny{\,R}}$ Processors E5462. Next we do the same but use double precision arithmetic. Lastly we use double precision on a distributed memory machine using MPI.

We survey shared memory simulations on distributed memory machines (DMMs), that use universal hashing to distribute the shared memory cells over the memory modules of the DMM. We measure their quality in terms of delay, time-processor efficiency, memory contention (how many requests have to be satisfied by one memory module per simulated step) and simplicity. Further we take into consideration different access conflict rules to the modules of the DMM, in particular the c-Collision rule motivated by the idea of communicating between processors and modules using an optical crossbar.

Simulating Spatially Explicit Problems on High Performance Architectures

Article Free Access Share on Communication strategies for out-of-core programs on distributed memory machines Authors: Rajesh Bordawekar ECE Dept. and NPAC, Syracuse University, Syracuse, NY ECE Dept. and NPAC, Syracuse University, Syracuse, NYView Profile , Alok Choudhary ECE Dept. and NPAC, Syracuse University, Syracuse, NY ECE Dept. and NPAC, Syracuse University, Syracuse, NYView Profile Authors Info & Claims ICS '95: Proceedings of the 9th international conference on SupercomputingJuly 1995 Pages 395–403https://doi.org/10.1145/224538.224642Published:03 July 1995Publication History 5citation191DownloadsMetricsTotal Citations5Total Downloads191Last 12 Months2Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

It is the goal of the Polaris project to develop a new parallelizing compiler that will overcome limitations of current compilers. While current parallelizing compilers may succeed on small kernels, they often fail to extract any meaningful parallelism from large applications. After a study of application codes, it was concluded that by adding a few new techniques to current compilers, automatic parallelization becomes possible. The techniques needed are interprocedural analysis, scalar and array privatization, symbolic dependence analysis, and advanced induction and reduction recognition and elimination, along with run-time techniques to allow data dependent behavior.