The Modula-2* environment for parallel programming

Genericity of parallel programming environments, enabling development of portable parallel programs, is expected to result in performance penalties. Furthermore, programmability and tool support of programming environments are important issues if a choice between programming environments has to be made. In this paper we propose a methodology to compare native and generic parallel programming environments, taking into account such competing issues as portability and performance. As a case study, this paper compares the Iserver-Occam, Parix, Express and PVM parallel programming environments on a 512-node Parasytec GCel. Furthermore, we apply our methodology to compare Parix and PVM on a new architecture, a 32-node Parsytec PowerXplorer, which is based on the PowerPC chip. In our approach we start with a representative application and isolate the basic (environment)-dependent building blocks. These basic building blocks, which depend on floating-point performance and communication capabilities of the environments, are analysed independently. We have measured point-to-point communication times, global communication times and floating-point performance. All information is combined into a time complexity analysis, allowing comparison of the environments on different degrees of functionality. Together with demands for portability of the code and development time (i.e. programmability), an overall judgement of the environments is given.

Partitioned Global Address Space (PGAS) parallel programming models can provide an efficient mechanism for managing shared data stored across multiple nodes in a distributed memory system. However, these models are traditionally directly addressed and, for applications with loosely-structured or sparse data, determining the location of a given data element within a PGAS can incur significant overheads. Applications incur additional overhead from the network latency of lookups from remote location resolution structures. Further, for large data, caching such structures locally incurs space and coherence overheads that can limit scaling. We observe that the matching structures used by implementations of the Message Passing Interface (MPI) establish a separation between incoming data writes and the location where data will be stored. In this work, we investigate extending such structures to add a layer of indirection between incoming data reads and the location from which data will be read, effectively extending PGAS models with logical addressing.

P 3 L is a parallel coordination language which is based on the emerging research on skeletons (templates). The design of P 3 L begun in 1990, just before skeletons started to interest a large body of the parallel processing community, and after that we developed a prototype P 3 L compiler based on innovative template—based organization. During the last years we made experiments with the prototype P 3 L environment and performed a set of tests to verify different results: the suitability of P 3 L for massively parallel programming, its efficiency in generating “good” parallel code, the performance achieved with respect to traditional parallel programming languages and its portability over different parallel architectures. This paper summarizes the experience gained by our group with the “P 3 L experiment”.

In this paper we present a new methodology for the performance prediction of parallel programs on parallel platforms ranging from shared-memory to distributed-memory (vector) machines. The methodology comprises a procedural program and machine specification paradigm based on PAMELA (PerformAnce ModEling LAnguage), along with a performance calculus, called “serialization analysis”. This calculus extends conventional parallel program analysis technology by explicitly accounting for resource contention, yet at the low evaluation cost typical for static techniques. It is shown that, where conventional techniques introduce fundamental errors, predictions from serialization analysis remain realistic. Apart from the merits of the methodology itself, this high reliability/cost ratio makes PAMELA an attractive candidate for compile-time application within the performance prediction hierarchy often found in parallel programming environments.

We are proposing a shared-memory communication infrastructure that provides a common parallel programming interface for FPGA and CPU components in a heterogeneous system. Our intent is to ease the integration of reconfigurable hardware into parallel programming models like Partitioned Global Address Space (PGAS). For this purpose, we introduce a remote memory access component based on Active Messages that implements the core API of the Berkeley GASNet communication library, and a simple controller that manages communication and synchronization for custom FPGA cores. We demonstrate how these components deliver a simple and easily configurable communication mechanism between distributed memories in a multi-FPGA system with processors as well as custom hardware nodes.

Global address space parallel programming models can be an effective alternative to send/receive style communication, simplifying programming or code generation and increasing performance for certain application types. Traditionally, global address space mechanisms have been implemented in hardware in order to provide the necessary communication performance and responsiveness. However new high performance cluster messaging systems now allow global address space mechanisms to be realized efficiently in software. We describe a high performance one sided communication model that is implemented as a software layer on top of the Illinois Fast Messages (FM) system. We evaluate several different software implementation architectures for the remote agent, characterizing their differing performance characteristics. Our Put/Get FM implementation achieves peak bandwidths for put/get operations of 67 MBytes/s, overheads of a few microseconds, and remote read latencies as low as 26 microseconds on a Myrinet connected PC cluster. This implementation was released publicly as part of HPVM 1.0 in August 1997, and is receiving significant usage. It has been used for an implementation of the Global Arrays library and also serves as a back-end target for PGI's commercial HPF compiler.

HPJava is an environment for scientific and parallel programming using Java. It is based on an extended version of the Java language. One feature that HPJava adds to Java is a multi-dimensional array, or multiarray, with properties similar to the arrays of Fortran. We are using Adlib as our high-level collective communication library. Adlib was originally developed using C++ by the Parallel Compiler Runtime Consortium (PCRC). Many functionalities of this high-level communication library is following its predecessor. However, many design issues are reconsidered and re-implemented according to Java environment. Detailed functionalities and implementation issues of this collective library will be described.

No abstract available.

Introduction. Many iterative algorithms allow a parallel implementation through multiprogramming or through multiprocessing where various programs can be executed with messages exchanges having the main objective of optimization. It is well known that through multilevel decomposition one may compute the solution of a large scale optimization problem and even observe the intrinsic parallelism there is among the various subsystems obtained through different Coordination Methods. Anyway, many questions remain unformulated or unsatisfactorily answered particularly when we consider the realization of this kind of optimization taking into consideration the increasing availability of multiprocessors and of operational systems for parallel programming. In addition, there is a need for gaining more and more experience on the real implementation, at least at the simulation level, on parallel processor vis a vis the now classical sequential computational solutions. The majority of computer programmers and users is relatively experient with sequential algorithms,'but has little or no experience at all with parallel computation. Traditionally, the efforts have been concentrated on finding sequential algorithms computationally more efficient. Up to recently, this was the only available option, but nowadays, if still the greater part of the programming languages is inherently sequential, there is an increasing tendency both in terms of hardware and of software for parallel computation studies and application, mainly after 1975, with the practical introduction of parallel processing.

While multicore architectures are used in the whole product range from server systems to handheld computers, the deployed software still undergoes the slow transition from sequential to parallel. This transition, however, is gaining more and more momentum due to the increased availability of more sophisticated parallel programming environments. Combined with the ever increasing complexity of multicore architectures, this results in a scheduling problem that is different from what it has been, because concurrently executing parallel programs and features such as non-uniform memory access, shared caches, or simultaneous multithreading have to be considered. In this paper, we compare different ways of scheduling multiple parallel applications on multicore architectures. Due to emerging parallel programming environments, we primarily consider applications where the parallelism degree can be changed on the fly. We propose TACO, a topology-aware scheduling scheme that combines equipartitioning and coscheduling, which does not suffer from the drawbacks of the individual concepts. Additionally, TACO is conceptually compatible with contention-aware scheduling strategies. We find that topology-awareness increases performance for all evaluated workloads. The combination with coscheduling is more sensitive towards the executed workloads and NUMA effects. However, the gained versatility allows new use cases to be explored, which were not possible before.

JCSPre is a highly reduced version of the JCSP (Communicating Sequential Processes for Java) parallel programming environment. JCSPre has been implemented on a LEGO Mindstorms NXT brick using the LeJOS Java runtime environment. The LeJOS environment provides an abstraction for the NXT Robot in terms of Sensors, Sensor Ports and Motors, amongst others. In the implementation described these abstractions have been converted into the equivalent active component that is much easier to incorporate into a parallel robot controller. Their use in a simple line following robot is described, thereby demonstrating the ease with which robot controllers can be built using parallel programming principles. As a further demonstration we show how the line following robot controls a slave robot by means of Bluetooth communications.

For better utilization of computing resources, it is important to consider parallel programming environments in which the number of available processors varies at runtime. In this paper, we discuss runtime support for data parallel programming in such an adaptive environment. Executing data parallel-programs in an adaptive environment requires redistributing data when the number of processors changes, and also requires determining new loop bounds and communication patterns for the new set of processors. We have developed a runtime library to provide this support. We also present performance results for a multiblock Navier-Stokes solver run on a network of workstations using PVM for message passing. Our experiments show that if the number of processors is not varied frequently, the cost of data redistribution is not significant compared to the time required for the actual computations. >

The Partitioned Global Address Space (PGAS) programming paradigm offers ease-of-use in expressing parallelism through a global shared address space while emphasizing performance by providing locality awareness through the partitioning of this address space. Therefore, the interest in PGAS programming languages is growing and many new languages have emerged and are becoming ubiquitously available on nearly all modern parallel architectures. Recently, new parallel machines with multiple cores are designed for targeting high performance applications. Most of the efforts have gone into benchmarking but there are a few examples of real high performance applications running on multicore machines. In this paper, we present and evaluate a parallelization technique for implementing a local DNA sequence alignment algorithm using a PGAS based language, UPC (Unified Parallel C) on a chip multithreading architecture, the UltraSPARC T1.

As cluster-based architectures are becoming a well-established platform to the development and execution of parallel applications which demand for high performance, new requirements are imposed to the software used in such architectures. Within this context, we have developed DECK, an environment conceived to allow the development of parallel applications on top of this new architecture. This work presents the internal structure and the functionalities of DECK, as well as a performance evaluation of its two available implementations.

Distributed memory parallel machines are increasingly being used by the scientific community to provide computational power for solving problems. Building a distributed memory parallel machine offers a tremendous advantage in terms of cost and scalability over building a shared memory parallel machine. Unfortunately, distributed memory machines are much more difficult to program, mainly due to the lack of a single global address space. Communication between processors in a distributed memory machine is accomplished by message passing. The time cost for acquiring data from another processor's memory is significantly greater than the time cost of acquiring data from local memory. To assist the programmer in using a distributed memory machine, compilers have been developed which alleviate the need for the programmer to manage the lack of a global address space. These compilers will generate semantically correct code containing communication routines to produce a logically shared memory machine on a physically distributed memory machine. However, the runtime performance of the code emitted from some of these compilers can be further improved. This thesis is primarily concerned with advanced compilation techniques which will provide better runtime performance of programs written in imperative languages for regular scientific programs. The optimization of communication generation for distributed memory machines is used to produce this result. We look at two "state of the art" compilation systems and find that improvement in the communication code generated by these compilers is still possible. We have simplified existing descriptions of the various formalisms found in these compilers, while still providing enough formalism to study the problems in depth. We have extended some existing optimization techniques by enhancing the elimination of masks, by extending the notion of relaxing the owner computes rule, by extending the notion of replicating computation, and by extending the notion of replicating data. This thesis has discussed a novel technique to split loops so that they do not contain any guarded statements. Finally, we have suggested avenues for future research, as it is clear from this thesis that more work is required before compilation systems for distributed memory parallel machines will be widely used for general purpose computing.