Performance prediction of parallel processing systems

Genericity of parallel programming environments, enabling development of portable parallel programs, is expected to result in performance penalties. Furthermore, programmability and tool support of programming environments are important issues if a choice between programming environments has to be made. In this paper we propose a methodology to compare native and generic parallel programming environments, taking into account such competing issues as portability and performance. As a case study, this paper compares the Iserver-Occam, Parix, Express and PVM parallel programming environments on a 512-node Parasytec GCel. Furthermore, we apply our methodology to compare Parix and PVM on a new architecture, a 32-node Parsytec PowerXplorer, which is based on the PowerPC chip. In our approach we start with a representative application and isolate the basic (environment)-dependent building blocks. These basic building blocks, which depend on floating-point performance and communication capabilities of the environments, are analysed independently. We have measured point-to-point communication times, global communication times and floating-point performance. All information is combined into a time complexity analysis, allowing comparison of the environments on different degrees of functionality. Together with demands for portability of the code and development time (i.e. programmability), an overall judgement of the environments is given.

Introduction. Many iterative algorithms allow a parallel implementation through multiprogramming or through multiprocessing where various programs can be executed with messages exchanges having the main objective of optimization. It is well known that through multilevel decomposition one may compute the solution of a large scale optimization problem and even observe the intrinsic parallelism there is among the various subsystems obtained through different Coordination Methods. Anyway, many questions remain unformulated or unsatisfactorily answered particularly when we consider the realization of this kind of optimization taking into consideration the increasing availability of multiprocessors and of operational systems for parallel programming. In addition, there is a need for gaining more and more experience on the real implementation, at least at the simulation level, on parallel processor vis a vis the now classical sequential computational solutions. The majority of computer programmers and users is relatively experient with sequential algorithms,'but has little or no experience at all with parallel computation. Traditionally, the efforts have been concentrated on finding sequential algorithms computationally more efficient. Up to recently, this was the only available option, but nowadays, if still the greater part of the programming languages is inherently sequential, there is an increasing tendency both in terms of hardware and of software for parallel computation studies and application, mainly after 1975, with the practical introduction of parallel processing.

While multicore architectures are used in the whole product range from server systems to handheld computers, the deployed software still undergoes the slow transition from sequential to parallel. This transition, however, is gaining more and more momentum due to the increased availability of more sophisticated parallel programming environments. Combined with the ever increasing complexity of multicore architectures, this results in a scheduling problem that is different from what it has been, because concurrently executing parallel programs and features such as non-uniform memory access, shared caches, or simultaneous multithreading have to be considered. In this paper, we compare different ways of scheduling multiple parallel applications on multicore architectures. Due to emerging parallel programming environments, we primarily consider applications where the parallelism degree can be changed on the fly. We propose TACO, a topology-aware scheduling scheme that combines equipartitioning and coscheduling, which does not suffer from the drawbacks of the individual concepts. Additionally, TACO is conceptually compatible with contention-aware scheduling strategies. We find that topology-awareness increases performance for all evaluated workloads. The combination with coscheduling is more sensitive towards the executed workloads and NUMA effects. However, the gained versatility allows new use cases to be explored, which were not possible before.

JCSPre is a highly reduced version of the JCSP (Communicating Sequential Processes for Java) parallel programming environment. JCSPre has been implemented on a LEGO Mindstorms NXT brick using the LeJOS Java runtime environment. The LeJOS environment provides an abstraction for the NXT Robot in terms of Sensors, Sensor Ports and Motors, amongst others. In the implementation described these abstractions have been converted into the equivalent active component that is much easier to incorporate into a parallel robot controller. Their use in a simple line following robot is described, thereby demonstrating the ease with which robot controllers can be built using parallel programming principles. As a further demonstration we show how the line following robot controls a slave robot by means of Bluetooth communications.

For better utilization of computing resources, it is important to consider parallel programming environments in which the number of available processors varies at runtime. In this paper, we discuss runtime support for data parallel programming in such an adaptive environment. Executing data parallel-programs in an adaptive environment requires redistributing data when the number of processors changes, and also requires determining new loop bounds and communication patterns for the new set of processors. We have developed a runtime library to provide this support. We also present performance results for a multiblock Navier-Stokes solver run on a network of workstations using PVM for message passing. Our experiments show that if the number of processors is not varied frequently, the cost of data redistribution is not significant compared to the time required for the actual computations. >

Parsec is a parallel programming environment whose goal is to simplify the development of multicomputer programs without, as is often the case, sacrificing performance. We have reconciled these objectives by “compiling” the structure of parallel applications into information to configure each of a small set of communication primitives on a context sensitive basis. In this chapter we show how Parsec can be used to implement a high-performance processor farm and compare Parsec and hand-optimized implementations to demonstrate that Parsec can achieve a similar level of performance. Extensive static analysis and optimization is necessary to achieve these results. We discuss both the tools which perform these tasks as well as the user interface that provides the necessary declarative structural information. Using the processor farm, we show how Parsec simplifies the task of specifying the structure of a parallel application and improves the result by supporting abstraction, reuse and scalability.

Parallel programs have traditionally been implemented on distributed systems using conventional programming languages, such as Fortran or C, plus a library with support for communication, such as, for instance, PVM [11]. Also, some effort has been put into developing parallel programming languages [13]. Usually, the different development platforms are compared as to application execution time or parallel efficiency. However, as pointed out in [4], when measuring the performance of a parallel application or programming tool, different metrics may need to be taken into consideration. The ease of construction of an application and the time a programmer spends to implement it may in some cases be more important than the final execution time. Working under this assumption, this work argues that the use of interpreted languages as a basis for parallel programming tools should be further investigated. As support for our argument, we show an example of use of an interpreted language in a classical application in parallel distributed programming, namely, that of detecting program termination. Two different algorithms for termination detection are implemented in Lua, an extension language developed at PUC-Rio [5].

Presents a portable parallel programming environment for Modula-2*, an explicitly parallel machine-independent extension of Modula-2. Modula-2* offers synchronous and asynchronous parallelism, a global single address space, and automatic data and process distribution. The Modula-2* system consists of a compiler, a debugger, a cross-architecture make, graphical X Windows control panel, run-time systems for different machines, and sets of scalable parallel libraries. The existing implementation targets the MasPar MP series of massively parallel processors (SIMD), the KSR-1 parallel computer (MIMD), heterogeneous LANs of workstations (MIMD), and single workstations (SISD). We describe the important components of the Modula-2* environment, and discuss selected implementation issues. We focus on how we achieve a high degree of portability for our system, while at the same time ensuring efficiency.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

P 3 L is a parallel coordination language which is based on the emerging research on skeletons (templates). The design of P 3 L begun in 1990, just before skeletons started to interest a large body of the parallel processing community, and after that we developed a prototype P 3 L compiler based on innovative template—based organization. During the last years we made experiments with the prototype P 3 L environment and performed a set of tests to verify different results: the suitability of P 3 L for massively parallel programming, its efficiency in generating “good” parallel code, the performance achieved with respect to traditional parallel programming languages and its portability over different parallel architectures. This paper summarizes the experience gained by our group with the “P 3 L experiment”.

We describe the approach we are pursuing to cope with parallelism in symbolic applications. Our purpose is to build a unified and user friendly parallel symbolic programming environment suited to distributed memory parallel systems (DMPS). This system is based on G.W. Sabot's (1987) Paralation model (in particular its LISP implementation, PARALATION LISP), whose main feature is to clearly differentiate the computational aspect from the communication one. In our opinion this distinction allows a clearer and cleaner exploitation of the opportunities of parallelism in symbolic applications. The simple semantics of the Paralation model parallel constructs seems to us well suited to run on a set of LISP interpreters. We chose to implement the Paralation model extensions on top of the Parallel Virtual Machine (PVM) package because this tool allows one to easily use a set of LAN interconnected powerful workstations as a general purpose parallel computational resource.

The parallel programs can be integrated into the parallel software in order to avoid different departments or different applications developing a same type of parallel programs many times. This article brought forward a parallel integration model. In which, the parallel programs need interoperate each other for the integration, and the interoperation ability can be achieved by the message communication in the parallel environment, and the parallel programs can compose parallel software or bigger parallel programs.

Due to the culture of sequential programming, the lack of tools and the inherent difficulties of parallel programming, programmers find hard to design and evaluate parallel real-time programs. As a result, a major problem found in the development of parallel real-time systems is the difficulty to predict the performance of the application, and the need to reduce rapid prototypes of the systems behind schedule. This work is part of a bigger project financed by FINEP, with the title Arquitetura Paralela Usando DSPs (Parallel Architecture Using DSPs), whose aim is to use a parallel system for processing and visualising tomography images from the solar atmosphere in real time. In this way, it can be possible to have the 3D reconstruction of the solar corona from the angular tomography, as well as the spectral lines. This paper presents a Visual Environment which should be able to provide continuity in the development of projects using the most common development methods, traditional or object-oriented, by offering support during the phases of implementation, debugging and testing. The environment has a visual interface and the main aim is to facilitate the development process and debugging of source code of applications developed for parallel kernels, particularly for Virtuoso (Virtual Single Processor Programming System – Virtuoso). The environment is composed by four tools: a Parallel Programs Generator, a Worst Case Execution Time Analyser, a Scheduling Analyser, and a Parallel Debugger.

Due to the development of several new technologies concerning high speed networks, one can get PC clusters for intensive computing at a low price. Parallel programming environments must take into account the heterogeneity of workstation networks into their communication kernels. New technologies must be easily integrated into such environments. The LANDA-HSN project has been designed for an efficient use of the available resources (network, processor, memory). The paper describes the mechanisms and the implementation of its communication kernel on SMP hosts and through interconnection networks. The contribution of the paper is to propose a flexible and efficient communication kernel.

For almost a decade we have been working at developing and using template-based models for parallel computing. Template-based models separate the specification of the parallel structuring aspects from the application code that is to be parallelized. A user provides the application code and specifies the parallel structure of the application using high-level icons, called templates. The parallel programming system then generates the code necessary for parallelizing the application. The goal here is to provide a mechanism for quick and reliable development of coarse-grain parallel applications that employ frequently occurring parallel structures. Our initial template-based system, FrameWorks, was positively received but had a number of shortcomings. The Enterprise parallel programming environment evolved out of this work. Now, after several years of experience with the system, its shortcomings are becoming evident. Controlled experiments have been conducted to assess the usability of our system in comparison with other systems. The paper outlines our experiences in developing and using these systems. A list of desirable characteristics of template-based models is given. The FrameWorks and Enterprise systems are discussed in the context of these characteristics and the results of our usability experiments. Many of our observations are relevant to other parallel programming systems, even though they may be based on different assumptions. Although template-base models have the potential for simplifying the complexities of parallel programming, they have yet to realize these expectations for high-performance applications. © 1998 John Wiley & Sons, Ltd.

In this paper, TOPAS, a new parallel programming environment for distributed systems is presented. TOPAS automatically analyzes data dependence among tasks and synchronizes data, which reduces the time needed for parallel program developments. TOPAS also provides supports for scheduling, dynamic load balancing and fault tolerance. Experiments show simplicity and efficiency of parallel programming in TOPAS environment.KeywordsFault ToleranceParallel ProgramTask GraphFaulty NodeDynamic Load BalanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.