Session 1: performance analysis

Parallel Computing on a network of workstations (NOW) is receiving a lot of attention from the research community. However, there is still a lack of tools for developing and running parallel applications on a NOW environment. In this paper, we develop and implement an object-oriented parallel programming system that facilitates the implementation of parallel applications on a NOW environment. In our programming system, a conceptual model, named Synchronous Object Model, is provided to help programmers write parallel programs. The object model makes use of the inherent parallel nature of an object-oriented model to express parallelism. In addition, an object-oriented framework is included into our programming system that reduces the size of parallel programs through code reuse and design reuse. Our system design further takes advantage of the underlying hardware architecture of the NOW environment to maximize performance by integrating multicast and load balancing support, and this is illustrated through experimental results.

Speeding up the communications on a cluster using MPI by means of Software Defined Networks

The analysis of parallel scientific applications allows us to understand their computational and communication behavior. One way of obtaining performance information is through performance tools. One such tool is parallel application signatures for performance prediction (PAS2P), based on parallel application repeatability, focusing on performance analysis and prediction. The same resources that execute the parallel application are used to perform its analysis, creating a machine independent model of the application and identifying its common patterns. However, the analysis is costly in terms of execution time due to the high number of synchronization communications performed by PAS2P, degrading performance as the number of processes increases. To solve this problem, we propose a model that reduces data dependency between processes, reducing the number of communications performed by PAS2P in the analysis stage and taking advantage of the characteristics of single program, multiple sata applications. Our analysis proposal allows us to decrease the analysis time by 29 times when the application scales to 256 processes, while keeping error levels below 11% in the runtime prediction. It is important to mention that the analysis time is not considerably affected by increasing the number of application processes.

We present a design methodology for real-time vision applications aiming at significantly reducing the design-implement-validate cycle time on dedicated parallel platforms. This methodology is based upon the concept of algorithmic skeletons, i.e., higher order program constructs encapsulating recurring forms of parallel computations and hiding their low-level implementation details. Parallel programs are built by simply selecting and composing instances of skeletons chosen in a predefined basis. A complete parallel programming environment was built to support the presented methodology. It comprises a library of vision-specific skeletons and a chain of tools capable of turning an architecture-independent skeletal specification of an application into an optimized, deadlock-free distributive executive for a wide range of parallel platforms. This skeleton basis was defined after a careful analysis of a large corpus of existing parallel vision applications. The source program is a purely functional specification of the algorithm in which the structure of a parallel application is expressed only as combination of a limited number of skeletons. This specification is compiled down to a parametric process graph, which is subsequently mapped onto the actual physical topology using a third-party CAD software. It can also be executed on any sequential platform to check the correctness of the parallel algorithm. The applicability of the proposed methodology and associated tools has been demonstrated by parallelizing several realistic real-time vision applications both on a multi-processor platform and a network of workstations. It is here illustrated with a complete road-tracking algorithm based upon white-line detection. This experiment showed a dramatic reduction in development times (hence the term fast prototyping), while keeping performances on par with those obtained with the handcrafted parallel version.

This paper proposes the application of parallelization techniques for solving the unit commitment problem (UCP) on NOWs (networks of workstations). A modified parallel dynamic programming method that takes advantage of NOWs is presented. Our algorithm performs better than the general parallel dynamic programming algorithm for UCP. In order to demonstrate the usefulness of our technique, we apply our parallel programs in some case studies. Studies reveal that our parallel dynamic programming algorithm has the potential to enhance the dynamic control of unit commitment in real time situations.

This paper presents a dynamic multi-application scheduling approach for running and scheduling parallel adaptive applications in Networks of Workstations (NOWs) and Clusters of Processors (COPs). Parallel adaptive applications have the property of varying their parallelism degree dynamically following availability of resources and changes in the underlying environment state. In our model, each parallel adaptive application is controlled by its own scheduler responsible for optimizing resources which it uses. Multi-application scheduling consists of sharing resources among applications fairly and uses a combined (time-sharing and space-sharing) scheduling approach. The scheduling approach is characterized especially by dynamic arrivals of applications, and remapping of allocation in order to handle dynamic arrivals and departures of applications and underlying environment state changes. Therefore, resources are fairly shared among applications. The multi-application scheduler interacts with the application schedulers in order to optimize the scheduling approach. The application schedulers are responsible for applying the multi-application scheduler decisions and internally optimizing the resources exploited by the application. Beyond the adaptive aspect which allows parallel applications to exploit idle cycles in NOWs with respect to the personal character of workstations, the proposed model provides a multi-application scheduling support which globally ensures better resource utilization and fair resource sharing.

Scalability and Performance of Two Large Linux Clusters

Today's massively parallel machines are typically message-passing systems consisting of hundreds or thousands of processors. Implementing parallel applications efficiently in this environment is a challenging task, and poor parallel design decisions can be expensive to correct. Tools and techniques that allow the fast and accurate evaluation of different parallelization strategies would significantly improve the productivity of application developers and increase throughput on parallel architectures. This paper investigates one of the major issues in building tools to compare parallelization strategies: determining what type of performance models of the application code and of the computer system are sufficient for a fast and accurate comparison of different strategies. The paper is built around a case study employing the performance prediction tool (PerPreT) to predict performance of the parallel spectral transform shallow water model code (PSTSWM) on the Intel Paragon. PSTSWM is a parallel application code that was designed to evaluate different parallel strategies for the spectral transform method as it is used in climate modeling and weather forecasting. Multiple parallel algorithms and algorithm variants are embedded in the code. PerPreT uses a relatively simple algebraic model to predict execution time for SPMD (single program multiple data) parallel applications. Applications are modeled through parameterized formulae for communication and computation, where the parameters include the problem size, the number of processors used to execute the program, and system characteristics (e.g. setup times for communication, link bandwidth and sustained computing performance per processor). In this paper we describe performance models that predict the performance of the different algorithms in PSTSWM accurately enough to allow them to be compared, establishing the feasibility of such a demanding application of performance modeling. We also discuss issues in generating and validating the performance models, emphasizing the practical importance of tools such as PerPreT in such studies. © 1998 John Wiley & Sons, Ltd.

Automating resource allocation for multiprocessors

Summary form only given. Scheduling policies are proposed for parallelizing data intensive particle physics analysis applications on computer clusters. Particle physics analysis jobs require the analysis of tens of thousands of particle collision events, each event requiring typically 200ms processing time and 600KB of data. Many jobs are launched concurrently by a large number of physicists. At a first view, particle physics jobs seem to be easy to parallelize, since particle collision events can be processed independently one from another. However, since large amounts of data need to be accessed, the real challenge resides in making an efficient use of the underlying computing resources. We propose several job parallelization and scheduling policies aiming at reducing job processing times and at increasing the sustainable load of a cluster server. Since particle collision events are usually reused by several jobs, cache based job splitting strategies considerably increase cluster utilization and reduce job processing times. Compared with straightforward job scheduling on a processing form, cache based first in first out job splitting speeds up average response times by an order of magnitude and reduces job waiting times in the system's queues from hours to minutes. By scheduling the jobs out of order, according to the availability of their collision events in the node disk caches, response times are further reduced, especially at high loads. In the delayed scheduling policy, job requests are accumulated during a time period, divided into subjob requests according to a parameterizable subjob size, and scheduled at the beginning of the next time period according to the availability of their data segments within the disk node caches. Delayed scheduling sustains a load close to the maximal theoretically sustainable load of a cluster, but at the cost of longer average response times. Finally we propose an adaptive delay scheduling approach, where the scheduling delay is adapted to the current load. This last scheduling approach sustains very high loads and offers low response times at normal loads.

Task parallelism raises the level of abstraction in shared memory parallel programming to simplify the development of complex applications. However, task parallel applications can exhibit poor performance due to thread idleness, scheduling overheads, and work time inflation -- additional time spent by threads in a multithreaded computation beyond the time required to perform the same work in a sequential computation. We identify the contributions of each factor to lost efficiency in various task parallel OpenMP applications and diagnose the causes of work time inflation in those applications. Increased data access latency can cause significant work time inflation in NUMA systems. Our locality framework for task parallel OpenMP programs mitigates this cause of work time inflation. Our extensions to the Qthreads library demonstrate that locality-aware scheduling can improve performance up to 3X compared to the Intel OpenMP task scheduler.

We describe the development of two parallel applications running on our PC cluster which consists of 64 Pentium Pro 200MHz microprocessors. One is an integrated parallel calculation system, called the PAPIA system, dedicated to protein information analysis. The PAPIA system enables very fast protein database searches (3-0 structure matching and sequence homology search) by fully utilizing the power of the 64 local hard-disks, as well as fast parallel calculations supported by the high-bandwidth and low-latency communication driver developed by RWCR We have starteda WWWsewice (http://www. rwcp.oKjp/papid) so any biologist can easily submit jobs to the PAPIA system through a WWW browse,: The service has been accessed from 55 countries in the world. The other application is parallel molecular dynamics simulation. We have eficiently ported the AMBER program to the PC cluster; and then appended an optional functionality of the Barnes-Hut tree code for quick and accurate calculation of Coulomb potentials without a cut-off approximation (PPPC method). The RWC PC Cluster shows good scalability in these two real parallel applications.

The performance of a multiprocessor system greatly depends on the bandwidth of its memory architecture. In this paper, uniform memory architectures with various interconnection networks including crossbar, multiple-buses and generalized shuffle networks are studied. We propose a general method based on the Markov chain model by assuming that the blocked memory requests will be redistributed to the memory modules in the next memory cycle. This assumption results in an analysis with lower complexity where the number of states is linearly proportional to the number of processors. Moreover, it can provide excellent estimation on the system power and memory bandwidth for all three types of interconnection networks as compared with the simulation results in which the blocked memory requests are resubmitted to the same memory module. Comparisons also show that our method is more general and precise than most existing analysis methods. The method is further extended to estimate the performance of multiprocessor system with caches. The approximation results are also shown to be remarkably good.

Performance analysis of SPMD algorithms on a network of workstations with virtual shared memory

High-level parallel programming is an active research topic aimed at promoting parallel programming methodologies that provide the programmer with high-level abstractions to develop complex parallel software with reduced time to solution. Pattern-based parallel programming is based on a set of composable and customizable parallel patterns used as basic building blocks in parallel applications. In recent years, a considerable effort has been made in empowering this programming model with features able to overcome shortcomings of early approaches concerning flexibility and performance. In this article, we demonstrate that the approach is flexible and efficient enough by applying it on 12 out of 13 PARSEC applications. Our analysis, conducted on three different multicore architectures, demonstrates that pattern-based parallel programming has reached a good level of maturity, providing comparable results in terms of performance with respect to both other parallel programming methodologies based on pragma-based annotations (i.e., Open mp and O mp S s ) and native implementations (i.e., P threads ). Regarding the programming effort, we also demonstrate a considerable reduction in lines of code and code churn compared to P threads and comparable results with respect to other existing implementations.