Load balancing based concurrent execution of NAS parallel benchmarks with BYTE sequential benchmarks on a non-dedicated cluster

Computers of a non-dedicated cluster are often idle (users attend meetings, have lunch or coffee breaks) or lightly loaded (users carry out simple computations). These underutilized computers can be employed to execute parallel applications not only during weekends and at nights but also during office hours. Thus, they have to be shared by parallel and sequential applications which could lead to the improvement of their execution performance. However, there is a lack of experimental study showing the behavior and performance of parallel and sequential applications executing concurrently on clusters. We present here the result of an experimental study into load balancing based scheduling of a mixture of parallel and sequential applications on a non-dedicated cluster.

Run time variability of parallel applications continues to be a significant challenge in high performance computing (HPC) systems. We are currently studying run time variability in the context of both systemic performance and energy management. Our perspective is from that of the application, focusing on the interactions of the inter-process communication system on the set of concurrently executing parallel applications. In such a scenario, application run time can be extended and become highly variable. While some applications may be more sensitive to these interactions, others may in fact be generating the interactions that cause inconsistent run time, thus forming the notion of application-level behavioral attributes. To gain insight into this problem, our earlier work developed a framework that emulates parallel applications, called PACE. We also introduced a Parallel Application Run time Sensitivity Evaluation (PARSE) function that uses the PACE framework to study the run time effects of controlled network performance degradation on applications. Inter-process communication has evolved over the last decade from network communication between single-processor, single-core nodes to hybrid systems whose compute nodes contain several multi-core processor units. Motivated by the evolution of compute hardware and systems software, this work introduces PARSE 2.0, which is a nearly complete re-write that extends PARSE capabilities to include fully automating the processes of evaluating and quantifying run time critical parallel application-level behavioral attributes. We present an overview of the tool and the attributes being evaluated, and present experimental results from tests conducted on several widely used parallel benchmarks and application code fragments. The results re-enforce our earlier work, demonstrating that parallel applications can be classified according to their behavioral attributes, in the context of communication system resources.

Non-intrusive data collection for load balancing of parallel applications

The development of an efficient checkpointing facility exploiting operating systems services of the GENESIS cluster operating system

We investigate proactive dynamic load balancing on multicore systems, in which threads are continually migrated to reduce the impact of processor/thread mismatches. Our goal is to enhance the flexibility of the SPMD-style programming model and enable SPMD applications to run efficiently in multiprogrammed environments. We present Juggle, a practical decentralized, user-space implementation of a proactive load balancer that emphasizes portability and usability. In this paper we assume perfect intrinsic load balance and focus on extrinsic imbalances caused by OS noise, multiprogramming and mismatches of threads to hardware parallelism. Juggle shows performance improvements of up to 80 % over static load balancing for oversubscribed UPC, OpenMP, and pthreads benchmarks. We also show that Juggle is effective in unpredictable, multiprogrammed environments, with up to a 50 % performance improvement over the Linux load balancer and a 25 % reduction in performance variation. We analyze the impact of Juggle on parallel applications and derive lower bounds and approximations for thread completion times. We show that results from Juggle closely match theoretical predictions across a variety of architectures, including NUMA and hyper-threaded systems.

The performance of SPMD parallel programs is strongly affected by dynamic load imbalancing factors. The use of a suitable load balancing strategy is essential in overcoming the effects of these imbalancing factors. This chapter deals with concepts and experiments related to load balancing in SPMD applications. Initially, we discuss a set of classification criteria for load balancing algorithms designed for SPMD applications. In addition, we define a load imbalancing index in order to measure the load imbalance of a parallel application execution. In the experimental part of this chapter, we describe the development of an SPMD parallel application which computes the macroscopic thermal dispersion in porous media. Nine versions of this scientific application were developed, each one adopting a different load balancing strategy. We evaluate and compare the performance of these nine versions and show the importance of using an appropriate load balancing strategy for the characteristics of a specific SPMD parallel application.

Adaptive data parallel computing on workstation clusters

Our interest is oriented towards keeping both local and parallel jobs together in a time-sharing non-dedicated cluster. In such systems, dynamic coscheduling techniques, without memory restriction, that consider the MultiProgramming Level for parallel applications (MPL), is a main goal in current cluster research. In this paper, a new technique called Cooperating Coscheduling (CCS), that combines a dynamic coscheduling system and a resource balancing schema, is applied.The main aim of CCS is to allow the efficient execution of parallel tasks from the system and parallel user points of view without disturbing the local jobs. Its feasibility is shown experimentally in a PVM-Linux cluster.KeywordsLocal UserParallel ApplicationParallel TaskIdle ResourceReady QueueThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Parallel processing in the era of many-core processors demands high-performance networks-on-chip and parallel communication based on intra-chip message passing. In this context, wireless networks-on-chip (WiNoCs) emerge to improve inter-core communication, bringing high bandwidth and low power consumption. In order to increase application performance, WiNoCs can support single-hop or multi-hop communication. The main issue is related to the performance of each communication architecture in the face of different parallel workloads. For this reason, the goal of this work is to evaluate and compare single- and multi-hop WiNoC architectures using parallel applications. The methodology is based on simulations done using the well-known simulator Network Simulator 2 (NS-2) and applications from NAS Parallel Benchmarks (NPB). The WiNoC architectures are based on 2-D mesh topology and ultra wide band (UWB) radio technology. The inter-core transmission is evaluated concerning unicast communication (1:1 and N:1) and broadcast communication (1:N and N:N). This work has, as contribution to the state-of-the-art, the evaluation of both WiNoC designs (single- and multi-hop architectures) with parallel applications. Based on our results, the single-hop architecture has lower communication delay than the multi-hop version, and for some workloads, there were no packet losses. However, to achieve high-performance communication, the single-hop architecture consumed 63.12 J for the 256-node network, versus 0.22 J consumed by the multi-hop version. Although single-hop WiNoCs reduce network bottlenecks and increase communication parallelism, they are recommended when energy consumption is not a critical factor.

he analysis of any hardware or software from the point of view of the actual and possible performance is crucial in the computer world. Huge efforts are made to create newer, better and faster computers and programs while minor design flaws can result in very low performance not even worth the invested money. Many scientists focus their research on the performance analysis of different areas like parallel programs, PC clusters, shared memory architectures, supercomputers, cache performance and so on. Basically, two approaches can be found in this effort. One group of researchers try to create analitical models to calculate the performance and limits “in theory” while the other group implements monitoring tools to collect as much data about real executions as they can and then process this data and find bottlenecks in the performance. The first session of the workshop deals with performance analysis issues of different areas in computer science. The performance of PC clusters is analysed here as well as parallel applications with the need of the power of a teraflop supercomputer. Simulator for low-level interconnection networks is presented as well as mathematical models for parallel programs. Every researcher aims at making possible to the users to improve the performance of their system and their applications. The first paper is “Beowulf Performance in CFD Multigrid Applications” authored by F. Tirado, C. Garcia, R.S. Montereo, M. Prieto and I.M. Llorente. Authors evaluated the performance of a Beowulf-class system from the viewpoint of Computational Fluid Dynamics applications. They used two multigrid solvers to test their PC cluster with different configurations. The results show that, not surprisingly, the best configuration depends on the target application. The second paper presents “SICOSYS: An Integrated Framework for studying Interconnection Network Performance in Multiprocessor Systems” written by V. Puente, J.A. Gregorio and R. Beivide. SICOSYS is a generalpurpose simulator that makes possible the analysis of the interconnection networks of multiprocessor systems. This simulator has been integrated into the RSIM shared-memory-architecture and processor simulator as well as into the SimOS simulator that also includes the effects of an operating system into the simulation. The third paper presents new ideas about “Performance Analysis for Teraflop Computers: A Distributed Automatic Approach” authored by M. Gerndt, A. Schmidt, M. Schulz and R. Wismuller. A new performance analysis system has been designed for the monitoring and analysis of extremely computationally intensive applications on a teraflop computer. In such environment, classical analysis techniques fail because they are not scalable for thousands of processors. This paper presents the design of a new, automatic performance tool that is scalable on teraflop computers using a distributed approach for the analysis process. The fourth paper is the “Execution Time Prediction for Parallel Data Processing Tasks” written by S. Juhasz. This paper presents a mathematical model for execution time prediction in the domain of parallel programs with moderate computational power needs but working on large amount of data. A parallel sorting application is then analysed as an example.

PackStealLB: A scalable distributed load balancer based on work stealing and workload discretization

Simulators and empirical profiling data are often used to understand how suitable a specific hardware architecture is for an application. However, simulators can be slow, and empirical profiling-based methods can only provide insights about the existing hardware on which the applications are executed. While the insights obtained in this way are valuable, such methods cannot be used to evaluate a large number of system designs efficiently. Analytical performance evaluation models are fast alternatives, particularly well-suited for system design-space exploration. However, to be truly application-specific, they need to be combined with a workload model that captures relevant application characteristics. In this paper we introduce PISA, a framework based on the LLVM infrastructure that is able to generate such a model for sequential and parallel applications by performing hardware-independent characterization. Characteristics such as instruction-level parallelism, memory access patterns and branch behavior are analyzed per thread or process during application execution. To illustrate the potential of the framework, we provide a detailed characterization of a representative benchmark for graph-based analytics, Graph 500. Finally, we analyze how the properties extracted with PISA across Graph 500 and SPEC CPU2006 applications compare to measurements performed on x86 and POWER8 processors.

We present the Operator design pattern which can be used for the design of both sequential and data parallel applications. To reach this goal, we show how the participants of this pattern can be implemented either for a sequential or a parallel execution. Besides, reusing the sequential design for a parallel application decreases the cost of parallelization by allowing the maintenance of a unique application for the two execution environments. The proposed approach for parallel programming does not require any dedicated compiler or code pre-processing. Nothing but object oriented features, such as inheritance and polymorphism, is used to provide the distributed behaviour of the parallel participants of the pattern. The Operator pattern can help to solve many design issues in relation with the development of reusable software components for data collection processing. Moreover, we are confident that many programmers who want to migrate their applications towards parallelism can find it helpful.

Improving memory energy consumption of programs that manipulate arrays is an important problem as these codes spend large amounts of energy in accessing off-chip memory. We propose a data-driven strategy to optimize the memory energy consumption in a banked memory system. Our compiler-based strategy modifies the original execution order of loop iterations in array-dominated applications to increase the length of the time period(s) in which memory banks are idle (i.e., not accessed by any loop iteration). To achieve this, it first classifies loop iterations according to their bank accesses patterns and then, with the help of a polyhedral tool, tries to bring the iterations with similar bank access patterns close together. Increasing the idle periods of memory banks brings two major benefits: first, it allows us to place more memory banks into low-power operating modes and, second, it enables us to use a more aggressive (i.e., more energy saving) operating mode (hence, saving more energy) for a given bank (instead of a less aggressive mode). The proposed strategy can reduce memory energy consumption in both sequential and parallel applications. Our strategy has been implemented in an experimental compiler using a polyhedral tool and evaluated using nine array-dominated applications on both a cacheless system and a system with cache memory. Our experimental results indicate that the proposed strategy is very successful in reducing the memory system energy and improves the memory energy by as much as 36.8 percent over a strategy that uses low-power modes without optimizing data access pattern. Our results also show that optimizations that target reducing off-chip memory energy can generate very different results from those that target at improving only cache locality.

New challenges in dynamic load balancing