- Research Article
92
- 10.1016/s0167-8191(02)00190-4
From patterns to frameworks to parallel programs
- Nov 13, 2002
- Parallel Computing
- S Macdonald + 5 more +5
From patterns to frameworks to parallel programs
Object-oriented programming, design patterns, and frameworks are common techniques that have been used to reduce the complexity of sequential programming. We have applied these techniques to the more difficult domain of parallel programming. This paper describes CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S, a pattern-based parallel programming system that generates parallel programs from parallel design patterns. We demonstrate CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S by applying a new design pattern called the Wavefront pattern to three problems. We show that it is quick and easy to use CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S to generate structurally correct parallel programs with good speed-ups on shared-memory computers.
From patterns to frameworks to parallel programs
From patterns to frameworks to parallel programs
Generating Parallel Program Frameworks from Parallel Design Patterns
Object-oriented programming, design patterns, and frameworks are abstraction techniques that have been used to reduce the complexity of sequential programming. The CO2P3S parallel programming system provides a layered development process that applies these three techniques to the more difficult domain of parallel programming. The system generates correct frameworks from pattern template specifications at the highest layer and provides performance tuning opportunities at lower layers. Each of these features is a solution to a major problem with current parallel programming systems. This paper describes CO2P3S and its highest level of abstraction using an example program to demonstrate the programming model and one of the supported pattern templates. Our results show that a programmer using the system can quickly generate a correct parallel structure. Further, applications built using these structures provide good speedups for a small amount of development effort.
Read moreDesign patterns for parallel computing using a network of processors
High complexity of building parallel applications is often cited as one of the major impediments to the mainstream adoption of parallel computing. To deal with the complexity of software development, abstractions such as macros, functions, abstract data types, and objects are commonly employed by sequential as well as parallel programming models. This paper describes the concept of a design pattern for the development of parallel applications. A design pattern in our case describes a recurring parallel programming problem and a reusable solution to that problem. A design pattern is implemented as a reusable code skeleton for quick and reliable development of parallel applications. A parallel programming system, called DPnDP (Design Patterns and Distributed Processes), that employs such design patterns is described. In the past, parallel programming systems have allowed fast prototyping of parallel applications based on commonly occurring communication and synchronization structures. The uniqueness of our approach is in the use of a standard structure and interface for a design pattern. This has several important implications: first, design patterns can be defined and added to the system's library in an incremental manner without requiring any major modification of the system (extensibility). Second, customization of a parallel application is possible by mixing design patterns with low level parallel code resulting in a flexible and efficient parallel programming tool (flexibility). Also, a parallel design pattern can be parameterized to provide some variations in terms of structure and behavior.
Read moreUsing generative design patterns to generate parallel code for a distributed memory environment
A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct Object-Oriented Pattern-based Parallel Programming System) and MetaCO2P3S, use generative design patterns. A programmer selects the parallel design patterns that are appropriate for an application, and then adapts the patterns for that specific application by selecting from a small set of code-configuration options. CO2P3S then generates a custom framework for the application that includes all of the structural code necessary for the application to run in parallel. The programmer is only required to write simple code that launches the application and to fill in some application-specific sequential hook routines. We use generative design patterns to take an application specification (parallel design patterns + sequential user code) and use it to generate parallel application code that achieves good performance in shared memory and distributed memory environments. Although our implementations are for Java, the approach we describe is tool and language independent. This paper describes generalizing CO2P3S to generate distributed-memory parallel solutions.
Read moreUsing generative design patterns to generate parallel code for a distributed memory environment
A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct Object-Oriented Pattern-based Parallel Programming System) and MetaCO2P3S, usegenerative design patterns. A programmer selects the parallel design patterns that are appropriate for an application, and then adapts the patterns for that specific application by selecting from a small set of code-configuration options. CO2P3S then generates a custom framework for the application that includes all of the structural code necessary for the application to run in parallel. The programmer is only required to write simple code that launches the application and to fill in some application-specific sequential hook routines. We use generative design patterns to take an application specification (parallel design patterns + sequential user code) and use it to generate parallel application code that achieves good performance in shared memory and distributed memory environments. Although our implementations are for Java, the approach we describe is tool and language independent. This paper describes generalizing CO2P3S to generate distributed-memory parallel solutions.
Read moreFrom Design Patterns to Parallel Architectural Skeletons
From Design Patterns to Parallel Architectural Skeletons
Parallel_dp
Intel Threading Building Blocks (TBB) is an ideal environment for implementation of the parallel dynamic programming design pattern. The task-based parallelism of TBB readily lends itself to the realization of the participants and participant collaboration of this design pattern. We propose the parallel_dp algorithm template, an implementation of the parallel dynamic programming design pattern using TBB. We define the participants and the participant collaboration of this design pattern and describe how the design pattern is implemented by parallel_dp. We analyze the performance of our solution by applying parallel_dp to create four TBB programs that are parallel versions of the four types of dynamic programming algorithms. Our experimental results prove that parallel_dp provides speedup to all of our TBB programs and near linear speedup to one of our TBB programs. We conclude that parallel_dp will improve the performance of many TBB programs that include a dynamic programming algorithm.
Read moreOOSA’98 ECOOP’98 Workshop on Object-Oriented Software Architectures Workshop Summary
Recently, one can recognize an increased awareness in the object-oriented research community with respect to the role of software architectures. Examples of this can, among others, be found in object-oriented frameworks and design patterns. In the call for papers for the workshop, the following topics were identified where software architecture, patterns and frameworks should be studied in more detail: Building OO software architectures: How can design patterns, frameworks and components be included in the traditional object-oriented analysis and design? How are architecture constraints processed to build OO software architectures? Which techniques can help to build an OO architecture that supports reuse and is scalable? Language role in architecture: What is the role of a particular object-oriented language when designing an architecture? Architecture documentation: It has been suggested that design patterns can be used to document the design and rationale of an object-oriented architecture. How well does this work in practice? OO architectural styles: What is the relation between the architectural styles of Garlan and Shaw and OO design? Composition of architectures: Since object-oriented frameworks are an accepted technology within the software industry, the composition of two or more frameworks in an application occurs increasingly often. The composition may be far from trivial and the problems that may appear as well as solutions should be investigated.
Read moreReview of Elements of Parallel Computing
As the title clearly states, this book is about parallel computing. Modern computers are no longer characterized by a single, fully sequential CPU. Instead, they have one or more multicore/manycore processors. The purpose of such parallel architectures is to enable the simultaneous execution of instructions, in order to achieve faster computations. In high performance computing, clusters of parallel processors are used to achieve PFLOPS performance, which is necessary for scientific and Big Data applications. Mastering parallel computing means having deep knowledge of parallel architectures, parallel programming models, parallel algorithms, parallel design patterns, and performance analysis and optimization techniques. The design of parallel programs requires a lot of creativity, because there is no universal recipe that allows one to achieve the best possible efficiency for any problem. The book presents the fundamental concepts of parallel computing from the point of view of the algorithmic and implementation patterns. The idea is that, while the hardware keeps changing, the same principles of parallel computing are reused. The book surveys some key algorithmic structures and programming models, together with an abstract representation of the underlying hardware. Parallel programming patterns are purposely not illustrated using the formal design patterns approach, to keep an informal and friendly presentation that is suited to novices.
Read moreIdentification of Suitable Parallelization Patterns for Sequential Programs
During the past decade, the degree of parallelism available in hardware has grown quickly and decisively. Unfortunately, transforming an existing sequential program into a parallel one is not an easy task. Although modern compilers are able to detect the parallelism available in simple loops, a wide range of sequential programs cannot profit from these strategies because such compilers may miss coarser-grained but sometimes more scalable parallelism. To strengthen programmer productivity, over the years software engineers created a comprehensive list of sequential design patterns. These patterns improve the quality, re-usability, and maintainability of the software. Similarly, to ease the burden of parallel programming, parallel design patterns have been introduced. These patterns provide reusable solutions for common problems and help avoid concurrency bugs such as deadlocks and data races, which are very difficult to locate. Although parallel patterns are helpful for programmers, much effort is still needed to find appropriate places to apply them in the software architecture. This problem is exacerbated if one’s job is to parallelize a sequential program written by someone else. A programmer or software architect needs to have deep knowledge of not only parallel patterns but also of the target program. Tools have been developed to identify parallel patterns in the data-dependence graphs of a sequential application but they usually support at most one or two parallel patterns. This dissertation presents a framework to automatically detect a much broader variety of parallel patterns in the algorithm structure design space of sequential applications. For the identification of parallel patterns, we analyze the dependence graphs of each region of the application under study. We use different novel approaches, such as template matching and regression analysis to detect parallel patterns in a region. Furthermore, code blocks in a region are classified according to the appropriate support structure of the detected pattern. This classification eases the transformation of a sequential application into its parallel version. Finally, we defined a metric that helps select the best suitable parallel pattern out of multiple patterns detected in the same region. We support the identification of six parallel patterns in the algorithm structure of sequential applications: pipeline, multi-loop pipeline, task parallelism, geometric decomposition, do-all and reduction. We successfully identified these patterns in applications from four different benchmark suites. We confirmed our results by comparing them with pre-existing parallel versions of these applications. We also implemented the detected patterns on our own in those cases where parallel implementations were not available and achieved considerable speedups.
Read moreAlgorithmic Skeletons and Parallel Design Patterns in Mainstream Parallel Programming
This paper discusses the impact of structured parallel programming methodologies in state-of-the-art industrial and research parallel programming frameworks. We first recap the main ideas underpinning structured parallel programming models and then present the concepts of algorithmic skeletons and parallel design patterns. We then discuss how such concepts have permeated the wider parallel programming community. Finally, we give our personal overview—as researchers active for more than two decades in the parallel programming models and frameworks area—of the process that led to the adoption of these concepts in state-of-the-art industrial and research parallel programming frameworks, and the perspectives they open in relation to the exploitation of forthcoming massively-parallel (both general and special-purpose) architectures.
Read moreParallel design pattern for computational biology and scientific computing applications
Dynamic programming is an important algorithm design technique in computational biology and scientific computing. Typical applications using this technique are very compute-intensive and suffer from long runtimes on sequential architectures. Parallel program design patterns provide a new tool to semiautomatically generate parallel programs. In this paper we present a new parallel programs called the "block-cyclic based wavefront" to parallelize typical biology and scientific computing. We show how this technique leads to significant runtime savings on PC clusters.
Read moreCompilers for Low Power with Design Patterns on Embedded Multicore Systems
Minimization of power dissipation can be considered at algorithmic, compiler, architectural, logic, and circuit level. Recent research trends for multicore programming models have come to the direction that parallel design patterns can be a solution to develop multicore applications. As parallel design patterns are with regularity, we view this as a great opportunity to exploit power optimizations in the software layer. In this paper, we investigate compilers for low power with parallel design patterns on embedded multicore systems. We evaluate four major parallel design patterns, Pipe and Filter, MapReduce with Iterator, Puppeteer, and Bulk Synchronous Parallel (BSP) Model. Our work attempts to devise power optimization schemes in compilers by exploiting the opportunities of the recurring patterns of embedded multicore programs. The proposed optimization schemes are rate-based optimization for Pipe and Filter pattern , early-exit power optimization for MapReduce with Iterator pattern, power aware mapping algorithm for Puppeteer pattern, and multi-phases power gating scheme for BSP pattern. In our experiments, real world multicore applications are evaluated on a multicore power simulator. Significant power reductions are observed from the experimental results. Therefore, we present a direction for power optimizations that one can further identify additional key design patterns for embedded multicore systems to explore power optimization opportunities via compilers.
Read moreCompilers for Low Power with Design Patterns on Embedded Multicore Systems
Minimization of power dissipation can be considered at algorithmic, compilers, architectural, logic, and circuit levels. Recent research trends for multicore programming models have come to the direction that parallel design patterns can be a solution to develop multicore applications. As parallel design patterns are with regularity, we view this as a great opportunity to exploit power optimizations in the software layer. In this paper, we present case studies to investigate compilers for low power with parallel design patterns on embedded multicore systems. We evaluate two major parallel design patterns, Pipe and Filter and MapReduce with Iterator. Our work, attempts to devise power optimization schemes in compilers by exploiting the opportunities of the recurring patterns of embedded multicore programs. In all two cases of the patterns investigated, the common recurring patterns of programs are exploited to seek the opportunity for compiler optimizations for low power. Proposed optimization schemes are rate-based optimization for Pipe and Filter pattern and early-exit power optimization for MapReduce with Iterator pattern. Our experiment is based on a power simulator simulating a heterogeneous multicore system under SID simulation framework. In our experiments, a finite impulse response (FIR) program with Pipe and Filter pattern and an image recognition application applied MapReduce with Iterator pattern are evaluated by incorporating our proposed power optimization schemes for each pattern. Significant power reductions are observed in all two cases. With the case study, we present a direction for power optimizations that one can further identify additional key design patterns for embedded multicore systems to explore power optimization opportunities via compilers.
Read morePatterns in the Field of Software Engineering
In the mid 1960’s, the architect Christopher Alexander (1964) came up with the idea of Patterns, as “a solution to a problem within a defined context” and developed this concept. He explains, in a very original way, his ideas of urban planning and building architecture, using patterns to explain the “what”, “when”, and “how” of a design. Alexander invented a Pattern Language that is the fundamental to good building and city designs, and describes it in a collection of repetitive schemas called patterns. In Computer Science, software is susceptible to conceptual patterns. Consequently, Ward Cunningham and Kent Beck, used Alexander’s idea to develop a programming pattern language composed of five patterns as an initiation guide for Smalltalk programming. This work was presented at the Object-Oriented Programming, Systems, Languages & Applications Conference (OOPSLA) in 1987. In the early 1990’s, Erich Gamma and Richard Helm did a joint research that resulted in the first specific design patterns catalog. They identified four patterns: Composite, Decider, Observer, and Constrainer patterns. According to many authors, OOPSLA ’91 highlighted the evolutionary process of design patterns. The synergy between Erich Gamma, Richard Helm, Rala Johnson, and John Vlissides (better known as the “Gang of Four” or GoF) and other reputable researchers (Ward Cunningham, Kant Beck or Doug Lea) definitively launched the study of and research into Object Oriented Design Patterns. At the same time, James Coplien, another software engineer, was compiling and shaping a programming patterns catalogue in C++, which was a significant advance in the implementation phase in software development. Coplien’s catalog was published in 1991 under the title “Advanced C++ Programming Styles and Idioms”. Between 1991 and 1994 the concept of pattern design was discussed at international congresses and conferences. All of these encounters culminated in OOPSLA ’94. The GoF took advantage of this event to present their compilation (Gamma, Helm, Johnson & Vlissides, 1995). This publication, considered at that time as the best book on Object Orientation, compiled a 23-pattern catalog, founding the basis of patterns design. The number of pattern-related works, studies and publications in general, but especially in design, has exponentially grown since. However, the different research groups being born must be cataloged into three fundamental paradigms: • Theoretical approximations to the software pattern design concept and pattern languages. Coplien’s work (Coplien, 1996; 2004; Coplien & Douglas, 1995) stands out in this field. • Analysis and compilation of software applications design patterns. Rising’s efforts (Rising, 1998; 2000) and Buschmman (Buschmann, Meunier, Rohnert, Sommerlad & Stal, 1996; Buschmann, Rohnert, Stal & Schmidt, 2000) are included in this classification. • The study of special purpose patterns, like antipatterns (Brown, 1998).
Read more