- Research Article
92
- 10.1016/s0167-8191(02)00190-4
From patterns to frameworks to parallel programs
- Nov 13, 2002
- Parallel Computing
- S Macdonald + 5 more +5
From patterns to frameworks to parallel programs
Object-oriented programming, design patterns, and frameworks are abstraction techniques that have been used to reduce the complexity of sequential programming. The CO2P3S parallel programming system provides a layered development process that applies these three techniques to the more difficult domain of parallel programming. The system generates correct frameworks from pattern template specifications at the highest layer and provides performance tuning opportunities at lower layers. Each of these features is a solution to a major problem with current parallel programming systems. This paper describes CO2P3S and its highest level of abstraction using an example program to demonstrate the programming model and one of the supported pattern templates. Our results show that a programmer using the system can quickly generate a correct parallel structure. Further, applications built using these structures provide good speedups for a small amount of development effort.
From patterns to frameworks to parallel programs
From patterns to frameworks to parallel programs
Generating parallel programs from the wavefront design pattern
Object-oriented programming, design patterns, and frameworks are common techniques that have been used to reduce the complexity of sequential programming. We have applied these techniques to the more difficult domain of parallel programming. This paper describes CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S, a pattern-based parallel programming system that generates parallel programs from parallel design patterns. We demonstrate CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S by applying a new design pattern called the Wavefront pattern to three problems. We show that it is quick and easy to use CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> S to generate structurally correct parallel programs with good speed-ups on shared-memory computers.
Read moreDesign patterns for parallel computing using a network of processors
High complexity of building parallel applications is often cited as one of the major impediments to the mainstream adoption of parallel computing. To deal with the complexity of software development, abstractions such as macros, functions, abstract data types, and objects are commonly employed by sequential as well as parallel programming models. This paper describes the concept of a design pattern for the development of parallel applications. A design pattern in our case describes a recurring parallel programming problem and a reusable solution to that problem. A design pattern is implemented as a reusable code skeleton for quick and reliable development of parallel applications. A parallel programming system, called DPnDP (Design Patterns and Distributed Processes), that employs such design patterns is described. In the past, parallel programming systems have allowed fast prototyping of parallel applications based on commonly occurring communication and synchronization structures. The uniqueness of our approach is in the use of a standard structure and interface for a design pattern. This has several important implications: first, design patterns can be defined and added to the system's library in an incremental manner without requiring any major modification of the system (extensibility). Second, customization of a parallel application is possible by mixing design patterns with low level parallel code resulting in a flexible and efficient parallel programming tool (flexibility). Also, a parallel design pattern can be parameterized to provide some variations in terms of structure and behavior.
Read moreFrom Design Patterns to Parallel Architectural Skeletons
From Design Patterns to Parallel Architectural Skeletons
Parallel Programming Using C++
Foreword by Bjarne Stroustrup Software is generally acknowledged to be the single greatest obstacle preventing mainstream adoption of massively-parallel computing. While sequential applications are routinely ported to platforms ranging from PCs to mainframes, most parallel programs only ever run on one type of machine. One reason for this is that most parallel programming systems have failed to insulate their users from the architectures of the machines on which they have run. Those that have been platform-independent have usually also had poor performance. Many researchers now believe that object-oriented languages may offer a solution. By hiding the architecture-specific constructs required for high performance inside platform-independent abstractions, parallel object-oriented programming systems may be able to combine the speed of massively-parallel computing with the comfort of sequential programming. Parallel Programming Using C++ describes fifteen parallel programming systems based on C++, the most popular object-oriented language of today. These systems cover the whole spectrum of parallel programming paradigms, from data parallelism through dataflow and distributed shared memory to message-passing control parallelism. For the parallel programming community, a common parallel application is discussed in each chapter, as part of the description of the system itself. By comparing the implementations of the polygon overlay problem in each system, the reader can get a better sense of their expressiveness and functionality for a common problem. For the systems community, the chapters contain a discussion of the implementation of the various compilers and runtime systems. In addition to discussing the performance of polygon overlay, several of the contributors also discuss the performance of other, more substantial, applications. For the research community, the contributors discuss the motivations for and philosophy of their systems. As well, many of the chapters include critiques that complete the research arc by pointing out possible future research directions. Finally, for the object-oriented community, there are many examples of how encapsulation, inheritance, and polymorphism can be used to control the complexity of developing, debugging, and tuning parallel software.
Read morePattern-based parallel programming
The advantages of pattern-based programming have been well-documented in the sequential programming literature. However patterns have yet to make their way into mainstream parallel computing, even though several research tools support them. There are two critical shortcomings of pattern (or template) based systems for parallel programming: lack of extensibility and performance. This paper describes our approach for addressing these problems in the CO/sub 2/P/sub 3/S parallel programming system. CO/sub 2/P/sub 3/S supports multiple levels of abstraction, allowing the user to design an application with high-level patterns, but move to lower levels of abstraction for performance tuning. Patterns are implemented as parameterized templates, allowing the user the ability to customize the pattern to meet their needs. CO/sub 2/P/sub 3/S generates code that is specific to the pattern/parameter combination selected by the user. The MetaCO/sub 2/P/sub 3/S tool addresses extensibility by giving users the ability to design and add new pattern templates to CO/sub 2/P/sub 3/S. Since the pattern templates are stored in a system-independent format, they are suitable for storing in a repository to be shared throughout the user community.
Read moreAn object-oriented approach to develop software fault-tolerant mechanisms for parallel programming systems
An object-oriented approach to develop software fault-tolerant mechanisms for parallel programming systems
Using generative design patterns to generate parallel code for a distributed memory environment
A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct Object-Oriented Pattern-based Parallel Programming System) and MetaCO2P3S, use generative design patterns. A programmer selects the parallel design patterns that are appropriate for an application, and then adapts the patterns for that specific application by selecting from a small set of code-configuration options. CO2P3S then generates a custom framework for the application that includes all of the structural code necessary for the application to run in parallel. The programmer is only required to write simple code that launches the application and to fill in some application-specific sequential hook routines. We use generative design patterns to take an application specification (parallel design patterns + sequential user code) and use it to generate parallel application code that achieves good performance in shared memory and distributed memory environments. Although our implementations are for Java, the approach we describe is tool and language independent. This paper describes generalizing CO2P3S to generate distributed-memory parallel solutions.
Read moreUsing generative design patterns to generate parallel code for a distributed memory environment
A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct Object-Oriented Pattern-based Parallel Programming System) and MetaCO2P3S, usegenerative design patterns. A programmer selects the parallel design patterns that are appropriate for an application, and then adapts the patterns for that specific application by selecting from a small set of code-configuration options. CO2P3S then generates a custom framework for the application that includes all of the structural code necessary for the application to run in parallel. The programmer is only required to write simple code that launches the application and to fill in some application-specific sequential hook routines. We use generative design patterns to take an application specification (parallel design patterns + sequential user code) and use it to generate parallel application code that achieves good performance in shared memory and distributed memory environments. Although our implementations are for Java, the approach we describe is tool and language independent. This paper describes generalizing CO2P3S to generate distributed-memory parallel solutions.
Read moreAlgorithmic Skeletons and Parallel Design Patterns in Mainstream Parallel Programming
This paper discusses the impact of structured parallel programming methodologies in state-of-the-art industrial and research parallel programming frameworks. We first recap the main ideas underpinning structured parallel programming models and then present the concepts of algorithmic skeletons and parallel design patterns. We then discuss how such concepts have permeated the wider parallel programming community. Finally, we give our personal overview—as researchers active for more than two decades in the parallel programming models and frameworks area—of the process that led to the adoption of these concepts in state-of-the-art industrial and research parallel programming frameworks, and the perspectives they open in relation to the exploitation of forthcoming massively-parallel (both general and special-purpose) architectures.
Read moreParallel_dp
Intel Threading Building Blocks (TBB) is an ideal environment for implementation of the parallel dynamic programming design pattern. The task-based parallelism of TBB readily lends itself to the realization of the participants and participant collaboration of this design pattern. We propose the parallel_dp algorithm template, an implementation of the parallel dynamic programming design pattern using TBB. We define the participants and the participant collaboration of this design pattern and describe how the design pattern is implemented by parallel_dp. We analyze the performance of our solution by applying parallel_dp to create four TBB programs that are parallel versions of the four types of dynamic programming algorithms. Our experimental results prove that parallel_dp provides speedup to all of our TBB programs and near linear speedup to one of our TBB programs. We conclude that parallel_dp will improve the performance of many TBB programs that include a dynamic programming algorithm.
Read moreReview of Elements of Parallel Computing
As the title clearly states, this book is about parallel computing. Modern computers are no longer characterized by a single, fully sequential CPU. Instead, they have one or more multicore/manycore processors. The purpose of such parallel architectures is to enable the simultaneous execution of instructions, in order to achieve faster computations. In high performance computing, clusters of parallel processors are used to achieve PFLOPS performance, which is necessary for scientific and Big Data applications. Mastering parallel computing means having deep knowledge of parallel architectures, parallel programming models, parallel algorithms, parallel design patterns, and performance analysis and optimization techniques. The design of parallel programs requires a lot of creativity, because there is no universal recipe that allows one to achieve the best possible efficiency for any problem. The book presents the fundamental concepts of parallel computing from the point of view of the algorithmic and implementation patterns. The idea is that, while the hardware keeps changing, the same principles of parallel computing are reused. The book surveys some key algorithmic structures and programming models, together with an abstract representation of the underlying hardware. Parallel programming patterns are purposely not illustrated using the formal design patterns approach, to keep an informal and friendly presentation that is suited to novices.
Read moreParallel design pattern for computational biology and scientific computing applications
Dynamic programming is an important algorithm design technique in computational biology and scientific computing. Typical applications using this technique are very compute-intensive and suffer from long runtimes on sequential architectures. Parallel program design patterns provide a new tool to semiautomatically generate parallel programs. In this paper we present a new parallel programs called the "block-cyclic based wavefront" to parallelize typical biology and scientific computing. We show how this technique leads to significant runtime savings on PC clusters.
Read moreDesign-Pattern Based Parallel Programming Model and System Implementation
Design patterns make it easier to reuse successful designs and architectures. Expressing proven techniques as design patterns makes them more accessible to developers of new systems and helps a designer get a design faster. In sequential and object-oriented programming domain design patterns have played a very important role but in parallel programming the application of design patterns is very few. We propose a design-pattern based parallel programming model and implement a parallel programming environment in the SMP platform to help the developers to build their parallel application system efficiently. The experiment results show our system is effective and competent.
Read moreOOSA’98 ECOOP’98 Workshop on Object-Oriented Software Architectures Workshop Summary
Recently, one can recognize an increased awareness in the object-oriented research community with respect to the role of software architectures. Examples of this can, among others, be found in object-oriented frameworks and design patterns. In the call for papers for the workshop, the following topics were identified where software architecture, patterns and frameworks should be studied in more detail: Building OO software architectures: How can design patterns, frameworks and components be included in the traditional object-oriented analysis and design? How are architecture constraints processed to build OO software architectures? Which techniques can help to build an OO architecture that supports reuse and is scalable? Language role in architecture: What is the role of a particular object-oriented language when designing an architecture? Architecture documentation: It has been suggested that design patterns can be used to document the design and rationale of an object-oriented architecture. How well does this work in practice? OO architectural styles: What is the relation between the architectural styles of Garlan and Shaw and OO design? Composition of architectures: Since object-oriented frameworks are an accepted technology within the software industry, the composition of two or more frameworks in an application occurs increasingly often. The composition may be far from trivial and the problems that may appear as well as solutions should be investigated.
Read more