- Conference Article
8
- 10.1109/pmmp.1993.315550
Compiling data parallel programs to message passing programs for massively parallel MIMD systems
- Sep 20, 1993
- T Brandes
S.100-107
We define an axiomatic semantics for a common kernel of existing data-parallel languages. We introduce an assertional language which enables us to define a weakest liberal precondition calculus which has the Definability Property, and a proof system ({\it a la\/} Hoare) which has the Completeness Property. Moreover, our axiomatic semantics integrates two previous works in the definition of proof systems for data-parallel programs. This work sheds a new light on the logical complexity of proving data-parallel programs.
Compiling data parallel programs to message passing programs for massively parallel MIMD systems
S.100-107
Data parallelism and functional programming
Data parallelism is often seen as a form of explicit parallelism for SIMD and vector machines, and data parallel programming as an explicit programming paradigm for these architectures. Data parallel languages possess certain software qualities as well, which justifies their use in higher level programming and specification closer to the algorithm domain. Thus, it is interesting to study how the data parallel paradigm can be best realized in a declarative setting, since declarative languages offer a pure view of computation which is good for these purposes. For numerical computing the functional programming paradigm is especially attractive, since numerical algorithms often are specified by recursion equations and thus can be translated more or less directly into recursive functional programs. Merging the data parallel and functional paradigms then yields languages and formalisms where many algorithms can be expressed in a very succinct fashion. In this paper we review data parallelism, functional programming, and existing approaches to the integration of the two paradigms. We then proceed to describe a formalism for data parallel functional programming, allowing very simple languages, where the view of aggregate data is particularly abstract. We explain how various data parallel operations can be expressed in this formalism. Finally, we conclude with a discussion of issues for languages based directly on the formalism.KeywordsData FieldSystolic ArrayFunctional ProgrammingFunctional LanguageData ParallelismThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Read moreProving proof rules: a proof system for concurrent programs
A methodology for developing sound proof systems for program verification is demonstrated by the development of a proof system for reasoning about concurrent programs based on the unity logic of K.M. Chandy and J. Misra (1988). This proof system has been validated by an automated theorem prover, a computer program that checks the correctness of proofs. The Boyer-Moore logic in which the proof system is formalized and the Boyer-Moore theorem prover which mechanizes this logic are described. The motivation behind the unity logic and the way in which it may be used to prove the correctness of concurrent programs are examined. A proof system for concurrent programs based on the unity logic system is provided as an example. An operational semantics of concurrency is formalized in Boyer-Moore logic by use of the transition system model. Unity's proof rules are proved as theorems about this operational semantics. The proofs of these theorems are mechanically checked. The entire proof system has been verified by the Boyer-Moore prover, making it possible to prove mechanically the consequence of other concurrent programs.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Read moreSIMD code generation in data-parallel programming
Today';s desktop PCs feature a variety of parallel processing units. Developing applications that exploit this parallelism is a demanding task, and a programmer has to obtain detailed knowledge about the hardware for efficient implementation. CGiS is a data-parallel programming language providing a unified abstraction for two parallel processing units: graphics processing units (GPUs) and the vector processing units of CPUs. The CGiS compiler framework fully virtualizes the differences in capability and accessibility by mapping an abstract data-parallel programming model on those targets. The applicability of CGiS for GPUs has been shown in previous work; this work focuses on applying the abstract programming model of CGiS to CPUs with SIMD (Single Instruction Multiple Data) instruction sets. We have identified, adapted and implemented a set of program analyses to expose and access the available parallelism. The code generation phase is based on selected optimization algorithms tailored to SIMD code generation. Via code generation profiles, it is possible to adapt the code generation strategy to different target architectures. To assess the effectiveness of our approach, we have implemented backends for the two most widespread SIMD instruction sets, namely Intel';s Streaming SIMD Extensions and Freescale';s AltiVec. Additionally, we integrated a prototypical backend for the Cell Broadband Engine as an example for a multi-core architecture. Our experimental results show excellent average performance gains by a factor of 3 compared to standard scalar C++ implementations and underline the viability of this approach: real-world applications can be implemented easily with CGiS and result in efficient code.
Read moreTowards a Unified Theory of Operational and Axiomatic Semantics
This paper presents a nine-rule language-independent proof system that takes an operational semantics as axioms and derives program reachability properties, including ones corresponding to Hoare triples. This eliminates the need for language-specific Hoare-style proof rules to verify programs, and, implicitly, the tedious step of proving such proof rules sound for each language separately. The key proof rule is Circularity, which is coinductive in nature and allows for reasoning about constructs with repetitive behaviors (e.g., loops). The generic proof system is shown sound and has been implemented in the MatchC verifier.
Read moreOn the completeness of a proof system for a simple data-parallel programming language (extended abstract)
We prove the completeness of an assertional proof system for a simple loop-free data-parallel language. This proof system is based on two-part assertions, where the predicate on the current value of variables is separated from the specification of the current extent of parallelism. The proof is based on a Weakest Precondition (WP) calculus. In contrast with the case of usual scalar languages, not all WP can be defined by an assertion. Yet, partial definability suffices to prove the completeness thanks to the introduction of hidden variables in assertions. The case of data-parallel programs with loops is briefly discussed in the conclusion.KeywordsConcurrent ProgrammingSpecifying and Verifying and Reasoning about ProgramsSemantics of Programming LanguagesData-Parallel LanguagesProof SystemHoare LogicWeakest Preconditions
Read moreAdaptor: A Compilation System for Data Parallel Fortran Programs
Data parallel programming stands for single threaEnd, global name space, and loosely synchronous parallel computation. This kind of parallel programming has been proven to be very user-friendly, easy to Enbug and easy to use. But this programming moEnl is not available for most message passing multiprocessor architectures. Adaptor (Automatic Data Parallelism Translator) is a compilation system that transforms data parallel programs written in Fortran with array extensions, parallel loops, and layout directives to parallel programs with explicit message passing. Therefore it does automatic partitioning directed by user specified data distributions . The current version supports especially the translation of Connection Machine Fortran and High Performance Fortran programs to message passing programs. After a short Enscription of the system it will be shown how efficient the compilation is. Therefore many results about speed-ups and efficiencies of real coEns are presented. Furthermore, the automatically generated message passing programs are compared with hand written message passing programs. The results will show that data parallel programs will be competitive to hand written message passing programs if the data parallelism in the program can be recognized and utilized by the compiler.
Read moreProgramming abstractions, compilation, and execution techniques for massively parallel data analysis
Aufgrund fallender Preise zur Speicherung von Daten kann man derzeit eine explosionsartige Zunahme in der Menge der verfügbaren Daten beobachten. Diese Entwicklung gibt Unternehmen und wissenschaftliche Institutionen die Möglichkeit empirische Daten in ungekannter Größenordnung zu analysieren. Für viele Firmen ist die Analyse der gesammelten Daten aus ihrem operationalen Geschäft längst zu einem zentralen strategischen Aspekt geworden. Im Gegensatz zu der seit längerem schon betriebenen Business Intelligence, bestehen diese Analysen nicht mehr nur aus traditionellen relationalen Anfragen. In zunehmendem Anteil kommen komplexe Algorithmen aus den Bereichen Data Mining und Maschinelles Lernen hinzu, um versteckte Muster in den Daten zu erkennen, oder Vorhersagemodelle zu trainieren. Mit zunehmender Datenmenge und Komplexität der Analysen wird jedoch eine neue Generation von Systemen benötigt, die diese Kombination aus Anfragekomplexität und Datenvolumen gewachsen sind. Relationale Datenbanken waren lange Zeit das Zugpferd der Datenanalyse im großen Stil. Grund dafür war zum großen Teil ihre deklarativen Anfragesprache, welche es ermöglichte die logischen und physischen Aspekte der Datenspeicherung und Verarbeitung zu trennen, und Anfragen automatisch zu optimieren. Das starres Datenmodell und ihre beschränkte Menge von möglichen Operationen schränken jedoch die Anwendbarkeit von relationalen Datenbanken für viele der neueren analytischen Probleme stark ein. Diese Erkenntnis hat die Entwicklung einer neuen Generation von Systemen und Architekturen eingeläutet, die sich durch sehr generische Abstraktionen für parallelisierbare analytische Programme auszeichnen; MapReduce kann hier beispielhaft genannt werden, als der zweifelsohne prominenteste Vertreter dieser Systeme. Zwar vereinfachte und erschloss diese neue Generation von Systemen die Datenanalyse in diversen neuen Anwendungsfeldern, sie ist jedoch nicht in der Lage komplexe Anwendungen aus den Bereichen Data Mining und Maschinelles Lernen effizient abzubilden, ohne sich dabei extrem auf spezifische Anwendungen zu spezialisieren. Verglichen mit den relationalen Datenbanken haben MapReduce und vergleichbare Systeme außerdem die deklarative Abstraktion aufgegeben und zwingen den Anwender dazu systemnahe Programme zu schreiben und diese manuell zu optimieren. In dieser Dissertation werden verschiedene Techniken vorgestellt, die es ermöglichen etliche der zentralen Eigenschaften von relationalen Datenbanken im Kontext dieser neuen Generation von daten-parallelen Analysesystemen zu realisieren. Mithilfe dieser Techniken ist es möglich ein Analysesystem zu beschreiben, dessen Programme gleichzeitig sowohl generische und ausdrucksstark, als auch prägnant und deklarativ sind. Im einzelnen stellen wir folgende Techniken vor: Erstens, eine Programmierabstraktion die generisch ist und mit komplexen Datenmodellen umgehen kann, aber gleichzeitig viele der deklarativen Eigenschaften der relationalen Algebra erhält. Programme, die gegen dies Abstraktion entwickelt werden können ähnlich optimiert werden wie relationale Anfragen. Zweitens stellen wir eine Abstraktion für iterative daten-parallele Algorithmen vor. Die Abstraktion unterstützt inkrementelle (delta-basierte) Berechnungen und geht mit zustandsbehafteteten Berechnungen transparent um. Wir beschreiben wie man einen relationalen Anfrageoptimierer erweitern kann so dass dieser iterative Anfragen effektiv optimiert. Wir zeigen dabei dass der Optimierer dadurch in die Lage versetzt wird automatisch Ausführungspläne zu erzeugen, die wohlbekannten, manuell erstellten Programmen entsprechen. Die Abstraktion subsumiert dadurch spezialisierte Systeme (wie Pregel) und bietet vergleichbare Performanz. Drittens stellen wir Methoden vor, um die Programmierabstraktion in eine funktionale Sprachen einzubetten. Diese Integration ermögliche es prägnante Programme zu schreiben und einfach wiederzuverwendenden Komponenten und Bibliotheken, sowie Domänenspezifische Sprachen, zu erstellen. Wir legen dar wie man die Übersetzung und Optimierung des daten-parallelen Programms mit dem Sprachübersetzer der funktionalen Sprache so integriert, dass maximales Optimierungspotenzial besteht.
Read moreAn overview of the Fortran D programming system
The success of large-scale parallel architectures is limited by the difficulty of developing machine-independent parallel programs. We have developed Fortran D, a version of Fortran extended with data decomposition specifications, to provide a portable data-parallel programming model. This paper presents the design of two key components of the Fortran D programming system: a prototype compiler and an environment to assist automatic data decomposition. The Fortran D compiler addresses program partitioning, communication generation and optimization, data decomposition analysis, run-time support for unstructured computations, and storage management. The Fortran D programming environment provides a static performance estimator and an automatic data partitioner. We believe that the Fortran D programming system will significantly ease the task of writing machine-independent data-parallel programs.
Read moreUsing Program Transformations to Provide Safety Properties for Real-Time Systems
The process of showing that a program satisfies some particular properties with respect to its specification is called program verification. Axiomatic semantics is a verification method that makes assertions describing properties about the states of a program. There exists a transformation from the assertions of a program’s verification proof to executable assertions. The latter may be embedded in the program to make it fault tolerant. An axiomatic proof system for concurrent programs is applied to generate executable assertions in a real time distributed environment. A train set example is used as modelproblem.
Read moreDeriving a Fault Tolerant Real Time Program using Program Verification
Deriving a Fault Tolerant Real Time Program using Program Verification
A comparison of message passing and shared memory architectures for data parallel programs
Shared memory and message passing are two opposing communication models for parallel multicomputer architectures. Comparing such architectures has been difficult, because applications must be hand-crafted for each architecture, often resulting in radically different sources for comparison. While it is clear that shared memory machines are currently easier to program, in the future, programs will be written in high-level languages and compiled to the specific parallel target, thus eliminating this difference.In this paper, we evaluate several parallel architecture alternatives --- message passing, NUMA, and cachecoherent shared memory --- for a collection of scientific benchmarks written in C*, a data-parallel language. Using a single suite of C* source programs, we compile each benchmark and simulate the interconnect for the alternative models. Our objective is to examine underlying, technology-independent costs inherent in each alternative. Our results show the relative work required to execute these data parallel programs on the different architectures, and point out where some models have inherent advantages for particular data-parallel program styles.
Read moreA comparison of message passing and shared memory architectures for data parallel programs
Shared memory and message passing are two opposing communication models for parallel multicomputer architectures. Comparing such architectures has been difficult, because applications must be hand-crafted for each architecture, often resulting in radically different sources for comparison. While it is clear that shared memory machines are currently easier to program, in the future, programs will be written in high-level languages and compiled to the specific parallel target, thus eliminating this difference.In this paper, we evaluate several parallel architecture alternatives --- message passing, NUMA, and cachecoherent shared memory --- for a collection of scientific benchmarks written in C*, a data-parallel language. Using a single suite of C* source programs, we compile each benchmark and simulate the interconnect for the alternative models. Our objective is to examine underlying, technology-independent costs inherent in each alternative. Our results show the relative work required to execute these data parallel programs on the different architectures, and point out where some models have inherent advantages for particular data-parallel program styles.
Read moreOn the consistency of distributed proofs with hidden subtrees
Previous work has shown that distributed authorization systems that fail to sample a consistent snapshot of the underlying system during policy evaluation are vulnerable to a number of attacks. Unfortuantely, the consistency enforcement solutions presented in previous work were designed for systems in which only CA-certified evidence is used during the decision-making process, all of which is available to the decision-making node at runtime. In this article, we generalize previous results and present light-weight mechanisms through which consistency constraints can be enforced in proof systems in which the full details of a proof may be unavailable to the querier due to information release policies, and the existence of certificate authorities for certifying evidence is unlikely; these types of distributed proof systems are likely candidates for use in pervasive computing and sensor network environments. We present modifications to one such distributed proof system that enable three types of consistency constraints to be enforced while still respecting the same confidentiality and integrity policies as the original proof system. We then discuss how these techniques can be adapted and applied to other, less restrictive, distributed proof systems. Further, we detail a performance analysis that illustrates the modest overheads of our consistency enforcement schemes.
Read moreAn axiomatic treatment of a parallel programming language
This paper describes a semantically-based axiomatic treatment of a parallel programming language with shared variable concurrency and conditional critical regions, essentially the language discussed by Owicki and Gries [20,21]. We use a structural operational semantics for this language, based on work of Hennessy and Plotkin [22,26], and we use the semantic structure to suggest a class of assertions for expressing properties of commands. We then define syntactic operations on assertions which correspond precisely to syntactic constructs of the programming language; in particular, we define sequential and parallel composition of assertions. This enables us to design a truly compositional proof system for program properties. Our proof system is sound and relatively complete. We examine the relationship between our proof system and the Owicki-Gries proof system. Our assertions are more expressive than Owicki's, and her proof outlines correspond roughly to a special subset of our assertion language. Owicki's parallel rule can be thought of as being based on a slightly different form of parallel composition of assertions; our form does not require interference-freedom, and our proof system is relatively complete without the need for auxiliary variables. Connections with other work, including the “Generalized Hoare Logic” of Lamport and Schneider [16,17], and with the Transition Logic of Gerth [11], are discussed briefly.KeywordsAuxiliary VariableProof SystemSequential CompositionParallel CompositionAtomic ActionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Read more