2DT-FP: A parallel functional programming language on two-dimensional data

We propose a new paradigm for programming tightly coupled multicomputer systems, 2DT. 2DT-programs are composed of local computations on linear data (columns) and global transformations on 2-dimensional combinations of the columns (2D-arrays). Local computations can be expressed in a functional or imperative base language; a typed variant of Backus' FP, 2DT-FP, is chosen in this paper.An interleaving semantics for 2DT-FP is given, exposing the potential for parallel execution of 2DT-FP programs. The claim is proved that any sequential and thus any parallel execution will deliver the same result.Keywordsparallel languageoperational semantics

The term "Logic Programming" refers to a variety of computer languages and execution models which are based on the traditional concept of Symbolic Logic. The expressive power of these languages offers promise to be of great assistance in facing the programming challenges of present and future symbolic processing applications in Artificial Intelligence, Knowledge-based systems, and many other areas of computing. The sequential execution speed of logic programs has been greatly improved since the advent of the first interpreters. However, higher inference speeds are still required in order to meet the demands of applications such as those contemplated for next generation computer systems. The execution of logic programs in parallel is currently considered a promising strategy for attaining such inference speeds. Logic Programming in turn appears as a suitable programming paradigm for parallel architectures because of the many opportunities for parallel execution present in the implementation of logic programs. This dissertation presents an efficient parallel execution model for logic programs. The model is described from the source language level down to an "Abstract Machine" level suitable for direct implementation on existing parallel systems or for the design of special purpose parallel architectures. Few assumptions are made at the source language level and therefore the techniques developed and the general Abstract Machine design are applicable to a variety of logic (and also functional) languages. These techniques offer efficient solutions to several areas of parallel Logic Programming implementation previously considered problematic or a source of considerable overhead, such as the detection and handling of variable binding conflicts in AND-Parallelism, the specification of control and management of the execution tree, the treatment of distributed backtracking, and goal scheduling and memory management issues, etc. A parallel Abstract Machine design is offered, specifying data areas, operation, and a suitable instruction set. This design is based on extending to a parallel environment the techniques introduced by the Warren Abstract Machine, which have already made very fast and space efficient sequential systems a reality. Therefore, the model herein presented is capable of retaining sequential execution speed similar to that of high performance sequential systems, while extracting additional gains in speed by efficiently implementing parallel execution. These claims are supported by simulations of the Abstract Machine on sample programs.

Concurrent Clean is an experimental, lazy, higher-order parallel functional programming language based on term graph rewriting. An important difference with other languages is that in Clean graphs are manipulated and not terms. This can be used by the programmer to control communication and sharing of computation. Cyclic structures can be defined. Concurrent Clean furthermore allows to control the (parallel) order of evaluation to make efficient evaluation possible. With help of sequential annotations the default lazy evaluation can be locally changed into eager evaluation. The language enables the definition of partially strict data structures which make a whole new class of algorithms feasible in a functional language. A powerful and fast strictness analyser is incorporated in the system. The quality of the code generated by the Clean compiler has been greatly improved such that it is one of the best code generators for a lazy functional language. Two very powerful parallel annotations enable the programmer to define concurrent functional programs with arbitrary process topologies. Concurrent Clean is set up in such a way that the efficiency achieved for the sequential case can largely be maintained for a parallel implementation on loosely coupled parallel machine architectures.

Parallel functional languages often use meta-linguistic annotations to provide control over parallel evaluation. In this paper we explore a flexible mechanism to control when an expression is evaluated: first-class monadic schedules . We discuss the advantages of using such first-class values over traditional annotation-based systems. In particular, it is often desirable to make decisions about the operational behavior of parallel programs depending on the dynamic state of the system. For example, we may want to measure the system load before deciding to evaluate expressions in parallel. For this purpose, we show how monads can be used to access dynamic system parameters in a referentially transparent manner (up to termination).As a mechanism to reason about schedules, we present a set of algebraic properties that any implementation of schedules must satisfy. We also describe an implementation that translates schedules into a dialect of Scheme extended with futures . We prove that this implementation satisfies the given set of algebraic properties, and give performance results for a parallel solution to the n -body problem using the Barnes--Hut method.Although our ideas were developed specifically for nonstrict functional languages such as Haskell, we briefly discuss how they can be used with strict functional languages and imperative languages as well.

Current (heterogeneous) multi-core environments such as gpGPU architectures are hard to program with normal imperative and object-oriented (OO) languages. There are two basic problems to tackle: (1) it is too easy to program race conditions and dead-locks with the available synchronization primitives, and (2) these environments do not support (or support inefficiently) the instructions required for efficient execution of OO programs, e.g., because function pointers and pointer arithmetic are lacking. We address both problems with a new language that comprises both Functional Programming (FP) and OO programming. We solve problem (1) by auto-parallelization in the functional core where all loops and non-dependent calls can be executed in parallel. FP is to be used to write computational intensive code with safe concurrent memory access. An alternative object model that does neither use pointer arithmetic nor function pointers but smart pointers/proxies (to implement polymorphism) as well as mixins and templates (to implement OO like code reuse) solves problem (2). To cleanly integrate the two language cores, we propose a new integration model that even grants some restricted ways to access state from within FP mode. With the new language and prototype compiler we can transparently parallelize code to target both Cuda and multi-core machines (without annotations from the programmer) and obtain good speedups.

It is well known that modern functional programming languages are naturally amenable to parallel programming. Achieving efficient parallelism using functional languages, however, remains difficult. Perhaps the most important reason for this is their lack of support for efficient in-place updates, i.e., mutation, which is important for the implementation of both parallel algorithms and the run-time system services (e.g., schedulers and synchronization primitives) used to execute them. In this paper, we propose techniques for efficient mutation in parallel functional languages. To this end, we couple the memory manager with the thread scheduler to make reading and updating data allocated by nested threads efficient. We describe the key algorithms behind our technique, implement them in the MLton Standard ML compiler, and present an empirical evaluation. Our experiments show that the approach performs well, significantly improving efficiency over existing functional language implementations.

It is well known that modern functional programming languages are naturally amenable to parallel programming. Achieving efficient parallelism using functional languages, however, remains difficult. Perhaps the most important reason for this is their lack of support for efficient in-place updates, i.e., mutation, which is important for the implementation of both parallel algorithms and the run-time system services (e.g., schedulers and synchronization primitives) used to execute them. In this paper, we propose techniques for efficient mutation in parallel functional languages. To this end, we couple the memory manager with the thread scheduler to make reading and updating data allocated by nested threads efficient. We describe the key algorithms behind our technique, implement them in the MLton Standard ML compiler, and present an empirical evaluation. Our experiments show that the approach performs well, significantly improving efficiency over existing functional language implementations.

Parallel programs have traditionally been implemented on distributed systems using conventional programming languages, such as Fortran or C, plus a library with support for communication, such as, for instance, PVM [11]. Also, some effort has been put into developing parallel programming languages [13]. Usually, the different development platforms are compared as to application execution time or parallel efficiency. However, as pointed out in [4], when measuring the performance of a parallel application or programming tool, different metrics may need to be taken into consideration. The ease of construction of an application and the time a programmer spends to implement it may in some cases be more important than the final execution time. Working under this assumption, this work argues that the use of interpreted languages as a basis for parallel programming tools should be further investigated. As support for our argument, we show an example of use of an interpreted language in a classical application in parallel distributed programming, namely, that of detecting program termination. Two different algorithms for termination detection are implemented in Lua, an extension language developed at PUC-Rio [5].

Binary search trees (BSTs) are one of the most important data structures in the field of computer science. We may easily write a parallel construction program of a BST by extending the sequential algorithm straightly. However, in such conventional approaches, the order of nodes inserted into a BST is determined dynamically, depending on the occasional state of the parallel thread execution. It results in a BST with a different structure (node position) generated on every execution of the parallel program. On the other hand, we have been developing parallel construction schemes of the BST with the same structure as a BST generated by the sequential algorithm. One is the speculatively parallel construction of a BST. And another is the purely (non-speculatively) parallel construction, but it was derived through the concept of thread-level speculation. This paper evaluates the performances of those construction schemes on several types of shared-memory multiprocessors. For the large enough size of BST, our new parallel programs can construct a BST with always the same structure on a little lower or sometimes higher performance than the program that makes a BST with a different structure on every execution. And in contrast with the general expectation that simply enlarging the size of parallel tasks increases misspeculation and damages the performance, we found that it sometimes enhances the performance of speculatively parallel execution.

Aufgrund fallender Preise zur Speicherung von Daten kann man derzeit eine explosionsartige Zunahme in der Menge der verfügbaren Daten beobachten. Diese Entwicklung gibt Unternehmen und wissenschaftliche Institutionen die Möglichkeit empirische Daten in ungekannter Größenordnung zu analysieren. Für viele Firmen ist die Analyse der gesammelten Daten aus ihrem operationalen Geschäft längst zu einem zentralen strategischen Aspekt geworden. Im Gegensatz zu der seit längerem schon betriebenen Business Intelligence, bestehen diese Analysen nicht mehr nur aus traditionellen relationalen Anfragen. In zunehmendem Anteil kommen komplexe Algorithmen aus den Bereichen Data Mining und Maschinelles Lernen hinzu, um versteckte Muster in den Daten zu erkennen, oder Vorhersagemodelle zu trainieren. Mit zunehmender Datenmenge und Komplexität der Analysen wird jedoch eine neue Generation von Systemen benötigt, die diese Kombination aus Anfragekomplexität und Datenvolumen gewachsen sind. Relationale Datenbanken waren lange Zeit das Zugpferd der Datenanalyse im großen Stil. Grund dafür war zum großen Teil ihre deklarativen Anfragesprache, welche es ermöglichte die logischen und physischen Aspekte der Datenspeicherung und Verarbeitung zu trennen, und Anfragen automatisch zu optimieren. Das starres Datenmodell und ihre beschränkte Menge von möglichen Operationen schränken jedoch die Anwendbarkeit von relationalen Datenbanken für viele der neueren analytischen Probleme stark ein. Diese Erkenntnis hat die Entwicklung einer neuen Generation von Systemen und Architekturen eingeläutet, die sich durch sehr generische Abstraktionen für parallelisierbare analytische Programme auszeichnen; MapReduce kann hier beispielhaft genannt werden, als der zweifelsohne prominenteste Vertreter dieser Systeme. Zwar vereinfachte und erschloss diese neue Generation von Systemen die Datenanalyse in diversen neuen Anwendungsfeldern, sie ist jedoch nicht in der Lage komplexe Anwendungen aus den Bereichen Data Mining und Maschinelles Lernen effizient abzubilden, ohne sich dabei extrem auf spezifische Anwendungen zu spezialisieren. Verglichen mit den relationalen Datenbanken haben MapReduce und vergleichbare Systeme außerdem die deklarative Abstraktion aufgegeben und zwingen den Anwender dazu systemnahe Programme zu schreiben und diese manuell zu optimieren. In dieser Dissertation werden verschiedene Techniken vorgestellt, die es ermöglichen etliche der zentralen Eigenschaften von relationalen Datenbanken im Kontext dieser neuen Generation von daten-parallelen Analysesystemen zu realisieren. Mithilfe dieser Techniken ist es möglich ein Analysesystem zu beschreiben, dessen Programme gleichzeitig sowohl generische und ausdrucksstark, als auch prägnant und deklarativ sind. Im einzelnen stellen wir folgende Techniken vor: Erstens, eine Programmierabstraktion die generisch ist und mit komplexen Datenmodellen umgehen kann, aber gleichzeitig viele der deklarativen Eigenschaften der relationalen Algebra erhält. Programme, die gegen dies Abstraktion entwickelt werden können ähnlich optimiert werden wie relationale Anfragen. Zweitens stellen wir eine Abstraktion für iterative daten-parallele Algorithmen vor. Die Abstraktion unterstützt inkrementelle (delta-basierte) Berechnungen und geht mit zustandsbehafteteten Berechnungen transparent um. Wir beschreiben wie man einen relationalen Anfrageoptimierer erweitern kann so dass dieser iterative Anfragen effektiv optimiert. Wir zeigen dabei dass der Optimierer dadurch in die Lage versetzt wird automatisch Ausführungspläne zu erzeugen, die wohlbekannten, manuell erstellten Programmen entsprechen. Die Abstraktion subsumiert dadurch spezialisierte Systeme (wie Pregel) und bietet vergleichbare Performanz. Drittens stellen wir Methoden vor, um die Programmierabstraktion in eine funktionale Sprachen einzubetten. Diese Integration ermögliche es prägnante Programme zu schreiben und einfach wiederzuverwendenden Komponenten und Bibliotheken, sowie Domänenspezifische Sprachen, zu erstellen. Wir legen dar wie man die Übersetzung und Optimierung des daten-parallelen Programms mit dem Sprachübersetzer der funktionalen Sprache so integriert, dass maximales Optimierungspotenzial besteht.

Proceedings of the 13th ACM SIGPLAN international conference on Functional programming

We describe PEPA, the first general on-line performance monitoring tool for Erlang. Erlang is a new non-lazy functional programming language with explicit concurrency. It has been designed especially for building fault tolerant industrial applications that have soft real-time requirements. It has already been used to build substantial software products in the telecommunications area. PEPA gives figures for hardware utilization and processing times as they are required for the development of real-time programs. The tool has a sophisticated graphical user interface that guides the user in a structured search for performance bottlenecks. The online mode of operation makes it possible to selectively record only data that is actually displayed. This reduces the probe effect introduced by the measurements to the achievable minimum and gives the user the possibility to control the amount of intrusion.

Programmers of sequential and parallel computers share the objective of developing fast, reliable programs as efficiently as possible. Already, considerable effort has been made to develop tools and techniques for sequential computers. Several approaches have been devised to meet the parallel programming challenge. Sequential languages have been adapted for parallel computers; new parallel languages designed; and tools developed which aid the programmer in detecting parallel code. A parallel programming assistant can aid the user in several ways: it can carry out tasks that are simple but mundane to the user, it can make comprehension easier for the user by summarizing program information; and it can cooperate with the user in doing a complex task, thus combining its own knowledge with that of the user's.

The popularity of multi-core systems makes software parallelization become an important way to improve performance. As a mainstream XML query language, XQuery is the core of XML processing. It is critical to take full advantage of multi-core computing to improve XML processing performance through parallelization of XQuery. However, usually it is difficult to parallelize XQuery programs because of the nested style of XQuery expressions. Moreover, implicit parallelism is necessary to simplify the development of parallel XML application. In this paper, we propose an automatic parallelization approach, which can automatically select proper types of parallelism for a specific XQuery query. Specifically, we propose a functional intermediate language called pFXQL (parallel Functional XML Query Language) to describe parallel query plans. pFXQL has parallel semantics and is well complied with XQuery. We propose a cost model to effectively support plan generation and the selection of a preferred plan. The model estimates both computational cost and parallel cost. We implement our approach in XQuery engine and conduct experiments on various multi-core systems. Experimental results verify the effectiveness of our approach.

In this dissertation, we address the problem of performance efficient multithreading execution on heterogeneous multicore embedded systems. By heterogeneous multicore embedded systems we refer to those, which have real-time requirements and consist of processor tiles with General Purpose Processor (GPP), local memory, and one or more coprocessors running on reconfigurable logic ((e)FPGA). We improve system performance by combining two common methods. The first method is to exploit the available application parallelism by means of multithreading program execution. The second method is to provide hardware acceleration for the most computationally intensive kernels. More specifically our scientific approach is as follows: we categorize the existing program execution models from the processor-coprocessor synchronization prospective and we introduce new parallel execution models. Then, we provide a high-level architectural abstraction of those execution models and programming paradigm that describes and utilizes them. Furthermore, we propose a microarchitectural support for the identified execution models. The functionality of the microarchitectural extensions is encapsulated in a new reconfigurable coprocessor, called Thread Interrupt State Controller (TISC). To improve the overall system performance, we employ the newly proposed program execution models to transfer highly time-variable and time-consuming Real-Time Operating System (RTOS) and application kernels from software, i.e., executed on the GPPs, to hardware, i.e., executed on the reconfigurable coprocessors. We refer to this reconfigurable coprocessor as Hardware Task Status Manager (HWTSM). Due to the properties of the newly introduced execution models such as parallel execution and constant response time, we preserve the predictability and composability at application level. Last but not least, we introduce a framework for distribution of slack information (idle processor time) among processor tiles. In the proposed framework we employ one of the newly introduced parallel processor-coprocessor execution models. We refer to the new reconfigurable coprocessor as RS. We use the extra slack information obtained through our framework for Dynamic Voltage Frequency Scaling that reduces the overall energy consumption. Based on the available experimental results with synthetic and real applications, we improve the system speedup up to 19.6 times with the help of the Thread Interrupt State Controller. Furthermore, we reduce RTOS cost with the help of the Hardware Task Status Manager, which results in additional application acceleration up to 13.3%. Last but not least, we improve the system energy consumption up to 56.7% over current state of the art with the help of inter-tile remote slack information distribution framework. Overall, with the help of our contributions, the system performance is improved, the predictability and composability are preserved, all with reduced energy consumption.