First-class monadic schedules

It is well known that modern functional programming languages are naturally amenable to parallel programming. Achieving efficient parallelism using functional languages, however, remains difficult. Perhaps the most important reason for this is their lack of support for efficient in-place updates, i.e., mutation, which is important for the implementation of both parallel algorithms and the run-time system services (e.g., schedulers and synchronization primitives) used to execute them. In this paper, we propose techniques for efficient mutation in parallel functional languages. To this end, we couple the memory manager with the thread scheduler to make reading and updating data allocated by nested threads efficient. We describe the key algorithms behind our technique, implement them in the MLton Standard ML compiler, and present an empirical evaluation. Our experiments show that the approach performs well, significantly improving efficiency over existing functional language implementations.

It is well known that modern functional programming languages are naturally amenable to parallel programming. Achieving efficient parallelism using functional languages, however, remains difficult. Perhaps the most important reason for this is their lack of support for efficient in-place updates, i.e., mutation, which is important for the implementation of both parallel algorithms and the run-time system services (e.g., schedulers and synchronization primitives) used to execute them. In this paper, we propose techniques for efficient mutation in parallel functional languages. To this end, we couple the memory manager with the thread scheduler to make reading and updating data allocated by nested threads efficient. We describe the key algorithms behind our technique, implement them in the MLton Standard ML compiler, and present an empirical evaluation. Our experiments show that the approach performs well, significantly improving efficiency over existing functional language implementations.

Current (heterogeneous) multi-core environments such as gpGPU architectures are hard to program with normal imperative and object-oriented (OO) languages. There are two basic problems to tackle: (1) it is too easy to program race conditions and dead-locks with the available synchronization primitives, and (2) these environments do not support (or support inefficiently) the instructions required for efficient execution of OO programs, e.g., because function pointers and pointer arithmetic are lacking. We address both problems with a new language that comprises both Functional Programming (FP) and OO programming. We solve problem (1) by auto-parallelization in the functional core where all loops and non-dependent calls can be executed in parallel. FP is to be used to write computational intensive code with safe concurrent memory access. An alternative object model that does neither use pointer arithmetic nor function pointers but smart pointers/proxies (to implement polymorphism) as well as mixins and templates (to implement OO like code reuse) solves problem (2). To cleanly integrate the two language cores, we propose a new integration model that even grants some restricted ways to access state from within FP mode. With the new language and prototype compiler we can transparently parallelize code to target both Cuda and multi-core machines (without annotations from the programmer) and obtain good speedups.

We propose a new paradigm for programming multiprocessor systems, 2DT (two-dimensional transformations). 2DT programs are composed of local computations on linear data (columns) and global transformations on 2-dimensional combinations of columns (2D-arrays). Local computations can be expressed in a functional or imperative base language; a typed variant of Backus' FP is chosen for 2DT-FP. The level of abstraction of 2DT makes it suitable for programming a relevant set of algorithms, in general any algorithms, whose data can be easily mapped to 2-dimensional representations. A set of examples tries to prove this claim. An interleaving semantics for 2DT-FP is given, exposing the potential for parallel execution of 2DT-FP programs. The claim is proved that any sequential and thus any parallel execution will deliver the same result. The implementation realized on the Intel Hypercube is described.

Concurrent Clean is an experimental, lazy, higher-order parallel functional programming language based on term graph rewriting. An important difference with other languages is that in Clean graphs are manipulated and not terms. This can be used by the programmer to control communication and sharing of computation. Cyclic structures can be defined. Concurrent Clean furthermore allows to control the (parallel) order of evaluation to make efficient evaluation possible. With help of sequential annotations the default lazy evaluation can be locally changed into eager evaluation. The language enables the definition of partially strict data structures which make a whole new class of algorithms feasible in a functional language. A powerful and fast strictness analyser is incorporated in the system. The quality of the code generated by the Clean compiler has been greatly improved such that it is one of the best code generators for a lazy functional language. Two very powerful parallel annotations enable the programmer to define concurrent functional programs with arbitrary process topologies. Concurrent Clean is set up in such a way that the efficiency achieved for the sequential case can largely be maintained for a parallel implementation on loosely coupled parallel machine architectures.

Proceedings of the 13th ACM SIGPLAN international conference on Functional programming

Explicitly parallel functional languages typically use metalinguistics annotations to provide control over parallel evaluation. In this paper we explore more flexible mechanisms to control when and where an expression is evaluated: firstclass schedules and processors. We discuss the advantages of using first-class objects over traditional annotation-based systems. In particular, we show how monads can be used to access dynamic system parameters like system load in a referentially transparent manner (up to termination). Finally, we describe an implementation of first-class schedules and virtual processors on a 16-processor Silicon Graphics Challenge multiprocessor.

Functional languages are not often associated with the development of network stacks, mainly due to the lower performance and lack of support for system programming than more conventional languages such as C. However, there are functional languages that offer features which make it easier to develop network protocols than using a more conventional approach based on an imperative language. Erlang, for instance, offers support for distribution, concurrency and soft real time built-in into the language. All those features are desirable to ease the development of network protocol implementations. It is also possible to implement a reasonably efficient network stack in this language, provided some precautions are taken in the design. By using Erlang distribution it is possible to support fault tolerant distributed Tcp connections that take advantage of the distributed nature of the applications implemented in Erlang to provide low cost synchronization, with just some support from the application itself.

The pedagogical desirability of functional programming is effectively swamped by an overwhelmingly imperative‐language‐oriented culture. Because a meaningful subset of the functional paradigm is accessible within imperative languages, it is possible to identify a subset of Modula‐2 that characterizes the functional paradigm subset. A series of list‐processing problems and their solutions in the Modula‐2 subset provides students with ample exposure to the practice and theory of functional programming. Extension to the full functional paradigm and migration to specialized functional languages are managed, in the worst case by initial pseudo‐Modula‐2 programming involving hypothetical extensions supporting higher‐order functions and polymorphism. More powerful solutions to this problem are foreshadowed, in the form of preprocessor implementations of Modula‐2‐ and Ada‐derivative functional languages.

Optimizing programs to run efficiently on modern parallel hardware is hard but crucial for many applications. The predominantly used imperative languages - like C or OpenCL - force the programmer to intertwine the code describing functionality and optimizations. This results in a portability nightmare that is particularly problematic given the accelerating trend towards specialized hardware devices to further increase efficiency. Many emerging DSLs used in performance demanding domains such as deep learning or high-performance image processing attempt to simplify or even fully automate the optimization process. Using a high-level - often functional - language, programmers focus on describing functionality in a declarative way. In some systems such as Halide or TVM, a separate schedule specifies how the program should be optimized. Unfortunately, these schedules are not written in well-defined programming languages. Instead, they are implemented as a set of ad-hoc predefined APIs that the compiler writers have exposed. In this functional pearl, we show how to employ functional programming techniques to solve this challenge with elegance. We present two functional languages that work together - each addressing a separate concern. RISE is a functional language for expressing computations using well known functional data-parallel patterns. ELEVATE is a functional language for describing optimization strategies. A high-level RISE program is transformed into a low-level form using optimization strategies written in ELEVATE . From the rewritten low-level program high-performance parallel code is automatically generated. In contrast to existing high-performance domain-specific systems with scheduling APIs, in our approach programmers are not restricted to a set of built-in operations and optimizations but freely define their own computational patterns in RISE and optimization strategies in ELEVATE in a composable and reusable way. We show how our holistic functional approach achieves competitive performance with the state-of-the-art imperative systems Halide and TVM.

We explore the design and implementation of Frank, a strict functional programming language with a bidirectional effect type system designed from the ground up around a novel variant of Plotkin and Pretnar's effect handler abstraction. Effect handlers provide an abstraction for modular effectful programming: a handler acts as an interpreter for a collection of commands whose interfaces are statically tracked by the type system. However, Frank eliminates the need for an additional effect handling construct by generalising the basic mechanism of functional abstraction itself. A function is simply the special case of a Frank operator that interprets no commands. Moreover, Frank's operators can be multihandlers which simultaneously interpret commands from several sources at once, without disturbing the direct style of functional programming with values. Effect typing in Frank employs a novel form of effect polymorphism which avoid mentioning effect variables in source code. This is achieved by propagating an ambient ability inwards, rather than accumulating unions of potential effects outwards. We introduce Frank by example, and then give a formal account of the Frank type system and its semantics. We introduce Core Frank by elaborating Frank operators into functions, case expressions, and unary handlers, and then give a sound small-step operational semantics for Core Frank. Programming with effects and handlers is in its infancy. We contribute an exploration of future possibilities, particularly in combination with other forms of rich type system.

We explore the design and implementation of Frank, a strict functional programming language with a bidirectional effect type system designed from the ground up around a novel variant of Plotkin and Pretnar's effect handler abstraction. Effect handlers provide an abstraction for modular effectful programming: a handler acts as an interpreter for a collection of commands whose interfaces are statically tracked by the type system. However, Frank eliminates the need for an additional effect handling construct by generalising the basic mechanism of functional abstraction itself. A function is simply the special case of a Frank operator that interprets no commands. Moreover, Frank's operators can be multihandlers which simultaneously interpret commands from several sources at once, without disturbing the direct style of functional programming with values. Effect typing in Frank employs a novel form of effect polymorphism which avoid mentioning effect variables in source code. This is achieved by propagating an ambient ability inwards, rather than accumulating unions of potential effects outwards. We introduce Frank by example, and then give a formal account of the Frank type system and its semantics. We introduce Core Frank by elaborating Frank operators into functions, case expressions, and unary handlers, and then give a sound small-step operational semantics for Core Frank. Programming with effects and handlers is in its infancy. We contribute an exploration of future possibilities, particularly in combination with other forms of rich type system.

The popularity of multi-core systems makes software parallelization become an important way to improve performance. As a mainstream XML query language, XQuery is the core of XML processing. It is critical to take full advantage of multi-core computing to improve XML processing performance through parallelization of XQuery. However, usually it is difficult to parallelize XQuery programs because of the nested style of XQuery expressions. Moreover, implicit parallelism is necessary to simplify the development of parallel XML application. In this paper, we propose an automatic parallelization approach, which can automatically select proper types of parallelism for a specific XQuery query. Specifically, we propose a functional intermediate language called pFXQL (parallel Functional XML Query Language) to describe parallel query plans. pFXQL has parallel semantics and is well complied with XQuery. We propose a cost model to effectively support plan generation and the selection of a preferred plan. The model estimates both computational cost and parallel cost. We implement our approach in XQuery engine and conduct experiments on various multi-core systems. Experimental results verify the effectiveness of our approach.

An important feature of functional programs is that they are parallel by default. Implementing an efficient parallel functional language, however, is a major challenge, in part because the high rate of allocation and freeing associated with functional programs requires an efficient and scalable memory manager. In this paper, we present a technique for parallel memory management for strict functional languages with nested parallelism. At the highest level of abstraction, the approach consists of a technique to organize memory as a hierarchy of heaps, and an algorithm for performing automatic memory reclamation by taking advantage of a disentanglement property of parallel functional programs. More specifically, the idea is to assign to each parallel task its own heap in memory and organize the heaps in a hierarchy/tree that mirrors the hierarchy of tasks. We present a nested-parallel calculus that specifies hierarchical heaps and prove in this calculus a disentanglement property, which prohibits a task from accessing objects allocated by another task that might execute in parallel. Leveraging the disentanglement property, we present a garbage collection technique that can operate on any subtree in the memory hierarchy concurrently as other tasks (and/or other collections) proceed in parallel. We prove the safety of this collector by formalizing it in the context of our parallel calculus. In addition, we describe how the proposed techniques can be implemented on modern shared-memory machines and present a prototype implementation as an extension to MLton, a high-performance compiler for the Standard ML language. Finally, we evaluate the performance of this implementation on a number of parallel benchmarks.

We describe PEPA, the first general on-line performance monitoring tool for Erlang. Erlang is a new non-lazy functional programming language with explicit concurrency. It has been designed especially for building fault tolerant industrial applications that have soft real-time requirements. It has already been used to build substantial software products in the telecommunications area. PEPA gives figures for hardware utilization and processing times as they are required for the development of real-time programs. The tool has a sophisticated graphical user interface that guides the user in a structured search for performance bottlenecks. The online mode of operation makes it possible to selectively record only data that is actually displayed. This reduces the probe effect introduced by the measurements to the achievable minimum and gives the user the possibility to control the amount of intrusion.