The Design and Implementation of GUMSMP

The Glasgow Haskell Compiler (GHC) is a well supported optimizing compiler for the Haskell programming language, along with its own extensions to the language and libraries. Haskell's lazy semantics imposes a runtime model which is in general difficult to implement efficiently. GHC achieves good performance across a wide variety of programs via aggressive optimization taking advantage of the lack of side effects, and by targeting a carefully tuned virtual machine. The Intel Labs Haskell Research Compiler uses GHC as a frontend, but provides a new whole-program optimizing backend by compiling the GHC intermediate representation to a relatively generic functional language compilation platform. We found that GHC's external Core language was relatively easy to use, but reusing GHC's libraries and achieving full compatibility were harder. For certain classes of programs, our platform provides substantial performance benefits over GHC alone, performing 2x faster than GHC with the LLVM backend on selected modern performance-oriented benchmarks; for other classes of programs, the benefits of GHC's tuned virtual machine continue to outweigh the benefits of more aggressive whole program optimization. Overall we achieve parity with GHC with the LLVM backend. In this paper, we describe our Haskell compiler stack, its implementation and optimization approach, and present benchmark results comparing it to GHC.

OpenMP is an architecture-independent language for programming in the shared memory model. OpenMP is designed to be simple and powerful in terms of programming abstractions. Unfortunately, the architecture-independent abstractions sometimes come with the price of low parallel performance. This is especially true for applications with an unstructured data access pattern running on distributed shared memory systems (DSM). Here, proper data distribution and algorithmic optimizations play a vital role for performance. In this article, we have investigated ways of improving the performance of an industrial class conjugate gradient (CG) solver, implemented in OpenMP running on two types of shared memory systems. We have evaluated bandwidth minimization, graph partitioning and reformulations of the original algorithm reducing global barriers. By a detailed analysis of barrier time and memory system performance, we found that bandwidth minimization is the most important optimization reducing both L2 misses and remote memory accesses. On a uniform memory system, we get perfect scaling. On a NUMA system, the performance is significantly improved with the algorithmic optimizations leaving the system dependent global reduction operations as a bottleneck.

We show how a shared memory architecture can be simulated using computers of different architectures. A description of a parallel Prolog (ROLOG) implementation for virtual shared memory machines is included. This paper reveals some general techniques and guidelines for (re)writing software for these machines.

Shared memory architectures have recently experienced a large increase in thread-level parallelism, leading to complex memory hierarchies with multiple cache memory levels and memory controllers. These new designs created a Non-Uniform Memory Access (NUMA) behavior, where the performance and energy consumption of memory accesses depend on the place where the data is located in the memory hierarchy. Accesses to local caches or memory controllers are generally more efficient than accesses to remote ones. A common way to improve the locality and balance of memory accesses is to determine the mapping of threads to cores and data to memory controllers based on the affinity between threads and data. Such mapping techniques can operate at different hardware and software levels, which impacts their complexity, applicability, and the resulting performance and energy consumption gains. In this article, we introduce a taxonomy to classify different mapping mechanisms and provide a comprehensive overview of existing solutions.

Efficient analysis of medical images to assist physician’s decision making is an important task. However, the analysis of such images often requires sophisticated segmentation and classification algorithms. An approach to speed up these time consuming operations is to use parallel processing. In this paper a new parallel system for medical image analysis is presented. The system combines distributed and shared memory architectures using MPI and the inter-process communication switching mechanism (IPC). MPI is used to communicate between nodes and shared-memory IPC is used to perform shared memory operations among processors within a node. We show how to map a clinical endoscopic image analysis algorithm efficiently onto this architecture. This results in an implementation with significant runtime savings.

Shared memory is the most popular parallel programming model for multi-core processors, while message passing is generally used for large distributed machines. However, as the number of cores on a chip increases, the relative merits of shared memory versus message passing change, and we argue that message passing becomes a viable, high performing, and parallel programming model. To demonstrate this hypothesis, we compare a shared memory architecture with a new message passing architecture on a suite of applications tuned for each system independently. Perhaps surprisingly, the fundamental behaviors of the applications studied in this work, when optimized for both models, are very similar to each other, and both could execute efficiently on multicore architectures despite many implementations being different from each other. Furthermore, if hardware is tuned to support message passing by supporting bulk message transfer and the elimination of unnecessary coherence overheads, and if effective support is available for global operations, then some applications would perform much better on a message passing architecture. Leveraging our insights, we design a message passing architecture that supports both memory-to-memory and cache-to-cache messaging in hardware. With the new architecture, message passing is able to outperform its shared memory counterparts on many of the applications due to the unique advantages of the message passing hardware as compared to cache coherence. In the best case, message passing achieves up to a 34% increase in speed over its shared memory counterpart, and it achieves an average 10% increase in speed. In the worst case, message passing is slowed down in two applications—CG (conjugate gradient) and FT (Fourier transform)—because it could not perform well on the unique data sharing patterns as its counterpart of shared memory. Overall, our analysis demonstrates the importance of considering message passing as a high performing and hardware-supported programming model on future multicore architectures.

MapReduce was originally proposed as a suitable and efficient approach for analyzing and processing large amounts of data. Since then, many researches contributed with MapReduce implementations for distributed and shared memory architectures. Nevertheless, different architectural levels require different optimization strategies in order to achieve high-performance computing. Such strategies in turn have caused very different MapReduce programming interfaces among these researches. This paper presents some research notes on coding productivity when developing MapReduce applications for distributed and shared memory architectures. As a case study, we introduce our current research on a unified MapReduce domain-specific language with code generation for Hadoop and Phoenix++, which has achieved a coding productivity increase from 41.84% and up to 94.71% without significant performance losses (below 3%) compared to those frameworks.

On-chip memory is a critical component of MPSoC architecture. Virtually Shared Scratch Pad Memory (VSSPM) architecture is a novel approach to close the processor-memory speed gap since it can take the advantages of both shared and private SPM. In this paper, we propose an efficient variable partitioning algorithms, Global View Prediction (GVP), to improve the overall throughput. The experimental results demonstrate the effectiveness of our approach.

Thread-parallel mesh improvement using face and edge swapping and vertex insertion

In mining supply chains, large combinatorial optimization problems arise. These are NP-hard and typically require a large number of computing resources to solve them. In particular, the run-time overheads can become increasingly prohibitive with increasing problem sizes. Parallel methods provide a way to manage such run-time issues by utilising several processors in independent or shared memory architectures. However it is not obvious how to adapt serial optimisation algorithms to perform best in a parallel environment. Here, we consider a resource constrained scheduling problem which is motivated in mining supply chains and present two popular meta-heuristics, ant colony optimization (ACO) and simulated annealing and investigate how best to parallelize these methods on a shared memory architecture consisting of several cores. ACO’s solution construction framework is inherently parallel allowing a relatively straightforward parallel implementation. However, for best performance, ACO needs an element of local search. This significantly complicates the paralellization. Several alternative schemes for parallel ACO with elements of local search are considered and evaluated empirically. We find that ACO with local search is the most effective single-threaded algorithm. The best parallel implementation can obtain similar quality results to the serial method in significantly less elapsed time.

The paper presents parallel implementation of the FDTD method that is used for computer simulation of high frequency electromagnetic field propagation. The parallel implementation is based on simultaneous application of message passing communication in distributed memory multi-processor environment (MPI library) and communication by shared variables in the environment with shared memory (OpenMP library). The methodology of mixed communication mode with message passing and shared memory is presented. Parallel versions of the FDTD computation programs were tested in the system composed of four shared memory processor clusters additionally connected by message passing networks. Program execution efficiency measurements and comparisons for different communication versions are presented.

To resolve the high memory bandwidth requirements presented by output-queued switches, several parallel shared-memory architectures have been recently proposed. The complexity of such systems lies in the algorithms used to assign arriving packets to available shared memories. To date, packet placement algorithms either require N sequential placement decisions per interval or necessitate O(N/sup 2/) memories, limiting switch scalability and speed. This paper presents a scalable multistage, pipelined memory management algorithm which offers O(logN) computational complexity while requiring only 3N dual-port memories, at a cost of fixed latency.

Bayesian approaches to decision trees (DTs) using Markov Chain Monte Carlo (MCMC) samplers have recently demonstrated state-of-the-art accuracy performance when it comes to training DTs to solve classification problems. Despite the competitive classification accuracy, MCMC requires a potentially long runtime to converge. A widely used approach to reducing an algorithm’s runtime is to employ modern multi-core computer architectures, either with shared memory (SM) or distributed memory (DM), and use parallel computing to accelerate the algorithm. However, the inherent sequential nature of MCMC makes it unsuitable for parallel implementation unless the accuracy is sacrificed. This issue is particularly evident in DM architectures, which normally provide access to larger numbers of cores than SM. Sequential Monte Carlo (SMC) samplers are a parallel alternative to MCMC, which do not trade off accuracy for parallelism. However, the performance of SMC samplers in the context of DTs is underexplored, and the parallelization is complicated by the challenges in parallelizing its bottleneck, namely redistribution, especially on variable-size data types such as DTs. In this work, we study the problem of parallelizing SMC in the context of DTs both on SM and DM. On both memory architectures, we show that the proposed parallelization strategies achieve asymptotically optimal O(log2N) time complexity. Numerical results are presented for a 32-core SM machine and a 256-core DM cluster. For both computer architectures, the experimental results show that our approach has comparable or better accuracy than MCMC but runs up to 51 times faster on SM and 640 times faster on DM. In this paper, we share the GitHub link to the source code.

The problem of partitioning task graphs in its general form is known to be NP-complete and it is extremely difficult to come up with simple but effective and fast heuristics too. In this paper, the tree task graphs are considered which arise from many important programming paradigms such as divide and conquer branch and bound, etc. The target architecture considered is shared memory architecture as it typifies a wide range of research prototypes and commercial products of multiprocessors. Optimal sequential and parallel algorithms for partitioning tree task graphs such that the bottleneck is minimized "with minimum number of processors are developed. The bandwidth minimization problem is NP-complete even for trees. Three effective, simple 2-pass heuristics and their parallel versions are given. The effectiveness and efficiency of those heuristics are validated through extensive simulations.

Triadic analysis encompasses a useful set of graph mining methods that are centered on the concept of a triad, which is a sub graph of three nodes. Such methods are often applied in the social sciences as well as many other diverse fields. Triadic methods commonly operate on a triad census that counts the number of triads of every possible edge configuration in a graph. Like other graph algorithms, triadic census algorithms do not scale well when graphs reach tens of millions to billions of nodes. To enable the triadic analysis of large-scale graphs, we developed and optimized a triad census algorithm to efficiently execute on shared memory architectures. We then conducted performance evaluations of the parallel triad census algorithm on three specific systems: CrayXMT, HP Superdome, and AMD multi-core NUMA machine. These three systems have shared memory architectures but with markedly different hardware capabilities to manage parallelism.