Data Layout and SIMD Abstraction Layers: Decoupling Interfaces from Implementations

Previous work has shown how the well-studied and SIMD-friendly Structure of Arrays (SOA) data layout strategy can speed up applications in high-performance computing compared to a traditional Array of Structures (AOS) data layout. However, a standard SOA layout cannot handle structures with inner arrays; such structures appear frequently in graph-based applications and object-oriented designs with associations of high multiplicity. This work extends the SOA data layout to structures with array-typed fields. We present different techniques for inlining (embedding) inner arrays into an AOS or SOA layout, as well as the design and implementation of an embedded C++/CUDA DSL that lets programmers write such layouts in a notation close to standard C++. We evaluate several layout strategies with a traffic flow simulation, an important real-world application in transport planning.

This paper focuses on designing and implementing parallel adaptive inverse distance weighting (AIDW) interpolation algorithms by using the graphics processing unit (GPU). The AIDW is an improved version of the standard IDW, which can adaptively determine the power parameter according to the data points’ spatial distribution pattern and achieve more accurate predictions than those predicted by IDW. In this paper, we first present two versions of the GPU-accelerated AIDW, i.e. the naive version without profiting from the shared memory and the tiled version taking advantage of the shared memory. We also implement the naive version and the tiled version using two data layouts, structure of arrays and array of aligned structures, on both single and double precision. We then evaluate the performance of parallel AIDW by comparing it with its corresponding serial algorithm on three different machines equipped with the GPUs GT730M, M5000 and K40c. The experimental results indicate that: (i) there is no significant difference in the computational efficiency when different data layouts are employed; (ii) the tiled version is always slightly faster than the naive version; and (iii) on single precision the achieved speed-up can be up to 763 (on the GPU M5000), while on double precision the obtained highest speed-up is 197 (on the GPU K40c). To benefit the community, all source code and testing data related to the presented parallel AIDW algorithm are publicly available.

Manycore processors and coprocessors with wide vector extensions, such as Intel Phi and Skylake devices, have become popular due to their high throughput capability. Performance optimization on these devices requires using both their x86-compatible cores and their vector units. While the x86-compatible cores can be programmed using traditional programming interfaces following the MIMD model, such as POSIX threads, MPI and OpenMP, the SIMD vector units are harder to program. The Intel software stack provides two approaches for code vectorization: automatic vectorization through the Intel compiler and manual vectorization through vector intrinsics. While the Intel compiler often fails to vectorize code with complex control flows and function calls, the manual approach is error-prone and leads to less portable code. Hence, there has been an increasing interest in SIMT programming tools allowing the simultaneous use of x86 cores and vector units while providing programmability and code portability. However, the effective implementation of the SIMT model on these hybrid architectures is not well understood. In this work, we target this problem. First, we propose a set of compiler techniques to transform programs written using a SIMT programming model (a subset of CUDA C) into code that leverages both the x86 cores and the vector units of a hybrid MIMD/SIMD architecture, thus providing programmability, high system utilization and performance. Second, we evaluate the proposed techniques on Xeon Phi and Skylake processors using micro-benchmarks and real-world applications. Third, we compare the resulting performance with that achieved by the same code on GPUs. Based on this analysis, we point out the main challenges in supporting the SIMT model on hybrid MIMD/SIMD architectures, while providing performance comparable to that of SIMT systems (e.g., GPUs).

A complete traversal of a container C (such as a set) is informally described by the iteration scheme for x in C F(x,C) where F is a function that might possibly modify C by inserting new elements into it. We assume that the order in which the elements are treated is not relevant, as long as the iteration continues until F has been applied to all elements currently in C, including those F has inserted. Standard iteration mechanisms, such as the iterators provided in the C++ Standard Template Library (STL), do not directly support complete traversals. In this paper we present two approaches to complete traversals, both extending the STL framework, one by means of generic algorithms and the other by means of a container adaptor.

In my last column (see ibid., January/February 2001), I talked about a notational device for matrix algebra called tensor diagrams. This time I write some C++ code to symbolically evaluate these quantities. This gives me a chance to play with some as yet untried features in the C++ standard library, such as strings and standard template library (STL) container classes.

Data locality exists everywhere in the memory hierarchy. Most applications show temporal and spatial locality, and computer system and architecture designers utilize this property to improve the system performance with better data layout, prefetching, and scheduling. The locality property can be represented by memory access patterns, which records the time and frequency of accessed addresses. From the security perspective, if an attacker can trace the access pattern, sensitive information inside of the application could be observed and leaked. Oblivious RAM is one of the most effective solutions to mitigate the access pattern leakage on the system, which adds redundant data blocks in space and time. With ORAM protection, the intrinsic data locality is broken by the randomly stored data. Therefore, the application cannot gain any performance benefits from locality if the ORAM protocol is used. In this work, we would like to study the potential to support multi-range accesses with new storage and access efficient ORAM construction. Our proposed designs include two major schemes: Lite-rORAM, which minimize the storage overhead of existing rORAM; and Hybrid-rORAM, which support multiple ranges accesses with minimum storage overhead. We achieve the goal to preserve the locality for consecutive data blocks with different ranges in the application while obfuscates the access pattern as well. We tested our proposed schemes with different workloads on local and remote backends. The experimental results show that, in the best case, our proposed ORAM construction can reduce the data block retrieval time to 0.24x of the baseline Path ORAM, with 87.5% storage overhead reduction compared to rORAM.

While regular algorithms, characterized by operations on dense matrices and arrays, have long been the mainstay of scientific, high-performance computing, irregular algorithms, which feature unpredictable accesses to pointer-based data structures, are becoming increasingly common in high performance computing, arising in graph analysis, data mining and visualization, among other domains. Unfortunately, the defining characteristics of irregular applications, their dynamic, unpredictable, data-dependent access patterns and data layouts, make achieving high performance on large scale systems difficult. Scaling applications to peta- and exa-scale requires carefully controlling communication and data movement and placement, an inherently difficult task when access patterns and data layouts are unpredictable! Most irregular applications that attain high performance must be painstakingly hand-written and hand-tuned, with few common principles or paradigms uniting various implementations and easing future development. Despite the increasing importance of irregular applications, there is little programmer knowledge, and even less compiler ability, devoted to optimizing them. This project aims to solve these problems. By allowing programmers to write irregular applications in high level forms, with at most a few annotations highlighting key structural properties, programmers can focus on developing their algorithms and methods. The compiler and run-time system can take on the tedious task of optimizing the application for execution at large scales, and can automatically provide efficient implementations. This will provide portability and ease maintenance for existing irregular applications, but, more importantly, open up whole new domains of computational science to large-scale, high-performance simulation codes.

Standard iteration mechanisms, even ones of such generality as those in the C++ Standard Template Library (STL), do not allow insertions to be made in a container during a traversal of the container. Since ad hoc ways of handling such iterations are tedious to program and error-prone, programmers should ideally have at their command an efficient generic solution. In earlier work we formally characterized the complete traversal problem and presented generic solutions by means of two generic algorithms and a container adaptor, all defined within the STL framework. Here we develop additional theory for reasoning about complete traversals and address the question of how general the complete traversal problem is by showing that it subsumes well-known graph (or relation) problems such as reachability and transitive closure.

This tutorial gives an overview of the C++ standard template library (STL), including its overall structure, functionality, and capabilities. Particular emphasis is placed on the ways in which STL ...

A functional style of programming has become common in C++, following the introduction of the Standard Template Library (STL) into C++'s standard library. C++ is, however, notably lacking in its support for this style and thus cannot take full advantage of its own standard libraries. C++'s mechanisms for defining functions or objects to pass to STL algorithms are overly verbose. The effective use of modern C++ libraries calls for lambda functions in the language.This paper describes a design and implementation of built-in lambda functions for C++. C++'s compilation model, where activation records are maintained in a stack, and the lack of automatic object lifetime management, make safe lambda functions and closures challenging: if a closure outlives its scope of definition, references stored in a closure dangle. Our design is careful to balance between conciseness of syntax and explicit annotations to guarantee safety.Lambda functions can be declared without annotating their parameter types. C++0x, the forthcoming revision of standard C++, supports constrained templates and modular type checking. We describe how to infer parameter types of lambda functions from the constraints of generic functions in order to support modular type checking in the presence of lambda functions. Our design is currently under consideration for adoption to C++0x.

Automatic vectorization is crucial for improving the performance of computationally intensive programs. Existing compilers use conservative optimization strategies for automatic vectorization, which, in many cases, lead to the loss of vectorization opportunity. Studies have shown that the use of machine learning algorithms to build a performance prediction model is beneficial to improve the program performance. The model input is program features, and the output is the predicted optimization strategies or the program performance related to the optimization. In this paper, we focus on a computational intensive loop structure-tensor contraction, which is common in quantum chemical simulations. Most existing machine learning methods rely on control and data flow graphs as features to represent programs, but different tensor contraction kernels have the same control and data flow graphs. In addition, the existing methods often use the same kind of learning algorithm to construct a learning model, which is prone to overfitting and low-precision problems. In this paper, we propose an automatic vectorization performance enhancement method based on ensemble learning. We construct an ensemble learning model to predict the performance of tensor contraction kernels with different vectorization strategies and select the best vectorization strategy for the kernels. According to the storage access patterns of the tensor contraction kernels, we propose a static method for features representation. Based on the multi-algorithm ensemble learning strategy, we obtain better learning results than the single learning algorithm. The experimental results show that the prediction model achieves 88% and 87% prediction efficiency on two different platforms with different instruction sets, data types, and compilers. Compared with the existing methods, the prediction efficiency is greatly improved. In addition, the average peak performance is $2.96 \times $ of Intel ICC 12.0 and $2.98 \times $ of GCC 4.6 compiler, respectively.

3D memories are becoming viable solutions for the memory wall problem and meeting the bandwidth requirements of memory intensive applications. The high bandwidth provided by 3D memories does not translate to a proportional increase in performance for all applications. For an application such as 2D FFT with strided access patterns, the data layout of the memory has a significant impact on the total execution time of the implementation. In this paper, we present a data layout for 2D FFT on 3D memory integrated FPGA that is both on-chip memory efficient as well as throughput-optimal. Our data layout ensures that consecutive accesses to 3D memory are sufficiently interleaved among layers and vaults to absorb latency due to activation overheads for both sequential (Row FFT) and strided (Column FFT) accesses. The current state-of-the-art implementation on 3D memory requires O(√cN) on-chip memory to reduce the strided accesses and achieve maximum bandwidth for an N × N FFT problem size and c columns in a 3D memory bank row. Our proposed data layout optimizes the throughput of both the Row FFT and Column FFT phases of 2D FFT with O(N) on-chip memory for the same problem size and memory parameters without decreasing the memory bandwidth thereby achieving a √c× reduction in on-chip memory. On architectures with limited on-chip memory, our data layout achieves 2× to 4× improvement in execution time compared with the state-of-art 2D FFT implementation on 3D memory.

The Vesta parallel file system is designed to provide parallel file access to application programs running on multicomputers with parallel I/O subsystems. Vesta uses a new abstraction of files: a file is not a sequence of bytes, but rather it can be partitioned into multiple disjoint sequences that are accessed in parallel. The partitioning—which can also be changed dynamically—reduces the need for synchronization and coordination during the access. Some control over the layout of data is also provided, so the layout can be matched with the anticipated access patterns. The system is fully implemented and forms the basis for the AIX Parallel I/O File System on the IBM SP2. The implementation does not compromise scalability or parallelism. In fact, all data accesses are done directly to the I/O node that contains the requested data, without any indirection or access to shared metadata. Disk mapping and caching functions are confined to each I/O node, so there is no need to keep data coherent across nodes. Performance measurements shown good scalability with increased resources. Moreover, different access patterns are show to achieve similar performance.

A short example of a Windows® application is modified to use the generic containers, iterators, and function adapters of the C++ Standard Template Library. Comparison is made with the containers and iteration mechanism provided by the Microsoft Foundation Classes.

Leveraging the SIMD capability of modern CPU architectures is mandatory to take full advantage of their increased performance. To exploit this capability, binary executables must be vectorized, either manually by developers or automatically by a tool. For this reason, the compilation research community has developed several strategies for transforming scalar code into a vectorized implementation. However, most existing automatic vectorization techniques in modern compilers are designed for regular codes, leaving irregular applications with non-contiguous data access patterns at a disadvantage. In this article, we present a new tool, Autovesk, that automatically generates vectorized code from scalar code, specifically targeting irregular data access patterns. We describe how our method transforms a graph of scalar instructions into a vectorized one, using different heuristics to reduce the number or cost of instructions. Finally, we demonstrate the effectiveness of our approach on various computational kernels using Intel AVX-512 and ARM SVE. We compare the speedups of Autovesk vectorized code over GCC, Clang LLVM, and Intel automatic vectorization optimizations. We achieve competitive results on linear kernels and up to 11× speedups on irregular kernels.