Evaluation of architectural paradigms for addressing theprocessor-memory gap

Emerging accelerating architectures, such as GPUs, have proved successful in providing significant performance gains to various application domains. This is done by exploiting data parallelism in existing algorithms. However, programming in a data-parallel fashion imposes extra burdens to programmers, who are used to writing sequential programs. New programming models and frameworks are needed to reach a balance between programmability, portability and performance. We start from stream processing domain and propose GStream, a general-purpose, scalable data streaming framework on GPUs. The contributions of GStream are as follows: (1) We provide powerful, yet concise language abstractions suitable to describe conventional algorithms as streaming problems. (2) We project these abstractions onto GPUs to fully exploit their inherent massive data-parallelism. (3) We demonstrate the viability of streaming on accelerators. Experiments show that the proposed framework provides flexibility, programmability and performance gains for various benchmarks from a collection of domains, including but not limited to data streaming, data parallel problems, numerical codes and text search. This work lays a foundation to our future work to develop more general data parallel programming models for many-core architectures.

An associative model to minimize matching and backtracking overhead in logic programs with large knowledge bases

The fast evolution of Machine Learning (ML) models requires flexible and efficient hardware solutions as hardwired accelerators face rapid obsolescence. Vector processors are fully programmable and achieve high energy efficiencies by exploiting data parallelism, amortizing instruction fetch and decoding costs. Hence, a promising design choice is to build accelerators based on shared L1-memory clusters of streamlined Vector Processing Elements (VPEs). However, current state-of-the-art VPEs are limited in L1 memory bandwidth and achieve high efficiency only for computational kernels with high data reuse in the Vector Register File (VRF), such as General Matrix Multiplication (GEMM). Performance is sub-optimal for workloads with lower data reuse like General Matrix-Vector Multiplication (GEMV). To fully exploit available bandwidth at the L1 memory interface, the VPE micro-architecture must be optimized to achieve nearideal utilization, i.e., to be as close as possible to the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{L 1}$</tex> memory roofline (at-the-roofline). In this work, we propose TROOP, a set of hardware optimizations that include decoupled load-store interfaces, improved vector chaining, shadow buffers to hide VRF conflicts, and address scrambling techniques to achieve at-the-roofline performance for VPEs without compromising their area and energy efficiency. We implement TROOP on an open-source streamlined vector processor in a 12 nm FinFET technology. TROOP achieves significant speedups of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1. 5} \boldsymbol{\times} \boldsymbol{,} \mathbf{2. 2} \boldsymbol{\times}$</tex>, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$2.6 \times$</tex>, respectively, for key memory-intensive kernels such as GEMV, DOTP and AXPY, achieving at-the-roofline performance. Additionally, TROOP enhances the energy efficiency by up to 45%, reaching 38 DP-GFLOPs/W (1 GHz, TT, 0.8 V) for DOTP while maintaining a high energy efficiency of 61 DP-GFLOPs/W for GEMMs, incurring only a minor area overhead of less than 7%.

The processing power and parallelism in hardware is expected to increase rapidly over the next years, whereas memory bandwidth per flop and the amount of main memory per flop will be falling behind. These trends will result in both more algorithms to become limited by memory bandwidth, and overall memory requirements to become an important factor for algorithm design. In this paper we study the Gaus-Seidel stencil as an example of a memory bandwidth limited algorithm. We consider GPUs and NUMA systems, which are both designed to provide high memory bandwidth at the cost of making algorithm design more complex. The mapping of the non-linear memory access pattern of the Gaus-Seidel stencil to the different hardware is important to achieve high performance. We show that there is a trade-off between overall performance and memory requirements when optimizing for optimal memory access pattern. Vectorizing on the NUMA system and optimizing to utilize all processors on the GPU does not pay off in terms of performance per memory used, which we consider an important measurement regarding the trends named before.

In the world of Performance Computing (newly renamed High Productivity Computing), where the race for performance is on, where the hunt for the last Flop rages and where the developers swear by the Cult of Power, the different applications have different needs. Some of them are single threaded, others are parallel, some are memory bandwidth, others processor speed dominated. Some are embarrassingly parallel, point-to-point or multicast dominated, all depending on the algorithms and numerical methods they implement. Computing resources are also different, some are low-cost clusters of PlayStations 3, clusters with more or less efficient networks, others are expensive vector supercomputers. To express that diversity, a parameterization of the applications and the resources is proposed. Based on this parameterization, it is possible to quantify the needs of the different types of applications in terms of memory, processor and network requirements. On the other hand, the parameterization of the resources quantifies what a resource in term of memory, processor or network performance. Comparing the needs of the application and the offers of the resources, we present a model that predicts the computation and the communication times, and consequently the execution times of an application on different resources. We define a cost model that finds the best matches between an application and the available resources in an HPC Grid. It takes into account the cost of execution time of the application (based on the prediction previously mentioned), the waiting time cost (based on the information given by the local resource management systems), the cost of the licenses of the different software the application may use, the cost of the data transfer, and the ecological cost. The choice is made on a QOS (Quality of Service) given by the user, such as minimum turnaround time or minimum cost. Computing these different costs requires a fine knowledge of the behavior of the application on the resource. Thus we present a detailed application-oriented monitoring that consists in mapping information from the hardware monitoring and the different middleware the application (or the resource) uses, such as MPI detailed information, the hardware counters or the information of the execution on a local resource management system. The monitored data is stored in an archive system. The archived monitoring data concerning the behavior of the applications on the different resources, can then be used to help deciders purchasing new best suited resources for the current (and future) applications. In the second part of this work, we go further in matching the applications and resources: if it is possible to predict the behavior of an application on a given resource, it is also possible to fit the resource to the needs of the application. Knowing the processor performance needs of the application during its execution, it is possible to adjust its frequency through the frequency stepping mechanism that authorizes an operating system kernel or a program to modify on the fly the frequency (and the voltage) of the present days processors. Hence the resource exactly fits the needs of the application and the energy consumption of the processor is reduced. Finally, we propose a methodology to determine poorly implemented applications and thus giving users and developers hints and advice to detect flaws in an automatic manner. We present a real-life example, a spectral element code in CFD, for which the complexity changed when more than 1024 processors were used.

SIMD engines are widely present in market processors aiming to improve performance of applications through Data Level Parallelism (DLP) exploitation. However, most SIMD engines rely on specific libraries and compilers to support DLP execution, which limits DLP gains since they are restricted to analyze static code. Dynamic SIMD Assembler (DSA) [8] is capable of exploiting DLP at runtime by identifying vectorizable loops to generate ARM NEON SIMD instructions. However, its DLP coverage capability is not fully exploited, since portion of code that depends on runtime information, such as dynamic range and conditional code loops are not exploited. In this work, we extend the DSA coverage by coupling the exploitation of conditional code and dynamic range loop vectorization. Results show that the proposed techniques improve the original DSA performance in 38% considering benchmarks with opportunities to exploit conditional code and dynamic range loops. In addition, the Extended DSA, besides keeping software productivity and binary compatibility, outperforms ARM compiler auto-vectorization by 12%.

We study the problem of contention for memory bandwidth between computation and communication in supercomputers that feature multicore CPUs. The problem arises when communication and computation are overlapped, and both operations compete for the same memory bandwidth. This contention is most visible at the limits of scalability, when communication and computation take similar amounts of time, and thus must be taken into account in order to reach maximum scalability in memory bandwidth bound applications. Typical examples of codes affected by the memory bandwidth contention problem are sparse matrix-vector computations, graph algorithms, and many machine learning problems, as they typically exhibit a high demand for both memory bandwidth and inter-node communication, while performing a relatively low number of arithmetic operations. The problem is even more relevant in truly heterogeneous computations where CPUs and accelerators are used in concert. In that case it can lead to mispredictions of expected performance and consequently to suboptimal load balancing between CPU and accelerator, which in turn can lead to idling of powerful accelerators and thus to a large decrease in performance. We propose a simple benchmark in order to quantify the loss of performance due to memory bandwidth contention. Based on that, we derive a theoretical model to determine the impact of the phenomenon on parallel memory-bound applications. We test the model on scientific computations, discuss the practical relevance of the problem and suggest possible techniques to remedy it.

Mining large data sets is no longer the prerogative of computer scientists - specialists in a wide variety of domains are performing analytics as a day-to-day activity. Often such analyses are specific to the domain and analysts are required to devise new algorithms or techniques. For such scenarios, providing a high-level programming environment that delivers high performance on clusters is a challenge. We propose a framework that supports high-level programming using domain abstractions in data mining while delivering scalable performance on commodity clusters i.e. clusters of multi-core workstations. This framework includes a domain specific programming language, DWARF, to enable data mining specialists to rapidly prototype algorithms. DWARF is supported by a compiler that automatically parallelizes code by identifying domain specific patterns and translating them to parallel code that exploits data parallelism and task parallelism. The compiler generates code for a hybrid virtual machine supporting distributed memory model at the top level and shared memory model nested within. The code generated by the compiler can be scheduled on commodity clusters. We compare the proposed framework with other frameworks commonly used for data mining on distributed platforms.

Distributed big data in scientific computing presents a major I/O performance bottleneck when exploiting data paral-lelism. Consumer and producer compute nodes are often throttled by saturated data channels when processing large numerical data. We describe ZHW, a hardware implementation of the ZFP numerical CODEC that can greatly reduce I/O pressure caused by large scientific datasets. Our ZHW design overcomes barriers that have prevented prior ZFP-like hardware accelerators from obtaining maximum compression in their implementations. The SystemC ZHW hardware library is available in an open source public repository. We demonstrate the practicality of ZHW by synthesizing our CODEC on an Ultrascale+ FPGA and analyzing performance.

An Approach to Correctness of Data Parallel Algorithms

This document presents key principles and practical considerations for optimizing CPU-bound code in the context of high-performance scientific computing. While classical algorithmic complexity provides a hardware-independent framework for evaluating scalability, real-world performance is largely determined by low-level factors such as memory access patterns, cache behavior, data locality, and compiler optimizations. The discussion highlights how modern architectures rely on hierarchical memory systems, making data movement often more critical than computation. Particular attention is given to issues such as aliasing, false sharing, and cache conflicts, which can significantly impact performance despite unchanged algorithms. The role of the compiler is emphasized as central in exploiting hardware capabilities, provided that code is well-structured and free from undefined behavior. Finally, the document underlines the limitations of general optimization rules and stresses the importance of empirical testing, as performance depends strongly on hardware characteristics and data layout.

The wide use of accelerators such as GPUs necessities their security support Recent works [17], [33], [34] pointed out that directly adopting the CPU secure memory design to GPUs could incur significant performance overheads due to the memory bandwidth contention between regular data and security metadata. In this paper, we analyze the security guarantees that used to defend against physical attacks, and make the observation that heterogeneous GPU memory system may not always need all the security mechanisms to achieve the security guarantees. Based on the memory types as well as memory access patterns either explicitly specified in the GPU programming model or implicitly detected at run time, we propose adaptive security memory support for heterogeneous memory on GPUs. Specifically, we first identify the read-only data and propose to only use MAC (Message Authentication Code) to protect their integrity. By eliminating the freshness checks on read-only data, we can use an on-chip shared counter for such data regions and remove the corresponding parts in the Bonsai Merkel Tree (BMT), thereby reducing the traffic due to encryption counters and the BMT. Second, we detect the common streaming data access pattern and propose coarse- grain MACs for such stream data to reduce the MAC access bandwidth. With the hardware-based detection of memory type (read-only or not) and memory access patterns (streaming or not), our proposed approach adapts the security support to significantly reduce the performance overhead without sacrificing the security guarantees. Our evaluation shows that our scheme can achieve secure memory on GPUs with low overheads for memory-intensive workloads. Among the fifteen memory-intensive workloads in our evaluation, our design reduces the performance overheads of secure GPU memory from 53.9% to 8.09% on average. Compared to the state-of- the-art secure memory designs for GPU [17], [33], our scheme outperforms PSSM by up to 41.63% and 9.5% on average and outperforms Common counters by 84.04% on average for memory-intensive workloads. We further propose to use the L2 cache as a victim cache for security metadata when the L2 is either underutilized or suffers from very high miss rates, which further reduces the overheads by up to 4% and 0.65% on average.

Face recognition is an algorithm that is capable of identifying or verifying a query face from multiple faces in the enrollment database. It poses a challenging problem in the field of image analysis and computer vision, especially for applications that deal with video sequences, face re-identification, or operate on intensity images and require fast processing. In this work, we introduce a high speed face recognition technique along with a high speed FPGA implementation. It uses a new similarity measure to estimate the distance between the query face and each of the database face images. The distance metric is the sum of the standard deviations between multiple histograms, which are calculated from each row of the query and database images. The lowest distance score refers to the database face that matches the query. The proposed technique is independent from the ambient illumination and outperforms the well-known face recognition algorithm “Eigenfaces” (it performs the face recognition 16× faster when both algorithms run on the same platform). Furthermore, we exploit data parallelism in our proposed algorithm to design a hardware accelerator and to implement it on an FPGA prototyping board. The results show 10x execution time improvement in comparison to the software version.

This paper describes and evaluates three architectural methods for accomplishing data parallel computation in a programmable embedded system. Comparisons are made between the well-studied Very Long Instruction Word (VLIW) and Single Instruction Multiple Packed Data (SIMpD) paradigms; the less-common Single Instruction Multiple Disjoint Data (SIMdD) architecture is described and evaluated. A taxonomy is defined for data-level parallel architectures, and patterns of data access for parallel computation are studied, with measurements presented for over 40 essential telecommunication and media kernels. While some algorithms exhibit data-level parallelism suited to packed vector computation, it is shown that other kernels are most efficiently scheduled with more flexible vector models. This motivates exploration of non-traditional processor architectures for the embedded domain.

The MMXTM technology and SSE/SSE2 (streaming-SIMD-extensions) introduced a variety of SIMD instructions that can exploit data parallelism in numerical and multimedia applications. In particular, new saturation and clipping instructions can boost the performance of applications that make extensive use of such operations. Unfortunately, due to the lack of support for saturation and clipping operators in e.g. C/C++ or Fortran, these operations must be explicitly coded with conditional constructs that test the value of operands before actual wrap-around arithmetic is performed. As a result, inlineassembly or language extensions are most commonly used to exploit the new instructions. In this paper, we explore an alternative approach, where the compiler automatically maps high-level saturation and clipping idioms onto efficient low-level instructions. The effectiveness of this approach is demonstrated with some experiments.