Enhancing Performance Optimization of Multicore/Multichip Nodes with Data Structure Metrics

The efficient use of the cache hierarchy of an execution platform often has a major impact on the performance of an application. It is often difficult for the application programmer or the compiler to determine a suitable memory layout for the application data, since the interactions between the memory accesses cannot be fully anticipated before the program execution. This paper introduces an approach to improve the cache efficiency by dynamically padding memory allocations by a post-compilation tool. Data structures like histograms are used to evaluate cache simulations of memory traces of the application considered and to compute optimized pad sets. These can then be used for later runs of the application with different data sets. The accumulated representation of the references' memory accesses additionally offers a visualization interface to the algorithm-specific memory access pattern of each reference captured. As implied above, the advantage of the method is that it also allows for an improvement of the cache usage of binary only applications for which no source code is available. Post optimization cache behavior analyses as well as run-time measurements show that the cache hit rates of the runtime-modified applications are considerably increased by applying the generated pad set.

The NAS Parallel Benchmarks for evaluating C++ parallel programming frameworks on shared-memory architectures

Modern processors rely on advanced memory management techniques such as predicting and prefetching to bridge the processor–memory gap. However, current mechanisms are primarily designed for a specific memory hierarchy level or access pattern, resulting in inefficiencies for diverse workloads with mixed memory access patterns. To address this challenge, this paper proposes DHCM, a Dynamic Hierarchy Coordination Mechanism that intelligently schedules prediction hierarchies and dynamically optimizes memory access processes to enhance system performance. DHCM integrates a hierarchy coordination mechanism driven by a State Trigger. This mechanism dynamically leverages system feedback to prioritize and coordinate memory operations, enabling the simultaneous management of both off-chip load requests and on-chip cache accesses. Through extensive evaluations on the ChampSim simulator, DHCM demonstrates its adaptability and efficiency with an average IPC improvement of 34.08% and 24.09% on single-core and multi-core systems, respectively. Additionally, DHCM contributes a 64.17% miss coverage and 89.33% DRAM-loads reduction.

The discrete wavelet transform with 5/3 filter of the JPEG2000 shows different memory access patterns according to the levels of decomposition. To make computation faster by hardware implementation, hiding memory access latency is important to improve a performance. The semi-programmable hardware (SPWH) as an intermediate hardware provides an efficient design method of a hardware with data prefetching to hide the memory access latency across the different memory access patterns. This paper describes the SPHW and demonstrates a mapping method for the DWT. The experimental result shows that the SPHW can significantly reduce a design burden and achieve a good performance.

Performance of applications running on GPUs is mainly affected by hardware occupancy and global memory latency. Scientific applications that rely on analysis using unstructured grids could benefit from the high performance capabilities provided by GPUs, however, its memory access pattern and algorithm limit the potential benefits.In this paper we analyze the algorithm for unstructured grid analysis on the basis of hardware occupancy and memory access efficiency. In general, the algorithm can be divided into three stages: cell-oriented analysis, edge-oriented analysis and information update, which present different memory access patterns. Based on the analysis we modify the algorithm to make it suitable for GPUs. The proposed algorithm aims for high hardware occupancy and efficient global memory access. Finally, through implementation we show that our design achieves up to 88 times speedup compared to the sequential CPU version.

With the rise of non-volatile memory (NVM) as a replacement for traditional main memories (e.g. DRAM), memory access analysis is becoming an increasingly important topic. NVMs suffer from technical shortcomings as such as reduced cell endurance which call for precise memory access analysis in order to design maintenance strategies that can extend the memory's lifetime. While existing memory access analyzers trace memory accesses at various levels, from the application level with code instrumentation, down to the hardware level where software is executed on special analysis hardware, they usually interpret main memory as a consecutive area, without investigating the application semantics of different memory regions. In contrast, this paper presents a memory access simulator, which splits the main memory into semantic regions and enriches the simulation result with semantics from the analyzed application. We leverage a library-based operating system called Unikraft by ascribing memory regions of the simulation to the relevant OS libraries. This novel approach allows us to derive a detailed analysis of which libraries (and thus functionalities) are responsible for which memory access patterns. Through offline profiling with our simulator, we provide a fine-granularity analysis of memory access patterns that provide insights for the design of efficient NVM maintenance strategies.

In the past decade, advances in the speed of commodity CPUs have far out-paced advances in memory latency. Main-memory access is therefore increasingly a performance bottleneck for many computer applications, including database systems. In this article, we use a simple scan test to show the severe impact of this bottleneck. The insights gained are translated into guidelines for database architecture, in terms of both data structures and algorithms. We discuss how vertically fragmented data structures optimize cache performance on sequential data access. We then focus on equi-join, typically a random-access operation, and introduce radix algorithms for partitioned hash-join. The performance of these algorithms is quantified using a detailed analytical model that incorporates memory access cost. Experiments that validate this model were performed on the Monet database system. We obtained exact statistics on events such as TLB misses and L1 and L2 cache misses by using hardware performance counters found in modern CPUs. Using our cost model, we show how the carefully tuned memory access pattern of our radix algorithms makes them perform well, which is confirmed by experimental results.

PatternS: An intelligent hybrid memory scheduler driven by page pattern recognition

The deployment of multi-core platforms in safety-critical avionic applications is hampered by the lack of means to ensure predictability when processes running on different cores can create interference effects, affecting worst-case execution time, due to shared memory accesses. One way to restrict these interferences is to allocate a budget for different processes prior to run-time and to monitor the adherence to this budget during run-time. While earlier works in adopting this approach seem promising, they focus on application level (user mode) accesses to shared memory and not the operating system accesses. In this paper we construct experiments for studying a multi-core platform running an ARINC 653 compliant operating system, and measure the impact of both application processes and operating system (supervisor mode) activities. In particular, as opposed to earlier works that considered networking applications, we select four avionic processes that exhibit different memory access patterns, namely, a navigation process, a matrix multiplication process, a math library process and an image processing one. The benchmarking on a set of avionic-relevant application processes shows that (a) the potential interference by the operating system cannot be neglected when allocating budgets that are to be monitored at run-time, and (b) the bounds for the allowed number of memory accesses should not always be based on the maximum measured count during profiling, which would lead to overly pessimistic budgets.

Tools to identify problems and improve MPI applications performance running on an HPC system require information from the application and the system. In this work we will focus on defining a methodology to analyze how memory usage affects an MPI application performance running on this HPC system. This methodology will obtain valid and comparable data based on different memory access patterns that will allow defining key performance values used to characterize the HPC system behavior in front of these access patterns and to characterize the Application Signature behavior, (PAS2P Signature that obtains the representative phases of the MPI application) in front of these same access patterns. With these key performance values, we will be able to detect memory access application problems, suggest improvements and define a mapping policy for this application in this HPC system to improve its performance and to determine limits to these improvements.

Leveraging large data sets, deep Convolutional Neural Networks (CNNs) achieve state-of-the-art recognition accuracy. Due to the substantial compute and memory operations, however, they require significant execution time. The massive parallel computing capability of GPUs make them as one of the ideal platforms to accelerate CNNs and a number of GPU-based CNN libraries have been developed. While existing works mainly focus on the computational efficiency of CNNs, the memory efficiency of CNNs have been largely overlooked. Yet CNNs have intricate data structures and their memory behavior can have significant impact on the performance. In this work, we study the memory efficiency of various CNN layers and reveal the performance implication from both data layouts and memory access patterns. Experiments show the universal effect of our proposed optimizations on both single layers and various networks, with up to 27.9× for a single layer and up to 5.6× on the whole networks.

The distributed shared memory (DSM) architecture is considered to be one of the most likely parallel computing environment candidate for the near future because of its ease of system scalability and facilitation for parallel programming. However, a naive program based on shared memory execution on a DSM machine often deteriorates performance, because of the overhead involved for maintaining cache coherency particularly with frequent remote memory accesses. We show that careful buffer management of parallel join processing on DSM can produce considerable performance improvements in comparison with a naive implementation. We propose four buffer management strategies for parallel hash join processing on the DSM architecture and actually implement them on the HP Exemplar SPP 1600. The basic strategy is to begin with the hash join algorithm for the shared everything architecture and then to consider the memory locality of DSM by distributing the hash table and data pool buffers among the nodes. The results of four buffering strategies are analyzed in detail. Consequently, we can conclude that, in order to achieve high performance on a DSM machine, our buffer management strategy in which the memory access pattern is extracted and buffers are allocated in the local memory of nodes to minimize memory access cost is very efficient.

Genetic algorithm based estimation of non–functional properties for GPGPU programs

The performance of data-intensive applications, when running on modern multi- and many-core processors, is largely determined by their memory access behavior. Its most important contributors are the frequency and latency of off-chip accesses and the extent to which long-latency memory accesses can be overlapped with useful computation or with each other. In this paper we present two methods to better understand application and microarchitectural interactions. An <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">epoch profile</i> is an intuitive way to understand the relationships between three important characteristics: theon-chip cache size, the size of the reorder window of an out-of-order processor, and the frequency of processor stalls caused by long-latency,off-chip requests (epochs). By relating these three quantities one can more easily understand an application’s memory reference behavior and thus significantly reduce the design space. While epoch profiles help to provide insight into the behavior of a single application, developing an understanding of a number of applications in the presence of area and core count constraints presents additional challenges. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Epoch-based microarchitectural analysis</i> is presented as a better way to understand the trade-offs for memory-bound applications in the presence of these physical constraints. Through epoch profiling and optimization, one can significantly reduce the multidimensional design space for hardware/software optimization through the use of high-level model-driven techniques.

Applications executing on multicore processors can now easily offload computations to many core processors, such as Intel Xeon Phi coprocessors. However, it requires high levels of expertise and effort to tune such offloaded applications to realize high-performance execution. Previous efforts have focused on optimizing the execution of offloaded computations on many core processors. However, we observe that the data transfer overhead between multicore and many core processors, and the limited device memories of many core processors often constrain the performance gains that are possible by offloading computations. In this paper, we present three source-to-source compiler optimizations that can significantly improve the performance of applications that offload computations to many core processors. The first optimization automatically transforms offloaded codes to enable data streaming, which overlaps data transfer between multicore and many core processors with computations on these processors to hide data transfer overhead. This optimization is also designed to minimize the memory usage on many core processors, while achieving the optimal performance. The second compiler optimization re-orders computations to regularize irregular memory accesses. It enables data streaming and factorization on many core processors, even when the memory access patterns in the original source codes are irregular. Finally, our new shared memory mechanism provides efficient support for transferring large pointer-based data structures between hosts and many core processors. Our evaluation shows that the proposed compiler optimizations benefit 9 out of 12 benchmarks. Compared with simply offloading the original parallel implementations of these benchmarks, we can achieve 1.16x-52.21x speedups.