Optimal Loop Tiling for Minimizing Write Operations on NVMs with Complete Memory Latency Hiding

Tuning cache hierarchies in platforms for embedded systems can significantly reduce energy consumption. In this paper we combined two optimization methods for tuning both instruction and data cache configurations in a two-level memory hierarchy, where both levels have separate instruction and data caches. This kind of hierarchy allows us to evaluate instruction and data caches branches separately, although previous approaches have applied the same method for both branches of the hierarchy. This work evaluates several methods intended for two-level hierarchies, and the results showed that when we combine different methods for each branch of the hierarchy, results can be improved. Experiments based on simulations were performed for 12 applications from the Mibench suite benchmark and the combined method achieved better efficiency in 60% of the evaluated cases compared with existing heuristics. The proposed solution is only 11% less economic in terms of energy consumption than optimal values and required, on average, 42 simulations to conclude optimization mechanism, representing only 9% of the design space.

Modern computing systems are embracing hybrid memory comprising of DRAM and non-volatile memory (NVM) to combine the best properties of both memory technologies, achieving low latency, high reliability, and high density. A prominent characteristic of DRAM-NVM hybrid memory is that it has NVM access latency much higher than DRAM access latency. We call this inter-memory asymmetry. We observe that parasitic components on a long bitline are a major source of high latency in both DRAM and NVM, and a significant factor contributing to high-voltage operations in NVM, which impact their reliability. We propose an architectural change, where each long bitline in DRAM and NVM is split into two segments by an isolation transistor. One segment can be accessed with lower latency and operating voltage than the other. By introducing tiers, we enable non-uniform accesses within each memory type (which we call intra-memory asymmetry), leading to performance and reliability trade-offs in DRAM-NVM hybrid memory. We extend existing NVM-DRAM OS in three ways. First, we exploit both inter- and intra-memory asymmetries to allocate and migrate memory pages between the tiers in DRAM and NVM. Second, we improve the OS's page allocation decisions by predicting the access intensity of a newly-referenced memory page in a program and placing it to a matching tier during its initial allocation. This minimizes page migrations during program execution, lowering the performance overhead. Third, we propose a solution to migrate pages between the tiers of the same memory without transferring data over the memory channel, minimizing channel occupancy and improving performance. Our overall approach, which we call MNEME, to enable and exploit asymmetries in DRAM-NVM hybrid tiered memory improves both performance and reliability for both single-core and multi-programmed workloads.

Nonvolatile memory (NVM) technologies are promising candidates as the next-generation main memory due to high scalability and low energy consumption. However, the performance bottlenecks, such as high write latency and low cell endurance, still exist in NVMs. To address these problems, frequent pattern compression schemes have been widely used, which however suffer from the lack of flexibility and adaptability. In order to overcome these shortcomings, we propose a well-adaptive NVM write scheme, called dynamic frequent pattern compression (DFPC), to significantly reduce the amount of write units and extend the lifetime. Instead of only using static frequent patterns in existing FPC schemes, which are predefined and not always efficient for all applications, the idea behind DFPC is to exploit the characteristics of data distribution in execution to obtain dynamic patterns, which often appear in the real-world applications. To further improve the compression ratio, we exploit the value locality in a cache line to extend the granularity of dynamic patterns. Hence, DFPC can encode the contents of cache lines with more kinds of frequent data patterns. Moreover, to further support efficient write and read operations in the context of multilevel/triple-level cell NVMs, we need to extend the DFPC to improve performance in terms of the access latency and energy consumption. We hence propose a latency-optimized and energy-efficient compression write scheme to encode the compressed data with low energy and latency states, i.e., enhanced DFPC, thus reducing the latency and energy consumption. We implement DFPC in GEM5 with NVMain and execute the applications from SPEC CPU2006 to evaluate our scheme. Experimental results demonstrate the efficacy and efficiency of DFPC. We have released the source codes for public use at Github https://github.com/dfpcscheme/DFPCScheme .

Checkpoint/recovery has been widely used in many high available and reliable systems. This paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> , an application-transparent and in-memory checkpointing mechanism based on hybrid memory system composed of DRAM and emerging Non-volatile Memory (NVM). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> adopts a pre-copy based checkpointing mechanism to reduce the system downtime. It supports fine-grained and incremental checkpointing at frequencies up to 100 times per second. Under this context, the checkpointing can significantly degrade application performance due to memory contention between applications and the checkpointing process. Previous checkpointing mechanisms on hybrid memory systems have focused on the performance of checkpointing, and have overlooked the impact of memory contention on the application performance. In this paper, we mitigate the memory contention at the bank level by carefully scheduling memory requests to fully leverage the idle time slots of different memory banks. Moreover, if bank conflicts are unavoidable, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> promotes the priority of applications’ memory requests to lessen their access latencies. By redesigning the memory controllers of DRAM and NVM, we implement a hardware-assisted checkpointing mechanism that can directly transfer data from working memory to the checkpoint in NVM, without any intervention of CPUs. Our evaluation shows that Shadow can reduce memory bank conflicts between applications and checkpointing by 75 percent, and decrease applications’ memory read request latency by 28 percent on average compared to the pre-copy based checkpointing. Moreover, Shadow can also reduce checkpointing overhead by 42 and 16 percent on average compared to the stop-and-copy and pre-copy based checkpointing approaches, respectively.

The emergency of non-volatile memory (NVM) can effectively address the performance gap between computing and storage due to its superior performance characteristics, including byte-addressability, low access latency and power consumption. However, NVM currently has high price and relative small capacity due to the process technology. In order to use the NVM in a cost-effective way, it is essential to improve the space utilization of NVM-based storage system. But, traditional file systems are not suitable for NVM as they are designed for block devices which incurs a large number of internal fragmentation and reduces the space utilization of storage system. On the other hand, the existing NVM-based file systems mostly focus on improving the system performance, failing to take full advantage of the byte-addressability of NVM. In this paper, we propose a Last Page Sharing (LPS) mechanism to improve the space utilization of NVM-based storage system. LPS utilizes the byte-addressability of NVM, and stores multiple small files or the tail data of files in the shared page together. We implement the Last Page Sharing scheme in an existing NVM-based file system and conduct several experiments to compare with other file systems to evaluate its efficiency. Experimental results show that, our approach reduces the space as much as 20% compared with Ext3, Ext4 and PMFS on average.

Non-Volatile Memory (NVM) technologies exhibit 4X the read access latency of conventional DRAM. When the working set does not fit in the processor cache, this latency gap between DRAM and NVM leads to more than 2X runtime increase for queries dominated by latency-bound operations such as index joins and tuple reconstruction. We explain how to easily hide NVM latency by interleaving the execution of parallel work in index joins and tuple reconstruction using coroutines. Our evaluation shows that interleaving applied to the non-trivial implementations of these two operations in a production-grade codebase accelerates end-to-end query runtimes on both NVM and DRAM by up to 1.7X and 2.6X respectively, thereby reducing the performance difference between DRAM and NVM by more than 60%.

In the development process of multi-core processors, the Network on Chip optimizes the storage structure, but the problem of “storage wall” still exists. There are many ways to alleviate this problem. The improvement of memory access performance can be obtained for appropriate address mapping between cache levels. As far as the performance index of address mapping is concerned, it can be considered from the perspectives of memory access bandwidth and latency in different levels of the storage system. Therefore, a module-level detection tool is designed to test the memory access performance. Based on the hardware emulation acceleration platform, this tool solves the online/offline data collection of memory access bandwidth and latency. And it implements the functions for extremely long simulation times, which is data capture, analysis, and processing. Under different address mapping and data types of different application scenarios, This tool tests memory access performance between cache modules. Based on the data-affinity address cross-mapping method, this thesis proposes an address mapping optimization scheme that binds the directory controller unit and the local memory. Through the module-level memory access performance test, the optimized data affinity and low-order address cross-mapping method increases the memory access bandwidth by 8% and the maximum bandwidth by 9% when processing scheduling-intensive data. In a word, this thesis establishes the memory access performance analysis method which is “benchmark test program, module-level memory access performance detection, address mapping optimization and configuration”.

Non volatile memory (NVM) is expected to enrich the next generation computer system. However, designers have difficulties in exploring new software and hardware design ideas based on NVM due to the limitations of current simulation-based evaluation, e.g., slow runtime. In order to resolve this problem, we present an open, reliable, and versatile hardware platform for NVM emulation. We built Zynq FPGA-based boards where software programs can run on the emulated NVM having parameterized read/write latency and non-volatility. They also support Linux-compatible OSs, Debian, Petalinux, and Ubuntu GUI. We report a case study where machine learning algorithms are evaluated while varying memory latency on the hardware platform.

Cache coherent non-uniform memory access (CC-NUMA) multiprocessors continue to suffer from remote memory access latencies due to comparatively slow memory technology and data transfer latencies in the interconnection network. We propose a novel hardware caching technique, called switch cache. The main idea is to implement small fast caches in crossbar switches of the interconnect medium to capture and store shared data as they flow from the memory module to the requesting processor. This stored data acts as a cache for subsequent requests, thus reducing the latency of remote memory accesses tremendously. The implementation of a cache in a crossbar switch needs to be efficient and robust, yet flexible for changes in the caching protocol. The design and implementation details of a CAche Embedded Switch ARchitecture, CAESAR, using wormhole routing with virtual channels is presented. Using detailed execution-driven simulations, we find that the CAESAR switch cache is capable of improving the performance of CC-NUMA multiprocessors by reducing the number of reads served at distant remote memories by up to 45% and improving the application execution time by as high as 20%. We conclude that the switch caches provide a cost-effective solution for designing high performance CC-NUMA multiprocessors.

Using nonvolatile memories in memory hierarchy has been investigated to reduce its energy consumption because nonvolatile memories consume zero leakage power in memory cells. One of the difficulties is, however, that the endurance of most nonvolatile memory technologies is much shorter than the conventional SRAM and DRAM technology. This has limited its usage to only the low levels of a memory hierarchy, e.g., disks, that is far from the CPU.

Towards new memory paradigms : integrating non-volatile main memory and remote direct memory access in modern systems

NVM (Non-Volatile Memory) that gains attention as an alternative memory technology these days has different memory characteristics in terms of latency and bandwidth, compared with the traditional DRAM. In this paper, we analyze how these different characteristics affect the performance of applications. For this purpose, we design a new memory analysis framework that can control the memory latency and bandwidth through hardware and software techniques in commercial computer systems. The framework allows an application to be executed as it is without any modification, leading to fast analysis under diverse configurations. In addition, we devise a novel memory performance model that investigates performance with the viewpoint of LLC (Last Level Cache) and MLP (Memory Level Parallelism). Real implementation based experiments show that our model analyzes performance changes accurately when we compare with the results measured by the proposed framework

With the rapidly growing demands for large capacity main memory in server systems and embedded systems, current DRAM-only approach is hitting the limit due to DRAM's capacity scaling issue and significant background power. With the emergence of new non-volatile memories, or storage-class memories (SCMs), we can now explore low power, high capacity memory subsystem by redesigning virtual memory system to be SCM-aware. Most research on virtual memory system design has focused on minimizing page fault frequency due to slow data transfers using storage such as HDD/SSD as virtual memory swap device. However with an SCM-based swap device, its near-DRAM access latency has potential for reducing requisite DRAM size by aggressively evicting pages from DRAM to SCM without sacrificing performance, and thus reducing background power by powering off the freed DRAM space for low power. To select an optimal SCM from among the many candidate SCM technologies, the impact of SCM characteristics was evaluated using full-system simulation. Results show that utilizing SCM with low access latency and low write energy can lead to significant potential reduction of memory subsystem energy by up to 83%, while maintaining performance degradation within acceptable range.

GPUs are being widely used to accelerate different workloads and multi-GPU systems can provide higher performance with multiple discrete GPUs interconnected together. However, there are two main communication bottlenecks in multi-GPU systems -- accessing remote GPU memory and the communication between GPU and the host CPU. Recent advances in multi-GPU programming, including unified virtual addressing and unified memory from NVIDIA, has made programming simpler but the costly remote memory access still makes multi-GPU programming difficult. In order to overcome the communication limitations, we propose to leverage the memory network based on hybrid memory cubes (HMCs) to simplify multi-GPU memory management and improve programmability. In particular, we propose scalable kernel execution (SKE) where multiple GPUs are viewed as a single virtual GPU as a single kernel can be executed across multiple GPUs without modifying the source code. To fully enable the benefits of SKE, we explore alternative memory network designs in a multi-GPU system. We propose a GPU memory network (GMN) to simplify data sharing between the discrete GPUs while a CPU memory network (CMN) is used to simplify data communication between the host CPU and the discrete GPUs. These two types of networks can be combined to create a unified memory network (UMN) where the communication bottleneck in multi-GPU can be significantly minimized as both the CPU and GPU share the memory network. We evaluate alternative network designs and propose a sliced flattened butterfly topology for the memory network that scales better than previously proposed alternative topologies by removing local HMC channels. In addition, we propose an overlay network organization for unified memory network to minimize the latency for CPU access while providing high bandwidth for the GPUs. We evaluate trade-offs between the different memory network organization and show how UMN significantly reduces the communication bottleneck in multi-GPU systems.

CC-NUMA is a widely adopted and deployed architecture of high performance computers. These machines are attractive for their transparent access to local and remote memory. However, the prohibitive latency gap between local and remote access deteriorates applications' performance seriously due to memory access stalls. File system cache, especially, being shared by all processes, inevitably triggers many remote accesses. To address this problem, we suggest and implement a mechanism that uses local memory to cache remote file cache, of which the main purpose is to improve data locality. Using realistic workload on a two-node cc-NUMA machine, we show that the cost of such a mechanism is as low as 0.5%, the performance can be increased 14.3% at most, and the local hit ratio can be improved as much as 40%.