Straggler-Aware Parallel Graph Processing in Hybrid Memory Systems

While the memory footprints of cloud and HPC applications continue to increase, fundamental issues with DRAM scaling are likely to prevent traditional main memory systems, composed of monolithic DRAM, from greatly growing in capacity. Hybrid memory systems can mitigate the scaling limitations of monolithic DRAM by pairing together multiple memory technologies (e.g., different types of DRAM, or DRAM and non-volatile memory) at the same level of the memory hierarchy. The goal of a hybrid main memory is to combine the different advantages of the multiple memory types in a cost-effective manner while avoiding the disadvantages of each technology. Memory pages are placed in and migrated between the different memories within a hybrid memory system, based on the properties of each page. It is important to make intelligent page management (i.e., placement and migration) decisions, as they can significantly affect system performance.In this paper, we propose utility-based hybrid memory management (UH-MEM), a new page management mechanism for various hybrid memories, that systematically estimates the utility (i.e., the system performance benefit) of migrating a page between different memory types, and uses this information to guide data placement. UH-MEM operates in two steps. First, it estimates how much a single application would benefit from migrating one of its pages to a different type of memory, by comprehensively considering access frequency, row buffer locality, and memory-level parallelism. Second, it translates the estimated benefit of a single application to an estimate of the overall system performance benefit from such a migration.We evaluate the effectiveness of UH-MEM with various types of hybrid memories, and show that it significantly improves system performance on each of these hybrid memories. For a memory system with DRAM and non-volatile memory, UH-MEM improves performance by 14% on average (and up to 26%) compared to the best of three evaluated state-of-the-art mechanisms across a large number of data-intensive workloads.

Checkpoint/recovery has been widely used in many high available and reliable systems. This paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> , an application-transparent and in-memory checkpointing mechanism based on hybrid memory system composed of DRAM and emerging Non-volatile Memory (NVM). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> adopts a pre-copy based checkpointing mechanism to reduce the system downtime. It supports fine-grained and incremental checkpointing at frequencies up to 100 times per second. Under this context, the checkpointing can significantly degrade application performance due to memory contention between applications and the checkpointing process. Previous checkpointing mechanisms on hybrid memory systems have focused on the performance of checkpointing, and have overlooked the impact of memory contention on the application performance. In this paper, we mitigate the memory contention at the bank level by carefully scheduling memory requests to fully leverage the idle time slots of different memory banks. Moreover, if bank conflicts are unavoidable, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Shadow</i> promotes the priority of applications’ memory requests to lessen their access latencies. By redesigning the memory controllers of DRAM and NVM, we implement a hardware-assisted checkpointing mechanism that can directly transfer data from working memory to the checkpoint in NVM, without any intervention of CPUs. Our evaluation shows that Shadow can reduce memory bank conflicts between applications and checkpointing by 75 percent, and decrease applications’ memory read request latency by 28 percent on average compared to the pre-copy based checkpointing. Moreover, Shadow can also reduce checkpointing overhead by 42 and 16 percent on average compared to the stop-and-copy and pre-copy based checkpointing approaches, respectively.

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.

PatternS: An intelligent hybrid memory scheduler driven by page pattern recognition

Hybrid memory composed of DRAM and persistent memory (PM) offers a promising way to realize large-capacity main memory supporting in-memory data storage and computing. However, traditional buffer management schemes focus on improving the hit ratio but lack awareness of the limitations of PM, e.g., slower write time and lower write endurance than DRAM. Therefore, developing new buffer management policies that can reduce costly write-backs of PM blocks while maintaining high performance for the hybrid buffer, is of paramount importance. Existing approaches mainly use a page-grained buffering policy, which will cause unnecessary data migrations between DRAM and PM, leading to a high number of disk I/Os and PM writes. Aiming to reduce I/O costs and PM writes, we propose a new buffer manager named HiBuffer for DRAM/PM-based hybrid memory systems. HiBuffer presents several novel ideas. First, it adopts multi-grained data layouts to manage the hybrid buffer cooperatively. In addition to the page granularity, we introduce Lines for the DRAM buffer and Sectors to the PM buffer, forming a buffer with three granularities, including Line, Sector, and Page. We prove that the multi-grained cooperative buffer management can deliver higher performance than existing page-grained schemes. Second, we propose a sector-grained method to migrate data from DRAM to PM, which can avoid unnecessary data movements and reduce PM writes. Third, we use an out-of-place updating mechanism to absorb updates in DRAM, which can further reduce the writes to PM. We compare HiBuffer with three existing schemes, including LRU, CLOCK-DWF, and MiniPage, on five synthetic workloads and the YCSB benchmark using real Intel Optane DC Persistent Memory. The results in terms of various metrics, including running time, PM writes, hit ratio, and disk I/Os, suggest the efficiency of HiBuffer. In particular, HiBuffer reduces the running time by up to 37.8% and the writes to PM by up to 83% compared to the competitors when evaluated on the YCSB benchmark.

The emerging of Non-Volatile Memory (NVM) has changed the traditional DRAM-only memory system. Compared to DRAM, NVM has the advantages of non-volatility and large capacity. However, as the read/write speed of NVM is still lower than that of DRAM, building DRAM/NVM-based hybrid memory systems is a feasible way of adding NVM into the current computer architecture. This paper aims to optimize the well-known B $${^+}$$ -tree for hybrid memory. We present a new index called HATree (Hotness-Aware Tree) that can identify and maintain hot keys with an in-node hotspot cache. The novel idea of HATree is using the unused space of the parent of leaf nodes (PLNs) as the hotspot data cache. Thus, no extra space is needed, but the in-node hotspot cache can efficiently improve query performance. We present the new node structures and operations of HATree and conduct experiments on synthetic workloads using real Intel Optane DC Persistent Memory. The comparative results with three existing state-of-the-art indices, including FPTree, LBTree, and BaseTree, suggest the efficiency of HATree.

Traditional scientific and emerging data analytics applications require fast, power-efficient, large, and persistent memories. Combining all these characteristics within a single memory technology is expensive and hence future supercomputers will feature different memory technologies side-by-side. However, it is a complex task to program hybrid-memory systems and to identify the best object-to-memory mapping. We envision that programmers will probably resort to use default configurations that only require minimal interventions on the application code or system settings. In this work, we argue that intelligent, fine-grained data placement can achieve higher performance than default setups. We present an algorithm for data placement on hybrid-memory systems. Our algorithm is based on a set of single-object allocation rules and global data placement decisions. We also present RTHMS, a tool that implements our algorithm and provides recommendations about the object-to-memory mapping. Our experiments on a hybrid memory system, an Intel Knights Landing processor with DRAM and HBM, show that RTHMS is able to achieve higher performance than the default configuration. We believe that RTHMS will be a valuable tool for programmers working on complex hybrid-memory systems.

This paper demonstrates Medusa, a programming framework for parallel graph processing on graphics processors (GPUs). Medusa enables developers to leverage the massive parallelism and other hardware features of GPUs by writing sequential C/C++ code for a small set of APIs. This simplifies the implementation of parallel graph processing on the GPU. The runtime system of Medusa automatically executes the user-defined APIs in parallel on the GPU, with a series of graph-centric optimizations based on the architecture features of GPUs. We will demonstrate the steps of developing GPU-based graph processing algorithms with Medusa, and the superior performance of Medusa with both real-world and synthetic datasets.

Emerging byte-addressable nonvolatile memories (NVMs) promise persistent memory, which allows processors to directly access persistent data in main memory. Yet, persistent memory systems need to guarantee a consistent memory state in the event of power loss or a system crash (i.e., crash consistency). To guarantee crash consistency, most prior works rely on programmers to (1) partition persistent and transient memory data and (2) use specialized software interfaces when updating persistent memory data. As a result, taking advantage of persistent memory requires significant programmer effort, e.g., to implement new programs as well as modify legacy programs. Use cases and adoption of persistent memory can therefore be largely limited. In this paper, we propose a hardware-assisted DRAM+NVM hybrid persistent memory design, Transparent Hybrid NVM (ThyNVM), which supports software-transparent crash consistency of memory data in a hybrid memory system. To efficiently enforce crash consistency, we design a new dual-scheme checkpointing mechanism, which efficiently overlaps checkpointing time with application execution time. The key novelty is to enable checkpointing of data at multiple granularities, cache block or page granularity, in a coordinated manner. This design is based on our insight that there is a tradeoff between the application stall time due to checkpointing and the hardware storage overhead of the metadata for checkpointing, both of which are dictated by the granularity of checkpointed data. To get the best of the tradeoff, our technique adapts the checkpointing granularity to the write locality characteristics of the data and coordinates the management of multiple-granularity updates. Our evaluation across a variety of applications shows that ThyNVM performs within 4.9% of an idealized DRAM-only system that can provide crash consistency at no cost.

In general, the performance of parallel graph processing is determined by three pairs of critical parameters, namely synchronous or asynchronous execution mode (Sync or Async), Push or Pull communication mechanism (Push or Pull), and Data-driven or Topology-driven traversing scheme (DD or TD), which increases the complexity and sophistication of programming and system implementation of GPU. Existing graph-processing frameworks mainly use a single combination in the entire execution for a given application, but we have observed their variable and suboptimal performance. In this paper, we present SEP-Graph, a highly efficient software framework for graph-processing on GPU. The hybrid execution mode is automatically switched among three pairs of parameters, with an objective to achieve the shortest execution time in each iteration. We also apply a set of optimizations to SEP-Graph, considering the characteristics of graph algorithms and underlying GPU architectures. We show the effectiveness of SEP-Graph based on our intensive and comparative performance evaluation on NVIDIA 1080, P100, and V100 GPUs. Compared with existing and representative GPU graph-processing framework Groute and Gunrock, SEP-Graph can reduce execution time up to 45.8 times and 39.4 times.

Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.

Current application domains, like mobile robotics, or internet of things require high computational power associated with low energy consumption. Therefore, MPSoCs are widely used as an adequate platform for high performance embedded computation. Recently, the emergence of RISC-V instruction set architecture drives SoC designers to adopt it in the design of MPSoCs as a cost-free, modular processor and suitable to be implemented in different hardware platforms. Furthermore, these characteristics make the RISC-V an interesting candidate for an FPGA soft-core processor. In this paper, we present a modular hybrid memory system for a lightweight RISC-V based MPSoC architecture. The implementation of the hybrid memory consists of a global scratchpad on-chip shared memory for both instruction and data for the purpose of communication and synchronization between the processing elements. In addition to a tightly coupled memory associated with each processing element for low latency memory access for private computation. Moreover, the complete MPSoC architecture is scalable and configurable, in terms of the number of PEs, shared/private memory sizes and the number of memory mapped peripherals. A benchmarking environment is developed to evaluate the performance of the proposed hybrid memory system in terms of memory access latency and memory bandwidth and their impact on the computation time. The complete MPSoC architecture is implemented and tested on a Xilinx Zynq 7000 FPGA device.

There are many researches and developments on byte accessible non-volatile memory (NVM). Byte accessible NVM can be used as not only storage but also main memory. Non-volatile main memory has two significant merits: reducing power consumption of main memory, flexible system standby and fast resume. However, NVM has bottlenecks in writing and limitation on the number of writing. To overcome this problem, there are some researches on hybrid memory architectures, which combine NVM and DRAM. On hybrid memory architectures, it needs to collect program characteristics of write access and allocate DRAM or NVM based on that characteristics. Previous works proposed to collect and allocate at hardware level or operating system level. At hardware or OS level, memory allocators can collect writing information of wide range of processes. However, they do not know contents of written data, thus they cannot allocate memory based on data characteristics. To solve this problem, we propose a method to obtain characteristics of writing and assign DRAM or NVM based on that information in previous work. The method applies write barrier of language runtimes to collect writing characteristics. This paper describes modification of our previous work and preliminary analysis for implementation the method on Jikes RVM.

CMOS compatible spin-transfer-torque magnetic random access memory (STT-MRAM) has demonstrated promising developments as the next-generation embedded non-volatile memory (eNVM). In this survey, we provide the state-of-the-art multi-modes reconfigurable techniques for energy-efficient STT-MRAM implementation. We resort to the bottom-up design approach with a twofold aim: 1) summarizing related work from bit-cell to circuit and system levels with conventional and emerging memories. 2) analyzing the present research status of multi-modes reconfigurable techniques in MRAM, which consist of tunable bit-cell configuration, in-memory computing and hybrid memory system. Experimental comparisons show that both reconfiguration and in-MRAM computing obtain extra design freedom. Nonvolatile data accessing cost can be well-alleviated, and thereby improving energy efficiency of MRAM.

This paper studies the problem of efficiently utilizing hybrid memory systems, consisting of both Dynamic Random Access Memory (DRAM) and novel Non-Volatile Memory (NVM) in database management systems (DBMS) for online analytical processing (OLAP) workloads. We present a methodology to determine the database operators that are responsible for most main memory accesses. Our analysis uses both cost models and empirical measurements. We develop heuristic decision procedures to allocate data in hybrid memory at the time that the data buffers are allocated, depending on the expected memory access frequency. We implement these heuristics in the MonetDB column-oriented database and demonstrate performance improvement and energy-efficiency as compared to state-of-the-art application-agnostic hybrid memory management techniques.