Off-chip prefetching based on Hidden Markov Model for non-volatile memory architectures.

With the rise of non-volatile memory (NVM) as a replacement for traditional main memories (e.g. DRAM), memory access analysis is becoming an increasingly important topic. NVMs suffer from technical shortcomings as such as reduced cell endurance which call for precise memory access analysis in order to design maintenance strategies that can extend the memory's lifetime. While existing memory access analyzers trace memory accesses at various levels, from the application level with code instrumentation, down to the hardware level where software is executed on special analysis hardware, they usually interpret main memory as a consecutive area, without investigating the application semantics of different memory regions. In contrast, this paper presents a memory access simulator, which splits the main memory into semantic regions and enriches the simulation result with semantics from the analyzed application. We leverage a library-based operating system called Unikraft by ascribing memory regions of the simulation to the relevant OS libraries. This novel approach allows us to derive a detailed analysis of which libraries (and thus functionalities) are responsible for which memory access patterns. Through offline profiling with our simulator, we provide a fine-granularity analysis of memory access patterns that provide insights for the design of efficient NVM maintenance strategies.

There are many researches and developments on byte accessible non-volatile memory (NVM). Byte accessible NVM can be used as not only storage but also main memory. Non-volatile main memory has two significant merits: reducing power consumption of main memory, flexible system standby and fast resume. However, NVM has bottlenecks in writing and limitation on the number of writing. To overcome this problem, there are some researches on hybrid memory architectures, which combine NVM and DRAM. On hybrid memory architectures, it needs to collect program characteristics of write access and allocate DRAM or NVM based on that characteristics. Previous works proposed to collect and allocate at hardware level or operating system level. At hardware or OS level, memory allocators can collect writing information of wide range of processes. However, they do not know contents of written data, thus they cannot allocate memory based on data characteristics. To solve this problem, we propose a method to obtain characteristics of writing and assign DRAM or NVM based on that information in previous work. The method applies write barrier of language runtimes to collect writing characteristics. This paper describes modification of our previous work and preliminary analysis for implementation the method on Jikes RVM.

Die-stacking technology enables multiple layers of DRAM to be integrated with multicore processors. A promising use of stacked DRAM is as a cache, since its capacity is insufficient to be all of main memory (for all but some embedded systems). However, a 1GB DRAM cache with 64-byte blocks requires 96MB of tag storage. Placing these tags on-chip is impractical (larger than on-chip L3s) while putting them in DRAM is slow (two full DRAM accesses for tag and data). Larger blocks and sub-blocking are possible, but less robust due to fragmentation.This work efficiently enables conventional block sizes for very large die-stacked DRAM caches with two innovations. First, we make hits faster than just storing tags in stacked DRAM by scheduling the tag and data accesses as a compound access so the data access is always a row buffer hit. Second, we make misses faster with a MissMap that eschews stacked-DRAM access on all misses. Like extreme sub-blocking, our implementation of the MissMap stores a vector of block-valid bits for each page in the DRAM cache. Unlike conventional sub-blocking, the MissMap (a) points to many more pages than can be stored in the DRAM cache (making the effects of fragmentation rare) and (b) does not point to the way that holds a block (but defers to the off-chip tags).For the evaluated large-footprint commercial workloads, the proposed cache organization delivers 92.9% of the performance benefit of an ideal 1GB DRAM cache with an impractical 96MB on-chip SRAM tag array.

Journaling techniques play an important role in addressing the reliability issue of filesystems caused by the volatile DRAM-based buffer cache. However, journaling techniques introduce a large number of extra storage writes, which greatly degrades the performance of the filesystem [10]. Emerging Non-Volatile Memory (NVM) technologies bring a new perspective of solving the write amplification issue caused by journaling. By adopting NVM as the buffer cache, the committed data can be maintained in NVM before being written back to the storage, thus eliminating the journaling overhead. However, simply replacing DRAM with NVM as the buffer cache suffers from the limited lifetime and relative slow writes of NVM. In this paper, we present a hybrid buffer cache architecture by combing NVM with DRAM to reduce the journaling overhead and overcome the constraints of NVM. In order to better utilize this novel architecture, we first propose a Journaling-Aware Page Management (JAPM) policy. JAPM puts infrequently updated data in NVM to reduce the journaling overhead and frequently updated data in DRAM to improve the write performance and lifetime of the hybrid buffer cache. In addition, data in one transaction may be dispersed in NVM and DRAM simultaneously and different committing policies are required for different storing media, NVM or DRAM. In order to guarantee the atomicity of the transactional execution in the hybrid cache architecture, a Partial In-Place Commit (PIPC) journaling scheme is proposed to coordinate the different committing patterns. We implement the proposed techniques on Linux 3.14.52 and measure the performance with representative I/O-intensive benchmarks. The experimental results show that our scheme effectively improves the I/O performance compared with the ext4 filesystem and prolongs the lifetime of the hybrid buffer cache compared with the Union of Buffer cache and Journaling (UBJ) scheme [10].

Applications executing on multicore processors can now easily offload computations to many core processors, such as Intel Xeon Phi coprocessors. However, it requires high levels of expertise and effort to tune such offloaded applications to realize high-performance execution. Previous efforts have focused on optimizing the execution of offloaded computations on many core processors. However, we observe that the data transfer overhead between multicore and many core processors, and the limited device memories of many core processors often constrain the performance gains that are possible by offloading computations. In this paper, we present three source-to-source compiler optimizations that can significantly improve the performance of applications that offload computations to many core processors. The first optimization automatically transforms offloaded codes to enable data streaming, which overlaps data transfer between multicore and many core processors with computations on these processors to hide data transfer overhead. This optimization is also designed to minimize the memory usage on many core processors, while achieving the optimal performance. The second compiler optimization re-orders computations to regularize irregular memory accesses. It enables data streaming and factorization on many core processors, even when the memory access patterns in the original source codes are irregular. Finally, our new shared memory mechanism provides efficient support for transferring large pointer-based data structures between hosts and many core processors. Our evaluation shows that the proposed compiler optimizations benefit 9 out of 12 benchmarks. Compared with simply offloading the original parallel implementations of these benchmarks, we can achieve 1.16x-52.21x speedups.

Non-volatile memory (NVM) provides a scalable and power-efficient solution to replace dynamic random access memory (DRAM) as main memory. However, because of the relatively high latency and low bandwidth of NVM, NVM is often paired with DRAM to build a heterogeneous memory system (HMS). As a result, data objects of the application must be carefully placed to NVM and DRAM for the best performance. In this paper, we introduce a lightweight runtime solution that automatically and transparently manages data placement on HMS without the requirement of hardware modifications and disruptive change to applications. Leveraging online profiling and performance models, the runtime solution characterizes memory access patterns associated with data objects, and minimizes unnecessary data movement. Our runtime solution effectively bridges the performance gap between NVM and DRAM. We demonstrate that using NVM to replace the majority of DRAM can be a feasible solution for future HPC systems with the assistance of a software-based data management.

Non-volatile memory (NVM) technology holds promise to replace SRAM and DRAM at various levels of the memory hierarchy. The interest in NVM is motivated by the difficulty faced in scaling DRAM beyond 22 nm and, long-term, lower cost per bit. While offering higher density and negligible static power (leakage and refresh), NVM suffers increased latency and energy per memory access. This paper develops energy and performance models of memory systems and applies them to understand the energy-efficiency of replacing or complementing DRAM with NVM. Our analysis focusses on the application of NVM in main memory. We demonstrate that NVM such as STT-RAM and RRAM is energy-efficient for memory sizes commonly employed in servers and high-end workstations, but PCM is not. Furthermore, the model is well suited to quickly evaluate the impact of changes to the model parameters, which may be achieved through optimization of the memory architecture, and to determine the key parameters that impact system-level energy and performance.

Computer hardware is currently moving towards heavily parallelized architectures with multiprocessors, multicore and chip multithreaded designs. Cache memory, the fastest component of the memory hierarchy, adapts to this new kind of parallel systems in order to provide the promised performance increase. Current cache designs have limitations that can be transformed into optimization opportunities both in hardware and software. This paper provides a detailed research of cache performance in multicore processors, considering critical hardware aspects. A new solution is proposed to improve the current performance: an optimized replacement policy for the shared cache level. From experiments run on four and eight core setups in a multicore simulator, the proposed enhancements achieve up to 30% execution speed increase over the default setup.

Although emerging non-volatile memories (NVMs) have been comprehensively studied to design next-generation memory systems, the symmetry of the crossbar structure adopted by most NVMs has not been addressed. In this work, we argue that crossbar-based NVMs can enable dual-addressing memory architecture, i.e., RC-NVM, to support both row- and column-oriented memory accesses for workloads with different access patterns. Through circuit-level analysis, we first prove that such a dual-addressing architecture is only practical with crossbar-based NVMs rather than DRAM. Then, we introduce the RC-NVM architecture from bank, chip and module levels, and propose RC-NVM aware memory controller. We also address the challenges to implement the end-to-end RC-NVM system. Especially, we design a novel protocol to solve the cache synonym problem with very little overhead. Finally, we introduce the deployment of RC-NVM for in-memory databases (IMDBs) and evaluate its performance with IMDBs and well-optimized general matrix multiply (GEMM) workloads. Experimental results show that with only 10 percent area overhead 1) the memory access performance of IMDBs can be improved up to 14.5X, and 2) for GEMM, RC-NVM naturally supports SIMD operations and outperforms the best tiled layout by 19 percent.

Large-capacity memory system allows big data applications to load as much data as possible for in-memory processing, which improves application performance. However, DRAM faces both scalability and energy challenges due to its inherent charging mechanism. Thus, DRAM-based memory system incurs excessive cost to meet both capacity and energy requirements for the emerging big data workloads. Fortunately, non-volatile memories(NVMs) are emerging with the advanced features of better scalability and lower power leakage. Integrating NVMs into main memory is non-trivial as NVMs have a few weakness, such as asymmetric read and write latency and power. Designing memory system comprising both DRAM and NVMs requires to understand the memory access behaviors of big data applications. In this paper, we first investigate the memory access patterns of both typical big data workloads and traditional parallel workloads. By doing so, we show the read/write intensity as well as temporal/spatial locality of big data workloads. We then replay memory access traces of big data applications to DRAM simulator and PCM simulator, respectively. We explore design implications of hybrid memory comprising DRAM and PCM.

The increasing interest in adopting cloud technologies for Industry 4.0 and mixed-criticality environments is paving the way for compelling new opportunities and challenges. However, cloud deployments use multicore processors that introduce interference between co-executing applications on different cores due to contention in shared resources, such as the memory subsystem. Such interference can cause critical applications to miss their deadlines.To enable the co-execution of critical and non-critical applications on the same multicore processor, we propose an approach that guarantees memory access time isolation for critical cores, while not jeopardizing the memory bandwidth of the non-critical ones. We prove the viability of our approach using Intel’s resource director technology for memory access detection and regulation. Experiments show that queue occupancy is an excellent metric to estimate the number of interfering cores co-accessing the memory. We also assess the indirect memory bandwidth limitation achievable by applying Intel’s Memory Bandwidth Allocation technology.

A new technique for clustering and classifying bioacoustics vocalizations from large audio recording data is presented. Over 30,000 terrestrial passive audio and ultrasonic recorders deployed worldwide for the monitoring of birds, frogs and bats generate many petabytes of data annually. Our methods seek to efficiently and automatically detect candidate vocalizations from these massive datasets and sort them into clusters of similar vocalization patterns. Researchers can then quickly survey biodiversity and find species of interest by examining each of the resulting clusters. After clustering, species-specific classifiers can be built by labeling whole clusters or training classifiers after labeling individual detections. Once a classifier is built, it can be used to search new data for species of interest. The algorithms detect vocalizations comprising one or more short syllables. A variable-length observation sequence is extracted comprising spectral features changing through time. A Hidden Markov Model is trained on these sequences and Fisher scores are calculated for each detection. The dot product of two Fisher scores provides a measure of similarity between two vocalizations and forms the basis for clustering. The algorithms are designed to run in parallel on multi-core processors and stream the data to minimize memory requirements.

Non-volatile memory (NVM) is expected to be the second level memory (named remote memory) in two-level memory hierarchy in the future. However, NVM has the limited write endurance, thus it is vital to reduce the number of write operations on NVM. Meanwhile, in two-level memory hierarchy, prefetch is widely used for fetching certain data before it is actually required, to hide the remote memory access latency. In general, large-scale nested loop is the performance bottleneck in one program due to the write operations on NVM caused by the first level memory (named local memory) miss and data reuse. Loop tiling is the key technique for grouping iterations so as to reduce the communication with remote memory used in compiler. In this paper, we propose a new loop tiling approach for minimizing the write operations on NVMs and completely hiding the NVM access latency. Specifically, we introduce a series of theorems to help loop tiling. Then, a legal tile shape and an optimal tile size selection strategy is proposed according to data dependency and local memory capacity. Furthermore, we propose a pipeline scheduling policy to completely hide the remote memory latency. Extensive experiments show that the proposed techniques can reduce write operations on NVMs by 95.1% on average, and NVM latency can be completely hidden.

The number of chip pins is limited due to the cost and reliability issues of sophisticated packages, and it is predicted that the chip pin count will be overstretched to satisfy the requirements of both power delivery and memory access. The gap between the achievable pin count and the demand will increase as the technology scales, due to the increasing computation resources and supply current. Pin reduction techniques are thus required for continued computing performance growth. In this article, we propose a chip pin constraint alleviation strategy, through on/off-chip power delivery system co-design, to effectively reduce the demand for power pins. An analytical model of a power delivery system, consisting of on/off-chip regulators and a power delivery network, is proposed to evaluate the influence of regulator design and package conduction loss. By combining this model with a multi-core processor model of performance and memory bandwidth requirements, we characterize the entire multi-core processor system to investigate the relationship between the chip pin constraint and performance in multi-core processor scaling and the effectiveness of our strategy. Experiments show that with the conventional power delivery system design, the chip pin constraint severely limits the performance growth as the technology scales. Using the on/off-chip power delivery system co-design, our strategy achieves a significant pin count reduction, for example, 31.3% at the 8nm technology node, compared to the conventional design with the same chip performance, while, provided with the same chip pin count, it is able to improve, by 35.0%, the chip performance at 8nm compared to the conventional design. For real applications of different parallelism, our strategy outperforms its counterpart, with a 23.7% performance improvement on average at the 8nm technology node.

In this letter, a flexible memory simulator - NVMain 2.0, is introduced to help the community for modeling not only commodity DRAMs but also emerging memory technologies, such as die-stacked DRAM caches, non-volatile memories (e.g., STT-RAM, PCRAM, and ReRAM) including multi-level cells (MLC), and hybrid non-volatile plus DRAM memory systems. Compared to existing memory simulators, NVMain 2.0 features a flexible user interface with compelling simulation speed and the capability of providing sub-array-level parallelism, fine-grained refresh, MLC and data encoder modeling, and distributed energy profiling.