A Novel Prefetching Scheme for Non-Volatile Cache in the AIoT Processor

Due to its large leakage power and low density, the conventional SARM becomes less appealing to implement the large on-chip cache due to energy issue. Emerging non-volatile memory technologies, such as phase change memory (PCM) and spin-transfer torque RAM (STT-RAM), have advantages of low leakage power and high density, which makes them good candidates for on-chip cache. In particular, STT-RAM has longer endurance and shorter access latency over PCM. There are two kinds of STT-RAM so far: single-level cell (SLC) STT-RAM and multi-level cell (MLC) STT-RAM. Compared to the SLC STT-RAM, the MLC STT-RAM has higher density and lower leakage power, which makes it a even more promising candidate for future on-chip cache. However, MLC STT-RAM improves density at the cost of almost doubled write latency and energy compared to the SLC STT-RAM. These drawbacks degrade the system performance and diminish the energy benefits. To alleviate these problems, we propose a novel cache organization, companion write cache (CWC), which is a small fully associative SRAM cache, working with the main MLC STT-RAM cache in a master-and-servant way. The key function of CWC is to absorb the energy-consuming write updates from the MLC STT-RAM cache. The experimental results are promising that CWC can greatly reduce the write energy and dynamic energy, improve the performance and endurance of MLC STT-RAM cache compared to a baseline.

Spin-Transfer Torque RAM (STT-RAM) is an emerging non-volatile memory technology that is a potential universal memory that could replace SRAM in processor caches. This paper presents a novel approach for redesigning STT-RAM memory cells to reduce the high dynamic energy and slow write latencies. We lower the retention time by reducing the planar area of the cell, thereby reducing the write current, which we then use with CACTI to design caches and memories. We simulate quad-core processor designs using a combination of SRAM- and STT-RAM-based caches. Since ultra-low retention STT-RAM may lose data, we also provide a preliminary evaluation for a simple, DRAM-style refresh policy. We found that a pure STT-RAM cache hierarchy provides the best energy efficiency, though a hybrid design of SRAM-based L1 caches with reduced-retention STT-RAM L2 and L3 caches eliminates performance loss while still reducing the energy-delay product by more than 70%.

Modern high performance Chip Multiprocessor (CMP) systems rely on large on-chip cache hierarchy. As technology scales down, the leakage power of present SRAM based cache gradually dominates the on-chip power consumption, which can severely jeopardize system performance. The emerging nonvolatile Spin Transfer Torque RAM (STT-RAM) is a promising candidate for large on-chip cache because of the ultra low leakage power. However, the write operations on STT-RAM suffer from considerably higher energy as well as longer latency compared with SRAM which will make STT-RAM in trouble for write-intensive workloads. In this paper, we propose to integrate SRAM with STT-RAM to construct a novel hybrid cache architecture for CMPs. We also propose dedicated microarchitectural mechanisms to make the hybrid cache robust to workloads with different write patterns. Extensive simulation results demonstrate that the proposed hybrid scheme is adaptive to variations of workloads. Overall power consumption is reduced by 37.1% and performance is improved by 23.6% on average compared with SRAM based static NUCA under the same area configuration.

There is growing interest in emerging non-volatile memory technologies such as Phase-Change Memory, Memristors, and Spin-Transfer Torque RAM (STT-RAM). STT-RAM, in particular, is experiencing rapid development that can be difficult for memory systems researchers to take advantage of. What is needed are techniques that enable designers to explore the potential of recent STT-RAM designs and adjust the performance without needing a detailed understanding of the physics. In this paper, we present the STeTSiMS STT-RAM Simulation and Modeling System to assist memory systems researchers. After providing background on the operation of STT-RAM magnetic tunnel junctions (MTJs), we demonstrate how to fit three different published MTJ models to our model and normalize their characteristics with respect to common metrics. The high-speed switching behavior of the designs is evaluated using macromagnetic simulations. We have also added a first-order model for STT-RAM memory arrays to the CACTI memory modeling tool, which we then use to evaluate the performance, energy consumption, and area for: (i) a high-performance cache, (ii) a high-capacity cache, and (iii) a high-density memory.

Spin-Transfer Torque RAM (STT-RAM) has emerged as a potential candidate for Universal memory. However, there are two challenges to using STT-RAM in memory system design: (1) the intrinsic variation in the storage element, the Magnetic Tunnel Junction (MTJ), and (2) the high write energy. In this paper, we present a physically based thermal noise model for simulating the statistical variations of MTJs. We have implemented it in HSPICE and validated it against analytical results. We demonstrate its use in setting the write pulse width for a given write error rate. We then propose two write-energy reduction techniques. At the device level, we propose the use of a low-M S ferromagnetic material that can reduce the write energy without sacrificing retention time. At the architecture level, we show that Invert Coding provides a 7% average reduction in the total write energy for the SPEC CPU2006 benchmark suite without any performance overhead.

Non-volatile memory (NVM) devices, such as Flash, phase change RAM, spin transfer torque RAM, and resistive RAM, offer several advantages and challenges when compared to conventional memory technologies, such as DRAM and magnetic hard disk drives (HDDs). In this paper, we present a survey of software techniques that have been proposed to exploit the advantages and mitigate the disadvantages of NVMs when used for designing memory systems, and, in particular, secondary storage (e.g., solid state drive) and main memory. We classify these software techniques along several dimensions to highlight their similarities and differences. Given that NVMs are growing in popularity, we believe that this survey will motivate further research in the field of software technology for NVMs.

Currently, commercial semiconductor-based memories face the problem of static energy consumption caused by leakage currents. Magnetoelectric random access memory (MeRAM), as a type of nonvolatile memory, provides a solution to this problem and can be used to replace the entire memory hierarchy as a universal memory. In contrast to spin transfer torque RAM (STT-RAM), MeRAM uses voltage controlled magnetic anisotropy (VCMA) effect to manipulate the switching of magnetic tunnel junctions (MTJs). MeRAM or VCMA-MTJ technology is more energy efficient and less likely to break down than STT-RAM. However, VCMA-MTJ suffers from serious write problems caused by variations in the VCMA coefficient, the external in-plane magnetic field, and the CMOS process. In this paper, a compact and SPICE-compatible VCMA-MTJ model is proposed. Then, a self-adaptive write circuit is proposed in which the write process is robust to variations in the VCMA coefficient and the external in-plane magnetic field. Finally, two practical methods are proposed to make our proposed self-adaptive write circuit also robust to 3$\sigma$ CMOS process variations, as demonstrated by Monte Carlo simulations. The proposed self-adaptive write circuit can assist in the design of next-generation high-speed, low-power MeRAM chips.

Research efforts to develop a novel memory technology that combines the desired traits of non-volatility, high endurance, high speed and low power have resulted in the emergence of Spin Torque Transfer-RAM (STTRAM) as a promising next generation universal memory. However, the prospect of developing a non-volatile FPGA framework with STTRAM exploiting its high integration density remains largely unexplored. In this paper, we propose a novel CMOS-STTRAM hybrid FPGA framework; identify the key design challenges; and propose optimization techniques at circuit, architecture and application mapping levels. Simulation results show that a STTRAM based optimized FPGA framework achieves an average improvement of 48.38% in area, 22.28% in delay and 16.1% in dynamic power for ISCAS benchmark circuits over a conventional CMOS based FPGA design.

ABSTRACTFog computing requires a large main memory capacity to decrease latency and increase the Quality of Service (QoS). However, dynamic random access memory (DRAM), the commonly used random access memory, cannot be included into a fog computing system due to its high consumption of power. In recent years, non-volatile memories (NVM) such as Phase-Change Memory (PCM) and Spin-transfer torque RAM (STT-RAM) with their low power consumption have emerged to replace DRAM. Moreover, the currently proposed hybrid main memory, consisting of both DRAM and NVM, have shown promising advantages in terms of scalability and power consumption. However, the drawbacks of NVM, such as long read/write latency give rise to potential problems leading to asymmetric cache misses in the hybrid main memory. Current last level cache (LLC) policies are based on the unified miss cost, and result in poor performance in LLC and add to the cost of using NVM. In order to minimize the cache miss cost in the hybrid main memory, we propose a cost aware cache replacement policy (CACRP) that reduces the number of cache misses from NVM and improves the cache performance for a hybrid memory system. Experimental results show that our CACRP behaves better in LLC performance, improving performance up to 43.6% (15.5% on average) compared to LRU.

International audience

Leakage power increase due to technology scaling has attracted a lot of attention to developing nonvolatile memory (NVM) technologies. Among the explored NVM candidates by the community, spintronic-based technologies such as magnetic RAM and spin-transfer torque RAM seem to be very promising. Conventionally, the main challenge of employing MRAMs as a possible candidate in universal memories is high power dissipation and long delay during write operation. However, in recent cutting-edge semiconductor technologies, the issue moves from write to read operation mainly due to significant suppression in read margin. Degradation in the read yield emerges from the increase in process variation and lowering of the supply voltage. Therefore, numerous research activities have been recently conducted to improve the read margin. This chapter reviews the sensing techniques, which have been developed to deal with the read margin degradation of MRAMs in scaled technology nodes. In addition, the chapter provides a background on different writing methods of MRAMs and possible solutions to improve the density of bit-cells.

Spin-transfer torque RAM (STT-RAM) has emerged as an energy-efficient and high-density alternative to SRAM for large on-chip caches. However, its high write energy has been considered as a serious drawback. Hybrid caches mitigate this problem by incorporating a small SRAM cache for write-intensive data along with an STT-RAM cache. In such architectures, choosing cache blocks to be placed into the SRAM cache is the key to their energy efficiency. This paper proposes a new hybrid cache architecture called prediction hybrid cache. The key idea is to predict write intensity of cache blocks at the time of cache misses and determine block placement based on the prediction. We design a write intensity predictor that realize the idea by exploiting a correlation between write intensity of blocks and memory access instructions that incur cache misses of those blocks. It includes a mechanism to dynamically adapt the predictor to application characteristics. We also design a hybrid cache architecture in which write-intensive blocks identified by the predictor are placed into the SRAM region. Evaluations show that our scheme reduces energy consumption of hybrid caches by 28 percent (31 percent) on average compared to the existing hybrid cache architecture in a single-core (quad-core) system.

GPUs provide high-bandwidth/low-latency on-chip shared memory and L1 cache to efficiently service a large number of concurrent memory requests. Specifically, concurrent memory requests accessing contiguous memory space are coalesced into warp-wide accesses. To support such large accesses to L1 cache with low latency, the size of L1 cache line is no smaller than that of warp-wide accesses. However, such L1 cache architecture cannot always be efficiently utilized when applications generate many memory requests with irregular access patterns especially due to branch and memory divergences that make requests uncoalesced and small. Furthermore, unlike L1 cache, the shared memory of GPUs is not often used in many applications, which essentially depends on programmers. In this article, we propose Elastic-Cache, which can efficiently support both fine- and coarse-grained L1 cache line management for applications with both regular and irregular memory access patterns to improve the L1 cache efficiency. Specifically, it can store 32- or 64-byte words in non-contiguous memory space to a single 128-byte cache line. Furthermore, it neither requires an extra memory structure nor reduces the capacity of L1 cache for tag storage, since it stores auxiliary tags for fine-grained L1 cache line managements in the shared memory space that is not fully used in many applications. To improve the bandwidth utilization of L1 cache with Elastic-Cache for fine-grained accesses, we further propose Elastic-Plus to issue 32-byte memory requests in parallel, which can reduce the processing latency of memory instructions and improve the throughput of GPUs. Our experiment result shows that Elastic-Cache improves the geometric-mean performance of applications with irregular memory access patterns by 104% without degrading the performance of applications with regular memory access patterns. Elastic-Plus outperforms Elastic-Cache and improves the performance of applications with irregular memory access patterns by 131%.

Non-volatile Memory (NVM) offers the opportunity to build large, durable B+ trees with markedly higher performance and faster post-crash recovery than is possible with traditional disk- or flash-based persistence. Unfortunately, cache flush and fence instructions, required for crash consistency and failure atomicity on many machines, introduce substantial overhead not present in non-persistent trees, and force additional NVM reads and writes. The overhead is particularly pronounced in workloads that benefit from cache reuse due to good temporal locality or small working sets---traits commonly observed in real-world applications. In this paper, we propose a buffered durable B+ tree (BD+Tree) that improves performance and reduces NVM traffic via relaxed persistence. Execution of a BD+Tree is divided into epochs of a few milliseconds each; if a crash occurs in epoch e, the tree recovers to its state as of the end of epoch e -2. (The persistence boundary can always be made current with an explicit sync operation, which quickly advances the epoch by 2.) NVM writes within an epoch are aggregated for delayed persistence, thereby increasing cache reuse and reducing traffic to NVM. In comparison to state-of-the-art persistent B+ trees, our micro-benchmark experiments show that BD+Tree can improve throughput by up to 2.4x and reduce NVM writes by up to 90% when working sets are small or workloads exhibit strong temporal locality. On real-world workloads that benefit from cache reuse, BD+Tree realizes throughput improvements of 1.1--2.4x and up to a 99% decrease in NVM writes. Even on uniform workloads, with working sets that significantly exceed cache capacity, BD+Tree still improves throughput by 1--1.3x. The performance advantage of BD+Tree increases with larger caches, suggesting ongoing benefits as CPUs evolve toward gigabyte cache capacities.

Nonvolatile memory such as magnetic RAM (MRAM) offers high cell density and low leakage power while suffering from long write latency and high write energy, compared with SRAM. 3-D integration technology using through-silicon vias enables stacking disparate memory technologies (e.g., SRAM and MRAM) together onto chip-multiprocessors (CMPs). The use of hybrid memories as an on-chip cache can take advantage of the best characteristics that each technology offers. However, the inherent high power density and heat removal limitation in 3-D integrated circuits may incur temperature-related problems. In this paper, we propose a runtime thermal management method for CMPs with the 3-D stacked hybrid SRAM/MRAM L2 cache. The proposed method combines dynamic cache management such as resource allocation, way-based power gating, and data migration with dynamic voltage and frequency scaling of processing cores in a temperature- and energy-aware manner. Experimental results show that the proposed runtime method with the 3-D stacked hybrid L2 cache offers up to 107.37% (55.28% on average) performance improvement and 88.47% (47.65% on average) energy efficiency improvement compared with existing thermal management methods with 3-D stacked SRAM-based L2 cache.