F1: Non-Volatile Memory Circuit Design and Technology

Using nonvolatile memories in memory hierarchy has been investigated to reduce its energy consumption because nonvolatile memories consume zero leakage power in memory cells. One of the difficulties is, however, that the endurance of most nonvolatile memory technologies is much shorter than the conventional SRAM and DRAM technology. This has limited its usage to only the low levels of a memory hierarchy, e.g., disks, that is far from the CPU.

Non-volatile memory (NVM) technology holds promise to replace SRAM and DRAM at various levels of the memory hierarchy. The interest in NVM is motivated by the difficulty faced in scaling DRAM beyond 22 nm and, long-term, lower cost per bit. While offering higher density and negligible static power (leakage and refresh), NVM suffers increased latency and energy per memory access. This paper develops energy and performance models of memory systems and applies them to understand the energy-efficiency of replacing or complementing DRAM with NVM. Our analysis focusses on the application of NVM in main memory. We demonstrate that NVM such as STT-RAM and RRAM is energy-efficient for memory sizes commonly employed in servers and high-end workstations, but PCM is not. Furthermore, the model is well suited to quickly evaluate the impact of changes to the model parameters, which may be achieved through optimization of the memory architecture, and to determine the key parameters that impact system-level energy and performance.

Hardware specialization is commonly used in data-centers to ameliorate the nearing end of CMOS technology scaling. While offering superior performance and energy-efficiency returns compared to general-purpose processors, specialized accelerators are bound to the same device technology constraints, and are thus prone to similar limitations in the future. Once technology scaling plateaus, accelerator and application tuning will reach a point of near-optimum, with no clear direction for further improvements. Emerging non-volatile memory (NVM) technologies follow different scaling trends due to different physical properties and manufacturing techniques. NVMs have inspired recent efforts of innovation in computer systems, as they possess appealing qualities such as high capacity and low energy. We present the COmpute-REuse Accelerators (COREx) architecture that shifts computations from the scalability-hindered transistor-based logic towards the continuing-to-scale storage domain. COREx leverages datacenter redundancy by integrating a storage layer together with the accelerator processing layer. The added layer stores the outcomes of previous accelerated computations. The previously computed results are reused in the case of recurring computations, thus eliminating the need to re-compute them. We designed COREx as a combination of an accelerator and specialized storage layer using emerging memory technologies, and evaluated it on a set of datacenter workloads. Our results show that, when integrated with a well-tuned accelerator, COREx achieves an average speedup of 6.4X and average savings of 50% in energy and 68% in energy-delay product. We expect further increase in gains in the future, as memory technologies continue to improve steadily.

Microcontroller units (MCUs) are often used in Internet of Things nodes that operate intermittently. Such nodes alternate active and inactive phases under strict energy constraints. Typically, the memory system has a significant impact on overall MCU energy consumption. Memory accesses and memory leakage power often dominate the consumption of active and inactive phases, respectively. Emerging Non-Volatile Memory (NVM) technologies have recently enabled the design of non-volatile MCUs that can significantly reduce energy consumption during inactive phases. However, replacing all memories with emerging NVMs is not necessarily the best solution, as it often results in dynamic power overhead during active phases. Instead, a hybrid memory architecture that combines volatile and non-volatile technologies is a promising alternative. However, designing hybrid memory MCUs is challenging because the technology that best fits a data segment depends on its access pattern during execution (e.g., program memory experiences mostly reads while the stack alternates reads and writes). For a given intermittent application, our goal is to find the best memory architecture based on a data mapping that takes advantage of the different properties of the available memory technologies. To this end, we present MemCork, a tool for hybrid memory architecture exploration in intermittent computing devices. Based on an instrumented execution on a technology-agnostic FPGA prototype, our tool exhaustively explores the possible data mapping and memory architecture combinations to find the most energy-efficient solution. We evaluate MemCork on two representative intermittent applications and find a customised memory architecture and data mapping that reduces energy consumption by up to 23% compared to a fully NVM solution.

Spin-Torque Transfer Magnetic RAM (STT-RAM), one of the emerging nonvolatile memory (NVM) technologies explored as the replacement for SRAM memory architectures, is particularly promising due to the fast access speed, high integration density, and zero standby power consumption. Recently, hybrid deigns with SRAM and STT-RAM buffers for routers in Network-on-Chip (NoC) systems have been widely implemented to maximize the mutually complementary characteristics of different memory technologies, and leverage the efficiency of intra-router latency and system power consumption. With the realization of Processing-in-Memory enabled by STT-RAM, in this paper, we novelly offload the execution from processors to the STT-RAM based on-chip routers to improve the application performance. On top of the hybrid buffer design in routers, we further present system-level approaches, including an ILP model and polynomial-time heuristic algorithms, to fine-tune the application mapping and scheduling on NoCs, with the objectives of improving system performance-energy efficiency. Network overhead caused by flit conflict in conventional communication circumstances can be ideally avoided by computing the contended flits in intermediate routers; meanwhile, the pressure of heavy workload on processors can be relieved by transferring partial operations to routers, such that network latency and system power consumption can be significantly reduced. Experimental results demonstrate that application schedule length and system energy consumption can be reduced by 35.62, 32.87 percent on average, respectively, in extensive evaluation experiments on PARSEC benchmark applications. In particular, the achievements of application performance and energy efficiency, averagely 36.44 and 33.19 percent, for the CNN application AlexNet have verified the practicability and effectiveness of our presented approaches.

New non-volatile memory (NVM) technologies, e.g., Phase Change Memory (PCM), are emerging with main features such as byte addressability, cheapness and large size, and will be deployed to serve as main memory. For fast processing of big data, block device drivers have been developed for NVM such that traditional file systems (e.g., EXT4) operated on non-volatile main memory. However, the existing block driver blindly caches pages, which incurs unnecessary access delays. To overcome this drawback, this paper develops a novel block device driver, called NVMBD, for byte-addressable non-volatile main memory. It adopts two methods of processing block I/O requests, i.e., traditional block I/O and BMAP. On the one hand, traditional block I/O reads/writes 512B sectors with multiple load/store instructions. On the other hand, BMAP inserts NVM pages into the page cache tree when processing block I/O requests, which removes the page movement between the page cache and the NVM. BMAP involves in the management of page cache, but it is transparent to file systems. BMAP provides file systems with the DAX-like feature, improving the file system performance. Through comprehensive experiments, we demonstrate that the developed block device driver significantly improves the file access performance.

Journaling techniques play an important role in addressing the reliability issue of filesystems caused by the volatile dynamic random access memory (DRAM)-based buffer cache. However, journaling techniques introduce a large number of extra storage writes, which greatly degrades system performance, especially for mobile devices. Emerging nonvolatile memory (NVM) technologies bring a new perspective of solving the write amplification issue caused by journaling. By adopting NVM as the buffer cache, the committed data can be maintained in NVM before being written back to the storage, thus, eliminating the journaling overhead. However, simply utilizing NVM as the buffer cache suffers from the limited lifetime and the relatively longer write latency of NVM. In this paper, we present a hybrid buffer cache architecture, named UniBuffer, by combing NVM with dynamic random access memory (DRAM) to reduce the journaling overhead and overcome the inherent constraints of NVM. In UniBuffer, we first propose a journaling-aware page management (JAPM) policy to smartly allocate data to DRAM and NVM pages. JAPM puts infrequently updated data in NVM to reduce the journaling overhead and frequently updated data in DRAM to improve the write performance and lifetime of the hybrid buffer cache. In addition, since data in one journaling transaction may be dispersed in NVM and DRAM simultaneously, different committing policies are required for different storage media, respectively. In order to guarantee the atomicity of the transaction execution in the hybrid cache architecture, a partial in-place commit (PIPC) journaling scheme is proposed to coordinate different committing patterns for NVM and DRAM. We have implemented the proposed techniques on Linux 3.14.52 and measured the performance with representative I/O-intensive benchmarks. The experimental results show that our scheme effectively improves the I/O performance compared with the ext4 filesystem and prolongs the lifetime of the hybrid buffer cache compared with the union of buffer cache and journaling (UBJ) scheme.

Journaling techniques play an important role in addressing the reliability issue of filesystems caused by the volatile DRAM-based buffer cache. However, journaling techniques introduce a large number of extra storage writes, which greatly degrades the performance of the filesystem [10]. Emerging Non-Volatile Memory (NVM) technologies bring a new perspective of solving the write amplification issue caused by journaling. By adopting NVM as the buffer cache, the committed data can be maintained in NVM before being written back to the storage, thus eliminating the journaling overhead. However, simply replacing DRAM with NVM as the buffer cache suffers from the limited lifetime and relative slow writes of NVM. In this paper, we present a hybrid buffer cache architecture by combing NVM with DRAM to reduce the journaling overhead and overcome the constraints of NVM. In order to better utilize this novel architecture, we first propose a Journaling-Aware Page Management (JAPM) policy. JAPM puts infrequently updated data in NVM to reduce the journaling overhead and frequently updated data in DRAM to improve the write performance and lifetime of the hybrid buffer cache. In addition, data in one transaction may be dispersed in NVM and DRAM simultaneously and different committing policies are required for different storing media, NVM or DRAM. In order to guarantee the atomicity of the transactional execution in the hybrid cache architecture, a Partial In-Place Commit (PIPC) journaling scheme is proposed to coordinate the different committing patterns. We implement the proposed techniques on Linux 3.14.52 and measure the performance with representative I/O-intensive benchmarks. The experimental results show that our scheme effectively improves the I/O performance compared with the ext4 filesystem and prolongs the lifetime of the hybrid buffer cache compared with the Union of Buffer cache and Journaling (UBJ) scheme [10].

In the rapidly evolving landscape of modern electronics, memory devices and circuits play a pivotal role in determining system performance, energy efficiency, and scalability. Traditional memory technologies, such as Dynamic Random Access Memory (DRAM) and NAND flash, face challenges related to scaling limits, latency, power consumption, and endurance, particularly as demands for faster and more efficient data storage grow. Emerging non-volatile memory (NVM) technologies like Resistive Random Access Memory (ReRAM), Phase-Change Memory (PCM), and Spin-Transfer Torque Magnetic Random Access Memory (STT-MRAM) offer potential solutions to these challenges, but their combination into existing systems poses design and manufacturing hurdles. The primary problem lies in balancing storage performance with power efficiency while ensuring scalability for advanced applications such as artificial intelligence (AI), Internet of Things (IoT), and high-performance computing (HPC). Innovations in circuit design, including advanced error-correction mechanisms, adaptive voltage scaling, and low-power architectures, are critical in enhancing the overall performance of memory systems. This study employs a hybrid approach, combining novel circuit design techniques with the combination of emerging NVM technologies. Simulations were performed to evaluate the performance metrics, including write latency, energy consumption, and endurance. Results show that ReRAM-based memory designs achieved a 35% reduction in write latency and a 50% improvement in energy efficiency compared to traditional DRAM, while PCM demonstrated superior endurance with up to 10^8 write cycles. Furthermore, STT-MRAM circuits showed promise with a 40% reduction in standby power consumption. These advancements in memory circuits and devices highlight the potential for significant improvements in storage performance and efficiency, addressing the growing needs of modern electronic systems and paving the way for future innovations in computing.

In order to mitigate the problem of the memory wall, various emerging nonvolatile memory (NVM) technologies have been proposed to replace the traditional ones. These emerging NVMs include spin torque transfer random access memory, PCRAM, resistive random access memory, racetrack memory, etc. Compared to traditional memory technologies, they have the advantages of near-zero standby power, high storage density and nonvolatility, which make them competitive for future memory hierarchy design. However, it is inefficient to directly apply these NVMs in existing memory architectures. On the one hand, these NVMs have their own limitations, such as long write latency, high write energy, limited write numbers, etc. Thus, proper architecture modification is required to adopt them into the traditional memory hierarchy. On the other hand, the unique features of these NVMs allow new memory architectures in memory subsystems and also introduce new challenges to be solved at the same time. In this paper, we not only review various emerging NVMs but also study typical related work on these topics to investigate their implications for memory architecture design.

This review article provides a comprehensive analysis of the structural and technological aspects of integrating various types of non-volatile memory into CMOS VLSI technology. It examines the fundamental operating principles and structural characteristics of the most widely used non-volatile memory architectures, including FLASH, EEPROM, SONOS, FeRAM, MRAM, and resistive memory (ReRAM, PCM). The paper also presents a detailed breakdown of the submicron CMOS VLSI fabrication process, distinguishing between the FEOL and BEOL stages. Based on an extensive review of current scientific and technical literature, the study identifies key challenges and technological constraints in embedding each type of memory, such as thermal incompatibility, material limitations, the need for additional photolithographic steps, and modifications to existing fabrication processes. The article systematizes modern strategies for overcoming these barriers, including low-temperature deposition techniques, optimized thermal processing, barrier layer implementation, and specialized architectural solutions. These findings contribute to the development of new fabrication approaches and the refinement of existing processes for integrating non-volatile memory into advanced semiconductor devices.

Polysilicon (poly-Si) grain size control is a critical issue with scaling of MOS transistors in integrated-circuit design, more so in embedded nonvolatile memory (NVM) technology. This paper investigates an approach to suppress poly-Si grain growth under a necessary additional thermal budget for 40-nm embedded NVM technology. Our studies reveal that carbon implant can suppress poly-Si grain size growth and that the implant dose rather than its energy plays a key role in controlling the grain size. Physical analysis using advanced planar transmission electron microscopy technique shows a reduction in the poly-Si grain size with an increasing carbon implant dose. The application of the carbon implant technique to sub-40 nm embedded NVM technology can therefore help to reduce SRAM ${V} _{{MIN}}$ -related failures significantly. In this paper, we present a complete process reliability case study to incorporate carbon implant as an effective solution for suppressing poly-Si grain growth, thereby eliminating the side effects of an additional thermal budget in advanced embedded NVM.

MariaDB is a community-developed fork of the MySQL relational database management system and originally designed and implemented to use traditional spinning disk architecture. Now that devices with Non-Volatile memory (NVM) technologies are available, MariaDB addresses this challenge by adding support for NVM devices and introduces NVM Compression method. NVM Compression is a novel hybrid technique that combines application level compression with flash awareness for optimal performance and storage efficiency. Utilizing new interface primitives exported by Flash Translation Layers (FTLs), we leverage the garbage collection available in flash devices to optimize the capacity management required by compression systems. We implement NVM Compression in the popular MariaDB database and use variants of commonly available POSIX file system interfaces to provide the extended FTL capabilities to the user space application. The experimental results show that the hybrid approach of NVM Compression can improve compression performance by 2-7x, deliver compression performance for flash devices that is within 5% of uncompressed performance, improves storage efficiency by 19% over legacy Row-Compression, reduce data writes by up to 4x when combined with other flash aware techniques such as Atomic Writes, and deliver further advantages in power efficiency and CPU utilization.

Emerging non-volatile memory (NVM) technologies promise persistence, byte-addressability, low power consumption and high density. The potential of NVM implies that it can improve persistence performance and availability for high-performance computing tasks in data centers. However, NVM suffers from limited write endurance; skewed writes can extremely curtail the lifetime of NVM. Although several wear-leveling algorithms have been proposed to address the wear-out problem, based on estimation of write counts, they come with limitations in accuracy and compatibility with existing hardware and softwares. In this paper, we propose several software-based ways of monitoring NVM writes and obtaining the write counts, which are more accurate and non-intrusive for existing programs. We compare our proposed monitoring approaches in the aspects of granularity and portability; we conduct extensive experiments to evaluate them and the results show that our proposed methods can obtain good accuracy with an affordable time overhead.

Non-Volatile Memory (NVM) technologies exhibit 4X the read access latency of conventional DRAM. When the working set does not fit in the processor cache, this latency gap between DRAM and NVM leads to more than 2X runtime increase for queries dominated by latency-bound operations such as index joins and tuple reconstruction. We explain how to easily hide NVM latency by interleaving the execution of parallel work in index joins and tuple reconstruction using coroutines. Our evaluation shows that interleaving applied to the non-trivial implementations of these two operations in a production-grade codebase accelerates end-to-end query runtimes on both NVM and DRAM by up to 1.7X and 2.6X respectively, thereby reducing the performance difference between DRAM and NVM by more than 60%.