Architecting On-Chip DRAM Cache for Simultaneous Miss Rate and Latency Reduction

Dynamic Random Access Memory (DRAM) has been prevalent over the past few decades as main memory component. The demand of higher memory capacity is increasing continuously, while scaling of DRAM is reaching its boundaries. As we scale DRAM to smaller feature size, difficulties in fabrication, leakage power, and energy consumption become significant. Therefore, memory technologies, which have better scalability feature, will be the future of memory systems. To overcome these limitations, search for a new memory technology is necessary and Phase Change Memory (PCM) is the most promising. PCM is non-volatile memory with better scalability and less leakage power than DRAM. It is a resistance-based memory, which doesn't need to be refreshed. PCM suffers more write latency and less write endurance. The write latency of PCM is higher than its read latency by almost 8 times. A large number of researches have been done on reducing write latency of PCM. This research will focus on studying different approaches and techniques in order to reduce write latency on various aspects. Then, techniques employed to reduce write latency of PCM such as PreSET, Partial-SET, Flip-N-Write, two-stage-write, and two-stage-write-inv are introduced. Conclusively, a performance comparison of each technique is discussed.

Stacked DRAM memories have become a reality in High-Performance Computing (HPC) architectures. These memories provide much higher bandwidth while consuming less power than traditional off-chip memories, but their limited memory capacity is insufficient for modern HPC systems. For this reason, both stacked DRAM and off-chip memories are expected to co-exist in HPC architectures, giving raise to different approaches for architecting the stacked DRAM in the system. This paper proposes a runtime approach to transparently manage stacked DRAM memories in task-based programming models. In this approach the runtime system is in charge of copying the data accessed by the tasks to the stacked DRAM, without any complex hardware support nor modifications to the application code. To mitigate the cost of copying data between the stacked DRAM and the off-chip memory, the proposal includes an optimization to parallelize the copies across idle or additional helper threads. In addition, the runtime system is aware of the reuse pattern of the data accessed by the tasks, and can exploit this information to avoid unworthy copies of data to the stacked DRAM. Results on the Intel Knights Landing processor show that the proposed techniques achieve an average speedup of 14% against the state-of-the-art library to manage the stacked DRAM and 29% against a stacked DRAM architected as a hardware cache.

Nearly pin-hole-free ferroelectric films of Pb(Zr0.6Ti0.4)O3 (PZT) with uniform compositions and thicknesses were synthesized using metallo-organic decomposition (MOD) technology. The metallo-organic precursors used for these PZT films were lead diethylhexanoate, (Pb(C7H15COO)2), titanium dimethoxy-dineodecanoate (Ti(OCH3)2 (C9 H19 COO)2), and zirconium tetraethylhexanoate (Zr(C7H15COO)4). The film thickness, structural development and electrical properties of these films were characterized using a surface profilometer, X-ray diffraction (XRD) and a standardized RT 66 ferroelectric test system. The effect of the processing conditions on the properties of ferroelectric 60/40 PZT films were investigated. The important processing parameters studied included the substrate history, the spinning speed and time, the film thickness, the pyrolysing temperature and the annealing conditions. The experimental results show that the structural development and electrical properties of these 60/40 PZT films are closely correlated with these processing parameters. The polarization-reversal characteristics and fatigue behaviour, which are important for non-volatile memory applications, were also studied. By optimizing the annealing conditions, a 60/40 PZT film exhibited a dielectric strength of 1 MVcm−1 and a sensed remanent polarization value of 9.3 ΜC cm2 after cycling 1010 times. This demonstrates the excellent potential of PZT films for non-volatile memory applications. The measured dielectric-constant values for these 60/40 PZT films were in the range of 300 to 900, which is about 75 to 220 times larger than that of the currently used dielectric, SiO2, in dynamic random-access memory (DRAM). This fact suggests that PZT film is an excellent alternative for the new generation, extremely high-density DRAM applications.

The eleven papers in this special section focus on both system- and circuit-level aspects of emerging memory and storage technologies. The overwhelmingly increasing demand for both storage and computation necessitates revisiting the traditional memory subsystems used in processors and storage systems to take advantage of emerging memory technologies. Dynamic Random Access Memory (DRAM) and Static Random Access Memory (SRAM) have been ubiquitously used for decades as main memory and as on-chip cache, respectively. Scaling and power issues of these traditional memory technologies have led to significant investment in emerging memory technologies. On the other hand, fundamental limitations of mechanical disk drives have brought great attention to further explore the design space of solid-state drives (SSDs). The promising features of emerging memory technologies such as low power consumption, increased performance, lower susceptibility to particle strikes, and higher bit density have prompted researchers to seek new organizations in the different memory-hierarchy levels, to propose new circuitry and algorithms to improve performance and reduce power consumption, and to develop new schemes to enhance system reliability.

Modern Internet routers have to handle a large number of packet classification rules, which requires classification schemes to be scalable both in time and space. In this paper, we present a scalable packet classification algorithm that is developed by combining two new concepts to the well‐known bit vector (BV) scheme. We propose a range search method based on a cache‐aware tree (CATree) which makes full use of processor's cache line to reduce the number of dynamic random access memory (DRAM) accesses. Theoretically, the number of DRAM accesses of CATree is about log(m+1) times lower than that of the widely used binary search algorithm, where m is the number of keys in a single cache line. Based on our computational results on a set of 1024 keys, the CATree algorithm is 36% faster than binary search algorithm and the performance is better when applied to a larger set of keys. In addition, we develop a rule re‐arrangement algorithm to reduce the bitmap space of BV. With this re‐arrangement, the rules for the same action may be assigned an identical priority. This reduces the number of priorities as well as the memory space of the bitmap. Furthermore, this also reduces the number of memory accesses and hence, increases the CPU cache utilization. With CATree and rule re‐arrangement, the cache‐aware bit vector with rule re‐arrangement algorithm achieves better performance in comparison with the regular BV scheme, both in space and time. In our experiments, the proposed algorithm reduces the bitmap memory space of a practical set of firewall rules by two orders of magnitude and is 91% faster than the regular BV.

The search for new dynamic RAM (DRAM) technologies to reduce memory access time and so unleash computer performance is discussed. The typical memory hierarchy of internal registers, cache, main memory, and mass storage is described. DRAM technologies that aim at simplifying this hierarchy by speeding up main memory to the point where the need for a separate, external cache is moot are examined. >

In embedded systems, off-chip buses and memory (i.e., L2 memory as opposed to the L1 memory which is usually on-chip cache) consume significant power, often more than the processor itself. In this paper, for the case of an embedded system with one processor chip and one memory chip, we propose frequency and voltage scaling of the off-chip buses and the memory chip and use a known micro-architectural enhancement called a store buffer to reduce the resulting impact on execution time. Our benchmarks show a system (processor + off-chip bus + off-chip memory) power savings of 28% to 36%, an energy savings of 13% to 35%, all while increasing the execution time in the range of 1% to 29%. Previous work in power-aware computing has focused on frequency and voltage scaling of the processors or selective power-down of sub-sets of off-chip memory chips. This paper quantitatively explores voltage/frequency scaling of off-chip buses and memory as a means of trading off performance for power/energy at the system level in embedded systems.

With the increasing number of cores in manycore accelerators and chip multiprocessors (CMPs), it gets more challenging to provide cache coherency efficiently. Although the snooping-based protocols are appropriate solutions to small-scale systems, they are inefficient for large systems because of the limited bandwidth. Therefore, large-scale manycores require directory-based solutions where a hardware structure called directory holds the information. This directory keeps track of all memory blocks and which cache stores a copy of these blocks. The directory sends messages only to caches that store relevant blocks and also coordinate simultaneous accesses to a cache block. As directory-based protocols scale to many cores, performance, network-on-chip (NoC) traffic, and bandwidth become major problems. In this paper, we present software mechanisms to improve the effectiveness of directory-based cache coherency in manycore and multicore systems with shared memory. In multithreaded applications, some of the data accesses do not disrupt cache coherency, but they still produce coherency messages among cores such as read-only (private) data. However, if data is accessed by at least two cores and at least one of them is a write operation, it is called shared data and requires cache coherency. In our proposed system, private data and shared data are determined at compile time, and cache coherency protocol only applies to shared data. We implement our approach in two stages. First, we use Andersen's static pointer analysis to analyze the program and mark its private instructions, i.e., instructions that load or store private data. Then, we use these analyses to decide if cache coherency protocol will be applied or not at runtime. Our simulation results on parallel benchmarks show that our approach reduces cycle count, dynamic random access memory (DRAM) accesses, and coherency traffic up to 13%.

This paper discusses the potential of an on-chip cache memory for modern vector supercomputers. The vector supercomputers can achieve the high computational efficiency for compute-intensive scientific applications. The most important factor affecting the computational performance is high memory bandwidth to provide a sufficient amount of data to the rich arithmetic units in time; the modern vector supercomputers such as NEC SX-7 and SX-8 have 4 bytes per flop (4B/FLOP) on the ratio of memory bandwidth to floating-point operations. However, the gap in performance between memory and processors has become remarkably exposed year by year in high performance computing. Therefore, it is getting harder to keep the 4B/FLOP memory bandwidth in design of future vector supercomputers. As a promising solution to cover a lack of the memory bandwidths of vector load/store units of the future vector supercomputers, we design an on-chip vector cache for the NEC SX vector processor architecture. This paper evaluates the performance of the on-chip cache memory system on the SX-7 system with 2B/FLOP or lower memory bandwidth across two kernel loops and five leading scientific applications. The results of the kernel loops demonstrate that a 2B/FLOP memory system with the on-chip cache whose hit ratio is 50% can achieve a performance comparable to that of a 4B/FLOP system without the cache. The results of the four applications indicate that the on-chip cache can improve sustained performance of the four applications by 20% to 98%. The experimental results regarding the last one show a conflicting effect of loop unrolling with vector caching, resulting in a poor hit rate. However, when loop-unrolling is disabled, its cache hit rate is improved, and the sustained performance comparable to that of the 4B/FLOP memory bandwidth without the loop-unrolling is obtained. In addition, selective caching, in which only a part of data with the high locality of reference are cached, is also effective for efficient use of the limited cache capacity.

Off-chip memory, such as DRAM, its access energy cost is orders of magnitude higher than other operations such as multiply and accumulate, thereby dominating the system energy consumption. Therefore, optimizing the access of the off-chip memory is crucial to further improve the energy efficiency of the deep neural network (DNN) accelerator. Towards this, this brief proposed an adaptive scheduling algorithm to minimize the DRAM access. Compared with the previous works, it can not only dynamically determine the data partition and the data type that will be reused, but also considered the constraints between adjacent layer, that is, if the output feature map of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i$ </tex-math></inline-formula> th layer is divided into N parts, the output feature map of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i+1$ </tex-math></inline-formula> th layer can only be divided into N parts or write back to the off-chip memory. Therefore, a minimize and realizable memory access solution can be obtained. Choosing 3 popular networks UNet, VGG-16 and MobileNet as benchmarks, the experiment results show that our scheduling algorithm can achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$34.46\% \sim 93.42\%$ </tex-math></inline-formula> reduction in energy consumption of DRAM access and a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$34.34\% \sim 93.37\%$ </tex-math></inline-formula> reduction in DRAM access latency when compared to previous works.

The performance of General Purpose Graphics Processing Units (GPGPUs) is frequently limited by the off-chip memory bandwidth. To mitigate this bandwidth wall problem, recent GPUs are equipped with on-chip L1 and L2 caches. However, there has been little work for better utilizing on-chip shared caches in GPGPUs. In this paper, we propose two cache management schemes: write-buffering and read-bypassing. The write buffering technique tries to utilize the shared cache for inter-block communication, and thereby reduces the DRAM accesses as much as the capacity of the cache. The read-bypassing scheme prevents the shared cache from being polluted by streamed data that are consumed only within a thread-block. The proposed schemes can be selectively applied to global memory instructions using newly defined cache operators. We evaluate the effects of the proposed schemes for a few GPGPU applications by simulations. We have shown that the off-chip memory accesses can be successfully reduced by the proposed techniques. We also analyze the effectiveness of these methods when the throughput gap between cores and off-chip memory becomes wider.

The performance of computing systems has been improved significantly for several decades. However, increasing the throughput of recent CPUs (Central Processing Units) is restricted by power consumption and thermal issues. GPUs (Graphics Processing Units) are recognized as efficient computing platform with powerful hardware resources to support CPUs in computing systems. Unlike CPUs, there is a large number of CUDA (Compute Unified Device Architecture) cores in GPUs, hence, some cache blocks are referenced many times repeatedly. If those cache blocks reside in the cache for long time, hit rates can be improved. On the other hand, many cache blocks are referenced only once and never referenced again in the cache. These blocks waste cache memory space, resulting in reduced GPU performance. Conventional LRU replacement policy cannot consider the problems from non-reused cache blocks and frequently-reused cache blocks. In this paper, a new cache replacement policy based on the reuse pattern of cache blocks is proposed. The proposed cache replacement policy manages cache blocks by separating reused cache blocks and thrashing cache blocks. According to simulation results, the proposed cache reuse replacement policy can increase IPC by up to 4.4% compared to the conventional GPU architecture.

With the availability of advanced MPSoC and emerging Dynamic RAM (DRAM) interface technologies, an optimal allocation of logical data buffers to physical memory cannot be handled manually anymore due to the huge design space. An allocation does not only need to decide between an on-or off-chip memory, but also needs to take an increasing number of available memory channels, different bandwidth capacities and several routing possibilities into account. We formalize this problem and introduce a Mixed Integer Linear Programming (MILP) model based on two different optimization criteria. We implement the MILP model into a retargetable tool and present a case study with representative data of the Long-Term-Evolution (LTE) standard to show the real-life applicability of our approach.

Performances of DRAM memories are characterized by memory latency and bandwidth. Contemporary DRAM memories more successfully satisfy demands for higher bandwidth than lower latency. In this paper solutions, which may reduce latency of these memories, are investigated. These solutions are two new controller policies called 'Write-miss Only Close-Page' and 'Write-miss Only Close-Page-Open previous Page' as well as several address remapping schemes. 'Write-miss Only Close-Page' policy is basically a combination of the policies 'Open-Page' and 'Close- Page-Autoprecharge'. For all DRAM reads 'Open-Page' policy is used. Also for all DRAM writes that cause row-buffer hits 'Open-Page' policy is used. For all DRAM writes that cause row-buffer misses 'Close- Page-Autoprecharge' policy is used. 'Write-miss Only Close-Page-Open previous Page' policy is the same as 'Writemiss Only Close-Page', except that after the precharge the previously open row is opened again. Simulations show improvements in using these combined policies. Permutation-based Page Interleaving scheme is known as an effective address remapping scheme for reducing row-buffer conflicts, which are consequence of conflict cache memory misses. This scheme is based on using xor circuits for changing bank indices of data blocks that fit into the same cache memory line set. We improve this scheme by proposing five similar schemes, with slightly better effectiveness. Three of the proposed schemes have approximately the same performances, but do not use xor circuits at all. Two of the proposed schemes use xor circuits but have slightly better performances.

Work on an open standard that defines a ring for linking fast memories and processors is discussed. The interface, called RamLink, will provide high performance even when dynamic RAMs (DRAMs) with internal caching mechanisms are ineffective. In addition to RAM, RamLink will support flash nonvolatile memory, read-only memory, and even memory-mapped disks. Mismatching, collision avoidance, and problems that must be addressed by RamLink are examined. A detailed simulation model for RamLink written in Simula, an object-oriented language, is briefly described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>