Evaluating Application Vulnerability to Soft Errors in Multi-level Cache Hierarchy

The complexity of multi-level cache hierarchies simulation is much more complicated than that of single-level cache simulation, so that it is highly desirable to develop some efficient simulation methods for multi-level cache hierarchies performance analysis. A one pass multi-level unified cache hierarchies simulation method is developed based on stack model and inclusion properties. Recently, many processors utilized split instruction and data caches in the first cache level. A one pass multi-level hybrid cache simulation method is proposed for more generalized hierarchies. These two one pass simulation methods have been implemented. The simulation results show the effectiveness of these two one pass simulation methods. >

Dead blocks are handled inefficiently in multi-level cache hierarchies because the decision as to whether a block is dead has to be taken locally at each cache level. This paper introduces runtime-assisted global cache management to quickly deem blocks dead across cache levels in the context of task-based parallel programs. The scheme is based on a cooperative hardware/software approach that leverages static and dynamic information about future data region reuse(s) available to runtime systems for task-based parallel programming models. We show that our proposed runtime-assisted global cache management approach outperforms previously proposed local dead-block management schemes for task-based parallel programs.

The amount of charge stored in an SRAM cell shrinks rapidly with each technology generation thus increasingly exposing caches to soft errors. Benchmarking the FIT rate of caches due to soft errors is critical to evaluate the relative merits of a plethora of protection schemes that are being proposed to protect against soft errors. The benchmarking of cache reliability introduces a unique challenge as compared to internal processor storage structures, such as the load/store queue. In the case of internal processor structures the time a data bit resides in the structure is so short that it is generally safe to assume that no more than one soft error strike can occur. Thus the reliability of such structures is overwhelmingly dominated by single bit errors. By contrast, a memory block may reside for millions of cycles in a last level cache. In this case it is important to consider the impact of the spatial and temporal distribution of multiple errors within the lifetime of a cache block in the presence of error protection.

Around 1970 Intel discovered it could put 2,000 transistorsor perhaps a few moreon a single NMOS chip. In retrospect, this may be said to mark the beginning of very large-scale integration (VLSI), ...

Two performance gaps in the memory hierarchy, between CPU cache and main memory, and main memory and mass storage, will become increasingly severe bottlenecks for computing-system performance. Although it is necessary to increase memory capacity to fill these gaps, power also increases when conventional volatile memories are used. A new nonvolatile memory for this purpose has been anticipated. Storage class memory is used to fill the second gap. Many candidates exist: ReRAM, PRAM, and 3D-cross point type with resistive change RAM. However, nonvolatile last level cache (LLC) is used to fill the first gap. Advanced STT-MRAM has achieved sub-4ns read and write accesses with perpendicular magnetic tunnel junctions (p-MTJ) [1–2]. Furthermore, mature integration processes have been developed and 8Mb STT-MRAM with sub-5ns operation has shown high reliability [3]. Moreover, because of its non-volatility, STT-MRAM can reduce operation energy by more than 81% compared to SRAM for cache [1]. This paper presents STT-MRAM-based last level cache memory (LLC) including MRAM memory core, peripherals and cache logic circuits, using novel power optimization with high-speed power gating (HS-PG),considering processor architectures and cache memory accesses. The STT-MRAM-based cache has high reliability to reduce the write-error rate with novel write-verify-write. Furthermore, a read-modify-write scheme is implemented to reduce active power without penalty. Figure 7.2.1 presents a block diagram of a 4Mb STT-MRAM based cache.

Soft error recovery in simplex and triplex memory systems

Prefetching and off-chip prediction are two techniques proposed to hide long memory access latencies in high-performance processors. In this work, we demonstrate that: (1) prefetching and off-chip prediction often provide complementary performance benefits, yet (2) naively combining these two mechanisms often fails to realize their full performance potential, and (3) existing prefetcher control policies (both heuristic- and learning-based) leave significant room for performance improvement behind. Our goal is to design a holistic framework that can autonomously learn to coordinate an off-chip predictor with multiple prefetchers employed at various cache levels, delivering consistent performance benefits across a wide range of workloads and system configurations. To this end, we propose a new technique called Athena, which models the coordination between prefetchers and off-chip predictor (OCP) as a reinforcement learning (RL) problem. Athena acts as the RL agent that observes multiple system-level features (e.g., prefetcher/OCP accuracy, bandwidth usage) over an epoch of program execution, and uses them as state information to select a coordination action (i.e., enabling the prefetcher and/or OCP, and adjusting prefetcher aggressiveness). At the end of every epoch, Athena receives a numerical reward that measures the change in multiple system-level metrics (e.g., number of cycles taken to execute an epoch). Athena uses this reward to autonomously and continuously learn a policy to coordinate prefetchers with OCP. Athena makes a key observation that using performance improvement as the sole RL reward, as used in prior work, is unreliable, as it confounds the effects of the agent's actions with inherent variations in workload behavior. To address this limitation, Athena introduces a composite reward framework that separates (1) system-level metrics directly influenced by Athena's actions (e.g., last-level cache misses) from (2) metrics primarily driven by workload phase changes (e.g., number of mispredicted branches). This allows Athena to autonomously learn a coordination policy by isolating the true impact of its actions from inherent variations in workload behavior. Our extensive evaluation using a diverse set of 100 memoryintensive workloads shows that Athena consistently outperforms multiple prior state-of-the-art coordination policies across a wide range of system configurations with various combinations of underlying prefetchers at various cache levels, OCPs, and main memory bandwidths, while incurring only modest storage overhead and design complexity. The source code of Athena is freely available at https://github.com/CMU-SAFARI/Athena.

By integrating multiple cores in a single chip, Chip Multiprocessors (CMP) provide an attractive approach to improve both system throughput and efficiency. This integration allows the sharing of on-chip resources which may lead to destructive interference between the executing workloads. Memorysubsystem is an important shared resource that contributes significantly to the overall throughput and power consumption. In order to prevent destructive interference, the cache capacity and memory bandwidth requirements of the last level cache have to be controlled. While previously proposed schemes focus on resource sharing within a chip, we explore additional possibilities both inside and outside a single chip. We propose a dynamic memory-subsystem resource management scheme that considers both cache capacity and memory bandwidth contention in large multi-chip CMP systems. Our approach uses low overhead, non-invasive resource profilers that are based on Mattson's stack distance algorithm to project each core's resource requirements and guide our cache partitioning algorithms. Our bandwidth-aware algorithm seeks for throughput optimizations among multiple chips by migrating workloads from the most resource-overcommitted chips to the ones with more available resources. Use of bandwidth as a criterion results in an overall 18% reduction in memory bandwidth along with a 7.9% reduction in miss rate, compared to existing resource management schemes. Using a cycle-accurate full system simulator, our approach achieved an average improvement of 8.5% on throughput.

Most modern microprocessors employ one or two levels of on-chip caches in order to improve performance. Caches typically are implemented with static RAM cells and often occupy a large portion of the chip area. Not surprisingly, these caches can consume a significant amount of power. In many applications, such as portable devices, energy efficiency is more important than performance. We propose sacrificing some performance in exchange for energy efficiency by filtering cache references through an unusually small first level cache. We refer to this structure as the filter cache. A second level cache, similar in size and structure to a conventional first level cache, is positioned behind the filter cache and serves to mitigate the performance loss. Extensive experiments indicate that a small filter cache still can achieve a high hit rate and good performance. This approach allows the second level cache to be in a low power mode most of the time, thus resulting in power savings. The filter cache is particularly attractive in low power applications, such as the embedded processors used for communication and multimedia applications. For example, experimental results across a wide range of embedded applications show that a direct mapped 255-byte filter cache achieves a 58 percent power reduction while reducing performance by 21 percent. This trade-off results in a 51 percent reduction in the energy-delay product when compared to a conventional design.

Spin-Transfer-Torque Magnetic Random Access Memory (STT-MRAM) is emerging as a leading candidate for a variety of embedded memory applications ranging from embedded NVM to working memory and last level cache. In this paper, we review recent breakthroughs that have brought perpendicular STT-MRAM to the cusp of mass production.

Transient errors caused by particle strikes have become a critical challenge for microprocessor design. Being the major consumer of on-chip real estate, cache memories are particularly susceptible to transient errors. However, not all cache soft errors can be propagated to the processor. For instance, soft errors can be corrected by write operations before they are read. In this paper, we define the cache vulnerability factor (CVF) to be the probability that a fault in the cache can be propagated to the processor or other memory hierarchy. We also propose an approach to compute the CVF based on the cache line access patterns. Building upon the CVF we evaluate the reliability for different cache memories. Our results show that 83.5% of soft errors from a write-through data cache can be masked without affecting other components. We also propose two early write-back strategies to improve the reliability (i.e., by reducing the CVF) of write-back data caches without compromising the high performance.

The cache hierarchy design in existing SMT and a superscalar processor is optimized for latency, but not for bandwidth. The size of the Level 1 (L1) data cache did not scale over the Past decade. Instead, larger unified Level 2 (L2) and Level 3 (L3) caches were introduced. The present paper is the part of the L3 cache. This paper describes last level cache memory decoding structure, which is designed mainly to improve performance. Functional verification as well as pre layout and post layout STA and ERC verifications are done on the 4MB cache. Results of each verification flow are presented. Based on the SDP methodology, good placement and routing, RC extraction, and noise analysis are achieved. SDP methodology is proven better than the RLS methodology for data path configurations. Cadence tools are used for performing the verification of SDP flows.

HP continues its development of low cost, high performance processors with an evolution of the PA7100LC which includes 128 kB of on-chip primary cache. It implements the full PA-RISC1.1 functionality including the little-endian, uncacheable memory, and multimedia extensions of the PA7100LC. The PA7300LC connects directly to an optional second level cache of 256 kB to 64 MB using plug-in cards. It also adds the ability to accelerate I/O stores to certain memory locations for greatly improved graphics performance. The cache system consists of on-chip, 2-way, separate instruction and data caches of 64 kB total each, plus the off-chip second level cache. Memory consists of 8 MB to 3.75 GB of standard DRAMs or SIMMs connecting directly to the processor chip, using either a 72 bit or 144 bit data path. The chip is fabricated in 0.5 micron, 4-level metal CMOS and is designed to run at frequencies up to 160 MHz. The PA7300LC exceeds performance levels of previous generation high-end workstations while lowering overall system cost and power consumption.

This article presents a framework for assessing the behavior and energy impact of cache hierarchies when encoding HEVC on general-purpose processors. The memory energy estimation framework estimates energy consumption of cache hierarchies based on mathematical models combined with memory access profiling tools. The energy analysis of several cache hierarchies targeting HEVC encoders with different input parameters is also carried out. This article provides relevant information on the energy consumption of HEVC encoders by taking into account the different tradeoffs between energy efficiency, coding efficiency, and other important cache memory design parameters, such as miss rates and access latency. The first analysis explores cache performance for different specifications, such as capacity, line size. Results show that most of the energy is spent on reading operations (almost 73% on the first level cache), indicating that HEVC encoders could benefit from memory technologies with low reading energy costs. This analysis also pointed that increasing the capacity affects more the energy performance of the first level cache, which represents 34.78% (on average) more energy consumption than the last level cache. Based on this investigation, we report the most suited cache specifications for HEVC encoders for each video resolution. The second analysis discusses the impact of HEVC input parameters in cache performance, demonstrating that it is possible to save up to 30% of energy with a small increase of 2% in BD-BR. A comparative analysis between HM (HEVC model) and x265 (H.265 video codec) HEVC software models is presented, demonstrating that x265 is faster (speedup to 648x), and more cache efficient providing less memory energy (31.38% on average) compared to the HM implementation. The results obtained with the proposed framework indicate that the management of video encoding parameters combined with application-tuned cache specifications has a high potential to reduce energy consumption of video coding systems while keeping video quality.

Although the use of Matrix-Multiplication is very extensive in research. In our paper it is being used for optimization in case of level one cache memory. Initially Open-MP (OMP) is used, followed by various optimizing techniques. Finally the results of Matrix-Multiplication are verified by the mathematical proof in terms of complexity.