Energy Efficient Last Level Caches via Last Read/Write Prediction

Efficient Cache Resizing policy for DRAM-based LLCs in ChipMultiprocessors

Closely coupled CPU and GPGPU system with the shared last level cache (LLC) enables fine-grained data exchange. However, traditional data exchange causes unnecessary LLC misses and degrades the entire system performance. We believe that the cache organization is not suitable for the temporary data exchange in the closely coupled system. We analyze the memory access pattern and discover the inefficiency data exchange. When the exchanged data cannot fit in the LLC, the low LLC hit rate exacerbates core stalls and memory contention. We also show that the stalls cannot be entirely covered by increasing the compute load or parallelism. In previous work, a simple LLC buffer is introduced to replace the cache with an architecture-supported data queue. However, the simple design limits the data element size and requires a potentially very large storage for pending requests. In this paper, we propose an improved LLC buffer. It adopts element-atom data organization to enable data exchange of arbitrary size. A simple hardware-software collaborated protocol is adopted, and eliminates the pending requests. The experiment results reveal that it has an average speedup of 48.2% compared with the traditional way, but incurs a 7.5% slowdown compared with the simple LLC buffer due to the overhead of the protocol. We also compare it with the fine-grain task, which implements a data exchange channel between CPU and GPGPU. The results show that the improved LLC buffer has less storage overhead but higher access efficiency than the fine-grain task.

In tiled Chip Multiprocessors (CMPs) the banks of the built-in last level cache (LLC) are usually distributed among the tiles and logically shared. A static mapping of cache blocks to the LLC banks leads to poor efficiency since a block can be mapped to a bank far away from the tiles which actually access it. Partially dynamic policies have been proposed, which however rely on the static mapping of blocks to a set of banks (D-NUCA) or rely on the OS to dynamically load pages to statically mapped addresses (first-touch).We propose a new dynamic approach where the LLC home bank is determined at runtime in hardware, with the memory controller in charge to perform the block mapping when fetched from main memory. To speed up the home bank lookup process, we use simple and lightweight NoC optimizations. When compared with alternative solutions (S-NUCA, D-NUCA, first touch, private LLCs) results with PARSEC and SPLASH-2 applications indicate improvement in locality of LLC blocks in the same tile (56.2% from 5.8%) and more than 33% reduction in load and store miss latencies. This leads to an average reduction of 24% in application’s execution time compared to static mapping.KeywordsCache LineBroadcast MessageMemory ControllerCache BlockLast Level CacheThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The number of cores in a single chip in the nodes of high-end computing systems is on rise, due, in part, to a number of constraints, such as power consumption. With this, the size of the last level cache (LLC) has also increased significantly. Since LLCs built with SRAM consume high leakage power, power consumption of LLCs is becoming a significant fraction of processor power consumption. To address this issue, researchers have used embedded DRAM (eDRAM) LLCs which consume low leakage power. However, eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we propose ESTEEM, an energy saving technique for embedded DRAM caches. ESTEEM uses dynamic cache reconfiguration to turn off a portion of the cache to save both leakage and refresh energy. It logically divides the cache sets into multiple modules and turns off possibly different number of ways in each module. Microarchitectural simulations confirm that ESTEEM is effective in improving performance and energy efficiency and provides better results compared to a recently-proposed eDRAM cache energy saving technique, namely Refrint. For single and dual-core simulations, the average energy saving in memory subsystem (LLC+main memory) with ESTEEM is 25.8% and 32.6% respectively, and the average weighted speedup is 1.09x and 1.22x respectively. Additional experiments confirm that ESTEEM works well for a wide-range of system and algorithm parameters.

RESTRAIN: A dynamic and cost-efficient resource management scheme for addressing performance interference in NFV-based systems

Nowadays, embedded device applications have become partially correlated and can share platform resources. Cross-execution and sharing resources can cause memory access conflicts, especially in the Last Level Cache (LLC). LLC is a promising candidate for improving system performance on multicore embedded systems. It leads to a reduction in the number of high-latency main memory accesses. Currently, commercial devices can use cache partitioning. The software could better utilize the LLC and conserve energy by caching. This paper proposes a new energy-optimization model for embedded multicore systems based on a reconfigurable artificial neural network LLC architecture. The proposed model uses a machine-learning approach to express the reconfiguration of LLC, and can predict each task’s next interval LLC partitioning factor at runtime. The obtained experimental results reveal that the proposed model - compared to other algorithms - improves energy consumption by 28%, and gives 33% reduction in the LLC miss rate.

As the size of on-chip SRAM caches is increasing rapidly and the physical dimension of the SRAM devices is decreasing, reliability of caches is becoming a growing concern. This is because with increased size of caches, the likelihood of radiation-induced soft faults also increases. As a result, information redundancy in the form of Error Correcting Codes (ECC) is becoming extremely important, especially to protect the larger sized last level caches (LLCs). In typical ECCs, extra redundancy bits are added to every row to detect and correct errors. There is additional encoding (while writing data) and decoding (while reading data) procedures required as well. In caches, these additional area, power and latency overheads need to be minimized as much as possible. To address this problem, we present in this paper Parity++: a novel unequal message protection scheme for last level caches that preferentially provides stronger error protection to certain "special messages". This protection scheme provides Single Error Detection (SED) for all messages and Single Error Correction (SEC) for a subset of messages. Thus, it is stronger than just a basic SED parity and has much lower parity storage overhead (4X lower for a 64-bit memory) and lower error detection energy than a traditional Single Error Correcting, Double Error Detecting (SECDED) code. We also evaluate Parity++ with a memory speculation procedure that can be used with any ECC scheme to hide the decoding latency while reading messages when there are no errors.

In recent years, features and applications of embedded systems have been increasing rapidly. Chip Multi-Processors (CMPs), have been used in these systems to meet the higher demand for performance and energy efficiency. In CMPs, the last level cache (LLC) and the memory bandwidth are usually shared by the cores. Despite the fact that CMPs improve performance of embedded systems, competition for the shared resources makes their performance unpredictable and suboptimal. In this paper, we propose PLSS: Phase-guided Locality Signature based Scheduler for arbitrating LLC requests in multi-core embedded processors. To achieve our goal, we perform phase-wise offline profiling to guide the runtime task scheduling scheme. Our approach can improve performance of dual core system by upto 11% over IPC based scheduler (5% on average) and 35% over LLC number-of-accesses based approach (6.5% on average).

ZeroCost-LLC (ZCLLC) is a shared inclusive lastlevel cache (LLC) architecture for predictable multicore platforms that does not incur additional cost to the worst-case latency (WCL) of memory requests when compared to the memory hierarchy without an LLC. Thus, the WCL remains the same as without an LLC in the memory hierarchy, but with the performance benefits of having an LLC, in the form of additional caching capacity. ZCLLC achieves this by eliminating all cache line invalidations, and proactively updating the main memory with cache lines to preserve an important vacancy invariant. Furthermore, ZCLLC does not impose any constraints on the way the LLC is used unlike other approaches such as LLC partitioning. Our analysis reveals that the WCL is 55.6%, 68.0%, and 80.2% lower, and the performance is 2.4%, 7.2%, and 25.6% better than the state-of-the-art LLC partition sharing techniques for 2, 4, and 8 cores, respectively.

The end of Dennard scaling has shifted the focus of performance enhancement in technology to power budgeting techniques, specifically in the nano-meter domain because, leakage power depletes the total chip budget. Therefore, to meet the power budget, the number of resources per die could be limited. With this emerging factor, power consumption of on-chip components is detrimental to the future of transistor scaling. Fortunately, earlier research has identified the Last Level Cache (LLC) as one of the major power consuming element. Consequently, there have been several efforts towards reducing power consumption in LLCs. This paper presents a survey of recent contribution towards reducing power consumption in the LLC.

Ideally, we could solve all memory performance problems by storing everything in high performance caches. However, the associated high-cost forces us to keep only a limited amount of working sets of data in these caches and the replacement of disposable blocks of data with the desired one is a fundamental property to caches. Ultimately, the performance of a cache is heavily influenced by the replacement policy it uses. In this paper, we conduct a comparative analysis of a few replacement policies currently in use from traditional strategies to the ones that attempt to emulate optimal replacement and simulate a few replacement strategies to test the performance of multilevel cache by configuring the cache memory subsystem in the Multi2sim simulator. We have focused on the shared nature of the last Level Cache (LLC) which includes many coherence problems. This is where directories and coherence protocols come into play. However, the blocks suffer when the directory entries are themselves evicted from the directory. In this paper, we look into a cache coherence mechanism that instead of evicting the block from the private caches when the directory entry is evicted, rather than storing the entry in the LLC slice. This guarantees freedom from the Directory Eviction Victim Blocks (DEVB). We analyze the performance of our proposed replacement technique, namely, coalesce private or spill shared (CPOSS) by varying the size and associativity of the sparse directory by providing the workload from Parsec, Splash2 and FFTW benchmarks and we observe that the CPOSS outperforms the existing least recently used (LRU) in slice LLC banks in Multicore processors.

The performance of modern multicore processes largely depends on the performance of their shared Last Level Cache (LLC). A larger LLC can reduce misses while increasing access latency. Tile-based chipmultiprocessors (TCMP) are also proposed with two levels of shared caches to handle such situations. Among the two levels of shared caches, the bottom level is designed with DRAM technology for better capacity, and the upper level is designed with SRAM technology for better latency. Because it is in the last level, the LLC is the DRAM-based shared cache. Both the shared caches are normally divided into multiple banks. The core in such TCMPs can have its own private cache above these two levels of shared caches. Process variation has been observed to have a much greater impact on DRAM-based memory chips than on SRAM-based memories. Hence, the banks of the DRAM-based LLC may not behave uniformly. Some of the banks may experience high energy costs and access latency because of PV. In this work, we have proposed a replacement policy for the SRAM-based shared cache (upper-level shared cache), which reduces the requirements to access the PV-affected LLC banks significantly. This is achieved by relaxing the blocks of the affected LLC banks to remain in the upper-level shared cache (SRAM-based) for a longer time. The experimental analysis found that the proposed replacement policy improves performance by 13%, 10%, and 6% as compared to three existing replacement policies.

Improving the performance of embedded systems can be supported by increasing the hit rates of last level caches (LLC). To enhance the hit rates of LLC, we propose a new prefetch technique. The proposed prefetch technique can fetch the data from main memory prior to actual requests to reduce the long latency to the main memory. To support the proposed technique, we introduce a new structure, LLC buffer which contains several memory blocks nearby the previous referenced memory block. In case that the LLC capacity is not enough, the proposed prefetch technique can improve the performance of embedded systems significantly.

Multi-core architectures are shaking the fundamental assumption that in real-time systems the WCET, used to analyze the schedulability of the complete system, is calculated on individual tasks. This is not even true in an approximate sense in a modern multi-core chip, due to interference caused by hardware resource sharing. In this work we propose (1) a complete framework to analyze and profile task memory access patterns and (2) a novel kernel-level cache management technique to enforce an efficient and deterministic cache allocation of the most frequently accessed memory areas. In this way, we provide a powerful tool to address one of the main sources of interference in a system where the last level of cache is shared among two or more CPUs. The technique has been implemented on commercial hardware and our evaluations show that it can be used to significantly improve the predictability of a given set of critical tasks.

Dynamic cache partitioning for shared Last Level Caches (LLC) is deployed in most modern multicore systems to achieve process isolation and fairness among the applications and avoid security threats. Since LLC has visibility of all cache blocks requested by several applications running on a multicore system, a malicious application can potentially threaten the system that can leverage the dynamic partitioning schemes applied to the LLCs by creating a timing-based covert channel attack. We call it as Cache Partitioned Covert Channel (CPCC) attack. The malicious applications may contain a trojan and a spy and use the underlying shared memory to create the attack. Through this attack, secret pieces of information like encryption keys or any secret information can be transmitted between the intended parties. We have observed that CPCC can target single or multiple cache sets to achieve a higher transmission rate with a maximum error rate of 5% only. The paper also addresses a few defense strategies that can avoid such cache partitioning based covert channel attacks.