Efficient Cache Resizing policy for DRAM-based LLCs in ChipMultiprocessors

The performance of modern multicore processes largely depends on the performance of their shared Last Level Cache (LLC). A larger LLC can reduce misses while increasing access latency. Tile-based chipmultiprocessors (TCMP) are also proposed with two levels of shared caches to handle such situations. Among the two levels of shared caches, the bottom level is designed with DRAM technology for better capacity, and the upper level is designed with SRAM technology for better latency. Because it is in the last level, the LLC is the DRAM-based shared cache. Both the shared caches are normally divided into multiple banks. The core in such TCMPs can have its own private cache above these two levels of shared caches. Process variation has been observed to have a much greater impact on DRAM-based memory chips than on SRAM-based memories. Hence, the banks of the DRAM-based LLC may not behave uniformly. Some of the banks may experience high energy costs and access latency because of PV. In this work, we have proposed a replacement policy for the SRAM-based shared cache (upper-level shared cache), which reduces the requirements to access the PV-affected LLC banks significantly. This is achieved by relaxing the blocks of the affected LLC banks to remain in the upper-level shared cache (SRAM-based) for a longer time. The experimental analysis found that the proposed replacement policy improves performance by 13%, 10%, and 6% as compared to three existing replacement policies.

Last-Level Cache (LLC) plays an important role in Chip Multi-Processor (CMP). The objective of this work is to optimize the structure and management strategy of LLC. Based on 8-core CMP, a LLC structure based on grouped cores is proposed, where 8 cores are divided into 4 groups. All LLC resources are classified into three types, which are fixed private cache, dynamic private cache and dynamic shared cache. The layout of the LLC structure and the corresponding dynamic partitioning strategy are designed to achieve low access latency and high efficiency. Experimental results on full-system simulator suggest that the proposed structure and method are able to reduce the access latency by 2% to 12% compared with previous works, such as tiled structure, cache-centered structure and core-centered structure. Consequently, performance measured by IPC is improved up to 7%. The contribution of this paper is useful for CMP performance, and applies to not only 8-core CMP but also all small-scale CMPs.

In recent years, features and applications of embedded systems have been increasing rapidly. Chip Multi-Processors (CMPs), have been used in these systems to meet the higher demand for performance and energy efficiency. In CMPs, the last level cache (LLC) and the memory bandwidth are usually shared by the cores. Despite the fact that CMPs improve performance of embedded systems, competition for the shared resources makes their performance unpredictable and suboptimal. In this paper, we propose PLSS: Phase-guided Locality Signature based Scheduler for arbitrating LLC requests in multi-core embedded processors. To achieve our goal, we perform phase-wise offline profiling to guide the runtime task scheduling scheme. Our approach can improve performance of dual core system by upto 11% over IPC based scheduler (5% on average) and 35% over LLC number-of-accesses based approach (6.5% on average).

Closely coupled CPU and GPGPU system with the shared last level cache (LLC) enables fine-grained data exchange. However, traditional data exchange causes unnecessary LLC misses and degrades the entire system performance. We believe that the cache organization is not suitable for the temporary data exchange in the closely coupled system. We analyze the memory access pattern and discover the inefficiency data exchange. When the exchanged data cannot fit in the LLC, the low LLC hit rate exacerbates core stalls and memory contention. We also show that the stalls cannot be entirely covered by increasing the compute load or parallelism. In previous work, a simple LLC buffer is introduced to replace the cache with an architecture-supported data queue. However, the simple design limits the data element size and requires a potentially very large storage for pending requests. In this paper, we propose an improved LLC buffer. It adopts element-atom data organization to enable data exchange of arbitrary size. A simple hardware-software collaborated protocol is adopted, and eliminates the pending requests. The experiment results reveal that it has an average speedup of 48.2% compared with the traditional way, but incurs a 7.5% slowdown compared with the simple LLC buffer due to the overhead of the protocol. We also compare it with the fine-grain task, which implements a data exchange channel between CPU and GPGPU. The results show that the improved LLC buffer has less storage overhead but higher access efficiency than the fine-grain task.

In tiled Chip Multiprocessors (CMPs) the banks of the built-in last level cache (LLC) are usually distributed among the tiles and logically shared. A static mapping of cache blocks to the LLC banks leads to poor efficiency since a block can be mapped to a bank far away from the tiles which actually access it. Partially dynamic policies have been proposed, which however rely on the static mapping of blocks to a set of banks (D-NUCA) or rely on the OS to dynamically load pages to statically mapped addresses (first-touch).We propose a new dynamic approach where the LLC home bank is determined at runtime in hardware, with the memory controller in charge to perform the block mapping when fetched from main memory. To speed up the home bank lookup process, we use simple and lightweight NoC optimizations. When compared with alternative solutions (S-NUCA, D-NUCA, first touch, private LLCs) results with PARSEC and SPLASH-2 applications indicate improvement in locality of LLC blocks in the same tile (56.2% from 5.8%) and more than 33% reduction in load and store miss latencies. This leads to an average reduction of 24% in application’s execution time compared to static mapping.KeywordsCache LineBroadcast MessageMemory ControllerCache BlockLast Level CacheThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Nowadays, embedded device applications have become partially correlated and can share platform resources. Cross-execution and sharing resources can cause memory access conflicts, especially in the Last Level Cache (LLC). LLC is a promising candidate for improving system performance on multicore embedded systems. It leads to a reduction in the number of high-latency main memory accesses. Currently, commercial devices can use cache partitioning. The software could better utilize the LLC and conserve energy by caching. This paper proposes a new energy-optimization model for embedded multicore systems based on a reconfigurable artificial neural network LLC architecture. The proposed model uses a machine-learning approach to express the reconfiguration of LLC, and can predict each task’s next interval LLC partitioning factor at runtime. The obtained experimental results reveal that the proposed model - compared to other algorithms - improves energy consumption by 28%, and gives 33% reduction in the LLC miss rate.

RESTRAIN: A dynamic and cost-efficient resource management scheme for addressing performance interference in NFV-based systems

Recent years have seen a large volume of proposals on managing the shared last-level cache (LLC) of chip-multiprocessors (CMPs). However, most of these proposals primarily focus on reducing the amount of destructive interference between competing independent threads of multi-programmed workloads. While very few of these studies evaluate the proposed policies on shared memory multi-threaded applications, they do not improve constructive cross-thread sharing of data in the LLC In this paper, we characterize a set of multi-threaded applications drawn from the PARSEC, SPEC OMP, and SPLASH-2 suites with the goal of introducing sharing-awareness in LLC replacement policies. We motivate our characterization study by quantifying the potential contributions of the shared and the private blocks toward the overall volume of the LLC hits in these applications and show that the shared blocks are more important than the private blocks. Next, we characterize the amount of sharing-awareness enjoyed by recent proposals compared to the optimal policy. We design and evaluate a generic oracle that can be used in conjunction with any existing policy to quantify the potential improvement that can come from introducing sharing-awareness. The oracle analysis shows that introducing sharing-awareness reduces the number of LLC misses incurred by the least-recently-used (LRU) policy by 6% and 10% on average for a 4MB and 8MB LLC respectively. A realistic implementation of this oracle requires the LLC controller to have the capability to accurately predict, at the time a block is filled into the LLC, whether the block will be shared during its residency in the LLC. We explore the feasibility of designing such a predictor based on the address of the fill and the program counter of the instruction that triggers the fill. Our sharing behavior predictability study of two history-based fill-time predictors that use block addresses and program counters concludes that achieving acceptable levels of accuracy with such predictors will require other architectural and/or high-level program semantic features that have strong correlations with active sharing phases of the LLC blocks.

Racetrack memory (RM, also known as DWM (domain wall memory)) is an emerging memory technology that has many advantages such as low power, high density, and low access latency. Recent studies have shown that it is promising to architect RM as last level cache (LLC). Given that a RM track consists of m domains (for storing m bits data) and n access heads (1≤n<m), one RM access often requires multiple hops of shift to move the access head above the domain to be accessed. This leads to variant access latency, which may be exploited to initiate covert channel attacks to leak sensitive information in secure computing environment. In this paper, we elaborate the feasibility of such attacks and propose secure head management policies to effectively mitigate the attacks in RM LLCs. Our experimental results show that the proposed schemes can reduce the new discovered shift-based covert channel's capacity by up to 260 times with modest performance overhead.

The size of the Last Level Caches (LLC) in multi-core architectures is increasing, and so is their power consumption. However, most of this power is wasted on unused or invalid cache lines. For dirty cache lines, the LLC waits until the line is evicted to be written back to memory. Hence, dirty lines compete for the memory bandwidth with read requests (prefetch and demand), increasing pressure on the memory controller. This paper proposes a Dead Line and Early Write-Back Predictor (DEWP) to improve the energy efficiency of the LLC. DEWP early evicts dead cache lines with an average accuracy of 94%, and only 2% false positives. DEWP also allows scheduling of dirty lines for early eviction, allowing earlier write-backs. Using DEWP over a set of single and multi-threaded benchmarks, we obtain an average of 61% static energy savings, while maintaining the performance, for both inclusive and non-inclusive LLCs.

As the size of on-chip SRAM caches is increasing rapidly and the physical dimension of the SRAM devices is decreasing, reliability of caches is becoming a growing concern. This is because with increased size of caches, the likelihood of radiation-induced soft faults also increases. As a result, information redundancy in the form of Error Correcting Codes (ECC) is becoming extremely important, especially to protect the larger sized last level caches (LLCs). In typical ECCs, extra redundancy bits are added to every row to detect and correct errors. There is additional encoding (while writing data) and decoding (while reading data) procedures required as well. In caches, these additional area, power and latency overheads need to be minimized as much as possible. To address this problem, we present in this paper Parity++: a novel unequal message protection scheme for last level caches that preferentially provides stronger error protection to certain "special messages". This protection scheme provides Single Error Detection (SED) for all messages and Single Error Correction (SEC) for a subset of messages. Thus, it is stronger than just a basic SED parity and has much lower parity storage overhead (4X lower for a 64-bit memory) and lower error detection energy than a traditional Single Error Correcting, Double Error Detecting (SECDED) code. We also evaluate Parity++ with a memory speculation procedure that can be used with any ECC scheme to hide the decoding latency while reading messages when there are no errors.

Improving the performance of embedded systems can be supported by increasing the hit rates of last level caches (LLC). To enhance the hit rates of LLC, we propose a new prefetch technique. The proposed prefetch technique can fetch the data from main memory prior to actual requests to reduce the long latency to the main memory. To support the proposed technique, we introduce a new structure, LLC buffer which contains several memory blocks nearby the previous referenced memory block. In case that the LLC capacity is not enough, the proposed prefetch technique can improve the performance of embedded systems significantly.

The number of cores in a single chip in the nodes of high-end computing systems is on rise, due, in part, to a number of constraints, such as power consumption. With this, the size of the last level cache (LLC) has also increased significantly. Since LLCs built with SRAM consume high leakage power, power consumption of LLCs is becoming a significant fraction of processor power consumption. To address this issue, researchers have used embedded DRAM (eDRAM) LLCs which consume low leakage power. However, eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we propose ESTEEM, an energy saving technique for embedded DRAM caches. ESTEEM uses dynamic cache reconfiguration to turn off a portion of the cache to save both leakage and refresh energy. It logically divides the cache sets into multiple modules and turns off possibly different number of ways in each module. Microarchitectural simulations confirm that ESTEEM is effective in improving performance and energy efficiency and provides better results compared to a recently-proposed eDRAM cache energy saving technique, namely Refrint. For single and dual-core simulations, the average energy saving in memory subsystem (LLC+main memory) with ESTEEM is 25.8% and 32.6% respectively, and the average weighted speedup is 1.09x and 1.22x respectively. Additional experiments confirm that ESTEEM works well for a wide-range of system and algorithm parameters.

The end of Dennard scaling has shifted the focus of performance enhancement in technology to power budgeting techniques, specifically in the nano-meter domain because, leakage power depletes the total chip budget. Therefore, to meet the power budget, the number of resources per die could be limited. With this emerging factor, power consumption of on-chip components is detrimental to the future of transistor scaling. Fortunately, earlier research has identified the Last Level Cache (LLC) as one of the major power consuming element. Consequently, there have been several efforts towards reducing power consumption in LLCs. This paper presents a survey of recent contribution towards reducing power consumption in the LLC.

Ideally, we could solve all memory performance problems by storing everything in high performance caches. However, the associated high-cost forces us to keep only a limited amount of working sets of data in these caches and the replacement of disposable blocks of data with the desired one is a fundamental property to caches. Ultimately, the performance of a cache is heavily influenced by the replacement policy it uses. In this paper, we conduct a comparative analysis of a few replacement policies currently in use from traditional strategies to the ones that attempt to emulate optimal replacement and simulate a few replacement strategies to test the performance of multilevel cache by configuring the cache memory subsystem in the Multi2sim simulator. We have focused on the shared nature of the last Level Cache (LLC) which includes many coherence problems. This is where directories and coherence protocols come into play. However, the blocks suffer when the directory entries are themselves evicted from the directory. In this paper, we look into a cache coherence mechanism that instead of evicting the block from the private caches when the directory entry is evicted, rather than storing the entry in the LLC slice. This guarantees freedom from the Directory Eviction Victim Blocks (DEVB). We analyze the performance of our proposed replacement technique, namely, coalesce private or spill shared (CPOSS) by varying the size and associativity of the sparse directory by providing the workload from Parsec, Splash2 and FFTW benchmarks and we observe that the CPOSS outperforms the existing least recently used (LRU) in slice LLC banks in Multicore processors.