Characterizing multi-threaded applications for designing sharing-aware last-level cache replacement policies

Efficient Cache Resizing policy for DRAM-based LLCs in ChipMultiprocessors

Ideally, we could solve all memory performance problems by storing everything in high performance caches. However, the associated high-cost forces us to keep only a limited amount of working sets of data in these caches and the replacement of disposable blocks of data with the desired one is a fundamental property to caches. Ultimately, the performance of a cache is heavily influenced by the replacement policy it uses. In this paper, we conduct a comparative analysis of a few replacement policies currently in use from traditional strategies to the ones that attempt to emulate optimal replacement and simulate a few replacement strategies to test the performance of multilevel cache by configuring the cache memory subsystem in the Multi2sim simulator. We have focused on the shared nature of the last Level Cache (LLC) which includes many coherence problems. This is where directories and coherence protocols come into play. However, the blocks suffer when the directory entries are themselves evicted from the directory. In this paper, we look into a cache coherence mechanism that instead of evicting the block from the private caches when the directory entry is evicted, rather than storing the entry in the LLC slice. This guarantees freedom from the Directory Eviction Victim Blocks (DEVB). We analyze the performance of our proposed replacement technique, namely, coalesce private or spill shared (CPOSS) by varying the size and associativity of the sparse directory by providing the workload from Parsec, Splash2 and FFTW benchmarks and we observe that the CPOSS outperforms the existing least recently used (LRU) in slice LLC banks in Multicore processors.

Last-Level Cache (LLC) plays an important role in Chip Multi-Processor (CMP). The objective of this work is to optimize the structure and management strategy of LLC. Based on 8-core CMP, a LLC structure based on grouped cores is proposed, where 8 cores are divided into 4 groups. All LLC resources are classified into three types, which are fixed private cache, dynamic private cache and dynamic shared cache. The layout of the LLC structure and the corresponding dynamic partitioning strategy are designed to achieve low access latency and high efficiency. Experimental results on full-system simulator suggest that the proposed structure and method are able to reduce the access latency by 2% to 12% compared with previous works, such as tiled structure, cache-centered structure and core-centered structure. Consequently, performance measured by IPC is improved up to 7%. The contribution of this paper is useful for CMP performance, and applies to not only 8-core CMP but also all small-scale CMPs.

The performance of modern multicore processes largely depends on the performance of their shared Last Level Cache (LLC). A larger LLC can reduce misses while increasing access latency. Tile-based chipmultiprocessors (TCMP) are also proposed with two levels of shared caches to handle such situations. Among the two levels of shared caches, the bottom level is designed with DRAM technology for better capacity, and the upper level is designed with SRAM technology for better latency. Because it is in the last level, the LLC is the DRAM-based shared cache. Both the shared caches are normally divided into multiple banks. The core in such TCMPs can have its own private cache above these two levels of shared caches. Process variation has been observed to have a much greater impact on DRAM-based memory chips than on SRAM-based memories. Hence, the banks of the DRAM-based LLC may not behave uniformly. Some of the banks may experience high energy costs and access latency because of PV. In this work, we have proposed a replacement policy for the SRAM-based shared cache (upper-level shared cache), which reduces the requirements to access the PV-affected LLC banks significantly. This is achieved by relaxing the blocks of the affected LLC banks to remain in the upper-level shared cache (SRAM-based) for a longer time. The experimental analysis found that the proposed replacement policy improves performance by 13%, 10%, and 6% as compared to three existing replacement policies.

In recent years, features and applications of embedded systems have been increasing rapidly. Chip Multi-Processors (CMPs), have been used in these systems to meet the higher demand for performance and energy efficiency. In CMPs, the last level cache (LLC) and the memory bandwidth are usually shared by the cores. Despite the fact that CMPs improve performance of embedded systems, competition for the shared resources makes their performance unpredictable and suboptimal. In this paper, we propose PLSS: Phase-guided Locality Signature based Scheduler for arbitrating LLC requests in multi-core embedded processors. To achieve our goal, we perform phase-wise offline profiling to guide the runtime task scheduling scheme. Our approach can improve performance of dual core system by upto 11% over IPC based scheduler (5% on average) and 35% over LLC number-of-accesses based approach (6.5% on average).

In tiled Chip Multiprocessors (CMPs) the banks of the built-in last level cache (LLC) are usually distributed among the tiles and logically shared. A static mapping of cache blocks to the LLC banks leads to poor efficiency since a block can be mapped to a bank far away from the tiles which actually access it. Partially dynamic policies have been proposed, which however rely on the static mapping of blocks to a set of banks (D-NUCA) or rely on the OS to dynamically load pages to statically mapped addresses (first-touch).We propose a new dynamic approach where the LLC home bank is determined at runtime in hardware, with the memory controller in charge to perform the block mapping when fetched from main memory. To speed up the home bank lookup process, we use simple and lightweight NoC optimizations. When compared with alternative solutions (S-NUCA, D-NUCA, first touch, private LLCs) results with PARSEC and SPLASH-2 applications indicate improvement in locality of LLC blocks in the same tile (56.2% from 5.8%) and more than 33% reduction in load and store miss latencies. This leads to an average reduction of 24% in application’s execution time compared to static mapping.KeywordsCache LineBroadcast MessageMemory ControllerCache BlockLast Level CacheThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The disparity between last-level cache and memory latencies motivates the search for efficient cache management policies. Recent work in predicting reuse of cache blocks enables optimizations that significantly improve cache performance and efficiency. However, the accuracy of the prediction mechanisms limits the scope of optimization. This paper proposes perceptron learning for reuse prediction. The proposed predictor greatly improves accuracy over previous work. For multi- programmed workloads, the average false positive rate of the proposed predictor is 3.2%, while sampling dead block prediction (SDBP) and signature-based hit prediction (SHiP) yield false positive rates above 7%. The improvement in accuracy translates directly into performance. For single-thread workloads and a 4MB lastlevel cache, reuse prediction with perceptron learning enables a replacement and bypass optimization to achieve a geometric mean speedup of 6.1%, compared with 3.8% for SHiP and 3.5% for SDBP on the SPEC CPU 2006 benchmarks. On a memory-intensive subset of SPEC, perceptron learning yields 18.3% speedup, versus 10.5% for SHiP and 7.7% for SDBP. For multi- programmed workloads and a 16MB cache, the proposed technique doubles the efficiency of the cache over LRU and yields a geometric mean normalized weighted speedup of 7.4%, compared with 4.4% for SHiP and 4.2% for SDBP.

Inclusive cache hierarchies are widely adopted in modern processors, since they can simplify the implementation of cache coherence. However, it sacrifices some performance to guarantee inclusion. Many recent intelligent management policies are proposed to improve the last-level cache (LLC) performance by evicting blocks with poor locality earlier. Unfortunately, they are inapplicable in inclusive LLCs. In this paper, we propose Two-level Eviction Priority (TEP) policy. Besides the eviction priority provided by the baseline replacement policy, TEP appends an additional high level of eviction priority to LLC blocks, which is decided at the insertion time and cannot be changed during their lifetime in the LLC. When blocks with high eviction priority are not in inner caches anymore, they get evicted from the LLC preferentially. Thus, the LLC can retain more useful blocks to improve performance. TEP can cooperate well with various baseline replacement policies. Our evaluation shows that TEP with NRU can improve the performance of inclusive LLCs significantly while requiring negligible extra storage. It also outperforms other recent proposals including QBS, DIP, and DRRIP.

Caches are used to bridge the gap between main memory and the significantly faster processor cores. In multi-core architectures, the last-level cache is often shared between cores. However, sharing a cache causes inter-core interference to emerge. Concurrently running tasks will experience additional cache misses as the competing tasks issue interfering accesses and trigger the eviction of data contained in the shared cache. Thus, to compute a task’s worst-case execution time (WCET), a safe bound on the effects of inter-core cache interference has to be determined. In this paper, we propose a novel analysis approach for shared caches using the least recently used (LRU) replacement policy. The presented analysis leverages timing information to produce tight bounds on the worst-case interference. We describe how inter-core cache interference may be expressed as a function of time using event-arrival curves. Thus, by determining the maximal duration between subsequent accesses to a cache block, it is possible to bound the inter-core interference. This enables us to classify accesses as cache hits or potential misses. We implemented the analysis in a WCET analyzer and evaluated its performance for multi-core systems containing 2, 4, and 8 cores using shared caches from 4 KB to 32 KB. The analysis achieves significant improvements compared to a standard interference analysis with WCET reductions of up to 60%. The average WCET reduction is 9% for dual-core, 15% for quad-core, and 11% for octa-core systems. The analysis runtime overhead ranges from a factor of 4 × to 7 × compared to the baseline analysis.

Secure hybrid replacement policy: Mitigating conflict-based cache side channel attacks

Hardware data prefetch is a very well known technique for hiding memory latencies. However, in a multicore system fitted with a shared Last-Level Cache (LLC), prefetch induced by a core consumes common resources such as shared cache space and main memory bandwidth. This may degrade the performance of other cores and even the overall system performance unless the prefetch aggressiveness of each core is controlled from a system standpoint. On the other hand, LLCs in commercial chip multiprocessors are more and more frequently organized in independent banks. In this contribution, we target for the first time prefetch in a banked LLC organization and propose ABS, a low-cost controller with a hill-climbing approach that runs stand-alone at each LLC bank without requiring inter-bank communication. Using multiprogrammed SPEC2K6 workloads, our analysis shows that the mechanism improves both user-oriented metrics (Harmonic Mean of Speedups by 27% and Fairness by 11%) and system-oriented metrics (Weighted Speedup increases 22% and Memory Bandwidth Consumption decreases 14%) over an eight-core baseline system that uses aggressive sequential prefetch with a fixed degree. Similar conclusions can be drawn by varying the number of cores or the LLC size, when running parallel applications, or when other prefetch engines are controlled.

With the recent advent of many-core architectures such as chip multiprocessors (CMPs), the number of processing units accessing a global shared memory is constantly increasing. Co-scheduling techniques are used to improve application throughput on such architectures, but sharing resources often generates critical interferences. In this article, we focus on the interferences in the last level of cache (LLC) and use the Cache Allocation Technology (CAT) recently provided by Intel to partition the LLC and give each co-scheduled application their own cache area. We consider m iterative HPC applications running concurrently and answer to the following questions: (i) How to precisely model the behavior of these applications on the cache-partitioned platform? and (ii) how many cores and cache fractions should be assigned to each application to maximize the platform efficiency? Here, platform efficiency is defined as maximizing the performance either globally, or as guaranteeing a fixed ratio of iterations per second for each application. Through extensive experiments using CAT, we demonstrate the impact of cache partitioning when multiple HPC applications are co-scheduled onto CMP platforms.

Employing an on-chip network in a manycore system (to improve scalability) makes the latencies of data accesses issued by a core non-uniform, which significant impact application performance. This paper presents a compiler strategy which involves exposing architecture information to the compiler to enable optimized computation-to-core mapping. Our scheme takes into account the relative positions of (and distances between) cores, last-level caches (LLCs) and memory controllers (MCs) in a manycore system, and generates a mapping of computations to cores with the goal of minimizing the on-chip network traffic. Our experiments of 12 multi-threaded applications reveal that, on average, our approach reduces the on-chip network latency in a 6x6 manycore system by 49.5% in the case of private LLCs and 52.7% in the case of shared LLCs. These improvements translate to the corresponding execution time improvements of 14.8% and 15.2% for the private LLC and shared LLC based systems.

The gap between processor and memory speeds is one of the greatest challenges that current designers face in order to develop more powerful computer systems. In addition, the scalability of the Dynamic Random Access Memory (DRAM) technology is very limited nowadays, leading one to consider new memory technologies as candidates for the replacement of conventional DRAM. Phase-Change Memory (PCM) is currently postulated as the prime contender due to its higher scalability and lower leakage. However, compared with DRAM, PCM also exhibits some drawbacks, like lower endurance or higher dynamic energy consumption and write latency, that need to be mitigated before it can be used as the main memory technology for the next generation of computers. This work addresses the PCM endurance constraint. For this purpose, we present an analysis of conventional cache replacement policies in terms of the amount of writebacks to main memory that they imply and we also propose some new replacement algorithms for the last-level cache (LLC) with the goal of cutting down the write traffic to memory and consequently, to increase PCM lifetime without degrading system performance. In this paper, we target general purpose processors provided with this kind of non-volatile main memory and we exhaustively evaluate our proposed policies in both single- and multi-core environments. Experimental results show that, on average, compared with a conventional Least Recently Used (LRU) algorithm, some of our proposals manage to reduce the amount of writes to main memory up to 20–30% depending on the scenario evaluated, which leads to memory endurance extensions of up to 20–45%, also reducing the energy consumption in the memory hierarchy by up to 9% and hardly degrading performance.

Recent studies on secure last-level cache (LLC) have advocated the fully associative organization to defend against conflict-based side-channel attacks. In a fully associative LLC, an attacker cannot extract any information about the cache location of the addresses evicted due to cross-core conflict. Use of the random replacement policy further guards against any deterministic eviction patterns. However, the fully associative LLC design remains vulnerable against timing-based covert channel attacks. In this article, we present LeakyRand , a high-bandwidth covert communication mechanism that exploits the fully associative LLC with random replacement while guaranteeing an ultra-low bit error rate (BER). Our proposal is a union of three unique contributions. First, we present an efficient algorithm with strong analytical guarantees that enables the attacker to quickly occupy nearly the whole LLC with high probability, a prerequisite for achieving high bandwidth and high fidelity. Second, we present a novel covert communication protocol that allows the attacker to maintain high LLC occupancy. Third, our proposal detects error syndromes and efficiently takes corrective measures leading to an ultra-low expected BER. Exploiting a 2 MB fully associative LLC with random replacement policy to set up a covert channel, LeakyRand experiences an average BER of 10 -4 or less while offering 3.3× to 5.7× higher channel bandwidth compared to the recently proposed Stochastic Prime+Probe attack. We also demonstrate that LeakyRand can be adopted to mount high-precision fine-grain fingerprinting attacks.