GPUs Cache Performance Estimation using Reuse Distance Analysis

Reuse distance (RD) analysis is a powerful memory analysis tool that can potentially help architects study multicore processor scaling. One key obstacle though is multicore RD analysis requires measuring concurrent reuse distance (CRD) profiles across thread-interleaved memory reference streams. Sensitivity to memory interleaving makes CRD profiles architecture dependent, preventing them from analyzing different processor configurations. For loop-based parallel programs, CRD profiles shift coherently to larger CRD values with core count scaling because interleaving threads are symmetric. Simple techniques can predict such shifting, making the analysis of numerous multicore configurations from a small set of CRD profiles feasible. Given the ubiquity and scalability of loop-level parallelism, such techniques will be extremely valuable for studying future large multicore designs. This paper investigates using RD analysis to efficiently analyze multicore cache performance for loop-based parallel programs, making several contributions. First, we provide in depth analysis on how CRD profiles change with core count scaling. Second, we develop techniques to predict CRD profile scaling, in particular employing reference groups to predict coherent shift, and evaluate prediction accuracy. Third, we show core count scaling only degrades performance for last level caches (LLCs) below 16MB for our benchmarks and problem sizes, increasing to 64 -- 128MB if problem size scales by 64x. Finally, we apply CRD profiles to analyze multicore cache performance. When combined with existing problem scaling prediction, our techniques can predict LLC MPKI to within 11.1% of simulation across 1,728 configurations using only 36 measured CRD profiles.

In high performance computing (HPC) applications, the speed of the L1 cache will typically determine the maximum frequency (/Max) of the processor core. Companies that mass produce high-performance microprocessors commonly have the L1 cache consist of fully-custom macros: to ensure that the performance of the L1 cache does not limit the f <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MAX</inf> or throughput of the processor. In addition, it is also common for the custom L1 cache designs to use a two-port 8T or a large 6T bitcell, along with domino read logic and very short BL [2,3]. These designs tradeoff density and area for high performance. This paper presents a different approach, one which can satisfy a range of different applications; a memory compiler that can generate more than 10,000 different high-speed L1 cache macro configurations is proposed. The 7nm L1-cache compiler described in this paper uses a high-current (HC) 6T bitcell, which is more area efficient than an 8T bitcell. The HC bitcell, along with small-signal sensing, allows for long BL (256b), leading to further area efficiency improvements. Since these L1 macros are just as likely to be used in mobile applications as they are to be used in HPC applications, they were implemented using the array dual-rail (ADR) architecture [4]. The ADR architecture (Fig. 11.3.1) allows the periphery circuits of the L1 macro to operate at the same voltage as the processor core: a lower l/ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DD</inf> results in dynamic power savings. ADR performance is also improved, over an interface dual-rail, when the SRAM and logic supplies are equivalent, as ADR design does not suffer from a level-shifter delays on the inputs or outputs.

Due to the ever increasing performance gap between the processor and the main memory, it becomes crucial to bridge that gap by designing an efficient memory hierarchy capable of reducing the average memory access time. The cache replacement algorithm plays a central role in designing an efficient memory hierarchy. Many of the recent studies in cache replacement algorithms have focused on improving L2 cache replacement algorithms by minimizing the miss count. However, depending on the dependency chain, cache miss bursts, and other factors, a processor's ability to partially hide the cost of an L2 cache miss varies; that is, cache miss costs are not uniform. Therefore, a better solution would account also for the aggregate miss cost in designing cache replacement algorithms. Our proposed solution combines the two principles of locality and cost-sensitivity into one which we call: LACS: Locality-Aware Cost-Sensitive cache replacement algorithm. LACS estimates a cache block's cost from the number of instructions the processor manages to issue during a cache miss on that block and then victimizes cache blocks with low cost and poor locality in order to maximize the overall cache performance. When LACS is evaluated using a uniprocessor architecture model, it speeds up 10 L2 cache performance-constrained SPEC CPU2000 benchmarks by up to 85% and 15% on average while not slowing down any of the 20 SPEC CPU2000 benchmarks evaluated. When evaluated using a dual-core CMP architecture model, LACS speeds up 6 SPEC CPU2000 benchmark pairs by up to 44% and 11% on average.

Memory trace analysis is an important technology for architecture research, system software (i.e., OS, compiler) optimization, and application performance improvements. Many approaches have been used to track memory trace, such as simulation, binary instrumentation and hardware snooping. However, they usually have limitations of time, accuracy and capacity.In this paper we propose a platform independent memory trace monitoring system, which is able to track virtual memory reference trace of full systems (including OS, VMMs, libraries, and applications). The system adopts a DIMM-snooping mechanism that uses hardware boards plugged in DIMM slots to snoop. There are several advantages in this approach, such as fast, complete, undistorted, and portable. Three key techniques are proposed to address the system design challenges with this mechanism: (1) To keep up with memory speeds, the DDR protocol state machine is simplified, and large FIFOs are added between the state machine and the trace transmitting logic to handle burst memory accesses; (2) To reconstruct physical-tovirtual mapping and distinguish one process' address space from others, an OS kernel module, which collects page table information, and a synchronization mechanism, which synchronizes the page table information with the memory race, are developed; (3) To dump massive trace data, we employ a straightforward method to compress the trace and use Gigabit Ethernet and RAID to send and receive the compressed trace.We present our implementation of an initial monitoring system, named HMTT (Hyper Memory Trace Tracker). Using HMTT, we have observed that burst bandwidth utilization is much larger than average bandwidth utilization, by up to 5X in desktop applications. We have also confirmed that the stream memory accesses of many applications contribute even more than 40% of L2 Cache misses and OS virtual memory management may decrease stream accesses in view of memory controller (or L2 Cache), by up to 30.2%. Moreover, we have evaluated OS impact on memory performance in real systems. The evaluations and case studies show the feasibility and effectiveness of our proposed monitoring mechanism and techniques.

Recent studies have shown that, in highly associative caches, the performance gap between the least recently used (LRU) and the theoretical optimal replacement algorithms is large, motivating the design of alternative replacement algorithms to improve cache performance. In LRU replacement, a line, after its last use, remains in the cache for a long time until it becomes the LRU line. Such deadlines unnecessarily reduce the cache capacity available for other lines. In addition, in multilevel caches, temporal reuse patterns are often inverted, showing in the L1 cache but, due to the filtering effect of the L1 cache, not showing in the L2 cache. At the L2, these lines appear to be brought in the cache but are never reaccessed until they are replaced. These lines unnecessarily pollute the L2 cache. This paper proposes a new counter-based approach to deal with the above problems. For the former problem, we predict lines that have become dead and replace them early from the L2 cache. For the latter problem, we identify never-reaccessed lines, bypass the L2 cache, and place them directly in the L1 cache. Both techniques are achieved through a single counter-based mechanism. In our approach, each line in the L2 cache is augmented with an event counter that is incremented when an event of interest such as certain cache accesses occurs. When the counter reaches a threshold, the line ";expires"; and becomes replaceable. Each line's threshold is unique and is dynamically learned. We propose and evaluate two new replacement algorithms: Access interval predictor (AIP) and live-time predictor (LvP). AIP and LvP speed up 10 capacity-constrained SPEC2000 benchmarks by up to 48 percent and 15 percent on average (7 percent on average for the whole 21 Spec2000 benchmarks). Cache bypassing further reduces L2 cache pollution and improves the average speedups to 17 percent (8 percent for the whole 21 Spec2000 benchmarks).

Growing complexity of applications pose new challenges to memory system design due to their data intensive nature, complex access patterns, larger footprints, etc. The slow nature of full-system simulators, challenges of simulators to run deep software stacks of many emerging workloads, proprietary nature of software, etc. pose challenges to fast and accurate microarchitectural explorations of future memory hierarchies. One technique to mitigate this problem is to create spatio-temporal models of access streams and use them to explore memory system tradeoffs. However, existing memory stream models have weaknesses such as they only model temporal locality behavior or model spatio-temporal locality using global stride transitions, resulting in high storage/metadata overhead. In this paper, we propose HALO, a Hierarchical memory Access LOcality modeling technique that identifies patterns by isolating global memory references into localized streams and further zooming into each local stream capturing multi-granularity spatial locality patterns. HALO also models the interleaving degree between localized stream accesses leveraging coarse-grained reuse locality. We evaluate HALO's effectiveness in replicating original application performance using over 20K different memory system configurations and show that HALO achieves over 98.3%, 95.6%, 99.3% and 96% accuracy in replicating performance of prefetcher-enabled L1 & L2 caches, TLB and DRAM respectively. HALO outperforms the state-of-the-art memory cloning schemes, WEST and STM, while using ~39X less metadata storage than STM.

The probability of timing failure in SRAM accessing becomes unacceptably high at low voltages, which makes the SRAM become the bottleneck of the system performance. Recently proposed timing speculation SRAM (SSRAM) can access bit cells much earlier than the conservative time margin and detect the potential timing failures. Unfortunately, the performance benefit brought by this speculative scheme might be nullified by the extra cycles used to correct the errors. Therefore, in a cache architecture, cachelines containing bit cells with timing failure bring significant performance degradation in a low-voltage scenario. In this paper, we propose the set-associative L1 and L2 Remapping and Reuse-aware timing Speculation caches (RRS caches) based on the cross-sensing SSRAM to reduce the weak cacheline access penalties under low supply voltages. RRS caches improve the proportion of the error-free cachelines (or strong cachelines) by clustering as more as possible bit cells with timing failures into weak cachelines. In addition, RRS caches swap the frequently reused data to the error-free cachelines to further reduce the average access latency and the energy consumption by an optimized filling/replacement policy. We compared the performance, energy consumption and area overhead of RRS caches with those of state-of-the-art approaches. According to our simulation results, RRS caches improve the system performance close to that of the Perfect caches in a four-core processor with only 0.12% extra energy consumption, 4.12% extra area overhead in the L1 cache and 2.69% extra energy consumption, 5.66% extra area overhead in the L2 cache compared to the cache designs with the raw SSRAM.

&lt;p&gt;The primary objective of this study is to develop new computational methods for robust design optimization (RDO) and reliability-based design optimization (RBDO) of high-dimensional, complex engineering systems. Four major research directions, all anchored in polynomial dimensional decomposition (PDD), have been defined to meet the objective. They involve: (1) development of new sensitivity analysis methods for RDO and RBDO; (2) development of novel optimization methods for solving RDO problems; (3) development of novel optimization methods for solving RBDO problems; and (4) development of a novel scheme and formulation to solve stochastic design optimization problems with both distributional and structural design parameters.&lt;/p&gt; &lt;p&gt;The major achievements are as follows. Firstly, three new computational methods were developed for calculating design sensitivities of statistical moments and reliability of high-dimensional complex systems subject to random inputs. The first method represents a novel integration of PDD of a multivariate stochastic response function and score functions, leading to analytical expressions of design sensitivities of the first two moments. The second and third methods, relevant to probability distribution or reliability analysis, exploit two distinct combinations built on PDD: the PDD-SPA method, entailing the saddlepoint approximation (SPA) and score functions; and the PDD-MCS method, utilizing the embedded Monte Carlo simulation (MCS) of the PDD approximation and score functions. For all three methods developed, both the statistical moments or failure probabilities and their design sensitivities are both determined concurrently from a single stochastic analysis or simulation. Secondly, four new methods were developed for RDO of complex engineering systems. The methods involve PDD of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms. The methods, depending on how statistical moment and sensitivity analyses are dovetailed with an optimization algorithm, encompass direct, single-step, sequential, and multi-point single-step design processes. Thirdly, two new methods were developed for RBDO of complex engineering systems. The methods involve an adaptive-sparse polynomial dimensional decomposition (AS-PDD) of a high-dimensional stochastic response for reliability analysis, a novel integration of AS-PDD and score functions for calculating the sensitivities of the failure probability with respect to design variables, and standard gradient-based optimization algorithms, resulting in a multi-point, single-step design process. The two methods, depending on how the failure probability and its design sensitivities are evaluated, exploit two distinct combinations built on AS-PDD: the AS-PDD-SPA method, entailing SPA and score functions; and the AS-PDD-MCS method, utilizing the embedded MCS of the AS-PDD approximation and score functions. In addition, a new method, named as the augmented PDD method, was developed for RDO and RBDO subject to mixed design variables, comprising both distributional and structural design variables. The method comprises a new augmented PDD of a high-dimensional stochastic response for statistical moment and reliability analyses; an integration of the augmented PDD, score functions, and finite-difference approximation for calculating the sensitivities of the first two moments and the failure probability with respect to distributional and structural design variables; and standard gradient-based optimization algorithms, leading to a multi-point, single-step design process.&lt;/p&gt; &lt;p&gt;The innovative formulations of statistical moment and reliability analysis, design sensitivity analysis, and optimization algorithms have achieved not only highly accurate but also computationally efficient design solutions. Therefore, these new methods are capable of performing industrial-scale design optimization with numerous design variables.&lt;/p&gt;

Several new formulations for structural design sensitivity analysis (DSA) of elastic solids are presented that consistently account for steady and transient thermal loading, along with centrifugal and gravity loading. The approach employs implicit differentiation of the discretized boundary integral equations and facilitates the production of accurate and efficient computational procedures that obviate the need for matrix factorizations in the DSA step. For centrifugal, gravity and general thermal loading, a new DSA concept employing a particular integral technique is developed. For centrifugal, gravity and steady thermal effects, a surface integration approach is also presented, and the two methods compared. Alternative strategies for modelling the behaviour of temperature distributions of evolving shapes undergoing shape optimization and the ramifications of these strategies in the DSA process are also discussed. It is shown that, for certain classes of thermoelastic problems, shape optimization can be performed without the need to incorporate thermal analysis or DSA into the overall process. A thermoelastic surface stress sensitivity analysis formulation is presented that does not require additional numerical integration. A new thermoelastic sharp corner traction component consistent sensitivity analysis formulation is also presented. This technique is shown to work for any initial stress distribution. Numerical sensitivity results are presented for many problems with analytical solutions.

Accuracy enhancement of fatigue damage counting using design sensitivity analysis

Three methods for design sensitivity such as numerical differentiation, analytical method and semianalytical method have been developed for the last three decades. Although analytical design sensitivity analysis is exact, it is hard to implement for practical design problems. Therefore, numerical method such as finite difference method is widely used to simply obtain the design sensitivity in most cases. The numerical differentiation is sufficiently accurate and reliable for most linear problems. However, it turns out that the numerical differentiation is inefficient and inaccurate because its computational cost depends on the number of design variables and large numerical errors can be included especially in nonlinear design sensitivity analysis. Thus semi-analytical method is more suitable for complicated design problems. Moreover semianalytical method is easy to be performed in design procedure, which can be coupled with an analysis solver such as commercial finite element package. In this paper, implementation procedure for the semi-analytical design sensitivity analysis outside of the commercial finite element package is studied and computational technique is proposed, which evaluates the pseudo-load for design sensitivity analysis easily by using the design variation of corresponding internal nodal forces. Errors in semi-analytical design sensitivity analysis are examined and numerical examples are illustrated to confirm the reduction of numerical error considerably.

Improved semi-analytic sensitivity analysis combined with a higher order approximation scheme in the framework of adjoint variable method

Modern computing systems typically have multiple users sharing hardware resources. While such shared hardware have typically been performance boosters, they have also led to inadvertent side-effects such as side channels. Caches, that present the largest attack surface, have been popular among adversaries for side channel attacks. In this work, we repurpose a classic cache performance metric namely, reuse distance, to capture the activity of an adversary in cache timing channels. We design Reuse-trap, an efficient cache side channel mitigation framework to record reuse distances during victim accesses and carefully inject noise to mislead the spy from inferring the victim’s activity. Our experimental results show that we can identify adversaries with zero false positives and make timing channels suffer from over 50% bit error rate on average.

The one- and two-dimensional inviscid Euler equations are formulated in an integro-differential form for shape design sensitivity analysis and optimization. The principal tool employed to derive the performance derivative sensitivity equations is the variational method, which is a continuous alternative to the discrete sensitivity analysis. Along with the sensitivity equations, the co-state equations and their boundary conditions are derived. Thereafter, based on the modal analysis of Von Neumann theory, the stability limits of the co-state equations are investigated. The stability characteristics of the co-state equations are compared with those of the Euler equations. Finally, using the criteria from the stability analysis of the state and co-state equations, some results of shape design optimization and sensitivity analysis for both quasi one- and two-dimensional Euler equations are presented. Copyright © 1999 John Wiley & Sons, Ltd.

Technological advances along with more complex and dynamic application behavior argue for revisiting mechanisms that adapt logical cache block size to application characteristics. This approach to bridging the processor/memory performance gap has been studied in the past, but most studies used trace-driven simulation and only looked at L1 caches. Given the changes in hardware and software since these seminal studies, we revisit the general approach: we present a transparent, phase-adaptive mechanism for L2 cache block superloading with minimal hardware complexity, evaluating it on a full-system simulator running 23 SPEC CPU2000 applications run to completion using training inputs.