Improving memory utilization in cache coherence directories

Several mechanisms have been proposed for maintaining cache coherence in large-scale shared memory multiprocessors. Two important factors that distinguish these coherence mechanisms are: 1) the coherence enforcement strategy, such as updating or invalidating, that is used to prevent access to stale data; and 2) the coherence detection strategy, which is used to detect the existence of incoherent cached copies either at compile-time or at run-time. This paper examines the range of performance that can be realized by a compiler-directed coherence mechanism using three different levels of compiler ability. This performance is compared to the performance of the updating and the invalidating directory schemes. In addition, the performance of all three coherence schemes is compared with respect to different cache block sizes.

Directory-based and software-assisted schemes are the two main approaches to solving the cache coherence problem in large scale shared-memory multiprocessors. Until now, the emphasis in software-assisted schemes has been on ascertaining consistency within parallel constructs such as DoAll or DoAcross loops. In this paper, we propose a timestamped-based approach which also allows caching within critical sections. This scheme combines the best features of software-assisted and directory-based cache coherence protocols. It is based on a compile time analysis of interactions among critical sections and execution time local detection of cache incoherence by comparing the times when a variable was last written and last updated in the cache. A quantitative evaluation based on synthetic traces shows that this scheme results in hit ratios almost as high as in directory-based solutions while significantly reducing the network traffic.

Recently multicore technology has enabled development of hundreds or thousands core processor on a single chip. However, on such multicore processor, cache coherence hardware will become very complex, hot and expensive. This paper proposes a parallelizing compiler directed software coherence scheme for shared memory multicore systems without hardware cache coherence control. The general idea of the proposed method is that an automatic parallelizing compiler parallelize coarse grain task, analyzes stale data and line sharing in the program, then solves those problems by simple program restructuring and data synchronization. The proposed method is a simple and efficient software cache coherent control scheme built on OSCAR automatic parallelizing compiler and evaluated on Renesas RP2 with 8 SH-4A cores processor. The cache coherence hardware on the RP2 processor is only available for up to 4 cores. The cache coherence hardware can also be turned off for non-coherence cache mode. Performance evaluation was performed using 10 benchmark programs from SPEC2000, SPEC2006, NAS Parallel Benchmark (NPB) and MediaBench II. The proposed method performed as good as or better than hardware cache coherence scheme while still provided correct result as the hardware coherent mechanism. For example, the proposed software cache coherent control (NCC) gave us 2.63 times speedup for SPEC 2000 equake with 4 cores against sequential execution while got only 2.52 times speedup for 4 cores MESI hardware coherent control. Also, the software coherence control gave us 4.37 speed up for 8 cores with no hardware coherent mechanism available.

Cache coherence enforcement and memory latency reduction and hiding are very important and challenging problems in the design of large-scale distributed shared-memory (DSM) multiprocessors. We propose an integrated framework to solve these problems through a compiler-directed cache coherence scheme called the Cache Coherence with Data Prefetching (CCDP) scheme. The CCDP scheme enforces cache coherence by prefetching the potentially stale references in a parallel program. It also prefetches the nonstale references to hide their memory latencies. To optimize the performance of the CCDP scheme, some prefetch hardware support is provided to efficiently handle these two forms of data prefetching operations. We also developed the compiler techniques utilized by the CCDP scheme for stale reference detection, prefetch target analysis and prefetch scheduling. We evaluated the performance of the CCDP scheme via execution-driven simulations of several applications from the SPEC CFP95 and the Perfect benchmark suites. The simulation results show that the CCDP scheme provides significant performance improvements for the applications studied.

Directory-based protocols are currently the method of choice to enforce cache coherence in large-scale shared-memory multiprocessors. The problems associated with these hardware schemes include their lack of scalability, although various suggestions have been made to ameliorate this drawback, and the loss of performance due to false sharing. Software controlled cache coherence (SCCC) is an alternative that solves the scalability problem at the possible expense of a serious loss in performance since many compile-time decisions on cacheability are conservative. The authors perform a comparative study of hardware and software cache coherence based on instruction-level simulations of two benchmarks. The studies do not shy away from realistic SCCC difficulties such as multi-word lines and DOAcross loops. The simplification in hardware and in protocols result in a slight performance loss for SCCC, a loss that becomes relatively smaller and negligible with larger memory latencies. >

The problem of cache coherence in large-scale shared-memory multiprocessors has been addressed using directory-schemes. Two problems arise when the number of processors increases: the network latency increases and the implementation cost must be kept acceptable. The authors present a tree-based cache coherence protocol called the scalable tree protocol (STP). They show that it can be implemented at a reasonable implementation cost and that the write latency is logarithmic to the size of the sharing set. How to maintain an optimal tree structure and how to handle replacements efficiently are critical issues the authors address for this type of protocol. They compare the performance of the STP with that of the scalable coherent interface (SCI) (IEEE standard P1596) by considering a classical matrix-oriented algorithm targeted for large-scale parallel processing. They show that the STP manages to reduce the execution time considerably by reducing the write latency. >

Open Core Protocol (OCP) is a standard on-chip core interface specification. The current release is flexible and configurable to support the communication needs of a wide range of Intellectual Property cores, and is now in widespread use. However, it does not support system-level coherence. This paper summarizes an effort within the OCP-IP cache coherence working group on incorporating cache coherence extensions into OCP, which is expected to have strong impact on the MPSoC industry. In this paper, we propose a backward-compatible coherent Open Core Protocol interface and discuss the design challenges and implications introduced. This interface is flexible and can support a range of coherence protocols and schemes: we show how it can specify a snoopy bus-based scheme as well as a directory-based scheme. The correctness of the specification and models was verified using NuSMV, via exploring the entire state space for the two basic coherence schemes.

In a shared-memory multiprocessor with private caches, cached copies of a data item must be kept consistent. This is called cache coherence. Both hardware and software coherence schemes have been proposed. Software techniques are attractive because they avoid hardware complexity and can be used with any processor-memory interconnection. This paper presents an analytical model of the performance of two software coherence schemes and, for comparison, snoopy-cache hardware. The model is validated against address traces from a bus-based multiprocessor. The behavior of the coherence schemes under various workloads is compared, and their sensitivity to variations in workload parameters is assessed. The analysis shows that the performance of software schemes is critically determined by certain parameters of the workload: the proportion of data accesses, the fraction of shared references, and the number of times a shared block is accessed before it is purged from the cache. Snoopy caches are more resilient to variations in these parameters. Thus when evaluating a software scheme as a design alternative, it is essential to consider the characteristics of the expected workload. The performance of the two software schemes with a multistage interconnection network is also evaluated, and it is determined that both scale well.

Latency and synchronization overheads have been identified as two fundamental problems in large-scale shared memory multiprocessors. The notion of partial sharing of variables is introduced, and architectures based on hierarchical memories which exploit this notion of partial sharing to reduce the latency and synchronization overheads significantly are suggested. A particular class of architectures, the tree structure, hierarchical memory multiprocessor architectures (THMMs), is examined by suggesting an implementation and considering the execution and the performance of several well-known applications such as matrix multiplication, solution of partial differential equations, solution of linear recurrence relations, barrier synchronization, and reduction operations. Speedup and cost figures for these examples are compared when executing on the THMM and on a conventional memory multiprocessor.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Dynamiically tagged directories have been recently proposed as a memory-efficient mechanism for maintaining cache coherence in large-scale shared-memory multiprocessors. In order to efficiently use these directories, the number of pointer operations must be minimized and pointers should be allocated as late as possible. If pointers are allocated too early, frequent pointer overflow will occur, which in turn may cause cache thrashing.

Appropriate solution of the well-known cache coherence problem in shared memory multiprocessors is one of the key issues in improving the performance and scalability of these systems. Hardware methods are convenient because of their transparency for software. They also offer good performance since they deal with the problem dynamically. Hardware cache coherence schemes can be principally divided into two large groups: directory and snoopy protocols. This survey underlines their principles and summarizes a relatively large number of relevant representatives from both groups. The coherence problem in multilevel caches is also briefly considered. Special attention is devoted to cache coherence maintenance in large scalable shared memory multiprocessors. >

Cache coherence enforcement and memory latency reduction and hiding are very important and challenging problems in the design of large-scale distributed shared-memory (DSM) multiprocessors. We propose an integrated approach to solve these problems through a compiler-directed cache coherence scheme called the Cache Coherence with Data Prefetching (CCDP) scheme. The CCDP scheme enforces cache coherence by prefetching the potentially-stale references in a parallel program. It also prefetches the non-stale references to hide their memory latencies. To optimize the performance of the CCDP scheme, some prefetch hardware support is provided to efficiently handle these two forms of data prefetching operations. We also developed the compiler techniques utilized by the CCDP scheme for stale reference detection, prefetch target analysis and prefetch scheduling. We evaluated the performance of the CCDP scheme via execution-driven simulations of several applications from the SPEC CFP95 and CFP92 benchmark suites. The simulation results show that the CCDP scheme provides significant performance improvements for the benchmark programs studied.KeywordsCompiler-directed Cache CoherenceData PrefetchingMemory Latency HidingShared-memory Multiprocessors

Operating systems for most current shared-memory multiprocessors must maintain translation lookaside buffer (TLB) consistency across processors. A processor that changes a shared page table must flush outdated mapping information from its own TLB, and it must force the other processors using the page table to do so as well. Published algorithms for maintaining TLB consistency on some popular commercial multiprocessors incur excessively high synchronization costs. We present an efficient TLB consistency algorithm that can be implemented on multiprocessors that include a small set of reasonable architectural features. This algorithm has been incorporated in a version of the MACH operating system developed for the IBM Research Parallel Processor Prototype (RP3).

The large latency of memory accesses is a major obstacle in obtaining high processor utilization in large scale shared-memory multiprocessors. Access to remote memory is likely to be slow, compared to the ever-increasing speeds of processors. Thus, any scalable architecture must rely on techniques that can cope with the large latency of memory accesses to reduce/hide/tolerate remote-memory-access latencies.

In this paper we present an analytical-based framework for parallel program performance prediction. The main thrust of this work is to provide a means for treating realistic applications within a single unified framework. Our approach is based upon the specification of a set of non-linear equations which describe the application, processor configuration, network and memory operations. These equations are solved iteratively since the application execution rate depends on the communication latencies. The iterative solution technique is found to be efficient as it typically requires only few iterations to reach convergence. Our modeling methodology achieves a good balance between abstraction and accuracy. This is attained by accounting for both time and space dimensions of memory references, while maintaining a simple description of the workload. We demonstrate both the practicality and the accuracy of our approach by comparing predicted results with measurements taken on a commercial multiprocessor system. We found the model to be faithful in reflecting changes in processor speed, and changes in the number and placement of allocated processors.