Utilizing home node prediction to improve the performance of software distributed shared memory

High speed networks and rapidly improving microprocessor performance make the network of workstations an extremely important tool for parallel computing in order to speedup the execution of scientific applications. Shared memory is an attractive programming model for designing parallel and distributed applications, where the programmer can focus on algorithmic development rather than data partition and communication. Based on this important characteristic, the design of systems to provide the shared memory abstraction on physically distributed memory machines has been developed, known as Distributed Shared Memory (DSM). DSM is built using specific software to combine a number of computer hardware resources into one computing environment. Such an environment not only provides an easy way to execute parallel applications, but also combines available computational resources with the purpose of speeding up execution of these applications. DSM systems need to maintain data consistency in memory, which usually leads to communication overhead. Therefore, there exists a number of strategies that can be used to overcome this overhead issue and improve overall performance. Strategies as prefetching have been proven to show great performance in DSM systems, since they can reduce data access communication latencies from remote nodes. On the other hand, these strategies also transfer unnecessary prefetching pages to remote nodes. In this research paper, we focus on the access pattern during execution of a parallel application, and then analyze the data type and behavior of parallel applications. We propose an adaptive data classification scheme to improve prefetching strategy with the goal to improve overall performance. Adaptive data classification scheme classifies data according to the accessing sequence of pages, so that the home node uses past history access patterns of remote nodes to decide whether it needs to transfer related pages to remote nodes. From experimental results, we can observe that our proposed method can increase the accuracy of data access in effective prefetch strategy by reducing the number of page faults and misprefetching. Experimental results using our proposed classification scheme show a performance improvement of about 9---25% over the same benchmark applications running on top of an original JIAJIA DSM system.

This paper presents the design and implementation of our novel home-based software Distributed Shared Memory (DSM) system. We call our system Lazy Home-based Scope Consistency (LHScC) since the system implements scope consistency lazily based on the home-based protocol. Our technique is different from others in that an optimum home node is decided before updating a home node. The performance evaluation shows that LHScC is more scalable than other DSM systems and removes the home assignment problem in the usual home-based protocol.

The distributed shared memory (DSM) architecture is considered to be one of the most likely parallel computing environment candidate for the near future because of its ease of system scalability and facilitation for parallel programming. However, a naive program based on shared memory execution on a DSM machine often deteriorates performance, because of the overhead involved for maintaining cache coherency particularly with frequent remote memory accesses. We show that careful buffer management of parallel join processing on DSM can produce considerable performance improvements in comparison with a naive implementation. We propose four buffer management strategies for parallel hash join processing on the DSM architecture and actually implement them on the HP Exemplar SPP 1600. The basic strategy is to begin with the hash join algorithm for the shared everything architecture and then to consider the memory locality of DSM by distributing the hash table and data pool buffers among the nodes. The results of four buffering strategies are analyzed in detail. Consequently, we can conclude that, in order to achieve high performance on a DSM machine, our buffer management strategy in which the memory access pattern is extracted and buffers are allocated in the local memory of nodes to minimize memory access cost is very efficient.

Distributed shared memory (DSM) systems allow programmers to use a shared memory programming model on distributed systems. However, DSM systems present new challenges to performance evaluation and debugging. While programmers write programs with a shared memory model using shared variables and shared memory synchronization primitives to coordinate processes, most performance problems arise from the implementation of these abstractions with messages. Because the DSM abstraction hides the underlying message passing, it is difficult to identify the sources of performance problems. We have developed an instrumentation system and performance evaluation techniques for DSM systems that correlate message passing behavior in the lower levels of a DSM system with the higher-level shared memory programming abstractions. We have developed a prototype tool called Xunify to demonstrate the feasibility of such an approach. Xunify provides graphical animations of DSM programs at three distinct levels and offers a powerful event filtering facility capable to help show system behavior corresponding to specific DSM constructs.

Distributed Shared Memory (DSM) systems have gained popular acceptance by combining the scalability and low cost of distributed system with the ease of use of single address space. Many new hardware DSM and software DSM systems have been proposed in recent years. In general, benchmarking is widely used to demonstrate the performance advantages of new systems. However, the common method used to summarize the measured results is the arithmetic mean of ratios, which is incorrect in some cases. Furthermore, many published papers list a lot of data only, and do not summarize them effectively, which confuse users greatly. In fact, many users want to get a single number as conclusion, which is not provided in old summarizing techniques. Therefore, a new data-summarizing technique based on confidence interval is proposed in this paper. The new technique includes two data-summarizing methods: (1) paired confidence interval method; (2) unpaired confidence interval method. With this new technique, it is concluded that at some confidence one system is better than others. Four examples are shown to demonstrate the advantages of this new technique. Furthermore, with the help of confidence level, it is proposed to standardize the benchmarks used for evaluating DSM systems so that a convincing result can be got. In addition, the new summarizing technique fits not only for evaluating DSM systems, but also for evaluating other systems, such as memory system and communication systems.

Workload distribution is an important issue for user applications to obtain a good performance on software distributed shared memory (DSM) systems. The workload distribution methods that were previously proposed for DSM systems always considered only CPU power and data sharing but ignored memory availability of processors. However, the cost of memory accesses is also an important factor of determining program performance especially when processors have not sufficient memory space to cache all the data needed by program threads. Therefore, the past methods might make wrong decisions in workload distribution to degrade program performance. To resolve this problem, this paper is dedicated to adding memory resource consideration into workload distribution for software DSM systems. Our experimental result shows that one workload distribution method with memory resource consideration is indeed more effective for minimizing the execution time of DSM applications than another without memory resource consideration.

The management of memory coherence is an important problem in distributed shared memory (DSM) system. In a cache-based coherence DSM system using linked list structure, the key to maintaining the coherence and improving system performance is how to manage the owner in the linked list. This paper presents the design of a new management protocol—NONH (New-Owner New-Head) and its performance evaluation. The analysis results show that this protocol can improve the scalability and performence of a coherent DSM system using linked list It is also suitable for managing the cache coherency in tree-like hierarchical architecture.

It is well known that contention is one of the factors that limit the performance of high performance parallel computing systems that implement distributed shared memory (DSM). On cache-coherent DSM multiprocessors, one of the most intrusive forms of contention is the one that occurs at the network interface of a node, due to simultaneous requests for remote memory accesses. This thesis focuses on the issue of resolving remote memory access contention on hardware DSM systems, and on the performance impact of implementing contention resolution algorithms. The Simultaneous Optical Multiprocessor Exchange Bus (SOME-Bus) is a low-latency, high-bandwidth, fiber-optic interconnection network supporting DSM with a unique feature that every processor is connected to the other processor through a dedicated output channel. The thesis presents simple but effective contention resolution algorithms that rely on the information of average channel utilization, average channel waiting time, number of messages in the channel queue or short-term average channel waiting time reported by each node, and uses simulation to examine the performance of the algorithms on the SOMEBus multiprocessor. The algorithms detect potential hot spots and resolve contention using dynamic block migration, which is a novel approach for migrating home nodes on a block-level basis and balancing remote memory accesses across the nodes of the system. The algorithms require minimal support from the operating system and all the necessary message transactions happen on the hardware level. Experiments with eight parallel codes on a 64-processor SOME-Bus show that algorithms yield significant performance improvements such as reduction in the execution times, number of remote memory accesses, average channel waiting times and average network latencies. Modified contention resolution algorithms are developed with the addition of data acknowledge message multicasting. Application of the modified algorithms yields more performance improvements in some codes. Architectural enhancements to the network interface and directory of the SOME-Bus multiprocessor to support the contention resolution algorithms are also presented.

The Simultaneous Optical Multiprocessor Exchange Bus (SOME-Bus) is a low-latency, high-bandwidth interconnection network that directly links arbitrary pairs of processor nodes without contention, and can efficiently interconnect over one hundred nodes. Each node has a dedicated output channel and an array of receivers, with one receiver dedicated to every other node's output channel. The SOME-Bus eliminates the need for global arbitration and provides bandwidth that scales directly with the number of nodes in the system. Under the Distributed Shared Memory (DSM) paradigm, the SOME-bus allows strong integration of the transmitter, receiver and cache controller hardware to produce a highly integrated system-wide cache coherence mechanism. Backward Error Recovery fault-tolerance techniques can rely on DSM data replication and SOME-Bus broadcasts with little additional network traffic and corresponding performance degradation. This paper presents three protocols for fault-tolerant DSM and uses simulation to examine the performance of the protocols on the SOME-Bus multiprocessor architecture.

Summary form only given. We present a broadcast-based architecture called the SOME-Bus interconnection network, which directly links processor nodes without contention, and can efficiently interconnect several hundred nodes. Each node has a dedicated output channel and an array of receivers, with one receiver dedicated to every other node's output channel. The SOME-Bus eliminates the need for global arbitration and provides bandwidth that scales directly with the number of nodes in the system. Under the distributed shared memory (DSM) paradigm, the SOME-bus allows strong integration of the transmitter, receiver and cache controller hardware to produce a highly integrated system-wide cache coherence mechanism. Backward error recovery fault-tolerance techniques can exploit DSM data replication and SOME-Bus broadcasts with little additional network traffic and corresponding performance degradation. Simulation results show that in the SOME-Bus architecture under the DSM paradigm, messages tend to wait at the node output network interface. Consequently, to minimize the effect of increased network traffic, messages can be combined at the node output queue to form a new message containing the payloads of all original messages. We use simulation to examine the effect of such message combining on the performance of SOME-Bus, in the presence of additional traffic due to fault tolerance, and we compare it to similar performance measures of a reduced SOME-Bus network where two nodes share one channel.

The paper proposes a new coherence protocol for a distributed shared memory (DSM) system, called the Selective Validity Control (SVC) protocol. There are two main obstacles that degrade the performance of a DSM system as follows: (1) it is difficult to hide access latency efficiently; (2) the excessive amount of unnecessary coherence traffic is generated. The SVC protocol is the coherence protocol that alleviates these problems by introducing fine-grain control of data transfer. The SVC protocol can also promote and improve the effectiveness of the prefetch operation by allowing cache lines to have partially valid/invalid states. The paper discusses the major problems of a conventional DSM system, and describes the design, characteristics, and primitives of the SVC protocol in detail. To evaluate the performance of the SVC protocol, a trace-driven simulator has been developed. Benchmark programs such as Gauss elimination, Fibonacci, FFT, and Jacobi iteration, were executed on the simulator to measure the amount of data transferred and the number of coherent messages issued. The simulation results show that the SVC protocol can maintain the coherence with less traffic and also minimizes the number of messages required compared to the conventional protocols.

Nowadays clusters are one of the most used platforms in High Performance Computing and most programmers use the Message Passing Interface (MPI) library to program their applications in these distributed platforms getting their maximum performance, although it is a complex task. On the other side, OpenMP has been established as the de facto standard to program applications on shared memory platforms because it is easy to use and obtains good performance without too much effort. So, could it be possible to join both worlds? Could programmers use the easiness of OpenMP in distributed platforms? A lot of researchers think so. And one of the developed ideas is the distributed shared memory (DSM), a software layer on top of a distributed platform giving an abstract shared memory view to the applications. Even though it seems a good solution it also has some inconveniences. The memory coherence between the nodes in the platform is difficult to maintain (complex management, scalability issues, high overhead and others) and the latency of the remote-memory accesses which can be orders of magnitude greater than on a shared bus due to the interconnection network. Therefore this research improves the performance of OpenMP applications being executed on distributed memory platforms using a DSM with sequential consistency evaluating thoroughly the results from the NAS parallel benchmarks. The vast majority of designed DSMs use a relaxed consistency model because it avoids some major problems in the area. In contrast, we use a sequential consistency model because we think that showing these potential problems that otherwise are hidden may allow the finding of some solutions and, therefore, apply them to both models. The main idea behind this work is that both runtimes, the OpenMP and the DSM layer, should cooperate to achieve good performance, otherwise they interfere one each other trashing the final performance of applications. We develop three different contributions to improve the performance of these applications: (a) a technique to avoid false sharing at runtime, (b) a technique to mimic the MPI behaviour, where produced data is forwarded to their consumers and, finally, (c) a mechanism to avoid the network congestion due to the DSM coherence messages. The NAS Parallel Benchmarks are used to test the contributions. The results of this work shows that the false-sharing problem is a relative problem depending on each application. Another result is the importance to move the data flow outside of the critical path and to use techniques that forwards data as early as possible, similar to MPI, benefits the final application performance. Additionally, this data movement is usually concentrated at single points and affects the application performance due to the limited bandwidth of the network. Therefore it is necessary to provide mechanisms that allows the distribution of this data through the computation time using an otherwise idle network. Finally, results shows that the proposed contributions improve the performance of OpenMP applications on this kind of environments.

Load balance is an area of current research in software distributed shared memory (DSM) systems. When threads are dynamically migrated from heavily loaded nodes to lightly loaded nodes to achieve load balance, the communication cost of maintaining data consistency is increased if migration threads are carelessly selected. Program performance is degraded when loss from increased communication exceeds the benefit from load balancing. Therefore, load balancing requires careful choice of migration threads. This study addresses the problem with a novel selection policy called Reduce Internode Sharing Cost (RISC). The main characteristic of this thread selection policy is simultaneous consideration of both thread memory access types and global sharing. Experimental application of this policy to a DSM system called Cohesion shows that simultaneous consideration of memory access types and global sharing is necessary for thread selection. RISC can reduce 50% data-consistency communication of benchmark applications during execution of the load balance mechanism.

Directory-based write-invalidate cache coherence protocols have been widely used in distributed shared memory (DSM) systems, in which cache invalidation overheads occupy a large part of the system overheads. In this paper, we propose a novel tree-based multidestination multicast scheme TBM which involves a new efficient multidestination message format. TBM combines the best features of two existing approaches: tree-based multicast and multidestination message passing. With the new scheme, only one invalidation message and less than 2/sup [log2(n+1)]-1/ acknowledgement messages are required in one cache invalidation transaction when n processor nodes have the copies of the cache block. Detailed analysis and simulation in 2D mesh show that TBM is preferable to traditional Umesh, Hamiltonian Path and BRCP-HL multicast schemes, which indicates that current and future DSM systems can take advantage of this scheme to deliver better performance.

One of the factors that can influence the performance of a DSM system is the efficiency of multipoint access on the interconnection network. This work presents an evaluation of the Brazos system, a software implemented distributed shared memory (DSM) system designed for x86 SMP nodes running Windows NT and that takes advantage of multicast communication, with regard to different configurations of the communication network: broadcast fast Ethernet, switched fast Ethernet and LANE-ATM. In this evaluation, although the bandwidth of the ATM is larger than the fast Ethernet, low performance using LANE-ATM was observed, compared to the Brazos system performance using broadcast or switched Fast Ethernet. It shows the importance of the communication subsystem and the necessity of improvements on aspects like multipoint access, which has been not much explored on networks like ATM.