Speculative defragmentation - a technique to improve the communication software efficiency for Gigabit Ethernet

Contemporary discrete GPUs support rich memory management features such as virtual memory and demand paging. These features simplify GPU programming by providing a virtual address space abstraction similar to CPUs and eliminating manual memory management, but they introduce high performance overheads during (1) address translation and (2) page faults. A GPU relies on high degrees of thread-level parallelism (TLP) to hide memory latency. Address translation can undermine TLP, as a single miss in the translation lookaside buffer (TLB) invokes an expensive serialized page table walk that often stalls multiple threads. Demand paging can also undermine TLP, as multiple threads often stall while they wait for an expensive data transfer over the system I/O (e.g., PCIe) bus when the GPU demands a page. In modern GPUs, we face a trade-o on how the page size used for memory management affects address translation and demand paging. The address translation overhead is lower when we employ a larger page size (e.g., 2MB large pages, compared with conventional 4KB base pages), which increases TLB coverage and thus reduces TLB misses. Conversely, the demand paging overhead is lower when we employ a smaller page size, which decreases the system I/O bus transfer latency. Support for multiple page sizes can help relax the page size trade-o so that address translation and demand paging optimizations work together synergistically. However, existing page coalescing (i.e., merging base pages into a large page) and splintering (i.e., splitting a large page into base pages) policies require costly base page migrations that undermine the benefits multiple page sizes provide. In this paper, we observe that GPGPU applications present an opportunity to support multiple page sizes without costly data migration, as the applications perform most of their memory allocation en masse (i.e., they allocate a large number of base pages at once). We show that this en masse allocation allows us to create intelligent memory allocation policies which ensure that base pages that are contiguous in virtual memory are allocated to contiguous physical memory pages. As a result, coalescing and splintering operations no longer need to migrate base pages. We introduce Mosaic, a GPU memory manager that provides application-transparent support for multiple page sizes. Mosaic uses base pages to transfer data over the system I/O bus, and allocates physical memory in a way that (1) preserves base page contiguity and (2) ensures that a large page frame contains pages from only a single memory protection domain. We take advantage of this allocation strategy to design a novel in-place page size selection mechanism that avoids data migration. This mechanism allows the TLB to use large pages, reducing address translation overhead. During data transfer, this mechanism enables the GPU to transfer only the base pages that are needed by the application over the system I/O bus, keeping demand paging overhead low. Our evaluations show that Mosaic reduces address translation overheads while efficiently achieving the benefits of demand paging, compared to a contemporary GPU that uses only a 4KB page size. Relative to a state-of-the-art GPU memory manager, Mosaic improves the performance of homogeneous and heterogeneous multi-application workloads by 55.5% and 29.7% on average, respectively, coming within 6.8% and 15.4% of the performance of an ideal TLB where all TLB requests are hits. CCS CONCEPTS • Computer systems organization → Parallel architectures;

Contemporary discrete GPUs support rich memory management features such as virtual memory and demand paging. These features simplify GPU programming by providing a virtual address space abstraction similar to CPUs and eliminating manual memory management, but they introduce high performance overheads during (1) address translation and (2) page faults. A GPU relies on high degrees of thread-level parallelism (TLP) to hide memory latency. Address translation can undermine TLP, as a single miss in the translation lookaside buffer (TLB) invokes an expensive serialized page table walk that often stalls multiple threads. Demand paging can also undermine TLP, as multiple threads often stall while they wait for an expensive data transfer over the system I/O (e.g., PCIe) bus when the GPU demands a page. In modern GPUs, we face a trade-off on how the page size used for memory management affects address translation and demand paging. The address translation overhead is lower when we employ a larger page size (e.g., 2MB large pages, compared with conventional 4KB base pages), which increases TLB coverage and thus reduces TLB misses. Conversely, the demand paging overhead is lower when we employ a smaller page size, which decreases the system I/O bus transfer latency. Support for multiple page sizes can help relax the page size trade-off so that address translation and demand paging optimizations work together synergistically. However, existing page coalescing (i.e., merging base pages into a large page) and splintering (i.e., splitting a large page into base pages) policies require costly base page migrations that undermine the benefits multiple page sizes provide. In this paper, we observe that GPGPU applications present an opportunity to support multiple page sizes without costly data migration, as the applications perform most of their memory allocation en masse (i.e., they allocate a large number of base pages at once).We show that this en masse allocation allows us to create intelligent memory allocation policies which ensure that base pages that are contiguous in virtual memory are allocated to contiguous physical memory pages. As a result, coalescing and splintering operations no longer need to migrate base pages.

With the increasing number of processing elements or IP cores in a System-on-Chip, Network-on-Chip has become a better solution for on-chip interconnection. RISC-V is an open ISA specification that is strongly growing in the industry as well as academia. In this paper, we have designed and implemented a Network Interface (NI) architecture that can adapt a 32-bit RISC-V core to the on-chip communication framework. The functionality of the design is verified with the behavioral simulations done using Xilinx Vivado software tool. The design is implemented on an FPGA using ZedBoard. The performance parameters of the designed NI architecture are evaluated using Cadence RTL compiler and are found to consume considerably lesser area and power compared with the existing NI architectures in the literature. The behavioural simulations show that the proposed design shows improvement in latency as well compared with the prior work in literature.

Streamlining communication is key to achieving good performance in shared-memory parallel programs. While full hardware support for cache coherence generally offers the best performance, not all parallel machines provide it. Instead, software layers using Shared Virtual Memory (SVM) can be built to enforce coherence at a higher level. In prior work, researchers have studied application-specific cache coherence protocols implemented either in SVM systems or as handlers run by programmable protocol processors. Since the protocols are specialized to the needs of a single application, they can be particularly helpful in reducing the long latencies and processing overhead that sometimes degrade performance in SVM systems. This paper studies implementing application-specific protocols in hardware, but not via an instruction-based protocol processor as is typical. Instead, we consider configurable implementations based on Field-Programmable Gate Arrays (FPGAs). This approach can be faster than software-based techniques and less expensive than some hardware-based techniques. We study one application, appbt, in detail, including a VHDL-level design of the configurable protocol design. We sketch out approaches for other applications as well. Implementing protocol operations in configurable hardware improves communication performance by roughly 11X for a 32-node system. While overall speedups are a more modest 12% our method is promising because of its flexibility and because it offers a new way of harnessing configurable hardware at the network interface, where it already exists or could be easily added to current systems.

Network interfaces for programmable NICs and multicore platforms

Although many valuable research works have investigated the properties of optical networks-on-chip (ONoCs), the vast majority of them lack an accurate exploration of the network interface architecture (NI) required to support optical communications on the silicon chip. The complexity of this architecture is especially critical for a specific kind of ONoCs: wavelength-routed ones. From a logical viewpoint, they can be considered as full nonblocking crossbars, thus the control complexity is implemented at the NIs. To our knowledge, this paper proposes the first complete NI architecture for wavelength-routed optical NoCs, by coping with the intricacy of networking issues such as flow control, buffering strategy, deadlock avoidance, serialization, and above all, with their codesign in a complete architecture.

The topology level fault tolerance of Network-on-Chip (NoC) can be improved with multi network interface (multi-NI) architectures. Multi-NI NoC architectures are based on connecting at least two network interfaces on each core. The aim is to improve fault tolerance on the architectural level which means the delivery of packets even when there are faulty links or routers in the network. This paper presents architectures and algorithms for multi-NI NoCs. The analysis of the proposed architectures and algorithms shows that some of them improve the fault tolerance of NoC with a reasonable overhead by decreasing the average hop counts and keeping the cores connectable even in the case of faults. With a multi-NI architecture the number of successfully delivered packets has been even doubled.

Communication networks, protocols, and application programming interfaces (APIs) are crucial factors for the performance of parallel and distributed computations. This topic of Euro-Par 2003 is therefore devoted to all aspects of communication in on-chip interconnects, parallel computers, networks of workstations, and more widely distributed systems such as grids. Papers were solicited that examine the design and implementation of interconnection networks and communication protocols, advances in system area and storage area networks, routing and communication algorithms, and the communication costs of parallel and distributed algorithms. On-chip and power-e.cient interconnects, I/O architectures and storage area networks, switch architectures as well as multimedia and QoS-aware communication were new topics introduced in this year’s Call for Papers (CfP).

Programming models based on messaging continue to be an important programming model for parallel machines. Messaging costs are strongly influenced by a machine's network interface architecture. We examine the impact of architectural support for messaging in two machines --- the TMC CM-5 and the Cray T3D --- by exploring the design and performance of several messaging implementations. The additional features in the T3D support remote operations: memory access, fetch-and-increment, atomic swaps, and prefetch.Experiments on the CM-5 show that requiring processor involvement for message reception can increase the communication overheads from 60% to 300% for moderate variations in computation grain size at the destination. In contrast, the T3D hardware for remote operations decouples message reception from processor activity, producing high-performance messaging independent of computation grain size or variability.In addition, hardware support for a shared address space in the T3D can be used to solve the output contention problem (output hot spots), producing messaging implementations that are robust over a wide variety of traffic patterns. Atomic swap hardware can be used to build a distributed message queue, enabling a "pull" messaging scheme where the destination requests data transfer upon receive. This scheme uses prefetches to mask receive latency. While this yields performance robust over output contention, its base cost is competitive only for small messages (up to 64 bytes) because of the high cost of issuing and resolving prefetches in the T3D. Emulation shows that if the interaction costs can be reduced by a factor of eight (250ns to 31ns), perhaps by moving the prefetch queue on chip, and there is a corresponding increase in the prefetch queue size, the pull scheme can give superior performance in all eases.

The paper describes the design and implementation of asynchronous communication library routines for distributed computing across networks of workstations. The new system is based on modifications of the existing PVM message-passing environment. An intensive and comparative study of synchronous, asynchronous and non-blocking communication protocols is addressed in terms of their design, implementation and applications. Experimental performance comparisons of an application program using the three communication protocols on a network of workstations, are also presented. The experimental results show the power of the asynchronous communication library and the effective enhancements of the PVM message-passing environment.

DIMMnet-2 is a network interface for PC cluster, plugged into a DIMM slot. Connecting network interface into commonly used memory bus reduces the cost of building PC cluster compared with using expensive machines with recent high performance I/O bus like PCIX. Moreover, low latency communication from the host CPU can be achieved. In this paper, implementation of the mechanisms for low latency communication on the DIMMnet-2 prototype board by making the best use of the memory slot is shown. Its latency for 4 Bytes data transfer is only 1.4 µs which is lower than those of InfiniBand and QsNET II on condition those host processes are Intel Xeon.

FPGAs are often used in high speed networking and telecommunications environments, where they have been shown to be very capable of line rate forwarding and routing. However, complex processes are more easily described in high-level software. In addition, many researchers do not have backgrounds in complex hardware design. NetThreads 10G is a solution to both of these problems - a soft, multithreaded multicore network processor implemented on the NetFPGA-10G[1], and software programmable using C. NefThreads10G is a port and upgrade of the original NetThreads [2] system designed for the NetFPGA: the number of cores has been doubled, packet buffer capacity increased, and a new Ethernet packet based programming system has been implemented. NetThreads 10G has a bus-based architecture connecting four MIPS-like processors to a shared data cache and a shared packet I/O buffer (Figure 1). Each core has a private instruction cache and four independent threads executed in a round robin fashion. Sixteen hardware locks are included for protecting critical code sections. The NetFPGA-10G onboard RLDRAM provides up to 128MB of main memory. During the demonstration, a sample application is developed and compiled using the NetThreads cross compiler tool. NefThreads10G is configured on the NetFPGA10G, and the application is downloaded remotely via Ethernet packets. The application is a deep packet inspection program that can detect suspicious keywords in packet payloads and keeps a record in shared memory. The demo shows how NetThreads affords us complete programmable and stateful control over OSI Layer 2 and above. The demonstration also shows NetThreads in the context of the SAVI (Smart Applications on Virtual Infrastructure) testbed. SAVI [3] is a new approach to network and Internet infrastructure - completely virtualized and extremely flexible, it views infrastructure as converged, where processing, compute, networking and reconfigurable resources are all part of a shared and managed pool. Having reconfigurable hardware in such a virtualized and programmable environment will open up new avenues of research in reconfigurable systems.

Recent advances in commodity network interface technology enable scientists and engineers to build clusters of workstations or PCs to execute parallel applications. However, raw-hardware network performance is rarely delivered to applications because of the overheads of communication software and operating systems. To reduce these overheads, a technique called user-level communication can be used to allow applications to access the network interface directly without intervention from the operating system. We examine two user-level communication systems, GM and BIP, on alpha based systems connected by a Myrinet network. In addition quantitative studies on how DMA initiation costs, flow control and reliable communication will effect the performance of communication software are presented.

Using multiple SDRAMs in MPSoCs and NoCs to increase memory parallelism is very common nowadays. In-order delivery, resource utilization, and latency are the most critical issues in such architectures. In this paper, we present a novel network interface architecture to cope with these issues efficiently. The proposed network interface exploits a resourceful reordering mechanism to handle the in-order delivery and to increase the resource utilization. A brilliant memory controller is efficiently integrated into this network interface to improve the memory utilization and reduce both memory and network latencies. In addition, to bring compatibility with existing IP cores the proposed network interface utilizes AXI transaction based protocol. Experimental results with synthetic test cases demonstrate that the proposed architecture gives significant improvements in average network latency (12%), average memory access latency (19%), and average memory utilization (22%).

Modern high speed local area networks offer great potential for communication intensive applications, but their performance is limited by the use of traditional communication protocols, such as TCP/IP. In most cases, these protocols require that all network access be through the operating system, which adds significant overhead to both the transmission path (typically a system call and data copy) and the receive path (typically an interrupt, a system call, and a data copy). To address this performance problem, several user level communication architectures have been developed that remove the operating system from the critical communication path. The article describes six important issues to consider in designing communication protocols for user level architectures. The issues discussed focus on the performance and semantics of a communication system. These issues include data transfer, address translation, protection, and control transfer mechanisms, as well as the issues of reliability and multicast. To provide a basis for analyzing these issues, the authors present a simple network interface protocol for Myricom's Myrinet network, which has a programmable network interface. Researchers can thus explore many protocol design options, and several groups have designed communication systems for Myrinet. The authors refer to 11 such systems, all of which differ significantly in how they resolve these design issues but all of which aim for high performance and provide a lean, low level, and more or less generic communication facility.