Helper thread prefetching for loosely-coupled multiprocessor systems

Data prefetching via helper threading has been extensively investigated on Simultaneous Multi- Threading (SMT) or Virtual Multi-Threading (VMT) architectures. Although reportedly large cache latency can be hidden by helper threads at runtime, most techniques rely on hardware support to reduce context switch overhead between the main thread and helper thread as well as rely on static profile feedback to construct the help thread code. This paper develops a new solution by exploiting helper threaded prefetching through dynamic optimization on the latest UltraSPARC Chip-Multiprocessing (CMP) processor. Our experiments show that by utilizing the otherwise idle processor core, a single user-level helper thread is sufficient to improve the runtime performance of the main thread without triggering multiple thread slices. Moreover, since the multiple cores are physically decoupled in the CMP, contention introduced by helper threading is minimal. This paper also discusses several key technical challenges of building a lightweight dynamic optimization/software scouting system on the UltraSPARC/Solaris platform.

Heterogeneous Many Cores (HMC) architectures that mix many simple/small cores with a few complex/large cores are emerging as a design alternative that can provide both fast sequential performance for single threaded workloads and power-efficient execution for through-put oriented parallel workloads. The availability of many small cores in a HMC presents an opportunity to utilize them as low-power helper cores to accelerate memory-intensive sequential programs mapped to a large core. However, the latency overhead of accessing small cores in a loosely coupled system limits their utility as helper cores. Also, it is not clear if small cores can execute helper threads sufficiently in advance to benefit applications running on a larger, much powerful, core. In this paper, we present a hardware/software framework called coretethering to support efficient helper threading on heterogeneous manycores. Core-tethering provides a co-processor like interface to the small cores that (a) enables a large core to directly initiate and control helper execution on the helper core and (b) allows efficient transfer of execution context between the cores, thereby reducing the performance overhead of accessing small cores for helper execution. Our evaluation on a set of memory intensive programs chosen from the standard benchmark suites show that, helper threads using moderately sized small cores can significantly accelerate a larger core compared to using a hardware prefacer alone. We find that a small core provides a good trade-off against using an equivalent large core to run helper threads in a HMC. Additionally, helper prefetching on small cores when used along with hardware prefetching, can provide an alternate design point to growing instruction window size for achieving higher sequential performance on memory intensive applications.

Pre-execution techniques have received much attention as an effective way of prefetching cache blocks to tolerate the ever-increasing memory latency. A number of pre-execution techniques based on hardware, compiler, or both have been proposed and studied extensively by researchers. They report promising results on simulators that model a simultaneous multithreading (SMT) processor. We apply the helper threading idea on a real multithreaded machine, i.e., Intel Pentium 4 processor with hyper-threading technology, and show that indeed it can provide wall-clock speedup on real silicon. To achieve further performance improvements via helper threads, we investigate three helper threading scenarios that are driven by automated compiler infrastructure, and identify several key challenges and opportunities for novel hardware and software optimizations. Our study shows a program behavior changes dynamically during execution. In addition, the organizations of certain critical hardware structures in the hyper-threaded processors are either shared or partitioned in the multithreading mode and thus, the tradeoffs regarding resource contention can be intricate. Therefore, it is essential to judiciously invoke helper threads by adapting to the dynamic program behavior so that we can alleviate potential performance degradation due to resource contention. Moreover, since adapting to the dynamic behavior requires frequent thread synchronization, having light-weight thread synchronization mechanisms is important.

Pre-execution techniques have received much attention as an effective way of prefetching cache blocks to tolerate the ever-increasing memory latency. A number of pre-execution techniques based on hardware, compiler, or both have been proposed and studied extensively by researchers. They report promising results on simulators that model a simultaneous multithreading (SMT) processor. We apply the helper threading idea on a real multithreaded machine, i.e., Intel Pentium 4 processor with hyper-threading technology, and show that indeed it can provide wall-clock speedup on real silicon. To achieve further performance improvements via helper threads, we investigate three helper threading scenarios that are driven by automated compiler infrastructure, and identify several key challenges and opportunities for novel hardware and software optimizations. Our study shows a program behavior changes dynamically during execution. In addition, the organizations of certain critical hardware structures in the hyper-threaded processors are either shared or partitioned in the multithreading mode and thus, the tradeoffs regarding resource contention can be intricate. Therefore, it is essential to judiciously invoke helper threads by adapting to the dynamic program behavior so that we can alleviate potential performance degradation due to resource contention. Moreover, since adapting to the dynamic behavior requires frequent thread synchronization, having light-weight thread synchronization mechanisms is important.

This paper describes multigrain parallel processing on a compiler cooperative chip multiprocessor. The multigrain parallel processing hierarchically exploits multiple grains of parallelism such as coarse grain task parallelism, loop iteration level parallelism and statement level near-fine grain parallelism. The chip multiprocessor has been designed to attain high effective performance, cost effectiveness and high software productivity by supporting the optimizations of the multigrain parallelizing compiler, which is developed by Japanese Millennium Project IT21 Advance Parallelizing Compiler. To achieve full potential of multigrain parallel processing, the chip multiprocessor integrates simple single-issue processors having distributed shared data memory for both optimal use of data locality and scalar data transfer, local data memory for processor private data, in addition to centralized shared memory for shared data among processors. This paper focuses on the scalability of the chip multiprocessor having up to eight processors on a chip by exploiting of the multigrain parallelism from SPECfp95 programs. When microSPARC like the simple processor core is used under assumption of 90 nm technology and 2.8 GHz, the evaluation results show the speedups for eight processors and four processors reach 7.1 and 3.9, respectively. Similarly, when 400 MHz is assumed for embedded usage, the speedups reach 7.8 and 4.0, respectively.

Technology scaling enables Chip Multiprocessors (CMPs) a promising approach to powerful performance within a chip. However, increasing transistor density makes them more susceptible to transient faults. Even worse, process variation due to unstable manufacturing makes the reliability problem more complicated. In recent years, there is a trend towards exploiting the inherent core redundancy characteristic in multi-core systems for error resilience. Nevertheless, traditional multi-core simulators lack of features to support reliability research. For fault environment simulation, high-level fault injection techniques make soft error effect unreal. Besides, core-variation error environment is not supported. For N Modular Redundancy (NMR) system simulation, there is no thread duplicate and thread affinity assignment feature. In this paper, we present a core-variation simulation platform with register-level fault injection based on intel's Sniper multi-core simulator. In addition to modifications of internal mechanism, an application program interface (API) is also provided. The validation result shows that the fault model in our simulator is consistent with exponential distribution model. In case study, NMR-system is validated. Hence, the proposed platform can be used for robust computing on multicore and other related researches in the future.

The network-on-chip (NoC) is a primary shared resource in a chip multiprocessor (CMP) system. As core counts continue to increase and applications become increasingly data-intensive, the network load will also increase, leading to more congestion in the network. This network congestion can degrade system performance if the network load is not appropriately controlled. Prior works have proposed source-throttling congestion control, which limits the rate at which new network traffic (packets) enters the NoC in order to reduce congestion and improve performance. These prior congestion control mechanisms have shortcomings that significantly limit their performance: either 1) they are not application-aware, but rather throttle all applications equally regardless of applications' sensitivity to latency, or 2) they are not network-load-aware, throttling according to application characteristics but sometimes under- or over-throttling the cores. In this work, we propose Heterogeneous Adaptive Throttling, or HAT, a new source-throttling congestion control mechanism based on two key principles: application-aware throttling and network-load-aware throttling rate adjustment. First, we observe that only network-bandwidth-intensive applications(those which use the network most heavily) should be throttled, allowing the other latency-sensitive applications to make faster progress without as much interference. Second, we observe that the throttling rate which yields the best performance varies between workloads, a single, static, throttling rate under-throttles some workloads while over-throttling others. Hence, the throttling mechanism should observe network load dynamically and adjust its throttling rate accordingly. While some past works have also used a closed-loop control approach, none have been application-aware. HAT is the first mechanism to combine application-awareness and network-load-aware throttling rate adjustment to address congestion in a NoC. We evaluate HAT using a wide variety of multiprogrammed workloads on several NoC-based CMP systems with 16-, 64-, and 144-cores and compare its performance to two state-of-the-art congestion control mechanisms. Our evaluations show that HAT consistently provides higher system performance and fairness than prior congestion control mechanisms.

Continuous improvements in semiconductor fabrication density are supporting new classes of Chip Multiprocessor (CMP) architectures that combine extensive processing logic/processor with high-density memory in a single chip. One of the architecture, called Processor-in-Memory (PIM) can support high-performance computing by combining various processors in a single system. Therefore, a new strategy is developed to identify their capabilities and dispatch the most appropriate jobs to them in order to exploit them fully. This paper presents a novel scheduling mechanism, called Swing Scheduling to fully utilize all of the heterogeneous processors in the PIM architecture. Integrated with our Octans system, this mechanism can decompose the original program into blocks and can produce a feasible execution schedule for the host and memory processors, even for other CMP architectures. The experimental results for real benchmarks are also proposed.

Green cloud is an emerging new technology in the computing world in which memory is a critical component. Phase-change memory (PCM) is one of the most promising alternative techniques to the dynamic random access memory (DRAM) that faces the scalability wall. Recent research has been focusing on the multi-level cell (MLC) of PCM. By precisely arranging multiple levels of resistance inside a PCM cell, more than one bit of data can be stored in one single PCM cell. However, the MLC PCM suffers from the degradation of performance compared to the single-level cell (SLC) PCM, due to the longer memory access time. In this paper, we present a genetic-based optimization algorithm for chip multiprocessor (CMP) equipped with PCM memory in green clouds. The proposed genetic-based algorithm not only schedules and assigns tasks to cores in the CMP system, but also provides a PCM MLC configuration that balances the PCM memory performance as well as the efficiency. The experimental results show that our genetic-based algorithm can significantly reduce the maximum memory usage by 76.8 percent comparing with the uniform SLC configuration, and improve the efficiency of memory usage by 127 percent comparing with the uniform 4 bits/cell MLC configuration. Moreover, the performance of the system is also improved by 24.5 percent comparing with the uniform 4 bits/cell MLC configuration in terms of total execution time.

Network-on-Chip (NoC) has been a viable solution for resolving the complexities associated with inter-processor communications in a Chip Multi Processor (CMP). NoC accounts for a significant portion of the total power consumed by a CMP. In a standard NoC system, only a fraction of power dissipation occurs due to static/leakage, while the rest is due to dynamic power. Dynamic dissipation includes self-switching and coupling switching dissipations, with the latter accounting for a significant part of total power dissipated. Several coding methods exist to minimize this dynamic power dissipation in NoC links. In this paper, we propose Algorithm 1 to reduce self-switching transitions and Algorithm 2 & 3 to lower coupling switching transitions in between the serial links, with Encoder/decoder modules placed at Network-Interface(NI) level. Simulations done on Xilinx Vivado design suite showed a maximum reduction of 64% in coupling switching activity compared to existing schemes.

It is well-known that multiprocessor systems are vastly more difficult to program than systems that support sequential programming models. In a 1998 paper[11] this author argued that six important principles for supporting modular software construction are often violated by the architectures proposed for multiprocessor computer systems. The Fresh Breeze project concerns the architecture and design of a multiprocessor chip that can achieve superior performance while honoring these six principles.The envisioned multiprocessor chip will incorporate three ideas that are significant departures from mainstream thinking about multiprocessor architecture: (1) Simultaneous multithreading has been shown to have performance advantages relative to contemporary superscalar designs. This advantage can be exploited through use of a programming model that exposes parallelism in the form of multiple threads of computation. (2) The value of a shared address space is widely appreciated. Through the use of 64-bit pointers, the conventional distinction between "memory" and the file system can be abolished. This can provide a superior execution environment in support of program modularity and software reuse, as well as supporting multi-user data protection and security that is consistent with modular software structure. (3) No memory update; cycle-free heap. Data items are created, used, and released, but never modified once created. The allocation, release, and garbage collection of fixed-size chunks of memory will be implemented by efficient hardware mechanisms. A major benefit of this choice is that the multiprocessor cache coherence problem vanishes: any object retrieved from the memory system is immutable. In addition, it is easy to prevent the formation of pointer cycles, simplifying the design of memory management support.

Major advances in merged logic DRAM (MLD) technology coupled with the popularization of memory-intensive applications provide fertile ground for architectures based on intelligent memory (IRAM) or processors-in-memory (PIM). The contribution of this paper is to explore one way to use the current state-of-the-art MLD technology for general-purpose computers. To satisfy requirements of general purpose and low programming cost, we place the PIM chips in the memory system and let them default to plain DRAM if the application is not enabled for intelligent memory. Since wide usability is crucial, we identify and analyze a range of real applications for PIM. Based on the requirements of these applications and current technological constraints, we design a PIM chip and a PIM-based memory system. We call the chip FlexRAM. We describe FlexRAM's design and floorplan, and the resulting memory system. Evaluation of the system through simulations shows that 4 FlexRAM chips often allow a workstation to run 25-40 times faster.

Commercial Off The Shelf (COTS) Chip Multi-Processor (CMP) systems are for cost reasons often used in industry for soft real-time stream processing. COTS CMP systems typically have a low timing predictability, which makes it difficult to develop software applications for these systems with tight temporal requirements. Restricting the way applications use the hardware and Operating System (OS) might alleviate this difficulty, so that certain types of applications could be run on COTS CMP systems with statistically verified temporal requirements. In this thesis we restrict the application domain to soft real-time medical image processing applications, which have a much more ‘stable’ usage of hardware resources than applications in general. Techniques at the application level are employed to improve the reproducibility (i.e. to reduce the variance) of the end-to-end latency of these imaging processing systems. Firstly, we study the effectiveness of a number of scheduling heuristics that are intended to improve the reproducibility of a stream processing application that is executed on COTS multiprocessor systems. Experiments show that the proposed heuristics can reduce the end-to-end latency with almost 60%, and reduce the variation in the latency with more than 90%, when compared with a naive scheduling heuristic that does not consider execution times, dependencies and the memory hierarchy. Secondly, we want to be able to integrate multiple real-time and best-effort applications on a single COTS CMP system without reducing the reproducibility of the real-time application too much. For this we examined the first component that is shared between different applications running on separate cores, the shared cache and in particular the bandwidth in the cache. We propose a technique that implements cache bandwidth reservation in software. This is achieved by dynamically duty-cycling best-effort applications based on their cache bandwidth usages measured with processor performance counters. With this technique we can control the latency increase of real-time applications that is caused by best-effort applications. Thirdly, we introduce the Probabilistic Time Triggered System (PTTS) model to analyze and optimize the end-to-end latency of a complete system that contains multiple time triggered interfaces. Our case study demonstrates the applicability of the PTTS model and the corresponding analysis techniques for an interventional X-ray system. We expect that the PTTS model is also applicable for other systems than medical image processing systems.

Instrumentation technology has been widely used in debugging interactive applications, such as interactive games and virtual reality. Debug codes are instrumented into a target program in order to collect run-time information. Although instrumentation provides detail information of the target program behavior, it can significantly prolong execution time, change program behavior and lead to incorrect debugging results, especially for time dependent and real-time applications. This paper aims to design a scalable parallel debugging mechanism to reduce instrumentation overhead while collecting detail run-time information. We design a new synchronization mechanism of instrumentation, named MDM, which uses multiple buffers to process debug messages. Also, a binding mechanism is used to specify the relationship between the target program, helper threads and cores. We conduct a case study of augmented reality interactive games on an Intel Core i7-2600 processor with Linux 2.6.38. Compared to existing methods, MDM can reduce instrumentation overhead by up to 19%.

The microprocessor industry is currently struggling with higher development costs and longer design times that arise from exceedingly complex processors that are pushing the limits of instruction-level parallelism. Meanwhile, such designs are especially ill suited for important commercial applications, such as on-line transaction processing (OLTP), which suffer from large memory stall times and exhibit little instruction-level parallelism. Given that commercial applications constitute by far the most important market for high-performance servers, the above trends emphasize the need to consider alternative processor designs that specifically target such workloads. The abundance of explicit thread-level parallelism in commercial workloads, along with advances in semiconductor integration density, identify chip multiprocessing (CMP) as potentially the most promising approach for designing processors targeted at commercial servers. This paper describes the Piranha system, a research prototype being developed at Compaq that aggressively exploits chip multi-processing by integrating eight simple Alpha processor cores along with a two-level cache hierarchy onto a single chip. Piranha also integrates further on-chip functionality to allow for scalable multiprocessor configurations to be built in a glueless and modular fashion. The use of simple processor cores combined with an industry-standard ASIC design methodology allow us to complete our prototype within a short time-frame, with a team size and investment that are an order of magnitude smaller than that of a commercial microprocessor. Our detailed simulation results show that while each Piranha processor core is substantially slower than an aggressive next-generation processor, the integration of eight cores onto a single chip allows Piranha to outperform next-generation processors by up to 2.9 times (on a per chip basis) on important workloads such as OLTP. This performance advantage can approach a factor of five by using full-custom instead of ASIC logic. In addition to exploiting chip multiprocessing, the Piranha prototype incorporates several other unique design choices including a shared second-level cache with no inclusion, a highly optimized cache coherence protocol, and a novel I/O architecture.