Speculative lock elision: enabling highly concurrent multithreaded execution

Current generation GPUs can accelerate high-performance, compute-intensive applications by exploiting massive thread-level parallelism. The high performance, however, comes at the cost of increased power consumption. Recently, commercial GPGPU architectures have introduced support for concurrent kernel execution to better utilize the computational/memory resources and thereby improve overall throughput. In this paper, we argue and experimentally validate the benefits of concurrent kernels towards energy-efficient execution. We design power-performance models to carefully select the appropriate kernel combinations to be executed concurrently, the relative contributions of the kernels to the thread mix, along with the frequency choices for the cores and the memory to achieve high performance per watt metric. Our experimental evaluation shows that the concurrent kernel execution in combination with DVFS can improve energy-efficiency by up to 34.5% compared to the most energy-efficient sequential execution.

Writing correct multithreaded programs is difficult. Existing tools for finding bugs in multithreaded programs primarily focus on finding generic concurrency problems such as data races, atomicity violations, and deadlocks. However, these generic bugs may sometimes be benign and may not help to catch other functional errors in multithreaded programs. In this paper, we focus on a high-level programming error, called typestate error, which happens when a program does not follow the correct usage protocol of an object. We present a novel technique that finds typestate errors in multithreaded programs by looking at a successful execution. An appealing aspect of our technique is that it not only finds typestate errors that occur during a program execution, but also many other typestate errors that could have occurred in a different execution. We have implemented this technique in a prototype tool for Java and have experimented it with a number of real-world Java programs.

With the 6/spl times/86 microprocessor, Cyrix's design team demands optimum functionality and performance at an acceptable cost. Cyrix maintains that the way to get high performance is to keep larger units of processing together and to incorporate as much concurrency of execution as possible. To do the former, the microprocessor attempts to keep together all parts of an /spl times/86 instruction as it passes through its seven processing steps. To do the latter, it attempts to initiate the processing of two /spl times/86 instructions each cycle, and pipelines the processing of each instruction in seven stages. Most of this paper deals with issues involving keeping these seven stages of the two pipelines busy so the microprocessor can attempt to approach the twofold increase in performance that is made possible by the underlying structure.

In multicore and multicached architectures, cache coherence is ensured with a coherence protocol. However, the performance benefits of caching diminishes due to the cost associated with the protocol implementation. In this paper, we propose a novel technique to improve the performance of multithreaded programs running on shared-memory multicore processors by embracing approximate computing. Our idea is to relax the coherence requirement selectively in order to reduce the cost associated with a cache-coherence protocol, and at the same time, ensure a bounded QoS degradation with probabilistic reliability. In particular, we detect instructions in a multithreaded program that write to shared data, we call them Shared-Write-Access-Points (SWAPs), and propose an automated statistical analysis to identify those which can tolerate coherence faults. We call such SWAPs approximable. Our experiments on 9 applications from the SPLASH 3.0 benchmarks suite reveal that an average of 57% of the tested SWAPs are approximable. To leverage this observation, we propose an adapted cache-coherence protocol that relaxes the coherence requirement on stores from approximable SWAPs. Additionally, our protocol uses stale values for load misses due to coherence, the stale value being the version at the time of invalidation. We observe an average of 15% reduction in CPU cycles and 11% reduction in energy footprint from architectural simulation of the 9 applications using our approximate execution scheme.

Multithreaded programming is an effective way to exploit concurrency, but it is difficult to debug and tune a highly threaded program. This paper describes a performance tool called Tmon for monitoring, analyzing and tuning the performance of multithreaded programs. The performance tool has two novel features: it uses "thread waiting time" as a measure and constructs thread waiting graphs to show thread dependencies and thus performance bottlenecks, and it identifies "semi-busy-waiting" points where CPU cycles are wasted in condition checking and context switching. We have implemented the Tmon tool and, as a case study, we have used it to measure and tune a heavily threaded file system. We used four workloads to tune different aspects of the file system. We were able to improve the file system bandwidth and throughput significantly. In one case, we were able to improve the bandwidth by two orders of magnitude.

Dynamic or Just-in-Time (JIT) compilation is crucial to achieve acceptable performance for applications written in traditionally interpreted languages, such as Java and C#. Such languages enable the generation of portable applications that are written and compiled once, and can be executed by a virtual machine on any supported architecture. However, by virtue of occurring at runtime, dynamic compilation adds to the overall execution time of the application, and can potentially slow down program execution if performed injudiciously. Selective compilation is a technique that was developed for single-core architectures to manage the compilation overhead by dynamically determining and compiling only the most critical program regions. Current processors have evolved from single-core machines to those supporting multiple tightly-integrated cores. Consequently, research is needed to explore the best JIT compilation policy on multi-core machines with several concurrent compiler threads. In this paper, we present novel experimental configurations to understand the benefits of dynamic compilation, and find the best JIT compilation policies on single and multi-core machines. Our results validate earlier claims that compiling a small percentage of methods has an inordinately large impact on performance. We show that spawning a greater number of simultaneous compiler threads can achieve better program performance on single-core machines. Our results also reveal that more than an increase in compiler aggressiveness, a small increase in the number of compiler threads achieves the best application performance on multi-core machines.

A coherent global address space in a distributed system enables shared memory programming in a much larger scale than a single multicore or a single SMP. Without dedicated hardware support at this scale, the solution is a software distributed shared memory (DSM) system. However, traditional approaches to coherence (centralized via active home-node directories) and critical-section execution (distributed across nodes and cores) are inherently unfit for such a scenario. Instead, it is crucial to make decisions locally and avoid the long latencies imposed by both network and software message handlers. Likewise, synchronization is fast if it rarely involves communication with distant nodes (or even other sockets). To minimize the amount of long-latency communication required in both coherence and critical section execution, we propose a DSM system with a novel coherence protocol, and a novel hierarchical queue delegation locking approach. More specifically, we propose an approach, suitable for Data-Race-Free programs, based on self-invalidation, self-downgrade, and passive data classification directories that require no message handlers, thereby incurring no extra latency. For fast synchronization we extend Queue Delegation Locking to execute critical sections in large batches on a single core before passing execution along to other cores, sockets, or nodes, in that hierarchical order. The result is a software DSM system called Argo which localizes as many decisions as possible and allows high parallel performance with little overhead on synchronization when compared to prior DSM implementations.

Loops are the main time-consuming part of numerical applications. The performance of the loops is limited either by the resources offered by the architecture or by recurrences in the computation. To execute more operations per cycle, current processors are designed with growing degrees of resource replication (replication technique) for memory ports and functional units. However, the high cost in terms of area and cycle time of this technique precludes the use of high degrees of replication. High values for the cycle time may clearly offset any gain in terms of number of execution cycles. High values for the area may lead to an unimplementable configuration. An alternative to resource replication is resource widening (widening technique), which has also been used in some recent designs in which the width of the resources is increased (i.e., a single operation is performed over multiple data). Moreover, several general-purpose superscalar microprocessors have been implemented with multiply-add fused floating-point units (fusion technique), which reduces the latency of the combined operation and the number of resources used. The authors evaluate a broad set of VLIW processor design alternatives that combine the three techniques. We perform a technological projection for the next processor generations in order to foresee the possible implementable alternatives. From this study, we conclude that if the cost is taken into account, combining certain degrees of replication and widening in the hardware resources is more effective than applying only replication. Also, we confirm that multiply-add fused units will have a significant impact in raising the performance of future processor architectures with a reasonable increase in cost.

The author presents strategies for static loop decomposition and scheduling as well as computer-assisted run-time scheduling that take into account, in addition to the cost of performing operations, the overhead costs associated with a decomposition and schedule. An algorithm for static decomposition of multidimensional loops based on the operation execution costs, communication costs, and synchronization costs is discussed. Synchronization instructions are introduced to ensure correct program execution following program decomposition. An algorithm for determining the explicit synchronization instruction that should be introduced to ensure correct execution of a program with arbitrarily nested loops is presented. Techniques for reducing run-time scheduling and communication and synchronization costs due to self-scheduling of multidimensional loops are also presented. Experiments performed on the Encore multiprocessor system demonstrate that the techniques developed can reduce overhead costs. >

Data with increasing bandwidth requires future general-purpose as well as application specific microprocessors to improve performance endlessly. Transistor scaling, novel transistor structures, novel state-of-art VLSI design techniques and new computer architectures are the key drivers for boosting power and performance of microprocessors. Unfortunately, the processor cooling technique is unable to keep pace with higher density of transistors and high performance. For appropriate trade-offs between performance and limitation of power dissipation, dark silicon has appeared in the current processors. With the number of transistors increasing in future chips, we could envision that next generation processors might be getting darker and darker. This compromise could reduce multiple-core processors' efficiency. In this paper, power dissipation and circuit optimization are discussed in an attempt to mitigate dark silicon for future processors. A power monitor and its algorithm are proposed mainly to explain how to efficiently regulate voltage and power in the future processors with multiple cores.

Value prediction improves instruction level parallelism in superscalar processors by breaking true data dependencies. Although this technique can significantly improve overall performance, most of the state-of-the-art value prediction approaches require high hardware cost, which is the main obstacle for its wide adoption in current processors. To tackle this issue, we revisit load value prediction as an efficient alternative to the classical approaches that predict all instructions. By speculating only on loads, the pressure over shared resources (e.g., the Physical Register File) and the predictor size can be substantially reduced (e.g., more than 90% reduction compared to recent works). We observe that existing value predictors cannot achieve very high performance when speculating only on load instructions. To solve this problem, we propose a new, accurate and low-cost mechanism for predicting the values of load instructions: the Address-first Value-next Predictor with Value Prefetching (AVPP). The key idea of our predictor is to predict the load address first (which, we find, is much more predictable than the value) and to use a small non-speculative Value Table (VT)—indexed by the predicted address—to predict the value next. To increase the coverage of AVPP, we aim to increase the hit rate of the VT by predicting also the load address of a future instance of the same load instruction and prefetching its value in the VT. We show that AVPP is relatively easy to implement, requiring only 2.5% of the area of a 32KB L1 data cache. We compare our mechanism with five state-of-the-art value prediction techniques, evaluated within the context of load value prediction, in a relatively narrow out-of-order processor. On average, our AVPP predictor achieves 11.2% speedup and 3.7% of energy savings over the baseline processor, outperforming all the state-of-the-art predictors in 16 of the 23 benchmarks we evaluate. We evaluate AVPP implemented together with different prefetching techniques, showing additive performance gains (20% average speedup). In addition, we propose a new taxonomy to classify different value predictor policies regarding predictor update, predictor availability, and in-flight pending updates. We evaluate these policies in detail.

As the smart cities emerged for more comfortable urban spaces, services, such as health, transportation, and so on, need to be promoted. In addition, the cloud computing provides flexible allocation, migration of services, and better security isolation; therefore, it is the infrastructure for the smart cities. Single instruction-set architecture (ISA) heterogeneous multi-core processors have higher performance per watt than their symmetric counterparts and are popular in current processors. In current cloud computing, which integrates a few fast out-of-order cores, coupled with a large number of simpler, slow cores, all cores expose the same ISA. The best way to leverage the effectiveness of these systems is to accelerate sequential CPU-bound threads using fast cores, and to improve the throughput of parallel memory-bound threads using slow cores. However, shared hardware resources, such as memory, respond to requests from all cores, which interfere with each other, leading to both low speed for fast cores and low throughput for slow cores. In this paper, we propose a dynamic resource partitioning (DRP) method for single-ISA heterogeneous multi-cores, which partitions the shared resources according to both threads' requirements for the shared resources and the performance of their running cores. The key principle is to profile both threads' resource characteristics at run-time and the performance of the cores that the threads are running on to estimate demands for resources. Then, we use the estimation to direct our resource partitioning. Moreover, we integrate our DRP with current memory scheduling policies to improve the system performance further for the two methods being orthogonal.

Dynamic binary translation which offers a solution for making an executable compiled on a platform to run on another platform automatically resolves the problem of code migration. The current single-threaded dynamic binary translation systems don't have enough scope to improve the performance. Therefore, the multi-core processors and the multi-threaded program are fully used for getting the high performance. Therefore, based on the performance analysis and experiments results of a single-threaded dynamic binary translation system, this paper shows the framework of MDT (Multi-Threaded Dynamic Binary Translation) system and the optimizations implemented on it. For achieving the speculatively translation scheme, the T-Tree (Translation Tree) is built by the servant thread which gives the direction of getting the next pc. Besides that, integrated with the merit of multi-level Cache scheme, the LRU scheme and the full flush scheme, a new management of T-Cache is presented which proves out managing the translated blocks efficiently. The framework and optimizations of MDT are evaluated wholly and partly across SPEC 2006 under the Alpha multi-core environment. The results compared with QEMU demonstrate that the speculative translation and the reducing of T-Cache missing rate are effectively.

In practice, it is quite difficult to write correct multithreaded programs due to the potential for unintended and nondeterministic interference between parallel threads. A fundamental correctness property for such programs is atomicity---a block of code in a program is atomic if, for any parallel execution of the program, there is an execution with the same overall program behavior in which the block is executed serially.We propose semantic atomicity, a generalization of atomicity with respect to a programmer-defined notion of equivalent behavior. We propose an assertion framework in which a programmer can use bridge predicates to specify noninterference properties at the level of abstraction of their application. Further, we propose a novel algorithm for systematically testing atomicity specifications on parallel executions with a bounded number of interruptions---i.e. atomic blocks whose execution is interleaved with that of other threads. We further propose a set of sound heuristics and optional user annotations that increase the efficiency of checking atomicity specifications in the common case where the specifications hold.We have implemented our assertion framework for specifying and checking semantic atomicity for parallel Java programs, and we have written semantic atomicity specifications for a number of benchmarks. We found that using bridge predicates allowed us to specify the natural and intended atomic behavior of a wider range of programs than did previous approaches. Further, in checking our specifications, we found several previously unknown bugs, including in the widely-used java.util.concurrent library.

The trend towards processors with more and more parallel cores is increasing the need for software that can take advantage of parallelism. The most widespread method for writing parallel software is to use explicit threads. Writing correct multithreaded programs, however, has proven to be quite challenging in practice. The key difficulty is nondeterminism. The threads of a parallel application may be interleaved nondeterministically during execution. In a buggy program, nondeterministic scheduling can lead to nondeterministic results—where some interleavings produce the correct result while others do not. We propose an assertion framework for specifying that regions of a parallel program behave deterministically despite nondeterministic thread interleaving. Our framework allows programmers to write assertions involving pairs of program states arisingfrom different parallel schedules. We describe an implementation of our deterministic assertions as a library for Java, and evaluate the utility of our specifications on a number of parallel Java benchmarks. We found specifying deterministic behavior to be quite simple using our assertions. Further, in experiments with our assertions, we were able to identify two races as true parallelism errors that lead to incorrect non-deterministic behavior. These races were distinguished from a number of benign races in the benchmarks.