Safe parallel programming using dynamic dependence hints

This paper introduces a technique which leverages value prediction and multithreading on a simultaneous multithreading processor to achieve higher performance in a single threaded application. By allowing the value-speculative execution to proceed in a separate thread, this technique overcomes barriers that make traditional value prediction relatively ineffective for tolerating long latency loads. It shows that this technique can be as much as 2-5 times more effective than traditional value prediction, achieving more than 40% average performance gain on the SPEC benchmarks with realistic hardware parameters. These gains come from two effects: allowing greater separation between the stalled load and the speculative execution, and the ability to speculate on multiple values for a single load.

Preventing execution of unauthorized software on a given computer plays a pivotal role in system security. The key problem is that although a program at the beginning of its execution can be verified as authentic, its execution flow can be redirected to externally injected malicious code us-ing, for example, a buffer overflow exploit. We introduce a novel, simplified, hardware-assisted in-trusion prevention platform. Our platform introduces over-lapping of program execution and MAC verification. It par-titions a program binary into blocks of instructions. Each block is signed using a keyed MAC that is attached as a footer to the block. When the control flow reaches a partic-ular block, its instructions are speculatively executed, while dedicated hardware verifies the attached MAC at run-time. The computation state is preserved during speculative exe-cution using a mediating buffer placed between the proces-sor and L1 data cache. Upon MAC verification, the results from this buffer are propagated externally. Central to this paper is the proposal of a novel optimization technique that initially identifies instructions that are likely to stall exe-cution, and reorders basic blocks within a given instruc-tion block to minimize the execution overhead. While the presented optimization technique is problem specific, it is flexible such that it can be adjusted for different optimiza-tion goals. Preliminary results showed that our optimiza-tion methods produced an average overhead reduction of 60 % on the SPEC2000 benchmark suite and Microsoft Vi-sual FoxPro. 1.

High-speed and high-accuracy control technology has been developed in Japan and applied to CNC (Computer Numerical Control) machine tools. Therefore, a CNC machine tool, which is made in Japan, has shown a successful history of its performance. Further development of these technologies is important for Japan to maintain its leadership in advanced manufacturing. The purpose of this research is to develop a high-speed and high-accuracy machining system. Two methods are implemented in the proposed prototype system. The first method is “geometric-model interpolation,” which is used to generate high-speed and highaccuracy command positions, and the second method is “predictive compensation,” which is used to correct machining errors. In the proposed prototype system, these errors are simulated preliminarily and compensated for in the servo data by controlling the position and the feed rate. This report describes the configuration of the proposed prototype system, and the performance of the prototype is evaluated by comparing its speed and accuracy with an existing commercial CNC machine tool. The results show that the prototype system is able to control a CNC machine tool with higher speed and greater accuracy than current CNC machine tool systems.

We present Chronos, a framework to build accelerators for applications with speculative parallelism. These applications consist of atomic tasks, sometimes with order constraints, and need speculative execution to extract parallelism. Prior work extended conventional multicores to support speculative parallelism, but these prior architectures are a poor match for accelerators because they rely on cache coherence and add non-trivial hardware to detect conflicts among tasks. Chronos instead relies on a novel execution model, Spatially Located Ordered Tasks (SLOT), that uses order as the only synchronization mechanism and limits task accesses to a single read-write object. This simplification avoids the need for cache coherence and makes speculative execution cheap and distributed. Chronos abstracts the complexities of speculative parallelism, making accelerator design easy. We develop an FPGA implementation of Chronos and use it to build accelerators for four challenging applications. When run on commodity AWS FPGA instances, these accelerators outperform state-of-the-art software versions running on a higher-priced multicore instance by 3.5x to 15.3x.

Software speculative parallelization has shown effectiveness in parallelizing certain applications. Prior techniques have mainly relied on simple exploitation of heuristics for speculation. In this work, we introduce probabilistic analysis into the design of speculation schemes. In particular, by tackling applications that are based on Finite State Machine (FSM) which have the most prevalent dependences among all programs, we show that the obstacles for effective speculation can be much better handled with rigor. We develop a probabilistic model to formulate the relations between speculative executions and the properties of the target computation and inputs. Based on the formulation, we propose two model-based speculation schemes that automatically customize themselves with the best configurations for a given FSM and its inputs. The new technique produces substantial speedup over the state of the art.

Speculative execution and instruction reuse are two important strategies that have been investigated for improving processor performance. Value prediction at the instruction level has been introduced to allow even more aggressive speculation and reuse than previous techniques. This study suggests that using compiler support to extend value reuse to a coarser granularity than a single instruction, such as a basic block, may have substantial performance benefits. We investigate the input and output values of basic blocks and find that these values can be quite regular and predictable. For the SPEC benchmark programs evaluated, 90 percent of the basic blocks have fewer than four register inputs, five live register outputs, four memory inputs, and two memory outputs. About 16 to 41 percent of all the basic blocks are simply repeating earlier calculations when the programs are compiled with the -O2 optimization level in the GCC compiler. Compiler optimizations, such as loop-unrolling and function inlining, affect the sizes of basic blocks, but have no significant or consistent impact on their value locality, nor the resulting performance. Based on these results, we evaluate the potential benefit of basic block reuse using a novel mechanism called the block history buffer. This mechanism records input and live output values of basic blocks to provide value reuse at the basic block level. Simulation results show that using a reasonably sized block history buffer to provide basic block reuse in a 4-way issue superscalar processor can improve execution time for the tested SPEC programs by 1 to 14 percent, with an overall average of 9 percent when using reasonable hardware assumptions.

Current parallelizing compilers cannot identify a significant fraction of parallelizable loops because they have complex or statically insufficiently defined access patterns. In our previously proposed framework we have speculatively executed a loop as a doall, and applied a fully parallel data dependence test to determine if it had any cross-processor dependences; if the test failed, then the loop was re-executed serially. While this method exploits doall parallelism well, it can cause slowdowns for loops with even one cross-processor flow dependence because we have to re-execute sequentially. Moreover, the existing, partial parallelism of loops is not exploited. We now propose a generalization of our speculative doall parallelization technique, called the Recursive LRPD test, that can extract and exploit the maximum available parallelism of any loop and that limits potential slowdowns to the overhead of the runtime dependence test itself. We present the base algorithm and an analysis of the different heuristics for its practical application and a few experimental results on loops from Track, Spice, and FMA3D.

This paper presents a new parallelization model, called coarse-grained thread pipelining, for exploiting speculative coarse-grained parallelism from general-purpose application programs in shared-memory multiprocessor systems. This parallelization model, which is based on the fine-grained thread pipelining model proposed for the superthreaded architecture, allows concurrent execution of loop iterations in a pipelined fashion with runtime data-dependence checking and control speculation. The speculative execution combined with the runtime dependence checking allows the parallelization of a variety of program constructs that cannot be parallelized with existing runtime parallelization algorithms. The pipelined execution of loop iterations in this new technique results in lower parallelization overhead than in other existing techniques. We evaluated the performance of this new model using some real applications and a synthetic benchmark. These experiments show that programs with a sufficiently large grain size compared to the parallelization overhead obtain significant speedup using this model. The results from the synthetic benchmark provide a means for estimating the performance that can be obtained from application programs that will be parallelized with this model. The library routines developed for this thread pipelining model are also useful for evaluating the correctness of the codes generated by the superthreaded compiler and in debugging and verifying the simulator for the superthreaded processor.

ReduxSTM: Optimizing STM designs for Irregular Applications

Speck (Speculative Parallel Check) is a system thataccelerates powerful security checks on commodity hardware by executing them in parallel on multiple cores. Speck provides an infrastructure that allows sequential invocations of a particular security check to run in parallel without sacrificing the safety of the system. Speck creates parallelism in two ways. First, Speck decouples a security check from an application by continuing the application, using speculative execution, while the security check executes in parallel on another core. Second, Speck creates parallelism between sequential invocations of a security check by running later checks in parallel with earlier ones. Speck provides a process-level replay system to deterministically and efficiently synchronize state between a security check and the original process.We use Speck to parallelize three security checks: sensitive data analysis, on-access virus scanning, and taint propagation. Running on a 4-core and an 8-core computer, Speck improves performance 4x and 7.5x for the sensitive data analysis check, 3.3x and 2.8x for theon-access virus scanning check, and 1.6x and 2x for the taint propagation check.

Speculative Execution (SE) runs loops in parallel even in the presence of a dependence. Using polyhedral dependence analysis, more speculation candidate loops can be discovered than normal OpenMP parallelization. In this research, a framework is implemented that can automatically perform speculative parallelization of loops using Polly's [15] polyhedral dependence analysis. The framework uses two different heuristics to find speculation candidates. The first heuristic allows loops with only may dependences to run speculatively in parallel while the second heuristic filters out cold loops and, using profile information, loops with actual run time dependences. The framework is fully automatic. Running SPEC2006 and the PolyBench/C benchmarks on the IBM BlueGene/Q [16] machine shows that the framework is able to discover more parallelization candidates than OpenMP parallelization and achieve better speedup.

Automatic parallelization techniques of sequential programs have focused on nests of for-loops. This work describes a method to extend some of these techniques to while-loops enclosing a nest of for-loops. This kind of nest is common in numerical applications, where a while-loop iterates a convergent computation on a discrete space scanned by the for-loop nest. The method relies on speculative execution expressed as an affine scheduling function, as in recent automatic parallelization techniques of for-loops. We show how this scheduling function can be found automatically. We also address the problem of mapping the operations onto the processors. >

Uncountable loops (such as while loops in C) and if-conditions are some of the most common constructs in programming. While-loops are widely used to determine the convergence in linear algebra algorithms or goal finding problems from graph algorithms, to name a few. In general while-loops are used whenever the loop iteration space, the number of iterations a loop executes is unknown. Usually in while-loops, the execution of the next iteration is decided inside the current loop iteration (i.e. the execution of iteration i depends on the values computed in iteration i-1). This precludes their parallel execution in today's ubiquitous multi-core architectures. In this paper a technique to speculatively create parallel tasks from the next iterations before the current one completes is proposed. If consecutive loop-iterations are only control dependent, then multiple iterations can be executed simultaneously; later in the execution path, the runtime system will decide to either commit the results of such speculatively executed iterations or undo the changes made by them. Data dependences within or between non-speculative and speculative work are honored to guarantee correctness. The proposed technique is implemented in SMPSs, a task-based dataflow programming model for shared-memory multiprocessor architectures. The approach is evaluated on a set of applications from graph algorithms and linear algebra. Results are promising with an average increase in the speedup of 1.2x with 16 threads when compared to non speculative execution of the applications. The increase in the speedup is significant, since the performance gain is achieved over an already parallelized version of the benchmarks.

Exploiting potential thread-level parallelism (TLP) is becoming the key factor to improving performance of programs on multicore or many-core systems. Among various kinds of parallel execution models, the software-based speculative parallel model has become a research focus due to its low cost, high efficiency, flexibility, and scalability. The performance of the guest program under the software-based speculative parallel execution model is closely related to the speculation accuracy, the control overhead, and the rollback overhead of the model. In this paper, we first analyzed the conventional speculative parallel model and presented an analytic model of its expectation of the overall overhead, then optimized the conventional model based on the analytic model, and finally proposed a novel speculative parallel model named HEUSPEC. The HEUSPEC model includes three key techniques, namely, the heuristic value prediction, the value based correctness checking, and the dynamic task granularity resizing. We have implemented the runtime system of the model in ANSI C language. The experiment results show that when the speedup of the HEUSPEC model can reach 2.20 on the average (15% higher than conventional model) when depth is equal to 3 and 4.51 on the average (12% higher than conventional model) when speculative depth is equal to 7. Besides, it shows good scalability and lower memory cost.

Speculative multithreading (SpMT) technology is an effective mechanism for automatic parallelization of irregular programs. While speculative parallelization can potentially deliver significant speedup for irregular programs, several overheads associated with this technique can limit these speedups in practice. Because finding optimum thread partitioning results is NP-complete, most existing thread partitioning methods are still based on simple heuristics rules to select speculative threads. However, these heuristics rules can only be used to indirectly estimate the speculative multithreaded execution overheads and tackle individual overheads, these methods can only get the empirical optimal speculative thread solution. For the limitation of existing methods, in this paper, we first propose a fuzzy c-means (FCM) algorithm based thread partitioning method which can be used to effectively search the solution space of speculative threads and get the optimal solution. With combining the heuristic rules, we compress the solution space of speculative threads based on the analysis of several overheads. Meanwhile, in order to apply the FCM algorithm, we design and implement the cluster validity function by introducing a cost model. The experimental results show that the proposed method can effectively search the solution space of speculative threads and we can indeed get better performance. We achieve an average speedup of 1.85 on Olden benchmark suits.