In search of speculative thread-level parallelism

Speculative plan execution for information gathering

The effective utilization of the abundant computation resources provided by Chip Multi-Processor (CMP? to speedup serial programs, has been received with various methods by many researchers making it a major research hotspot. However, embedded applications have not yet been thoroughly examined in thread level speculation (TLS), as compared to how researchers have focused and addressed other areas. In this paper, we propose kernel data structures of loop and procedure level speculation to accelerate serial programs. To verify the hypothesis, we choose some applications from Mibench, discuss codes and the impact of TLS technology features on the speedup such as coverage parallelism, dependence features, threads size, and core numbers. The experiment results prove that firstly, speculative thread level parallelism is better than instruction level parallel technology. Secondly, the best dijkstra application has a result 13.3x speedup in loop level speculation and a 29.7x speedup in procedure level. Thirdly, in the field of embedded applications, the TLS technology can effectively utilize resources of 4 to 8 core computing and procedure level speculation is better than loop level speculation.

The schedulability analysis of real-time embedded systems requires worst case execution time (WCET) analysis for the individual tasks. Bounding WCET involves not only language-level program path analysis, but also modeling the performance impact of complex micro-architectural features present in modern processors. In this paper, we statically analyze the execution time of embedded software on processors with speculative execution. The speculation of conditional branch outcomes (branch prediction) significantly improves a program's execution time. Thus, accurate modeling of control speculation is important for calculating tight WCET estimates. We present a parameterized framework to model the different branch prediction schemes. We further consider the complex interaction between speculative execution and instruction cache performance, that is, the fact that speculatively executed blocks can generate additional cache hits/misses. We extend our modeling to capture this effect of branch prediction on cache performance. Starting with the control flow graph of a program, our technique uses integer linear programming to estimate the program's WCET. The accuracy of our method is demonstrated by tight estimates obtained on realistic benchmarks.

Speculative thread-level parallelism has been recently proposed as a source of parallelism to improve the performance in applications where parallel threads are hard to find. However, the efficiency of this execution model strongly depends on the performance of the control and data speculation techniques. Several hardware-based schemes for partitioning the program into speculative threads are analyzed and evaluated. In general, we find that spawning threads associated to loop iterations is the most effective technique. We also show that value prediction is critical for the performance of all of the spawning policies. Thus, a new value predictor, the increment predictor, is proposed. This predictor is specially oriented for this kind of architecture and clearly outperforms the adapted versions of conventional value predictors such as the last value, the stride, and the context-based, especially for small-sized history tables.

Chip multiprocessors (CMPs), which recently became available with the advance of LSI technology, can outperform current superscalar processors by exploiting thread-level parallelism (TLP). However, the effectiveness of CMPs unfortunately depends greatly on their applications. In particular, they have so far not brought any significant benefit to non-numerical programs. This study explores what techniques are required to extract large amounts of TLP in non-numerical programs. We focus particularly on three techniques: thread partitioning with various control structure levels, speculative thread execution, and speculative register communication. We evaluate these techniques by examining the upper bound of the TLP, using trace-driven simulations. Our results are as follows. First, little TLP can be extracted without both of the speculations in any of the partitioning levels. Second, with the speculations, available TLP is still limited in conventional function-level and loop-level partitioning. However, it increases considerably with basic block-level partitioning. Finally, in basic block-level partitioning, focusing on control-equivalence instead of post-domination can significantly reduce the compile time, with a modest degradation of TLP.

A COOL-1 multiprocessor shared memory system based on superconductor Rapid Single-Flux Quantum (RSFQ) technology is being developed at SUNY (Stony Brook, USA) within the framework of the Hybrid Technology Multithreaded architecture (HTMT) petaflops project led by JPL. This paper describes a multithreading approach proposed in the COOL-I architecture and mechanisms to exploit the thread level parallelism in RSFQ processors called SPELL-1. Up to 128 fine-grain threads called (instruction) streams arranged in 16 groups of 8 streams each can run in parallel within a SPELL-1 processor. All eight streams comprising each COOL stream cluster can communicate and synchronize directly via shared registers. Fast creation and termination of streams including speculative stream execution are also supported.

Exploiting potential thread-level parallelism (TLP) is becoming the key factor to improving performance of programs on multicore or many-core systems. Among various kinds of parallel execution models, the software-based speculative parallel model has become a research focus due to its low cost, high efficiency, flexibility, and scalability. The performance of the guest program under the software-based speculative parallel execution model is closely related to the speculation accuracy, the control overhead, and the rollback overhead of the model. In this paper, we first analyzed the conventional speculative parallel model and presented an analytic model of its expectation of the overall overhead, then optimized the conventional model based on the analytic model, and finally proposed a novel speculative parallel model named HEUSPEC. The HEUSPEC model includes three key techniques, namely, the heuristic value prediction, the value based correctness checking, and the dynamic task granularity resizing. We have implemented the runtime system of the model in ANSI C language. The experiment results show that when the speedup of the HEUSPEC model can reach 2.20 on the average (15% higher than conventional model) when depth is equal to 3 and 4.51 on the average (12% higher than conventional model) when speculative depth is equal to 7. Besides, it shows good scalability and lower memory cost.

Instruction-level parallelism in a single stream of code for non-numerical applications has been the subject of many recent researches. This work extends the analysis to symbolic applications described with logic programming. In particular, the authors analyze the effects on performance of speculative execution, memory alias disambiguation, renaming and flow prediction. The obtained results indicate that one can reach a sustained parallelism of 4 (comparable with imperative languages), with the proper optimizations. The authors also show a comparison between static and dynamic scheduled approaches, outlining the conditions under which a dynamic solution can reach substantial improvements over a static one. In this way, they point out some important optimizations and parameters of a dynamic scheduling approach, indicating a guideline for future architectural implementations. >

In Speculative Multithreading, data dependence that limits the speedup of speculative parallelization needs to be resolved to achieve a high performance. This paper designs a synchronization execution model, with the support of compiler, to synchronize store and load instructions that frequently have data dependence on each other. We use hardware profiler to gather dependence violation information of memory data, and the profiler information is fed back to the compiler. The compiler analyzes the synchronization efficiency, select store/load pairs of great synchronization potential, and inserts synchronization instructions using insertion algorithm. Loop threads and non-loop threads can both be synchronized. The hardware support is also given in the paper. The experimental results show that the synchronization under the compiler support can effectively resolve some memory data dependence and improve the performance of the speculative execution.

As the complexity of applications continues to grow, each new generation of GPUs has been equipped with advanced architectural features and more resources to sustain its performance acceleration capability. Recent GPUs have been featured with concurrent kernel execution, which is designed to improve the resource utilization by executing multiple kernels simultaneously. However, prior systems only achieve limited performance improvement as they do not optimize the thread-level parallelism (TLP) and model the resource contention for the concurrently executing kernels. In this paper, we design a framework that optimizes the performance and energy-efficiency for multiple kernel execution on GPUs. It employs two key techniques. First, we develop an algorithm to adjust the TLP for the concurrently executing kernels. Second, we employ cache bypassing to mitigate the cache contention. Experiments indicate that our framework can improve performance by 1.42X on average (energy-efficiency by 1.33X on average), compared with the default concurrent kernel execution framework.

The simulation of lattice model systems for quantum materials is one of the most important approaches to understand quantum properties of matter in condensed matter physics. The main task in the simulation is to diagonalize a Hamiltonian matrix for the system and evaluate the electronic density of energy states. Kernel polynomial method (KPM) is one of the promising simulation methods. Because KPM contains a fine-grain recursive part in the algorithm, it is hard to parallelize it under the thread level parallelism such as on a supercomputer or a cluster computer. This paper focuses on methods to parallelize KPM on a massively parallel environment of GPU, aiming to achieve high parallelism for more speedups than the recent CPUs. This paper proposes two implementation methods called the full map and the sliding window methods, and evaluates the performances in the recent GPU platform. To enlarge available simulation sizes and at the same time to enhance the performance, this paper also describes additional optimization techniques depending on the GPU architecture.

A COOL-1 multiprocessor shared memory system based on superconductor Rapid Single-Flux Quantum (RSFQ) technology is being developed at SUNY (Stony Brook, USA) within the framework of the Hybrid Technology Multithreaded architecture (HTMT) petaflops project led by JPL. This paper describes a multithreading approach proposed in the COOL-I architecture and mechanisms to exploit the thread level parallelism in RSFQ processors called SPELL-1. Up to 128 fine-grain threads called (instruction) streams arranged in 16 groups of 8 streams each can run in parallel within a SPELL-1 processor. All eight streams comprising each COOL stream cluster can communicate and synchronize directly via shared registers. Fast creation and termination of streams including speculative stream execution are also supported.

In this paper, the scheduled dataflow (SDF) architecture-a decoupled memory/execution, multithreaded architecture using nonblocking threads-is presented in detail and evaluated against superscalar architecture. Recent focus in the field of new processor architectures is mainly on VLIW (e.g., IA-64), superscalar, and superspeculative designs. This trend allows for better performance, but at the expense of increased hardware complexity and, possibly, higher power expenditures resulting from dynamic instruction scheduling. Our research deviates from this trend by exploring a simpler, yet powerful execution paradigm that is based on dataflow and multithreading. A program is partitioned into nonblocking execution threads. In addition, all memory accesses are decoupled from the thread's execution. Data is preloaded into the thread's context (registers) and all results are poststored after the completion of the thread's execution. While multithreading and decoupling are possible with control-flow architectures, SDF makes it easier to coordinate the memory accesses and execution of a thread, as well as eliminate unnecessary dependencies among instructions. We have compared the execution cycles required for programs on SDF with the execution cycles required by programs on SimpleScalar (a superscalar simulator) by considering the essential aspects of these architectures in order to have a fair comparison. The results show that SDF architecture can outperform the superscalar. SDF performance scales better with the number of functional units and allows for a good exploitation of Thread Level Parallelism (TLP) and available chip area.

In recent years, speedup by the thread level parallel processing becomes more and more important with the spread of multi-core processors, and various techniques for parallelizing the single thread code into the efficient multithreaded code that can achieve efficient thread level parallel processing have been developed. The speculative multithreading is an important technology for achieving the high performance by the thread level parallel processing. To improve the execution performance by speculative multithreading, it is necessary to appropriately decompose the program code. Against the background of this, T. A. Johnson, et al. proposed a program decomposition technique (hereafter, we refer to their method as min-cut method) for finding the decomposition pattern that can minimize the effects of the performance degradation factors, by finding the minimum cut set in the weighted control flow graph (CFG) of the program. The min-cut method is wide coverage and a very promising technique since the whole program can be decomposed without being restricted to the logical structures, such as loop. However, the min-cut method has a problem that the loop level parallelism cannot be utilized enough. In this paper, we propose an improved method for the min-cut method to enhance the loop execution performance. Our method is based on the min-cut method and tries to apply the loop unrolling to the loop in the target program codes during the process of the decomposition. We apply our method to the practical program codes selected from the SPEC CINT2000 benchmarks. The results show that the loops, that are not decomposed with the min-cut method, are decomposed, and the possibilities of making use of the loop level parallelism increased. In addition, the results of the performance evaluation by using a cycle-based simulator show that our method can improve the loop execution performance, as compared to the min-cut method.

In this paper, we present the first Federated Learning (FL) framework which is secure against both confidentiality and integrity threats from the aggregation server, in the case where the resulting model is not disclosed to the latter. We do so by combining Homomorphic Encryption (HE) and Verifiable Computing (VC) techniques in order to perform a Federated Averaging operator directly in the encrypted domain (by means of HE) and produce formal proofs that the operator was correctly applied (by means of VC). Due to the simplicity of the aggregation function, we are able to ground our approach in additive HE techniques which are highly mature in terms of security and decently efficient. We also introduce a number of optimizations which allows to reach practical execution performances on the larger deep learning models end of the spectrum. The paper also provides extensive experimental results on the FEMNIST dataset demonstrating that the approach preserves the quality of the resulting models at the cost of practically meaningful computing and communication overheads, at least in the cross-silo setting for which higher-end machines can be involved on both the client and server sides.