Data Dependence Profiling for Speculative Optimizations

Loops are the main source of parallelism in scientific programs. Hence, several techniques were developed to detect parallelism in these loops and transform them into parallel forms. In this dissertation, compile time transformations and efficient parallel execution of loops with various type of dependencies are investigated. First, Doacross loops with uniform dependencies are considered for execution on distributed memory parallel machines (multicomputers). Most known Doacross loop execution techniques can be applied efficiently only to shared memory parallel machines. In this thesis, code reordering technique, improvements to partitioning strategies, and finding a balance between communication and parallelism are presented to reduce the execution time of Doacross loops on multicomputers. As with most parallelizing transformation techniques, only single loopnests are considered in the first part of this dissertation. However, parallelizing each loopnest in a program separately, even if an optimal execution can be obtained for each loopnest, may not result in an efficient execution of all the code in the program because of communication overhead across the loops in a multicomputer environment. Hence, across loop data dependence representation and analysis are introduced in this work to improve the parallel execution of the whole code. Our contribution consists of finding and representing data dependencies whose sources and destinations are subspaces of the iteration space mainly common across the loops. This type of dependence information is used in this thesis to improve global iteration space partitioning, automatic generation of communication statements across loops, and index alignment. The final part of this dissertation presents new parallelizing techniques for loops with irregular and complex dependencies. Various data dependence analysis algorithms can be found in the literature even for loops with complex array indexing. However, the improvement in data dependence testing has not been followed by similar amelioration in restructuring transformations for loops with complex dependencies. Such loops are mostly executed in serial mode. Our parallelizing techniques for these loops consists of identifying regions of the iteration space where all iterations can be executed in parallel. The advantages of all the transformations presented in this dissertation are: (1) they significantly reduce the execution time of loops with various types of dependencies as shown in this work using the MasPar machine (2) they can be implemented at compile, time which makes the task of parallel programming easier.

Thread Level Speculation(TLS) is a hardware/software technique that guarantees correct parallel execution of loops even in the presence of dependence and has potential to lead to performance gains through the parallelization of loops that cannot be proven to be free of dependencies at compile time. However, given the overhead of TLS execution, the selection of loops to be speculated is important to avoid performance degradation. Data-dependence profiling is often used to find out if the may dependencies reported by the static analysis of a compiler materialize at runtime. A cost analysis may conclude that some loops with a lower probability of dependence should be speculatively parallelized. This report addressed the question as to whether a loops’ dependence behaviour changes when the input to the program changes — a study of 57 different benchmarks indicates that it usually does not change. Then the report describes SpecEval, a new automatic speculative parallelization framework that uses single-input data-dependence profiles to find speculation candidates in the SPEC2006 and PolyBench/C benchmarks. This report also presents the first performance evaluation of TLS implementation in IBM’s BlueGene/Q supercomputer and shows that the performance of TLS is affected by several factors, including: number and coverage of speculated loops, missspeculation overhead due to function calls in a speculated loop, L1 cache miss rate and dynamic instruction path length affects.

Speculative optimizations relax conservative constraints, like ambiguous memory-carried dependences that will rarely occur at runtime, to allow compilers to generate higher performing code. Data dependence profiling enables these techniques by providing runtime dependence information. However, prior data dependence profilers (DDPs) tend to be slow or have significant inaccuracy. Such techniques try to track all pairs of dependences that occur at runtime. To have high accuracy, it usually requires significant storage and performance overhead. Of course, accuracy can be sacrificed via sampling or fixed size storage to provide higher performance; but, these knobs are tough to control and often lead to a significant loss in accuracy. Hence, we search for a better trade-off between speed and accuracy, and we achieve this by considering a new approach based on efficient set operations using software signatures.Rather than tracking pair-wise dependences, we identify important dependence relationships at compile time, group said relationships into sets, and then construct and operate directly on the sets at runtime. By grouping many pair-wise relationships into a single set operation, we achieve greater accuracy with higher performance. Furthermore, we can adjust the size of signatures and the number of set operations through various optimization techniques to trade off speed and accuracy. Using a manual design space search, we evaluated many promising configurations. We found a fast profiler which matches the accuracy of prior works with a slowdown of 2.97x. Also, the most accurate and balanced profilers have slowdowns of 18x and 8x respectively, with an accuracy of 0.15 and 0.17 Normalized Average Euclidean Distance, respectively. To the best of our knowledge, the set-based profiler is the most accurate one to date, with equal or better performance than prior works.

Algorithms are often parallelized based on data dependence analysis manually or by means of parallel compilers. Some vector/matrix computations such as the matrix‐vector products with simple data dependence structures (data parallelism) can be easily parallelized. For problems with more complicated data dependence structures, parallelization is less straightforward. The data dependence graph is a powerful means for designing and analyzing parallel algorithms. However, for sparse matrix computations, parallelization based on solely exploiting the existing parallelism in an algorithm does not always give satisfactory results. For example, the conventional Gaussian elimination algorithm for the solution of a tri‐diagonal system is inherently sequential, so algorithms specially for parallel computation has to be designed. After briefly reviewing different parallelization approaches, a powerful graph formalism for designing parallel algorithms is introduced. This formalism will be discussed using a tri‐diagonal system as an example. Its application to general matrix computations is also discussed. Its power in designing parallel algorithms beyond the ability of data dependence analysis is shown by means of a new algorithm called ACER (Alternating Cyclic Elimination and Reduction algorithm).

Data dependence testing is the basic step in detecting loop level parallelism in numerical programs. The problem is undecidable in the general case. Therefore, work has been concentrated on a simplified problem, affine memory disambiguation. In this simpler domain, array references and loops bounds are assumed to be linear integer functions of loop variables. Dataflow information is ignored. For this domain, we have shown that in practice the problem can be solved accurately and efficiently.(1) This paper studies empirically the effectiveness of this domain restriction, how many real references are affine and flow insensitive. We use Larus's llpp system(2) to find all the data dependences dynamically. We compare these to the results given by our affine memory disambiguation system. This system is exact for all the cases we see in practice. We show that while the affine approximation is reasonable, memory disambiguation is not a sufficient approximation for data dependence analysis. We propose extensions to improve the analysis.

The data dependence of the delete-and-insert sort algorithm for median filtering is analyzed in this paper. It is shown that because of data dependence, the fastest throughput rate and the most efficient pipeline scheme cannot be used concurrently. A modified delete-and insert sort algorithm avoiding the above dilemma and its bit-level systolic array implementation are proposed in this paper. The throughput rate of the proposed architecture is equal to one-half (output/clocks) the maximum throughput allowed by the delete-and-insert sort algorithm, and the clock cycle time is equal to the propagation delay of a simple combinational circuit. Its speed is about 1.5 times faster than the existing bit-level systolic array designed by using the same delete-and-insert sort algorithm. The proposed architecture can be designed to operate at different word lengths and different window sizes. It is modular, regular, and of local interconnections and therefore amenable for VLSI implementation.

Data dependence analysis is a fundamental step in an optimizing compiler. The results of the analysis enable the compiler to identify code fragments that can be executed in parallel. A number of data dependence tests have been proposed in the literature. In each test there are different tradeoffs between accuracy and efficiency. In this paper we present an experimental evaluation of several data dependence tests, including the Banerjee test, the I-Test and the Omega test. We compare these tests in terms of accuracy and efficiency. We run various experiments using the Perfect Club Benchmarks and the scientific libraries Eispack, Linpack and Lapack. Several observations and conclusions are derived from the experimental results, which are displayed and analyzed in this paper.

GridSolve employs a standard RPC-based model for solving computational problems. There are two deficiencies associated with this model when a computational problem essentially forms a workflow consisting of a set of tasks, among which there exist data dependencies. First, intermediate results are passed among tasks going through the client, resulting in additional data transport between the client and the servers, which is pure overhead. Second, since the execution of each individual task is a separate RPC session, it is difficult to exploit the potential parallelism among tasks. This paper presents a request sequencing technique that eliminates those limitations and solves the above problems. The core features of this work include automatic DAG construction and data dependency analysis, direct inter-server data transfer, and the capability of parallel task execution.

In loop parallelization, data dependence relations are used to decide which pair of statement instances should be allocated to a same processor or should have a synchronization communication. However, in existing researches, little attention has been paid to the widespread symmetrical patterns of data dependence implied in the loop iteration. These patterns are usually induced by the regular expressions as array indices. If these expressions are all of the same type, the transitive calculations of them are always commutative. In this paper, we introduce a permutation group model to represent data dependences and discuss the application of the model. We focus on three issues: 1) the basic permutation model and the symmetrical patterns 2) the application of Abelian group theory for commutative relations such as some uniform (addition) relations, multiplication relations and hybrids relations, and 3) an approach to obtaining the iteration slices for parallelization based on previous analyses.

To parallelize simulations, independent events have to be identified, which can be executed concurrently. The highest level of parallelism is achieved if the number of events identified as independent is maximized. Traditionally, this identification is based on time and location of events, only allowing parallelization if events on the same simulation entity are executed in timestamp order. To increase the level of parallelism, we propose a novel approach investigating another criterion for independence: If two events on the same simulation entity do not access the same data items in a conflicting manner, they can as well be executed in parallel. To this end, we propose static analysis of the model code for data access. To ease this process we develop the simulation language PSimLa similar to C++ but modified where necessary to increase analyzability without removing essential C++ features. First evaluation results show the potential of this approach and increase the confidence that data-dependency analysis can improve future parallel simulation.

A data dependence test based on the projection of paths over shape graphs

This paper introduces a new static data dependence constraint, called dependence difference inequality, which can deal with coupled subscripts for multi-dimensional array references. Unlike direction vectors, dependence difference inequalities are related to not only the iteration space for a loop program but also the operation distance between two operations. They are more strict than other methods, and can act as additional constraints to each variable in a linear system on their own or with others. As a result, the solution space for a linear system can be compressed heavily. So long as dependence difference inequalities do not satisfy simultaneously, the loop can be software-pipelined with any initiation interval even if there exists a data dependence between two operations. Meanwhile, by replacing direction vectors with dependence difference inequalities some conservative estimations made by other traditional data dependence analysis approaches can be eliminated.

Data dependence and data-flow analysis of arrays

International audience

The sparse matrix solver has become the bottleneck in a Simulation Program with Integrated Circuit Emphasis circuit simulator. It is difficult to parallelize the sparse matrix solver because of the high data dependence during the numerical LU factorization. In this brief, a parallel LU factorization algorithm is developed on shared-memory computers with multicore central processing units, based on KLU algorithms. An Elimination Scheduler (EScheduler) is proposed to represent the data dependence during the LU factorization. Based on the EScheduler, the parallel tasks are scheduled in two modes to achieve a high level of concurrence, i.e., cluster mode and pipeline mode . The experimental results on 26 circuit matrices reveal that the developed algorithm can achieve speedup of 1.18-4.55t (on geometric average), as compared with KLU, with 1-8 threads. The result analysis indicates that for different data dependence, different parallel strategies should be dynamically selected to obtain optimal performance.