A Hierarchical Tridiagonal System Solver for Heterogenous Supercomputers

Modern GPUs can achieve high computing power at low cost, but still requires much time and effort. Tridiagonal system and scan solvers are one example of widely used algorithms which can take advantage of these devices. In this article, one tridiagonal system solver and two scan primitive operators are implemented on CUDA GPUs. To do so, a tuning strategy based on three phases is developed. Additionally, a performance analysis is performed for two different CUDA GPU architectures, resulting in a huge improvement with respect to the state of the art.

Various tridiagonal solvers have been proposed in recent years for different parallel platforms. In this paper, the performance of three tridiagonal solvers, namely, the parallel partition LU algorithm, the parallel diagonal dominant algorithm, and the reduced diagonal dominant algorithm, is studied. These algorithms are designed for distributed-memory machines and are tested on an Intel Paragon and an IBM SP2 machines. Measured results are reported in terms of execution time and speedup. Analytical study are conducted for different communication topologies and for different tridiagonal systems. The measured results match the analytical results closely. In addition to address implementation issues, performance considerations such as problem sizes and models of speedup are also discussed.

For linear systems with coefficient matrices of classical structure, WZ factorizations for matrices are basic mathematical theories to design a class of parallel algorithms. Based on WZ factorization, a parallel algorithm is provided for symmetric tridiagonal linear systems. The method estimates the computation task carefully so that it assigns the system skillfully to get even load balance. In addition, the algorithm makes full use of the overlap between computation and communication to reduce waiting time in each processor. Both the subsystem assigned in each processor and the reduced subsystem have the same computing logic, as a result, a two-level method forms. By theory analysis and experiment results, it can be concluded that our method is effective in load balance and efficiency.

Three parallel algorithms, namely the parallel partition LU (PPT) algorithm, the parallel partition hybrid (PPH) algorithm, and the parallel diagonal dominant (PDD) algorithm are proposed for solving tridiagonal linear systems on multicomputers. These algorithms are based on the divide-and-conquer parallel computation model. The PPT and PPH algorithms support both pivoting and non-pivoting. The PPT algorithm is good when the number of processors is small; otherwise, the PPH algorithm is better. When the system is diagonal dominant, the PDD algorithm is highly parallel and provides an approximate solution which equals to the exact solution within machine accuracy. Both computation and communication complexities of the three algorithms are presented. All three methods proposed in this paper outperform other known parallel algorithms and have been implemented on a 64-node Ncube multicomputer. The analytic results matches closely with the results measured from the Ncube machine.

In this work, we present a stable parallel algorithm based on WZ factorization for solving diagonally dominant tridiagonal linear system of algebraic equations, using divide and conquer approach. Existence results are given and the backward error analysis of the method is presented. Numerical stability of the algorithm is proved. The given parallel algorithm for diagonally dominant tridiagonal linear systems is compared with the Truncated SPIKE version of the SPIKE algorithm [12].

Algorithms are often parallelized based on data dependence analysis manually or by means of parallel compilers. Some vector/matrix computations such as the matrix‐vector products with simple data dependence structures (data parallelism) can be easily parallelized. For problems with more complicated data dependence structures, parallelization is less straightforward. The data dependence graph is a powerful means for designing and analyzing parallel algorithms. However, for sparse matrix computations, parallelization based on solely exploiting the existing parallelism in an algorithm does not always give satisfactory results. For example, the conventional Gaussian elimination algorithm for the solution of a tri‐diagonal system is inherently sequential, so algorithms specially for parallel computation has to be designed. After briefly reviewing different parallelization approaches, a powerful graph formalism for designing parallel algorithms is introduced. This formalism will be discussed using a tri‐diagonal system as an example. Its application to general matrix computations is also discussed. Its power in designing parallel algorithms beyond the ability of data dependence analysis is shown by means of a new algorithm called ACER (Alternating Cyclic Elimination and Reduction algorithm).

A class of novel parallel algorithms for the solution of tridiagonal systems

The solution of tridiagonal system of equations using graphic processing units (GPU) is assessed. The parallel-Thomas-algorithm (PTA) is developed and the solution of PTA is compared to two known parallel algorithms, i.e. cyclic-reduction (CR) and parallel-cyclic-reduction (PCR). Lid-driven cavity problem is considered to assess these parallel approaches. This problem is also simulated using the classic Thomas algorithm that runs on a central processing unit (CPU). Runtimes and physical parameters of the mentioned GPU and CPU algorithms are compared. The results show that the speedup of CR, PCR and PTA against the CPU runtime is 4.4x,5.2x and 38.5x, respectively. Furthermore, the effect of coalesced and uncoalesced memory access to GPU global memory is examined for PTA, and a 2x-speedup is achieved for the coalesced memory access. Additionally, the PTA performance in a time dependent problem, the unsteady flow over a square, is assessed and a 9x-speedup is obtained against the CPU.

A new parallel direct algorithm for solving general linear systems of equations ist proposed in this paper. For sparse systems our algorithm requires less computations than the classical Jordan algorithm. Particularly we have also derived two related algorithms for linear recurrence problems of order 1 and tridiagonal systems. Each of the two algorithms has the same computational complexity as that of the corresponding recursive doubling algorithm or Even/Odd elimination algorithm, but requires half of the processors required by the corresponding algorithm.

The authors present a new parallel algorithm, based on the divide-and-conquer (DC) strategy, for solving tridiagonal systems. Through a comparative study between their DC method and other well known tridiagonal solvers: cyclic odd-even reduction (CR), recursive doubling (RD), and the partition method, they show that for the binary hypercube architecture, the communication complexity of their DC method is the lowest among all, and therefore the most efficient tridiagonal solver for communication-expensive massively parallel hypercube computers.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This paper gives an overview of the network sharing options available for 3G and beyond network operators. Sharing of network infrastructure among operators offers an alternative solution to reduce the investment in the coverage phase of WCDMA, allows increased coverage, reduces time to market, and allows earlier user acceptance for WCDMAII and its related services. Furthermore, in this paper general radio resource management (GRRM) strategies are proposed to cope with the implied new architectural changes. These GRRM strategies are devoted to achieve an efficient usage of the available pool of radio resources while satisfying the required quality of service (QoS) in heterogeneous traffic 3G wireless networks. These GRRM strategies include complete partitioning (CP), dynamic partitioning (DP), and adaptive partitioning with borrowing (APB). The Grade of service (GoS) and the resources utilization are considered in this study to evaluate the network performance. Simulation results indicate that the proposed DP resource allocation provides higher resource utilization under all load conditions leading in turn to increased revenue. While the CP resource allocation method provides the best performance in terms of fairness in assigning resources among customers of diferent operators, APB provides the best balance between the system utilization and the required QoS.

The parallel recursive decoupling algorithm for solving tridiagonal linear systems

We study the performance of three parallel algorithms and their hybrid variants for solving tridiagonal linear systems on a GPU: cyclic reduction (CR), parallel cyclic reduction (PCR) and recursive doubling (RD). We develop an approach to measure, analyze, and optimize the performance of GPU programs in terms of memory access, computation, and control overhead. We find that CR enjoys linear algorithm complexity but suffers from more algorithmic steps and bank conflicts, while PCR and RD have fewer algorithmic steps but do more work each step. To combine the benefits of the basic algorithms, we propose hybrid CR+PCR and CR+RD algorithms, which improve the performance of PCR, RD and CR by 21%, 31% and 61% respectively. Our GPU solvers achieve up to a 28x speedup over a sequential LAPACK solver, and a 12x speedup over a multi-threaded CPU solver.

A recursive doubling algorithm for solution of tridiagonal systems on hypercube multiprocessors

In the last years, large-scale graph processing has gained increasing attention, with most recent systems placing particular emphasis on latency. One possible technique to improve runtime performance in a distributed graph processing system is to reduce network communication. The most notable way to achieve this goal is to partition the graph by minimizing the number of edges that connect vertices assigned to different machines, while keeping the load balanced. However, real-world graphs are highly dynamic, with vertices and edges being constantly added and removed. Carefully updating the partitioning of the graph to reflect these changes is necessary to avoid the introduction of an extensive number of cut edges, which would gradually worsen computation performance. In this paper we show that performance degradation in dynamic graph processing systems can be avoided by adapting continuously the graph partitions as the graph changes. We present a novel highly scalable adaptive partitioning strategy, and show a number of refinements that make it work under the constraints of a large-scale distributed system. The partitioning strategy is based on iterative vertex migrations, relying only on local information. We have implemented the technique in a graph processing system, and we show through three real-world scenarios how adapting graph partitioning reduces execution time by over 50% when compared to commonly used hash-partitioning.