Parallel algebraic domain decomposition solver for the solution of augmented systems

We consider the parallel iterative solution of indefinite linear systems given as augmented systems. Our numerical technique is based on an algebraic non-overlapping domain decomposition technique that only exploits the graph of the sparse matrix. This approach to high-performance, scalable solution of large sparse linear systems in parallel scientific computing, is to combine direct and iterative methods. We report numerical and parallel performance of the scheme on large matrices arising from the finite element discretization of linear elasticity in structural mechanics problems.

Computer Solution of Large Linear Systems

The goal of this paper is to develop a parallel algorithm for the direct solution of large sparse linear systems and integrate it into domain decomposition methods. The computational effort for these linear systems, often encountered in numerical simulation of structural mechanics problems by finite element codes, is very significant in terms of run-time and memory requirements.In this paper, a two-level parallelism is exploited. The exploitation of the lower level of parallelism is based on the development of a parallel direct solver with a nested dissection algorithm and to introduce it into the FETI methods. This direct solver has the advantage of handling zero-energy modes in floating structures automatically and properly. The upper level of parallelism is a coarse-grain parallelism between substructures of FETI. Some numerical tests are carried out to evaluate the performance of the direct solver.

In this paper we present an algebraic dimension-oblivious two-level domain decomposition solver for discretizations of elliptic partial differential equations. The proposed parallel domain decomposition solver is based on a space-filling curve partitioning approach that is applicable to any discretization, i.e., it directly operates on the assembled matrix equations. Moreover, it allows for the effective use of arbitrary processor numbers independent of the dimension of the underlying partial differential equation while maintaining its optimal convergence behavior. This is the core property required to attain a combination technique solver with extreme scalability which can utilize exascale parallel systems efficiently. Moreover, this approach provides a basis for the development of a fault-tolerant solver for the numerical treatment of high-dimensional Problems. To achieve the required data redundancy we are therefore concerned with large overlaps of our domain decomposition, which we construct in any dimension via space-filling curves. In this paper, we propose a space-filling curve based domain decompositoin solver and present its convergence properties and scaling behavior. The results of our numerical experiments clearly show that our approach provides optimal convergence and scaling behavior in an arbitrary dimension utilizing arbitrary processor numbers.

Developments and trends in the parallel solution of linear systems

We study the role of preconditioning strategies recently developed for coercive problems in connection with a two-step iterative method, based on the Hermitian skew-Hermitian splitting (HSS) of the coefficient matrix, proposed by Bai, Golub and Ng for the solution of nonsymmetric linear systems whose real part is coercive. As a model problem we consider Finite Differences (FD) matrix sequences {A n (a,p)} n discretizing the elliptic (convection-diffusion) problem with Ω being a plurirectangle of R d with a(x) being a uniformly positive function and p(x) denoting the Reynolds function: here for plurirectangle we mean a connected union of rectangles in d dimensions with edges parallel to the axes. More precisely, in connection with preconditioned HSS/GMRES like methods, we consider the preconditioning sequence {P n (a)} n , P n (a):= D n 1/2(a)A n (1,0) D n 1/2(a) where D n (a) is the suitably scaled main diagonal of A n (a,0). If a(x) is positive and regular enough, then the preconditioned sequence shows a strong clustering at unity so that the sequence {P n (a)} n turns out to be a superlinear preconditioning sequence for {A n (a,0)} n where A n (a,0) represents a good approximation of Re(A n (a,p)) namely the real part of A n (a,p). The computational interest is due to the fact that the preconditioned HSS method has a convergence behavior depending on the spectral properties of {P n -1(a)Re(A n (a,p))} n ≈ {P n -1(a)A n (a,0)} n : therefore the solution of a linear system with coefficient matrix A n (a,p) is reduced to computations involving diagonals and to the use of fast Poisson solvers for {A n (1,0)} n . Some numerical experimentations confirm the optimality of the discussed proposal and its superiority with respect to existing techniques.

Parallel Newton--Krylov FETI-DP (Finite Element Tearing and Interconnecting---Dual-Primal) domain decomposition methods are fast and robust solvers, e.g., for nonlinear implicit problems in structural mechanics. In these methods, the nonlinear problem is first linearized and then decomposed into loosely coupled (linear) problems, which can be solved in parallel. By changing the order of the operations, new parallel communication can be constructed, where the loosely coupled local problems are nonlinear. We discuss different nonlinear FETI-DP methods which are equivalent when applied to linear problems but which show a different performance for nonlinear problems. Moreover, a new unified framework is introduced which casts all nonlinear FETI-DP domain decomposition approaches discussed in the literature into a single algorithm. Furthermore, the equivalence of nonlinear FETI-DP methods to specific nonlinearly right-preconditioned Newton--Krylov methods is shown. For the methods using nested Newton iterations, a strategy is presented to stop the inner Newton iteration early, resulting in an approximate local nonlinear elimination. Additionally, the nonlinear BDDC (Balancing Domain Decomposition by Constraint) method is presented as a right-preconditioned Newton approach. Finally, for the first time, parallel weak scaling results for four different nonlinear FETI-DP approaches are compared to standard Newton--Krylov FETI-DP in two and three dimensions, using both exact as well as highly scalable inexact linear FETI-DP preconditioners and up to 131,072 message passing interface (MPI) ranks on the JUQUEEN supercomputer at Forschungszentrum Jülich. For a model problem with nonlocal nonlinearities, nonlinear FETI-DP methods are shown to be up to five times faster than the standard Newton--Krylov FETI-DP approach.

Advanced Krylov subspace methods are investigated for the solution of large sparse linear systems arising from stiff adjoint-based aerodynamic shape optimization problems. A special attention is paid to the flexible inner–outer GMRES strategy combined with most relevant preconditioning and deflation techniques. The choice of this specific class of Krylov solvers for challenging problems is based on its outstanding convergence properties. Typically in our implementation the efficiency of the preconditioner is enhanced with a domain decomposition method with overlapping. However, maintaining the performance of the preconditioner may be challenging since scalability and efficiency of a preconditioning technique are properties often antagonistic to each other. In this paper we demonstrate how flexible inner–outer Krylov methods are able to overcome this critical issue. A numerical study is performed considering either a Finite Volume (FV), or a high-order Discontinuous Galerkin (DG) discretization which affect the arithmetic intensity and memory-bandwidth of the algebraic operations. We consider test cases of transonic turbulent flows with RANS modeling over the two-dimensional supercritical OAT15A airfoil and the three-dimensional ONERA M6 wing. Benefits in terms of robustness and convergence compared to standard GMRES solvers are obtained. Strong scalability analysis shows satisfactory results. Based on these representative problems a discussion of the recommended numerical practices is proposed.

FoSSI: the family of simplified solver interfaces for the rapid development of parallel numerical atmosphere and ocean models

On the application of orthogonal polynomials to the iterative solution of linear systems of equations with indefinite or non-Hermitian matrices

This paper describes and tests a parallel implementation of a factorized approximate inverse preconditioner (FSAI) to accelerate iterative linear system solvers. Such a preconditioner reveals an efficient accelerator of both Conjugate gradient and BiCGstab iterative methods in the parallel solution of large linear systems arising from the discretization of the advection-diffusion equation. The resulting message passing code allows the solution of large problems leading to a very cost-effective algorithm for the solution of large and sparse linear systems.

Accelerated generalized successive overrelaxation method for least squares problems

The solution of large sparse linear systems arises in many applications, such as computational fluid dynamics and oil reservoir simulation. In realistic cases the matrices are often so large that they require large scale distributed parallel computing to obtain the solution of interest in a reasonable time. In this paper we discuss the design and implementation of the AmgX library, which provides drop-in GPU acceleration of distributed algebraic multigrid (AMG) and preconditioned iterative methods. The AmgX library implements both classical and aggregation-based AMG methods with different selector and interpolation strategies, along with a variety of smoothers and preconditioners, including block-Jacobi, Gauss--Seidel, and incomplete-LU factorization. The library contains many of the standard and flexible preconditioned Krylov subspace iterative methods, which can be combined with any of the available multigrid methods or simpler preconditioners. The parallelism in the aggregation scheme exploits parallel graph matching techniques, while the smoothers and preconditioners often rely on parallel graph coloring algorithms. The AMG algorithm implemented in the AmgX library achieves $2$--$5\times$ speedup on a single GPU against a competitive implementation on the CPU. As will be shown in the numerical experiments section, both setup and solve phases scale well across multiple nodes, sustaining this performance advantage.

This paper presents a parallel implementation of the Hybrid Bi-Conjugate Gradient Stabilized (BiCGStab(2)) iterative method in Graphics Processing Unit (GPU) for solution of large and sparse linear systems. This implementation uses the CUDA-Matlab integration, in which the method operations are performed in a GPU cores using Matlab built-in functions. The goal is to show that the exploitation of parallelism by using this new technology can provide a significant computational performance. For the validation of the work we compared the proposed implementation with a BiCGStab(2) sequential and parallelized implementation in the C and CUDA-C languages, respectively. The results showed that the proposed implementation is more efficient and can be indispensable for simulations being carried out with quality and in a timely manner. The gains in computational efficiency were, respectively, 76x and 6x compared to the implementation in C and CUDA-C.

Conventional implementations of 3D finite-difference (FD) migration use splitting techniques to accelerate performace and save computational cost. However, such techniques are plagued with numerical anisotropy that jeopardizes correct positioning of dipping reflectors in the directions not used for the splitted operators. We implement 3D downward continuation migration without splitting in the space coordinates using a complex Pad´e approximation and implicit finite differences. In this way, the numerical anisotropy is eliminated at the expense of a computationally more intensive solution of a large banded linear system. We compare the performance of the iterative stabilized biconjugate gradient (BICGSTAB) and the multifrontal massively parallel direct solver (MUMPS). It turns out that the use of the complex Pad´e approximation is an effective preconditioner for the BICGSTAB, reducing the number of iterations relative to the real Pad´e expansion of the FD operator. As a consequence, the iterative BICGSTAB method was more efficient than the direct MUMPS method when solving for a single term in the Pad´e expansion. The properties of these algorithms are evaluated computing the migration impulse response in the SEG/EAGE salt model.