Parallel Algebraic Multigrid Methods on Distributed Memory Computers

We are interested in the design of efficient algebraic multigrid (AMG) methods for the solution of large sparse systems of linear equations arising from finite element (FE) discretization of second-order elliptic partial differential equations (PDEs). In particular, we introduce the concept of so-called “edge matrices”, which–in the present context–are extracted from the individual element matrices. This allows for the construction of spectrally equivalent approximations of the original stiffness matrix that can be utilized in the framework of AMG. The edge matrices give rise to modify the definition of “strong” and “weak” connections (edges), which provides a basis for selecting the coarse-grid nodes in algebraic multigrid methods. Moreover, a reproduction of edge matrices on coarse levels offers the opportunity to combine classical coarsening algorithms with effective (energy minimizing) interpolation principles involving small-sized “computational molecules” (small collections of edge matrices). This yields a flexible and robust new variant of AMG, which we refer to as AMGm.

In this report we summarize research into new parallel algebraic multigrid (AMG) methods. We first provide a introduction to parallel AMG. We then discuss our research in parallel AMG algorithms for very large scale platforms. We detail significant improvements in the AMG setup phase to a matrix-matrix multiplication kernel. We present a smoothed aggregation AMG algorithm with fewer communication synchronization points, and discuss its links to domain decomposition methods. Finally, we discuss a multigrid smoothing technique that utilizes two message passing layers for use on multicore processors.

Algebraic multigrid (AMG) methods have proven to be effective for solving the linear algebraic system of equations that arise from many classes of unstructured discretized elliptic PDEs. Standard AMG methods, however, are not suitable for shifted linear systems from elliptic PDEs, such as discretized Helmholtz operators, due to the indefiniteness of the system and the presence of low energy modes which are difficult for multigrid methods to resolve effectively. This paper investigates simple methods to adapt existing standard AMG methods to these shifted systems from direct frequency response analyses in solid mechanics.

The need to solve linear systems arising from problems posed on extremely large, unstructured grids has sparked great interest in parallelizing algebraic multigrid (AMG). To date, however, no parallel AMG algorithms exist. We introduce a parallel algorithm for the selection of coarse-grid points, a crucial component of AMG, based on modifications of certain parallel independent set algorithms and the application of heuristics designed to insure the quality of the coarse grids. A prototype serial version of the algorithm is implemented, and tests are conducted to determine its effect on multigrid convergence, and AMG complexity.

The efficiency of numerical simulation is of great importance for the development of tight oil reservoirs. In this paper, a nonlinear seepage model of tight oil reservoirs was presented, and the physical model of multi-fractured horizontal well is meshed by unstructured triangular grids, and the mathematical model is discretized by finite element method. Then, a fully implicit iterative solution scheme was derived. A novel algebraic multigrid method (AMG-CG) was proposed to solve the large sparse linear equation system during the iteration process. In order to demonstrate the computational efficiency of the novel algebraic multigrid method, a comparison was made between the novel method and others. The results reveal that the AMG-CG method has a faster calculation speed in solving linear equation systems. The calculation time of the AMG-CG method is 1029 s, the calculation time of the AMG method is 2029 s, the calculation time of the generalized minimum residual method (GMRES) method is 3709 s, the calculation time of the Gauss–Seidel (GS) method is 8115 s. The calculation speed of the novel method is almost 7.88 times faster than the Gauss–Seidel method, the novel method is more effective in the numerical simulation of tight oil.

The algebraic multigrid (AMG) method is an efficient solver for linear systems arising in finite element analyses. The AMG method is applicable at a matrix level, different from the geometric multigrid solvers. This paper proposes a combination of the parallel processing technique and the AMG method as a fast solver for electromagnetic field analyses. While the AMG method consists of a setup phase and a solution phase, parallel processing of the former phase is difficult. We present the use of long-range interpolation instead of the conventional direct interpolation for improvement of the parallel efficiency of the AMG setup phase. A magnetostatic analysis and an eddy-current analysis show the solver performance. The numerical results show that parallelized AMG is a fast solver and has sufficient scalability, as compared with the conventional solver.

Computer Solution of Large Linear Systems

Fast parallel solvers for fourth-order boundary value problems

For the numerical solution of linear systems that arise from discretized linear partial differential equations, multigrid and domain decomposition methods are well established. Multigrid methods are known to have optimal complexity and domain decomposition methods are in particular useful for parallelization of the implemented algorithm. For linear random operator equations, the classical theory is not directly applicable, since condition numbers of system matrices may be close to degenerate due to non-uniform random input. It is shown that iterative methods converge in the strong, i.e. $$L^p$$, sense if the random input satisfies certain integrability conditions. As a main result, standard multigrid and domain decomposition methods are applicable in the case of linear elliptic partial differential equations with lognormal diffusion coefficients and converge strongly with deterministic bounds on the computational work which are essentially optimal. This enables the application of multilevel Monte Carlo methods with rigorous, deterministic bounds on the computational work.

Numerical modeling of important phenomena and processes in science and engineering has emerged in recent years as an alternative to the experimental approach. The ever‐increasing computer speed and storage capabilities enable modeling of large‐scale problems in continuum mechanics, electromagnetism, quantum physics, and other important areas. To perform the modeling task efficiently, sophisticated algorithms and numerical techniques for the solution of the underlying models are essential. Such models are expressed frequently as mathematical equations. To represent a continuous mathematical model on a computer, which operates on discrete binary‐coded data, a mathematical procedure referred to as the discretization needs to be deployed. A common result of the discretization procedure is a system of linear or non linear equations that needs to be solved. Numerical modeling of complex systems frequently requires the solution of large systems of equations (numbering several million). Such systems need to be solved repeatedly, as part of an iterative procedure, such as linearization, time‐stepping, optimization, or inverse problems. This reason is why the solution of linear systems is a crucial phase of the overall numerical modeling process. As a result, substantial research effort has been devoted to the development of effective numerical algorithms for the solution of linear systems.This article describes an important class of iterative algorithms for the solution of linear systems, referred to as multigrid (MG) methods. MG was developed originally in the context of the solution of discretized differential equations, with the aim to overcome the difficulties that standard iterative solvers encounter in this context. Over time, MG methods have evolved into an independent field of research, which has a major impact on many scientific and engineering disciplines. In this article, we present a brief review of MG and its applications to some realistic problems from numerical modeling practice.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.

The bidomain equations are widely used for the simulation of electrical activity in cardiac tissue but are computationally expensive, limiting the size of the problem which can be modeled. The purpose of this study is to determine more efficient ways to solve the elliptic portion of the bidomain equations, the most computationally expensive part of the computation. Specifically, we assessed the performance of a parallel multigrid (MG) preconditioner for a conjugate gradient solver. We employed an operator splitting technique, dividing the computation in a parabolic equation, an elliptical equation, and a nonlinear system of ordinary differential equations at each time step. The elliptic equation was solved by the preconditioned conjugate gradient method, and the traditional block incomplete LU parallel preconditioner (ILU) was compared to MG. Execution time was minimized for each preconditioner by adjusting the fill-in factor for ILU, and by choosing the optimal number of levels for MG. The parallel implementation was based on the PETSc library and we report results for up to 16 nodes on a distributed cluster, for two and three dimensional simulations. A direct solver was also available to compare results for single processor runs. MG was found to solve the system in one third of the time required by ILU but required about 40% more memory. Thus, MG offered an attractive tradeoff between memory usage and speed, since its performance lay between those of the classic iterative methods (slow and low memory consumption) and direct methods (fast and high memory consumption). Results suggest the MG preconditioner is well suited for quickly and accurately solving the bidomain equations.

SummaryWe construct new robust and efficient preconditioned generalized minimal residual solvers for the monolithic linear systems of algebraic equations arising from the finite element discretization and Newton's linearization of the fully coupled fluid–structure interaction system of partial differential equations in the arbitrary Lagrangian–Eulerian formulation. We admit both linear elastic and nonlinear hyperelastic materials in the solid model and cover a large range of flows, for example, water, blood, and air, with highly varying density. The preconditioner is constructed in form of, where,, andare proper approximations to the matricesL,D, andUin theLDUblock factorization of the fully coupled system matrix, respectively. The inverse of the corresponding Schur complement is approximated by applying a few cycles of a special class of algebraic multigrid methods to the perturbed fluid sub‐problem, which is obtained by modifying corresponding entries in the original fluid matrix with an explicitly constructed approximation to the exact perturbation coming from the sparse matrix–matrix multiplications. The numerical studies presented impressively demonstrate the robustness and the efficiency of the preconditioner proposed in the paper. Copyright © 2016 John Wiley & Sons, Ltd.

Solving singularly perturbed convection-diffusion equations, especially for 3D problems, is a challenging problem. In this paper, we extend our work on the extrapolation cascadic multigrid (EXCMG) method for solving the 3D Poisson equation — cf. [Pan et al., J. Sci. Comput. 2017], to 3D convection-diffusion equations with singularly perturbed parameters. First, we introduce an exponential higher order compact finite difference scheme to discretize the 3D convection-diffusion equation with variable convection coefficients, resulting in a larger-scale nonsymmetric linear system. Then, we propose an EXCMG method combined with the biconjugate gradient stabilized smoother to solve the larger-scale nonsymmetric system efficiently. Numerical experiments demonstrate that the EXCMG method is a highly effective solver for convection-dominated problems, which outperforms the existing multigrid methods such as aggregation-based algebraic multigrid method.

With the rise in popularity of compatible finite element, finite difference, and finite volume discretizations for the time domain eddy current equations, there has been a corresponding need for fast solvers of the resulting linear algebraic systems. However, the traits that make compatible discretizations a preferred choice for the Maxwell's equations also render these linear systems essentially intractable by truly black-box techniques. We propose an algebraic reformulation of the discrete eddy current equations along with a new algebraic multigrid (AMG) technique for this reformulated problem. The reformulation process takes advantage of a discrete Hodge decomposition on cochains to replace the discrete eddy current equations by an equivalent $2\times2$ block linear system whose diagonal blocks are discrete Hodge–Laplace operators acting on 1-cochains and 0-cochains, respectively. While this new AMG technique requires somewhat specialized treatment on the finest mesh, the coarser meshes can be handled using standard methods for Laplace-type problems. Our new AMG method is applicable to a wide range of compatible methods on structured and unstructured grids, including edge finite elements, mimetic finite differences, covolume methods, and Yee-like schemes. We illustrate the new technique, using edge elements in the context of smoothed aggregation AMG, and present computational results for problems in both two and three dimensions.