Optimization of serial and parallel communications for parallel geometric multigrid method

In this paper we discuss the implementation of parallel multigrid methods on unstructured locally refined meshes for 2D linear elasticity calculations. The problem of rebalancing the workload of the processors during execution of the program is discussed in detail and a load balancing algorithm suited for hierarchical meshes is proposed. For large problems the efficiency per multigrid iteration ranges from 65% on 64 processors in the locally refined case to 85% on 256 processors in the uniformly refined case. All calculations were carried out on a CRAY T3D machine. © 1997 John Wiley & Sons, Ltd.

In this report we summarize research into new parallel algebraic multigrid (AMG) methods. We first provide a introduction to parallel AMG. We then discuss our research in parallel AMG algorithms for very large scale platforms. We detail significant improvements in the AMG setup phase to a matrix-matrix multiplication kernel. We present a smoothed aggregation AMG algorithm with fewer communication synchronization points, and discuss its links to domain decomposition methods. Finally, we discuss a multigrid smoothing technique that utilizes two message passing layers for use on multicore processors.

The algebraic multigrid method (AMG) can be applied as a preconditioner for the conjugate gradient method. Since no special hierarchical mesh structure has to be specified, this method is very well suited for the implementation into a standard finite element program. A general concept for the parallelization of a finite element code to a parallel machine with distributed memory of the MIMD class is presented. Here, a non-overlapping domain decomposition is employed. A non-linear shell theory involving elastoplastic material behaviour of von Mises type with linear isotropic hardening is briefly introduced and a parallel algebraic multigrid method is derivated. As a numerical example we discuss the pinching of a cylinder undergoing large elastoplastic deformations. The performance of the solver is shown by using speed-up and scale-up investigation, as well as the influence of the problem size and the plasticity. © 1997 John Wiley & Sons, Ltd.

In this paper we consider parallel adaptive multigrid methods on unstructured meshes using MIMD computers with distributed memory. We present two load balancing algorithms designed for additive (BPX) and multiplicative multigrid. Both methods will be compared in terms of parallel and numerical efficiency and we also compare uniform with adaptive computation including the overhead introduced by load balancing and load migration.KeywordsLoad BalanceMultigrid MethodUnstructured MeshGrid LevelMulti Grid MethodThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A parallel multigrid solver for 3D convection and convection–diffusion problems

In this article we present a complete parallelization approach for simulations of PDEs with applications in optimization and uncertainty quantification. The method of choice for linear or nonlinear elliptic or parabolic problems is the geometric multigrid method since it can achieve optimal (linear) complexity in terms of degrees of freedom, and it can be combined with adaptive refinement strategies in order to find the minimal number of degrees of freedom. This optimal solver is parallelized such that weak and strong scaling is possible for extreme scale HPC architectures. For the space parallelization of the multigrid method we use a tree based approach that allows for an adaptive grid refinement and online load balancing. Parallelization in time is achieved by SDC/ISDC or a space-time formulation. As an example we consider the permeation through human skin which serves as a diffusion model problem where aspects of shape optimization, uncertainty quantification as well as sensitivity to geometry and material parameters are studied. All methods are developed and tested in the UG4 library.

Commercial reservoir simulators must be very robust and fast. Moreover, current hardware requires the simulators to scale over multiple number of computing nodes and for a fixed (‘strong scalability’) as well as an increasing problem size per computing node (‘weak scalability’). In most current commercial reservoir simulators, due to the different geological structures and properties of hydrocarbon reservoirs and the use of enhanced oil recovery (EOR) techniques, the governing equations are strongly nonlinear and hard to solve. The Jacobian system is solved by FGMRES preconditioned by the two-level constrained pressure residual (CPR) preconditioner. The driving force of the CPR preconditioner is the solution of the pressure equation. The industry standard for solving the pressure equation is the algebraic multigrid (AMG) solver. AMG is well known for its ‘weak scalability’. However, in these applications, AMG has unfavorable ‘strong’ scalability properties. This degradation in scalability is due to the increased level of inter-processor communication in the algorithm. In this paper, a monotone non-Galerkin AMG (MNG-AMG) method is presented. The aim of the method is to reduce the overall communication in MNG-AMG by enforcing a predefined nonzero pattern and monotonicity property (i.e., M-matrices) on each multigrid level. This paper describes the application of the MNG-AMG method in the context of reservoir simulations. We will compare the parallel scalability of the default solver with the MNG-AMG solver and discuss the optimal values for the MNG-AMG solver for a variety of test cases based on full field reservoir simulations.

Scalable implementation of the parallel multigrid method on massively parallel computers

An implicit, second-order space and time discretization scheme together with a parallel multigrid method involving a strip grid domain partitioning has been developed to solve fully coupled, nonlinear phase field equations involving solute and heat transport for multiple solidifying dendrites. The computational algorithm has been shown to be stable and monotonously convergent, and allowed time marching steps that were 3–4 orders of magnitude larger than those employed in similar explicit approaches, resulting in an increase of 3–4 orders of magnitude in computing efficiency. Full solute and thermal coupling was achieved for metallic alloys with a realistic, high Lewis number of >10 4. The parallel multigrid computing scheme is shown to provide a scalable methodology that allowed the efficient use of distributed supercomputing resource to simulate the evolution of tens of complex shaped 2D dendrites in a computational domain containing tens or even hundreds of millions of grid points. The simulations have provided insight into the dynamic interplay of many growing dendrites in a more realistic fully coupled thermal-solute condition, capturing for the first time fine scale features such as dendrite splitting.

Parallel multigrid finite volume computation of three-dimensional thermal convection

A two-scale approach for efficient on-the-fly operator assembly in massively parallel high performance multigrid codes

The need to solve linear systems arising from problems posed on extremely large, unstructured grids has sparked great interest in parallelizing algebraic multigrid (AMG). To date, however, no parallel AMG algorithms exist. We introduce a parallel algorithm for the selection of coarse-grid points, a crucial component of AMG, based on modifications of certain parallel independent set algorithms and the application of heuristics designed to insure the quality of the coarse grids. A prototype serial version of the algorithm is implemented, and tests are conducted to determine its effect on multigrid convergence, and AMG complexity.

Computer Solution of Large Linear Systems

Mathematical models of physical problems are becoming increasingly complex and computationally intensive. At the same time, computing hardware is becoming more parallelized with an increasing number of cores promoting simultaneous tasks. In this work, we present a parallel, equation of state (EOS), compositional flow simulator for evaluating CO2 sequestration, enhanced oil recovery techniques such as gas flooding, and other subsurface porous media applications. Using the multipoint flux mixed finite element (MFMFE) method for spatial discretization, it can handle complex reservoir geometries using general distorted hexahedral grid elements, as well as satisfy local mass conservation and compute accurate phase fluxes. A parallel framework for the MFMFE is presented that has been extended to the highly non-linear, EOS, compositional flow model. Much of the non-linearity is due to the local flash and stability calculations associated with interphase mass transfer and phase behavior. Parallel multigrid linear solver libraries such as HYPRE are utilized to solve the algebraic problems on each Newton step. We perform a variety of strong and weak parallel scaling studies up to 10 million elements and 1024 processors, and discuss possible load balancing issues.

Scalable line and plane relaxation in a parallel structured multigrid solver