Towards a Lanczos’ $$\tau $$ τ -Method Toolkit for Differential Problems

Application demands and grand challenges in numerical simulation require for both highly capable computing platforms and efficient numerical solution schemes. Power constraints and further miniaturization of modern and future hardware give way for multi- and manycore processors with increasing fine-grained parallelism and deeply nested hierarchical memory systems -- as already exemplified by recent graphics processing units. Accordingly, numerical schemes need to be adapted and re-engineered in order to deliver scalable solutions across diverse processor configurations. Portability of parallel software solutions across emerging hardware platforms is another challenge. This work investigates multi-coloring and re-ordering schemes for block Gauß-Seidel methods and, in particular, for incomplete LU factorizations with and without fill-ins. We consider two matrix re-ordering schemes that deliver flexible and efficient parallel preconditioners. The general idea is to generate block decompositions of the system matrix such that the diagonal blocks are diagonal itself. In such a way, parallelism can be exploited on the block-level in a scalable manner. Our goal is to provide widely applicable, out-of-the-box preconditioners that can be used in the context of finite element solvers. We propose a new method for anticipating the fill-in pattern of ILU($p$) schemes which we call the power($q$)-pattern method. This method is based on an incomplete factorization of the system matrix $A$ subject to a predetermined pattern given by the matrix power $|A|^(p+1)$ and its associated multi-coloring permutation $. We prove that the obtained sparsity pattern is a superset of our modified ILU($p$) factorization applied to pi A pi^(-1). As a result, this modified ILU($p$) applied to multi-colored system matrix has no fill-ins in its diagonal blocks. This leads to an inherently parallel execution of triangular ILU($p$) sweeps. In addition, we describe the integration of the preconditioners into the HiFlow$^3$ open-source finite element package that provides a portable software solution across diverse hardware platforms. On this basis, we conduct performance analysis across a variety of test problems on multi-core CPUs and GPUs that proves efficiency, scalability and flexibility of our approach. Our preconditioners achieve a solver acceleration by a factor of up to 1.5, 8 and 85 for three different test problems. The GPU versions of the preconditioned solver are by a factor of up to 4 faster than an OpenMP parallel version on eight cores.

Modern petroleum reservoir simulation serves as a primary tool for quantitatively managing reservoir production and planning new fields. It involves repeatedly solving the Jacobian of a set of strong nonlinear partial differential equations governing the mass and energy conduction and conservation. Most of the existing reservoir simulators adopt iterative solver with multiple stages of preconditioners, in which the incomplete LU (ILU) factorization is an outstanding universal smoother. However, it turns out that when the degree of freedom of each grid grows, ILU usually becomes the bottleneck of the solver. Moreover, ILU is difficult to parallelize due to its inherent data dependency. In this paper, we developed a sparse iterative solver with parallelized ILU and triangular solve using block-wise data structure. Compared with the state of art iterative solver on 14 industrial reservoir simulation matrices, the proposed ILU is 5.2x faster (on average) than the state of art iterative solver because of the block-wise data structure, which leads to 2.2x speedup on the total solver runtime. In addition, parallel ILU and triangular solve are developed to further accelerate the solver. To tackle the strong data dependency in ILU and triangular solve, we first partition the algorithm into separated tasks and construct a data flow graph to represent the data dependency. Then, tasks are scheduled in parallel according to the topological order of the data flow graph. On an 8-thread multicore architecture, we achieved another 3.6x speedup on ILU factorization, and 3.3x on triangular solve with good scalability.

A parallel preconditioned conjugate gradient package for solving sparse linear systems on a Cray Y-MP

The purpose of the article is to present and evaluate usefulness of a new preconditioning technique (for the Gauss-Seidel algorithm), namely incomplete WZ factorization, for iterative solving of sparse and singular linear equations systems, which arise during modeling with Markov chains. The incomplete WZ factorization proposed here will be compared with the incomplete LU factorization in respect of amount of fill-in (newly created non-zeros) and in respect of the accuracy improvement of preconditioned algorithms in relation to not preconditioned ones. In the paper, the results of some numerical experiments will be presented, which were conducted for various matrices representing Markov chains. The experiments show that the incomplete WZ factorization can be a real alternative - because it is faster than incomplete LU factorization and the fill-in generated in the process is smaller (the output matrices are sparser).

When solving large linear systems of equations arising from the discretization of elliptic boundary value problems, a combination of iterative methods and preconditioners based on incomplete LU factorizations is frequently used. Given a model problem with variable coefficients, we investigate a class of incomplete LU factorizations depending on a relaxation parameter. We show that the associated preconditioner and the factorization itself both are numerically stable. The theoretical results are complemented by numerical experiments.

A highly efficient and robust scheme is proposed for analyzing electromagnetic scattering from electrically large arbitrary shaped conductors in a half space. This scheme is based on the electric field integral equation (EFIE) with a half-space Green’s function. The precorrected fast Fourier transform (p-FFT) is first extended to a half space for general three-dimensional scattering problems. A novel enhanced dual threshold incomplete LU factorization (ILUT) is then constructed as an effective preconditioner to improve the convergence of the half-space EFIE. Inspired by the idea of the improved electric field integral operator (IEFIO), the geometrical-optics current/principle value term of the magnetic field integral equation is used as a physical perturbation to stabilize the traditional ILUT perconditioning matrix. The high accuracy of EFIE is maintained, yet good calculating efficiency comparable to the combined field integral equation (CFIE) can be achieved. Furthermore, this approach can be applied to arbitrary geometrical structures including open surfaces and requires no extra types of Sommerfeld integrals needed in the half-space CFIE. Numerical examples are presented to demonstrate the high performance of the proposed solver among several other approaches in typical half-space problems.

This paper addresses the problem of computing preconditioners for solving linear systems of equations with a sequence of slowly varying matrices. This problem arises in many important applications. For example, a common situation in computational fluid dynamics, is when the equations change only slightly, possibly in some parts of the physical domain. In such situations it is wasteful to recompute entirely any LU or ILU factorizations computed for the previous coefficient matrix. A number of techniques for computing incremental ILU factorizations are examined. For example we consider methods based on approximate inverses as well as alternating techniques for updating the factors L and U of the factorization. Copyright © 2010 John Wiley & Sons, Ltd.

Chapter 9 - Conjugate Gradient-Type Methods

This paper presents the results of the Connection Machine implementation of a number of preconditioners for the preconditioned conjugate gradient method. The preconditioners implemented include those based on the incomplete LU factorization, the modified incomplete LU factorization, the symmetric successive overrelaxation, and others such as several polynomial preconditioners and the hierarchical basis preconditioner. Results based on numerical experiments show that both the degree of parallelism inherent in a preconditioner and its convergence rate improvement play important roles on the overall execution time performance on parallel computers. Factors that affect the performance of the preconditioners will also be discussed. We conclude that to search for the best preconditioner on a parallel machine, we have to consider the tradeoffs between fast convergence rate and high degree of parallelism as well as the architecture of the target parallel computer.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

This paper investigates parallel iterative solvers with preconditioning for the computation of magnetic and electric fields on a printed circuit board (PCB) containing a magnetic film. Special scaled symmetric quasi-minimal residual (QMR) methods and a bi-conjugate gradient stabilized (BICGSTAB) method, preconditioned by an incomplete LU factorization with threshold (ILUT), are presented. The results show that these solvers are well suited for PCB simulations on parallel computers, requiring both real and complex arithmetic.

Parallelization of the GESIMA mesoscale atmospheric model

In this study, a hybrid Chebyshev pseudo-spectral finite-difference time-domain (CPS-FDTD) algorithm is proposed for simulating 2D acoustic wave propagation in heterogeneous media, which is different from the other traditional numerical schemes such as finite element and finite difference. This proposed hybrid method integrates the efficiency of the FDTD approach in the time domain and the high accuracy of the CPS technique in the spatial domain. We present the calculation formulas of this novel approach and conduct simulation experiments to test it. The biconjugate gradient is solved by combining the large symmetric sparse systems stabilized algorithm with an incomplete LU factorization. Three numerical experiments are further presented to illustrate the accuracy, efficiency, and flexibility of the hybrid CPS-FDTD algorithm.

Incomplete LU factorization preconditioning techniques often have difficulty on indefinite sparse matrices. We present hybrid reordering strategies to deal with such matrices, which include new diagonal reorderings that are in conjunction with a symmetric nondecreasing degree algorithm. We first use the diagonal reorderings to efficiently search for entries of single element rows and columns and/or the maximum absolute value to be placed on the diagonal for computing a nonsymmetric permutation. To augment the effectiveness of the diagonal reorderings, a nondecreasing degree algorithm is applied to reduce the amount of fill-in during the ILU factorization. With the reordered matrices, we achieve a noticeable improvement in enhancing the stability of incomplete LU factorizations. Consequently, we reduce the convergence cost of the preconditioned Krylov subspace methods on solving the reordered indefinite matrices.

Parallel iterative algorithms based on the Newton method and on two of its variants, the Shamanskii method and the Chord method, for solving nonlinear systems are proposed. These algorithms are based on two‐stage multisplitting methods where incomplete LU factorizations are considered as a mean of constructing the inner splittings. Convergence properties of these parallel methods are studied for H‐matrices. Computational results of these methods on two parallel computing systems are discussed. The reported experiments show the effectiveness of these methods. Copyright © 2006 John Wiley & Sons, Ltd.