Parallel Newton Iterative Methods Based on Incomplete LU Factorizations for Solving Nonlinear Systems

Parallel iterative algorithms based on the Newton method and on two of its variants, the Shamanskii method and the Chord method, for solving nonlinear systems are proposed. These algorithms are based on two‐stage multisplitting methods where incomplete LU factorizations are considered as a mean of constructing the inner splittings. Convergence properties of these parallel methods are studied for H‐matrices. Computational results of these methods on two parallel computing systems are discussed. The reported experiments show the effectiveness of these methods. Copyright © 2006 John Wiley & Sons, Ltd.

The purpose of the article is to present and evaluate usefulness of a new preconditioning technique (for the Gauss-Seidel algorithm), namely incomplete WZ factorization, for iterative solving of sparse and singular linear equations systems, which arise during modeling with Markov chains. The incomplete WZ factorization proposed here will be compared with the incomplete LU factorization in respect of amount of fill-in (newly created non-zeros) and in respect of the accuracy improvement of preconditioned algorithms in relation to not preconditioned ones. In the paper, the results of some numerical experiments will be presented, which were conducted for various matrices representing Markov chains. The experiments show that the incomplete WZ factorization can be a real alternative - because it is faster than incomplete LU factorization and the fill-in generated in the process is smaller (the output matrices are sparser).

Modern petroleum reservoir simulation serves as a primary tool for quantitatively managing reservoir production and planning new fields. It involves repeatedly solving the Jacobian of a set of strong nonlinear partial differential equations governing the mass and energy conduction and conservation. Most of the existing reservoir simulators adopt iterative solver with multiple stages of preconditioners, in which the incomplete LU (ILU) factorization is an outstanding universal smoother. However, it turns out that when the degree of freedom of each grid grows, ILU usually becomes the bottleneck of the solver. Moreover, ILU is difficult to parallelize due to its inherent data dependency. In this paper, we developed a sparse iterative solver with parallelized ILU and triangular solve using block-wise data structure. Compared with the state of art iterative solver on 14 industrial reservoir simulation matrices, the proposed ILU is 5.2x faster (on average) than the state of art iterative solver because of the block-wise data structure, which leads to 2.2x speedup on the total solver runtime. In addition, parallel ILU and triangular solve are developed to further accelerate the solver. To tackle the strong data dependency in ILU and triangular solve, we first partition the algorithm into separated tasks and construct a data flow graph to represent the data dependency. Then, tasks are scheduled in parallel according to the topological order of the data flow graph. On an 8-thread multicore architecture, we achieved another 3.6x speedup on ILU factorization, and 3.3x on triangular solve with good scalability.

When solving large linear systems of equations arising from the discretization of elliptic boundary value problems, a combination of iterative methods and preconditioners based on incomplete LU factorizations is frequently used. Given a model problem with variable coefficients, we investigate a class of incomplete LU factorizations depending on a relaxation parameter. We show that the associated preconditioner and the factorization itself both are numerically stable. The theoretical results are complemented by numerical experiments.

A highly efficient and robust scheme is proposed for analyzing electromagnetic scattering from electrically large arbitrary shaped conductors in a half space. This scheme is based on the electric field integral equation (EFIE) with a half-space Green’s function. The precorrected fast Fourier transform (p-FFT) is first extended to a half space for general three-dimensional scattering problems. A novel enhanced dual threshold incomplete LU factorization (ILUT) is then constructed as an effective preconditioner to improve the convergence of the half-space EFIE. Inspired by the idea of the improved electric field integral operator (IEFIO), the geometrical-optics current/principle value term of the magnetic field integral equation is used as a physical perturbation to stabilize the traditional ILUT perconditioning matrix. The high accuracy of EFIE is maintained, yet good calculating efficiency comparable to the combined field integral equation (CFIE) can be achieved. Furthermore, this approach can be applied to arbitrary geometrical structures including open surfaces and requires no extra types of Sommerfeld integrals needed in the half-space CFIE. Numerical examples are presented to demonstrate the high performance of the proposed solver among several other approaches in typical half-space problems.

Application demands and grand challenges in numerical simulation require for both highly capable computing platforms and efficient numerical solution schemes. Power constraints and further miniaturization of modern and future hardware give way for multi- and manycore processors with increasing fine-grained parallelism and deeply nested hierarchical memory systems -- as already exemplified by recent graphics processing units. Accordingly, numerical schemes need to be adapted and re-engineered in order to deliver scalable solutions across diverse processor configurations. Portability of parallel software solutions across emerging hardware platforms is another challenge. This work investigates multi-coloring and re-ordering schemes for block Gauß-Seidel methods and, in particular, for incomplete LU factorizations with and without fill-ins. We consider two matrix re-ordering schemes that deliver flexible and efficient parallel preconditioners. The general idea is to generate block decompositions of the system matrix such that the diagonal blocks are diagonal itself. In such a way, parallelism can be exploited on the block-level in a scalable manner. Our goal is to provide widely applicable, out-of-the-box preconditioners that can be used in the context of finite element solvers. We propose a new method for anticipating the fill-in pattern of ILU($p$) schemes which we call the power($q$)-pattern method. This method is based on an incomplete factorization of the system matrix $A$ subject to a predetermined pattern given by the matrix power $|A|^(p+1)$ and its associated multi-coloring permutation $. We prove that the obtained sparsity pattern is a superset of our modified ILU($p$) factorization applied to pi A pi^(-1). As a result, this modified ILU($p$) applied to multi-colored system matrix has no fill-ins in its diagonal blocks. This leads to an inherently parallel execution of triangular ILU($p$) sweeps. In addition, we describe the integration of the preconditioners into the HiFlow$^3$ open-source finite element package that provides a portable software solution across diverse hardware platforms. On this basis, we conduct performance analysis across a variety of test problems on multi-core CPUs and GPUs that proves efficiency, scalability and flexibility of our approach. Our preconditioners achieve a solver acceleration by a factor of up to 1.5, 8 and 85 for three different test problems. The GPU versions of the preconditioned solver are by a factor of up to 4 faster than an OpenMP parallel version on eight cores.

This paper presents a new fine-grained parallel algorithm for computing an incomplete LU factorization. All nonzeros in the incomplete factors can be computed in parallel and asynchronously, using one or more sweeps that iteratively improve the accuracy of the factorization. Unlike existing parallel algorithms, the amount of parallelism is large irrespective of the ordering of the matrix, and matrix ordering can be used to enhance the accuracy of the factorization rather than to increase parallelism. Numerical tests show that very few sweeps are needed to construct a factorization that is an effective preconditioner.

This paper presents the results of the Connection Machine implementation of a number of preconditioners for the preconditioned conjugate gradient method. The preconditioners implemented include those based on the incomplete LU factorization, the modified incomplete LU factorization, the symmetric successive overrelaxation, and others such as several polynomial preconditioners and the hierarchical basis preconditioner. Results based on numerical experiments show that both the degree of parallelism inherent in a preconditioner and its convergence rate improvement play important roles on the overall execution time performance on parallel computers. Factors that affect the performance of the preconditioners will also be discussed. We conclude that to search for the best preconditioner on a parallel machine, we have to consider the tradeoffs between fast convergence rate and high degree of parallelism as well as the architecture of the target parallel computer.

The aim of this work is to build a numerical software library based on the \(\tau \)-method to solve differential problems using MATLAB. The \(\tau \)-method can be very effective in the solution of certain type of these problems, and therefore, the existence of a numerical library for its dissemination is of major importance. Furthermore, the method has been used for the solution of particular problems but has not yet been explored as a general technique. Focus will be on stability issues, namely those issued from the solution of algebraic linear systems required for the process. Additionally, preconditioners for the solution with the \(\tau \)-method will be tackled, with emphasizes on incomplete LU factorizations and (block) Jacobi preconditioners. We also propose an iterative approach, build upon an LU factorization over a moderate initial size, generating better approximations and providing a priori error estimate at each iteration. Numerical results enlightening the efficiency of the proposed methods will be presented.

This paper addresses the problem of computing preconditioners for solving linear systems of equations with a sequence of slowly varying matrices. This problem arises in many important applications. For example, a common situation in computational fluid dynamics, is when the equations change only slightly, possibly in some parts of the physical domain. In such situations it is wasteful to recompute entirely any LU or ILU factorizations computed for the previous coefficient matrix. A number of techniques for computing incremental ILU factorizations are examined. For example we consider methods based on approximate inverses as well as alternating techniques for updating the factors L and U of the factorization. Copyright © 2010 John Wiley & Sons, Ltd.

This paper investigates parallel iterative solvers with preconditioning for the computation of magnetic and electric fields on a printed circuit board (PCB) containing a magnetic film. Special scaled symmetric quasi-minimal residual (QMR) methods and a bi-conjugate gradient stabilized (BICGSTAB) method, preconditioned by an incomplete LU factorization with threshold (ILUT), are presented. The results show that these solvers are well suited for PCB simulations on parallel computers, requiring both real and complex arithmetic.

Parallelization of the GESIMA mesoscale atmospheric model

In this study, a hybrid Chebyshev pseudo-spectral finite-difference time-domain (CPS-FDTD) algorithm is proposed for simulating 2D acoustic wave propagation in heterogeneous media, which is different from the other traditional numerical schemes such as finite element and finite difference. This proposed hybrid method integrates the efficiency of the FDTD approach in the time domain and the high accuracy of the CPS technique in the spatial domain. We present the calculation formulas of this novel approach and conduct simulation experiments to test it. The biconjugate gradient is solved by combining the large symmetric sparse systems stabilized algorithm with an incomplete LU factorization. Three numerical experiments are further presented to illustrate the accuracy, efficiency, and flexibility of the hybrid CPS-FDTD algorithm.

Incomplete LU factorization preconditioning techniques often have difficulty on indefinite sparse matrices. We present hybrid reordering strategies to deal with such matrices, which include new diagonal reorderings that are in conjunction with a symmetric nondecreasing degree algorithm. We first use the diagonal reorderings to efficiently search for entries of single element rows and columns and/or the maximum absolute value to be placed on the diagonal for computing a nonsymmetric permutation. To augment the effectiveness of the diagonal reorderings, a nondecreasing degree algorithm is applied to reduce the amount of fill-in during the ILU factorization. With the reordered matrices, we achieve a noticeable improvement in enhancing the stability of incomplete LU factorizations. Consequently, we reduce the convergence cost of the preconditioned Krylov subspace methods on solving the reordered indefinite matrices.

Practical considerations for adaptive implicit methods in reservoir simulation