Fine-Grained Parallel Incomplete LU Factorization

The purpose of the article is to present and evaluate usefulness of a new preconditioning technique (for the Gauss-Seidel algorithm), namely incomplete WZ factorization, for iterative solving of sparse and singular linear equations systems, which arise during modeling with Markov chains. The incomplete WZ factorization proposed here will be compared with the incomplete LU factorization in respect of amount of fill-in (newly created non-zeros) and in respect of the accuracy improvement of preconditioned algorithms in relation to not preconditioned ones. In the paper, the results of some numerical experiments will be presented, which were conducted for various matrices representing Markov chains. The experiments show that the incomplete WZ factorization can be a real alternative - because it is faster than incomplete LU factorization and the fill-in generated in the process is smaller (the output matrices are sparser).

When solving large linear systems of equations arising from the discretization of elliptic boundary value problems, a combination of iterative methods and preconditioners based on incomplete LU factorizations is frequently used. Given a model problem with variable coefficients, we investigate a class of incomplete LU factorizations depending on a relaxation parameter. We show that the associated preconditioner and the factorization itself both are numerically stable. The theoretical results are complemented by numerical experiments.

Application demands and grand challenges in numerical simulation require for both highly capable computing platforms and efficient numerical solution schemes. Power constraints and further miniaturization of modern and future hardware give way for multi- and manycore processors with increasing fine-grained parallelism and deeply nested hierarchical memory systems -- as already exemplified by recent graphics processing units. Accordingly, numerical schemes need to be adapted and re-engineered in order to deliver scalable solutions across diverse processor configurations. Portability of parallel software solutions across emerging hardware platforms is another challenge. This work investigates multi-coloring and re-ordering schemes for block Gauß-Seidel methods and, in particular, for incomplete LU factorizations with and without fill-ins. We consider two matrix re-ordering schemes that deliver flexible and efficient parallel preconditioners. The general idea is to generate block decompositions of the system matrix such that the diagonal blocks are diagonal itself. In such a way, parallelism can be exploited on the block-level in a scalable manner. Our goal is to provide widely applicable, out-of-the-box preconditioners that can be used in the context of finite element solvers. We propose a new method for anticipating the fill-in pattern of ILU($p$) schemes which we call the power($q$)-pattern method. This method is based on an incomplete factorization of the system matrix $A$ subject to a predetermined pattern given by the matrix power $|A|^(p+1)$ and its associated multi-coloring permutation $. We prove that the obtained sparsity pattern is a superset of our modified ILU($p$) factorization applied to pi A pi^(-1). As a result, this modified ILU($p$) applied to multi-colored system matrix has no fill-ins in its diagonal blocks. This leads to an inherently parallel execution of triangular ILU($p$) sweeps. In addition, we describe the integration of the preconditioners into the HiFlow$^3$ open-source finite element package that provides a portable software solution across diverse hardware platforms. On this basis, we conduct performance analysis across a variety of test problems on multi-core CPUs and GPUs that proves efficiency, scalability and flexibility of our approach. Our preconditioners achieve a solver acceleration by a factor of up to 1.5, 8 and 85 for three different test problems. The GPU versions of the preconditioned solver are by a factor of up to 4 faster than an OpenMP parallel version on eight cores.

This paper describes a multilevel incomplete QR factorization for solving large sparse least‐squares problems. The algorithm builds the factorization by exploiting structural orthogonality in general sparse matrices. At any given step, the algorithm finds an independent set of columns, i.e., a set of columns that have orthogonal patterns. The other columns are then block orthogonalized against columns of the independent set, and the process is repeated recursively for a certain number of levels on these remaining columns. The final level matrix is processed with a standard QR or incomplete QR factorization. Dropping strategies are employed throughout the levels in order to maintain a good level of sparsity. A few improvements to this basic scheme are explored. Among these is the relaxation of the requirement of independent sets of columns. Numerical tests are proposed which compare this scheme with the standard incomplete QR preconditioner, the robust incomplete factorization preconditioner, and the algebraic recursive multilevel solver (on normal equations).

This paper presents the results of the Connection Machine implementation of a number of preconditioners for the preconditioned conjugate gradient method. The preconditioners implemented include those based on the incomplete LU factorization, the modified incomplete LU factorization, the symmetric successive overrelaxation, and others such as several polynomial preconditioners and the hierarchical basis preconditioner. Results based on numerical experiments show that both the degree of parallelism inherent in a preconditioner and its convergence rate improvement play important roles on the overall execution time performance on parallel computers. Factors that affect the performance of the preconditioners will also be discussed. We conclude that to search for the best preconditioner on a parallel machine, we have to consider the tradeoffs between fast convergence rate and high degree of parallelism as well as the architecture of the target parallel computer.

Parallel iterative algorithms based on the Newton method and on two of its variations, the Shamanskii method and the Chord method, for solving nonlinear systems are proposed. These algorithms also use techniques from the non–stationary multisplitting methods. Concretely, in order to construct the multisplitting, ILU factorizations are considered. Convergence properties of these parallel methods are studied for H–matrices. Computational results, on a distributed multiprocessor IBM RS/6000 SP, that show the effectiveness of these methods are included to illustrate the theoretical results. Topics: Numerical methods (nonlinear algebra), Parallel and distributed computing.

A highly efficient and robust scheme is proposed for analyzing electromagnetic scattering from electrically large arbitrary shaped conductors in a half space. This scheme is based on the electric field integral equation (EFIE) with a half-space Green’s function. The precorrected fast Fourier transform (p-FFT) is first extended to a half space for general three-dimensional scattering problems. A novel enhanced dual threshold incomplete LU factorization (ILUT) is then constructed as an effective preconditioner to improve the convergence of the half-space EFIE. Inspired by the idea of the improved electric field integral operator (IEFIO), the geometrical-optics current/principle value term of the magnetic field integral equation is used as a physical perturbation to stabilize the traditional ILUT perconditioning matrix. The high accuracy of EFIE is maintained, yet good calculating efficiency comparable to the combined field integral equation (CFIE) can be achieved. Furthermore, this approach can be applied to arbitrary geometrical structures including open surfaces and requires no extra types of Sommerfeld integrals needed in the half-space CFIE. Numerical examples are presented to demonstrate the high performance of the proposed solver among several other approaches in typical half-space problems.

Robust linear and nonlinear strategies for solution of the transonic Euler equations

The iterative solution of linear equation systems resulting from Method of Moments (MoM) discretizations of integral equations is of particular attractiveness because of the possibility to employ fast integral methods such as the Multilevel Fast Multipole Method (MLFMM). However, the robustness of the iterative solvers is often still not satisfying and the search for improved preconditioners is an ongoing process. In this paper, we concentrate on two classical near-zone preconditioning techniques: Gauss-Seidel smoothing and a special form of incomplete LU factorization. It is found that Gauss-Seidel smoothing is a relatively cheap preconditioner working well for fine meshes. Our special form of incomplete LU factorization gives reliable convergence for complex problems with very bad convergence behavior, regardless of mesh density but for the cost of increased memory requirements.

Application of two preconditioned generalizedconjugate gradient methods to three-dimensional neutron and photon transport equations

Chapter 9 - Conjugate Gradient-Type Methods

A parallel preconditioned conjugate gradient package for solving sparse linear systems on a Cray Y-MP

О р д е н а Л е н и н а ИНСТИТУТ ПРИКЛАДНОЙ МАТЕМАТИКИ имени М. В. Келдыша Р о с с и й с к о й а к а д е м и и н а у к И.Е.Капорин, О. Ю. Милюкова Неполное обратное треугольное разложение в параллельных алгоритмах

The aim of this work is to build a numerical software library based on the \(\tau \)-method to solve differential problems using MATLAB. The \(\tau \)-method can be very effective in the solution of certain type of these problems, and therefore, the existence of a numerical library for its dissemination is of major importance. Furthermore, the method has been used for the solution of particular problems but has not yet been explored as a general technique. Focus will be on stability issues, namely those issued from the solution of algebraic linear systems required for the process. Additionally, preconditioners for the solution with the \(\tau \)-method will be tackled, with emphasizes on incomplete LU factorizations and (block) Jacobi preconditioners. We also propose an iterative approach, build upon an LU factorization over a moderate initial size, generating better approximations and providing a priori error estimate at each iteration. Numerical results enlightening the efficiency of the proposed methods will be presented.

This paper addresses the problem of computing preconditioners for solving linear systems of equations with a sequence of slowly varying matrices. This problem arises in many important applications. For example, a common situation in computational fluid dynamics, is when the equations change only slightly, possibly in some parts of the physical domain. In such situations it is wasteful to recompute entirely any LU or ILU factorizations computed for the previous coefficient matrix. A number of techniques for computing incremental ILU factorizations are examined. For example we consider methods based on approximate inverses as well as alternating techniques for updating the factors L and U of the factorization. Copyright © 2010 John Wiley & Sons, Ltd.