Numerical study on incomplete orthogonal factorization preconditioners

There has been much recent interest in the use of incomplete factorizations of matrices, in conjunction with applications of the generalized conjugate gradient method, for approximating solutions of large sparse systems of linear equations. Underlying many of these recent developments is the theory of H-matrices, introduced by A. M. Ostrowski. In this note, further connections of the theory of incomplete factorizations of matrices with the theory of H-matrices are derived.

This chapter is concerned with efficient methods for iterative solution of large sparse systems of linear equations, typically derived from the discretization of an elliptic boundary value problem. In particular, attention is given to the family of Krylov subspace methods, as well as to several preconditioning strategies that are suitable for improving the convergence rates of such iterations.

We are interested in the design of efficient algebraic multigrid (AMG) methods for the solution of large sparse systems of linear equations arising from finite element (FE) discretization of second-order elliptic partial differential equations (PDEs). In particular, we introduce the concept of so-called “edge matrices”, which–in the present context–are extracted from the individual element matrices. This allows for the construction of spectrally equivalent approximations of the original stiffness matrix that can be utilized in the framework of AMG. The edge matrices give rise to modify the definition of “strong” and “weak” connections (edges), which provides a basis for selecting the coarse-grid nodes in algebraic multigrid methods. Moreover, a reproduction of edge matrices on coarse levels offers the opportunity to combine classical coarsening algorithms with effective (energy minimizing) interpolation principles involving small-sized “computational molecules” (small collections of edge matrices). This yields a flexible and robust new variant of AMG, which we refer to as AMGm.

The application of the finite element method invariably involves the solution of large sparse systems of linear algebraic equations, and the solution of these systems often represents a significant or even dominant component of the total cost of applying the method. The object of this paper is to describe and relate various sparse matrix techniques which have been developed to make the solution of these equations more efficient.KeywordsStiffness MatrixSparse MatrixDiagonal BlockInterior NodeStorage SchemeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Let Ax = b be a large and sparse system of linear equations where A is a nonsingular matrix. An approximate solution is frequently obtained by applying preconditioned iterations. Consider the matrix B = A + PQT where P,Qźźn×k$P, Q \in \mathbb {R}^{n \times k}$ are full rank matrices. In this work, we study the problem of updating a previously computed preconditioner for A in order to solve the updated linear system Bx = b by preconditioned iterations. In particular, we propose a method for updating a Balanced Incomplete Factorization preconditioner. The strategy is based on the computation of an approximate Inverse Sherman-Morrison decomposition for an equivalent augmented linear system. Approximation properties of the preconditioned matrix and an analysis of the computational cost of the algorithm are studied. Moreover, the results of the numerical experiments with different types of problems show that the proposed method contributes to accelerate the convergence.

Computer subroutines for solving large sparse systems of linear equations usually involve fairly complicated data structures and storage management. In most cases, the user of such subroutines simply wants top solve his problem and does not have to understand how the storage management is done, or how the matrix components are stored. Thus software for solving sparse systems of linear equations should be designed in such a way that it can insulate the user from these considerations, while still allowing the software to be used in a variety of ways.

: This report documents a program for the numerical solution of large sparse systems of algebraic and implicitly defined stiff differential equations. The principal use is intended to be the solution of differential equations arising from time dependent partial differential equations when the finite element method is used to discretize the space domain. The use of compact matrix storage techniques and iteration for the solution of the quasi-Newton iterates in Gear's method makes the program extremely efficient both in terms of storage requirements and execution times. (Author)

We consider preconditioned Krylov subspace iteration methods, e.g., CG, LSQR and GMRES, for the solution of large sparse least‐squares problems min ∥Ax – b ∥2, with A ∈ Rm×n, based on the Krylov subspaces Kk (BA, Br) and Kk (AB, r), respectively, where B ∈ Rn×m is the preconditioning matrix. More concretely, we propose and implement a class of incomplete QR factorization preconditioners based on the Givens rotations and analyze in detail the efficiency and robustness of the correspondingly preconditioned Krylov subspace iteration methods. A number of numerical experiments are used to further examine their numerical behaviour. It is shown that for both overdetermined and underdetermined least‐squares problems, the preconditioned GMRES methods are the best for large, sparse and ill‐conditioned matrices in terms of both CPU time and iteration step. Also, comparisons with the diagonal scaling and the RIF preconditioners are given to show the superiority of the newly‐proposed GMRES‐type methods. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

We present a class of new preconditioners based on the incomplete Givens orthogonalization (IGO) methods for solving large sparse systems of linear equations. In the new methods, instead of dropping entries and accepting fill-ins according to the magnitudes of values and the sparsity patterns, we adopt a diagonal compensation strategy, in which the dropped entries are re-used by adding to the main diagonal entries of the same rows of the incomplete upper-triangular factors, possibly after suitable relaxation treatments, so that certain constraints on the preconditioning matrices are further satisfied. This strategy can make the computed preconditioning matrices possess certain desired properties, e.g., having the same weighted row sums as the target matrices. Theoretical analysis shows that these modified incomplete Givens orthogonalization (MIGO) methods can preserve certain useful properties of the original matrix, and numerical results are used to verify the stability, the accuracy, and the efficiency of the MIGO methods employed to precondition the Krylov subspace iteration methods such as GMRES. Both theoretical and numerical studies show that the MIGO methods may have the potential to present high-quality preconditioners for large sparse nonsymmetric matrices.

This paper presents a novel hardware solution for solving sparse matrices using emulation technology. The proposed solution introduces a bio-inspired technique, which is Genetic Algorithm (GA) to solve the sparse system of linear equations (SLEs). The proposed solution can be used with any physical system that can be modeled as SLEs. Computations involved in Finite Element Method (FEM) consume too much time, which affects the final time-to-market value. Profiling shows that the most time-consuming part in the simulation process is the solver part which is responsible for solving the resultant SLEs generated from the FEM. The total number of equations may reach thousands or millions of linear equations. Hardware implementation of GA and making a good use of parallelism in implementing the architecture accelerate the time taken by the solver part. Solving the system by classical numerical methods, such as conjugate gradient (CG) implemented in software is performed sequentially, here the architecture allows the parallelism in performing GA operators, such as selection, crossover, and mutation. The parallelism in the proposed architecture is the superiority point which we rely on. The results show that our proposed method to solve the SLEs outperforms the classical CG software implementation on run time and number of equations.

An integrated approach for the parallel solution of large sparse systems arisen in finite element computations is presented. The approach includes a three-phase preprocessor and a macro dataflow execution scheme. The three phases of the preprocessor are: (1) Extracting parallelism by means of an automatic domain decomposer; (2) Building the distributed data structure and (partial) scheduling for parallel computation during symbolic factorization; (3) Assigning processes (tasks) onto processors. The proposed approach has been implemented in the finite element analysis software package DIANA. Experimental results show that this integrated approach is an efficient method for both shared-and distributed-memory parallel systems.

Many of the fast methods for factoring integers and computing discrete logarithms require the solution of large sparse linear systems of equations over finite fields. This paper presents the results of implementations of several linear algebra algorithms. It shows that very large sparse systems can be solved efficiently by using combinations of structured Gaussian elimination and the conjugate gradient, Lanczos, and Wiedemann methods.

During the past few years, algorithmic improvements alone have reduced the time required for the direct solution of unsymmetric sparse systems of linear equations by almost an order of magnitude. This paper compares the performance of some well-known software packages for solving general sparse systems. In particular, it demonstrates the consistently high level of performance achieved by WSMP---the most recent of such solvers. It compares the various algorithmic components of these solvers and discusses their impact on solver performance. Our experiments show that the algorithmic choices made in WSMP enable it to run more than twice as fast as the best among similar solvers and that WSMP can factor some of the largest sparse matrices available from real applications in only a few seconds on a 4-CPU workstation. Thus, the combination of advances in hardware and algorithms makes it possible to solve those general sparse linear systems quickly and easily that might have been considered too large until recently.

Over the past decade as Moore's Law has slowed, the need for new forms of computation that can provide sustainable performance improvements has risen. A new method, called in situ computing, has shown great potential to accelerate matrix vector multiplication (MVM), an important kernel for a diverse range of applications from neural networks to scientific computing. Existing in situ accelerators for scientific computing, however, have a significant limitation: these accelerators provide no acceleration for preconditioning-a key bottleneck in linear solvers and in scientific computing workflows. This paper enables in situ acceleration for state-of-the-art linear solvers by demonstrating how to use a new in situ matrix inversion accelerator for analog preconditioning. As existing techniques that enable high precision and scalability for in situ MVM are inapplicable to in situ matrix inversion, new techniques to compensate for circuit non-idealities are proposed. Additionally, a new approach to bit slicing that enables splitting operands across multiple devices without external digital logic is proposed. For scalability, this paper demonstrates how in situ matrix inversion kernels can work in tandem with existing domain decomposition techniques to accelerate the solutions of arbitrarily large linear systems. The analog kernel can be directly integrated into existing preconditioning workflows, leveraging several well-optimized numerical linear algebra tools to improve the behavior of the circuit. The result is an analog preconditioner that is more effective (up to 50% fewer iterations) than the widely used incomplete LU factorization preconditioner, ILU(0), while also reducing the energy and execution time of each approximate solve operation by 1025x and 105x respectively.

This paper investigates the performance of a class of methods based on the Preconditioned Conjugate Gradient (PCG) method for the solution of large and sparse systems of linear equations which arise from the p‐version of the adaptive finite element method. The hierarchical type of preconditioning used in the past is extended to the SSOR and incomplete factorization type preconditioning and applied with a global handling of the stiffness matrix. The paper also presents some new ideas on Domain Decomposition (DD) matrix‐handling techniques, whereby the explicit formulation of the resulting Schur complement is avoided by utilizing the PCG method with efficient preconditioners and by processing separately on the coarse and the fine mesh. The suitability of the methods to the p‐version of the adaptive finite element method is demonstrated through extensive numerical tests in two dimensions.