Low-rank updates of balanced incomplete factorization preconditioners

Introduction. Kron [1] has developed a technique for tearing large, sparse linear systems of algebraic equations into smaller systems, then putting the solutions of these smaller systems together to form the solution of the original problem. This can result in a great savings over solving the whole system in one piece. The trick is to find where to tear so that on the one hand the remaining pieces are small and easily solvable, but on the other hand not so much violence is done to the equations so that the task of putting their solutions back together becomes too difncult. Kron's pro? cedure for solving the pieces and putting their solutions together is based upon representing systems of equations as electric networks. He makes his choice of where to tear from his insight into these networks and the physical processes the equations represent. In the hands of someone such as Kron who has an understanding of the methods and good insight into the prob? lem, this technique has had outstanding success. The widespread use of Kron's technique has perhaps been limited severely for lack of practitioners skilled in this art of determining effective ways of tearing specific problems. Many problems are too large to submit to such insight. It is the purpose of this paper to contribute to the development of procedures which can be performed on a computer for analyzing the structures of the systems of equations themselves as an aid in choosing how to break them up for easier solution. The methods developed here require only that we consider what variables occur in each equation, but not how they appear. Thus, the tech? niques are not limited to linear systems.

This paper presents a novel hardware solution for solving sparse matrices using emulation technology. The proposed solution introduces a bio-inspired technique, which is Genetic Algorithm (GA) to solve the sparse system of linear equations (SLEs). The proposed solution can be used with any physical system that can be modeled as SLEs. Computations involved in Finite Element Method (FEM) consume too much time, which affects the final time-to-market value. Profiling shows that the most time-consuming part in the simulation process is the solver part which is responsible for solving the resultant SLEs generated from the FEM. The total number of equations may reach thousands or millions of linear equations. Hardware implementation of GA and making a good use of parallelism in implementing the architecture accelerate the time taken by the solver part. Solving the system by classical numerical methods, such as conjugate gradient (CG) implemented in software is performed sequentially, here the architecture allows the parallelism in performing GA operators, such as selection, crossover, and mutation. The parallelism in the proposed architecture is the superiority point which we rely on. The results show that our proposed method to solve the SLEs outperforms the classical CG software implementation on run time and number of equations.

During the past few years, algorithmic improvements alone have reduced the time required for the direct solution of unsymmetric sparse systems of linear equations by almost an order of magnitude. This paper compares the performance of some well-known software packages for solving general sparse systems. In particular, it demonstrates the consistently high level of performance achieved by WSMP---the most recent of such solvers. It compares the various algorithmic components of these solvers and discusses their impact on solver performance. Our experiments show that the algorithmic choices made in WSMP enable it to run more than twice as fast as the best among similar solvers and that WSMP can factor some of the largest sparse matrices available from real applications in only a few seconds on a 4-CPU workstation. Thus, the combination of advances in hardware and algorithms makes it possible to solve those general sparse linear systems quickly and easily that might have been considered too large until recently.

A projection method for solving nonsymmetric linear systems on multiprocessors

Element-by-element iterative solution methods have been shown to be effective for solving large, sparse linear systems of equations with a positive definite stiffness matrix. In this paper the iterative method GMRES will be described. This solution method is also suited for solving nonsymmetric systems of equations. As acceleration technique an element-by-element preconditioner has been employed. Attention is focused mainly on the convergence behaviour of the method. The calculations described in this paper have been performed with the finite element package DIANA.

It is known that the main computational costs in the FEM implementation are computing the global system of linear equations (material response computation and stiffness matrix ensemble) and its solution. With the aim of speeding up the total solving boundary value problem time with the help of the developed solver, algorithms of the parallel stiffness matrix ensemble and the system of linear equations solution were implemented. In this paper the results of using GPU to accelerate computations related to the solution of direct boundary value problems of crystal plasticity are present. The existing algorithms for solving the sparse systems of linear equations and parallel ensemble of the global stiffness matrix were implemented on both the CPU and GPU. As a result, a total acceleration of up to 10 times was obtained compared to the software implementation on the CPU.

During my visit to LLNL during the summer of 2010, I worked on algebraic multilevel solvers for large sparse systems of linear equations arising from discretizations of partial differential equations. The particular solver of interest is based on ILU decomposition. The setup phase for this AMG solve is just the single ILU decomposition, and its corresponding error matrix. Because the ILU uses a minimum degree or similar sparse matrix ordering, most of the fill-in, and hence most of the error, is concentrated in the lower right corner of the factored matrix. All of the major multigrid components - the smoother, the coarse level correction matrices, and the fine-to-coarse and coarse-to-fine rectangular transfer matrices, are defined in terms of various blocks of the ILU factorization. Although such a strategy is not likely to be optimal in terms of convergence properties, it has a relatively low setup cost, and therefore is useful in situations where setup costs for more traditional AMG approaches cannot be amortized over the solution of many linear systems using the same matrix. Such a situation arises in adaptive methods, where often just one linear system is solved at each step of an adaptive feedback loop, or in solving nonlinear equations by approximate Newton methods, where the approximate Jacobian might change substantially from iteration to iteration. In general terms, coarse levels are defined in terms of successively smaller lower right blocks of the matrix, typically decreasing geometrically in order. The most difficult issue was the coarse grid correction matrix. The preconditioner/smoother for a given level is just the corresponding lower right blocks of the ILU factorization. The coarse level matrix itself is just the Schur complement; this matrix is not known exactly using just the ILU decomposition in the setup phase. Thus we approximate this matrix using various combinations of the preconditioning matrix and the error matrix. During my visit, several approximations of this type were implemented and tested. While some improved the convergence rate of the overall method, these gains had to be balanced against the additional costs involved in creating and applying these matrices. By this more stringent criterion, none of the improved approximations could be characterized as an unqualified success.

For the large sparse block two-by-two real nonsingular matrices, we establish a general framework of practical and efficient structured preconditioners through matrix transformation and matrix approximations. For the specific versions such as modified block Jacobi-type, modified block Gauss-Seidel-type, and modified block unsymmetric (symmetric) Gauss-Seidel-type preconditioners, we precisely describe their concrete expressions and deliberately analyze eigenvalue distributions and positive definiteness of the preconditioned matrices. Also, we show that when these structured preconditioners are employed to precondition the Krylov subspace methods such as GMRES and restarted GMRES, fast and effective iteration solvers can be obtained for the large sparse systems of linear equations with block two-by-two coefficient matrices. In particular, these structured preconditioners can lead to efficient and high-quality preconditioning matrices for some typical matrices from the real-world applications.

Computer subroutines for solving large sparse systems of linear equations usually involve fairly complicated data structures and storage management. In most cases, the user of such subroutines simply wants top solve his problem and does not have to understand how the storage management is done, or how the matrix components are stored. Thus software for solving sparse systems of linear equations should be designed in such a way that it can insulate the user from these considerations, while still allowing the software to be used in a variety of ways.

Image acquisition of magnetic resonance imaging (MRI) can be accelerated by using multiple receiving coils simultaneously. The problem of reconstructing an unaliased image from partially sampled k-space data can be formulated as a large system of sparse linear equations. The k-space sparse matrix (kSPA) algorithm proposes to solve the system of equations by finding a sparse approximate inverse. This algorithm has been shown to accelerate the image reconstruction for a large number of coils. The original kSPA algorithm requires knowledge of coil sensitivities. Here, we propose and demonstrate an auto-calibrated kSPA algorithm that does not require the explicit computation of the coil sensitivity maps. We have also shown that calibration data, in principle, can be acquired at any region of k-space. This property applies to arbitrary sampling trajectories and all reconstruction algorithms based on k-space. In practice, because of its higher SNR, calibration data acquired at the center of k-space performed more favorably. Such auto-calibration can be advantageous in cases where an accurate sensitivity map is difficult to obtain.

In this paper, an FPGA-based single precision floating point hybrid iterative architecture for solving a linear system of equations is proposed. The novel architecture implements Gauss-Seidel method using a Jacobi method based building block. The design takes advantage of the fast convergence of Gauss-Seidel Method conflated with parallel, pipelined architecture of Jacobi Iterative solver resulting in a much efficient architecture with acceptable hardware augmentation. The whole design has been implemented in Verilog HDL, having Virtex 7 XCV2000T as targeted device. Other design optimizations include using modified high-speed radix 4 multiplier and optimized high-speed 2's complementer. The efficacy of the design is tested and implemented in solving nonsingular, exactly determined and strictly diagonally dominant coefficient matrix based dense and sparse linear system of equations with different number of variables. The design results in reduced number of iterations and equivalent speedups are presented, which are calculated factoring in, the increased delay effects.

The DEFLATED-GMRES(m, k) method is one of the major iterative solvers for the large sparse linear systems of equations, x = b. This algorithm assembles a preconditioner adaptively for the GMRES(m) method based on eigencomponents gathered from the Arnoldi process during iterations. It is usually known that if a restarted GMRES(m) method is used to solve linear systems of equations, the information of the smallest eigencomponents is lost at each restart and the super-linear convergence may also be lost. In this paper, we propose an adaptive procedure that combines the DEFLATED-GMRES(m, k) algorithm and the determination of a restart frequency automatically. It is shown that a new algorithm combining elements of both will reduce the negative effects of the restarted procedure. The numerical experiments are presented on three test problems by using the MIMD parallel machine AP3000. From these numerical results, we show that the proposed algorithm leads to faster convergence than the conventional DEFLATED-GMRES(m, k) method. Copyright © 2000 John Wiley & Sons, Ltd.

We examine the application of QMR methods to the solution of linear systems of equations arising from the use of implicit solution methods in computational fluid dynamics. We will deal with implicit finite difference schemes for solving the Euler equations. These schemes may arise from the implicit treatment of the time dependent equations or from the use of Newton`s method for the solution of the steady state equations. In both situations it is necessary to solve a large sparse nonsymmetric linear system of equations at each iteration. We will examine the effectiveness of QMR in the solution of these systems. W e compare the resulting methods to methods which rely on some other simplifying technique to solve the linear systems. Our goal is to show that the QMR method is a viable alternative to the more ad-hoc schemes for solving implicit computational fluid dynamics problems.

FIRE5: A C++ implementation of Feynman Integral REduction

Many applications in science and engineering give rise to large sparse linear systems of equations that need to be solved as efficiently as possible. As the size of the problems of interest increases, it can become necessary to consider exploiting multiprocessors to solve these systems. We report on the design and development of parallel frontal solvers for the numerical solution of large sparse linear systems. Three codes have been developed for the mathematical software library HSL (www.cse.clrc.ac.uk/Activity/HSL). The first is for unsymmetric finite-element problems; the second is for symmetric positive definite finite-element problems; and the third is for highly unsymmetric linear systems such as those that arise in chemical process engineering. In each case, the problem is subdivided into a small number of loosely connected subproblems and a frontal method is then applied to each of the subproblems in parallel. We discuss how our software is designed to achieve the goals of portability, ease of use, efficiency, and flexibility, and illustrate the performance using problems arising from real applications.