A general purpose sparse matrix parallel solvers package

CHAPTER 5 - Ordering for Gauss Elimination: General Matrices

In several applications in science and engineering, different types of matrix problems emerge from the discretization of partial differential equations. This thesis is devoted to the development of new algorithms to solve this kind of problems. In particular, when the matrices involved are sparse and non-symmetric. The new algorithms are based on the Induced Dimension Reduction method [IDR(s)]. IDR(s) is a Krylov subspace method originally proposed in 2008 to solve systems of linear equations. IDR(s) has received considerable attention due to its stable and fast convergence. It is, therefore, natural to ask if it is possible to extend IDR(s) to solve other matrix problems, and if so, to compare those extensions with other well-established methods. This work aims to answer these questions. The main matrix problems considered in this dissertation are: the standard eigenvalue problem, the quadratic eigenvalue problem, the solution of systems of linear equations, the solution of sequences of systems of linear equations, and linear matrix equations. We focus on examples that arise from the discretization of partial differential equations.

Scalable line and plane relaxation in a parallel structured multigrid solver

Simulators of semiconductor devices have to solve systems of equations generated by the discretization of partial differential equations, which are the most time-consuming part of the simulation process. Therefore, the use of an effective method to solve these linear systems is essential. In this work we have evaluated the efficiency of different parallel direct and iterative solvers used for the solution of the drift–diffusion equations in semiconductor device simulation. Several preconditioning techniques have been applied in order to minimize the execution times. We have found that FGMRES and BCGSTAB solvers preconditioned with Additive Schwarz are the most suitable for these types of problems. The results were obtained in an HP Superdome cluster with 128 Itanium2 1.5 GHz. Copyright © 2005 John Wiley & Sons, Ltd.

The Lanczos algorithm is among the most frequently used iterative techniques for computing a few dominant eigenvalues of a large sparse non-symmetric matrix. At the same time, it serves as a building block within biconjugate gradient (BiCG) and quasi-minimal residual (QMR) methods for solving large sparse non-symmetric systems of linear equations. It is well known that, when implemented on distributed-memory computers with a huge number of processes, the synchronization time spent on computing dot products increasingly limits the parallel scalability. Therefore, we propose synchronization-reducing variants of the Lanczos, as well as BiCG and QMR methods, in an attempt to mitigate these negative performance effects. These so-called s-step algorithms are based on grouping dot products for joint execution and replacing time-consuming matrix operations by efficient vector recurrences. The purpose of this paper is to provide a rigorous derivation of the recurrences for the s-step Lanczos algorithm, introduce s-step BiCG and QMR variants, and compare the parallel performance of these new s-step versions with previous algorithms.

In this paper we study the parallel performance of some nonlinear additive Schwarz preconditioned inexact Newton methods for solving large sparse system of nonlinear equations arising from the discretization of partial differential equations. The main idea of nonlinear preconditioning is to replace an ill-conditioned nonlinear system by an equivalent nonlinear system that has more balanced nonlinearities. In addition to balance the nonlinearities through nonlinear preconditioning, we also need to make sure that the multilayered iterative solver is scalable with respect to the number of processors. We focus on some two-level nonlinear additive Schwarz preconditioners, and show numerically that these two-level methods can reduce the nonlinearities and at the same time maintain the parallel scalability. Parallel numerical results for some high Reynolds number incompressible Navier-Stokes equations will be presented.

Efficient integration algorithms and solvers specially devised for boundary‐element procedures have been established over the last two decades. A good deal of quadrature techniques for singular and quasi‐singular boundary‐element integrals have been developed and reliable Krylov solvers have proven to be advantageous when compared to direct ones, also in case of non‐Hermitian matrices. The former has implied in CPU‐time reduction during the assembling of the system of equations and the latter in its faster solution. Here, a triangular polar co‐ordinate transformation and the Telles co‐ordinate transformation are employed separately and combined to develop the matrix‐assembly routines (integration routines). In addition, the Jacobi‐preconditioned Biconjugate Gradient solver (J‐BiCG) is used along with a generic substructuring boundary element algorithm. Thus, solution CPU time and computer memory can be considerably reduced. Discontinuous boundary elements are also included to simplify the coupling of the BE models (substructures). Numerical experiments involving 3D thin‐walled domains (shell‐like structural elements) are carried out to show the performance of the computer code with respect to accuracy and efficiency of the system solution. Precision, CPU‐time and potential applications of the BE code developed are commented upon. Copyright © 2006 John Wiley & Sons, Ltd.

Three-dimensional finite-element modelling of magnetotelluric data with a divergence correction

In this paper a preconditioned iterative method suitable for the solution of the generalised eigenvalue problem is presented. The proposed method in which no change in the structure of the original matrices occurs is suitable for the determination of the extreme eigenvalues and their corresponding eigenvectors of large sparse matrices derived from finite element/difference discretisation of partial differential equations. The new method when coupled with the conjugate gradient algorithm yields a powerful algorithm for this class of problems.

We propose a new approximate factorization for solving linear systems with symmetric positive definite sparse matrices. In a nutshell the algorithm is to apply hierarchically block Gaussian elimination and additionally compress the fill-in. The systems that have efficient compression of the fill-in mostly arise from discretization of partial differential equations. We show that the resulting factorization can be used as an efficient preconditioner and compare the proposed approach with state-of-art direct and iterative solvers.

During my visit to LLNL during the summer of 2010, I worked on algebraic multilevel solvers for large sparse systems of linear equations arising from discretizations of partial differential equations. The particular solver of interest is based on ILU decomposition. The setup phase for this AMG solve is just the single ILU decomposition, and its corresponding error matrix. Because the ILU uses a minimum degree or similar sparse matrix ordering, most of the fill-in, and hence most of the error, is concentrated in the lower right corner of the factored matrix. All of the major multigrid components - the smoother, the coarse level correction matrices, and the fine-to-coarse and coarse-to-fine rectangular transfer matrices, are defined in terms of various blocks of the ILU factorization. Although such a strategy is not likely to be optimal in terms of convergence properties, it has a relatively low setup cost, and therefore is useful in situations where setup costs for more traditional AMG approaches cannot be amortized over the solution of many linear systems using the same matrix. Such a situation arises in adaptive methods, where often just one linear system is solved at each step of an adaptive feedback loop, or in solving nonlinear equations by approximate Newton methods, where the approximate Jacobian might change substantially from iteration to iteration. In general terms, coarse levels are defined in terms of successively smaller lower right blocks of the matrix, typically decreasing geometrically in order. The most difficult issue was the coarse grid correction matrix. The preconditioner/smoother for a given level is just the corresponding lower right blocks of the ILU factorization. The coarse level matrix itself is just the Schur complement; this matrix is not known exactly using just the ILU decomposition in the setup phase. Thus we approximate this matrix using various combinations of the preconditioning matrix and the error matrix. During my visit, several approximations of this type were implemented and tested. While some improved the convergence rate of the overall method, these gains had to be balanced against the additional costs involved in creating and applying these matrices. By this more stringent criterion, none of the improved approximations could be characterized as an unqualified success.

The existing emergency rescue systems face the following problems: rescue systems belong to different rescue brigades and are independent of each other, there is no guarantee that the data interface is consistent with the National Safety Supervision Bureau, the bureau and provincial rescue centers can't get the important rescue information timely and accurately; In addition, the rescue brigades cannot share the large-scale equipment, human resources, training materials, and rescue prearranged planning cases to reduce the cost. We put forward the mine emergency rescue platform based on SOA, which supports national, provincial level and brigade level rescue management, with unified interface, entirely sharing of resources such as large equipment, online examination, simulation training, rescue prearranged planning case and so on to reduce the maintenance costs. The test results of platform show that the platform can easily support all the national emergency rescue management units/users to concurrent access, the throughput is 97.6 transactions/sec, the number of concurrent users is 2100, and 99% of transactions are finished in 81ms.

Approximate sparsity patterns for the inverse of a matrix and preconditioning

New parallel reservoir simulator software designed for Linux clusters enable to overcome hardware limitation and to simulate models with large amount of data. Reservoir engineering industry is very interested in using ever growing dataset with more and more complex physics and detailed models. The key issue still remains running simulations in an acceptable CPU time. As, the trend in hardware technologies is not to improve drastically the performance of individual CPUs but to facilitate the aggregation of computation facilities (with high bandwidth network, multi-core architectures …), the challenge is to improve the efficiency of reservoir simulation software on a large number of processors. New numerical difficulties and performance problems appear when the number of cells and the number of processors are growing. As a matter of fact, the architecture of Linux clusters is very sensible to memory distribution and load balancing: the cost of parallel solver algorithm is usually sensible to the size of the reservoir model (lack of scalability) and the consequences on CPU performance can no more be neglected;the domain decomposition algorithms used to distribute data between processors have a great influence on the computing load balancing between processors;using adaptive numerical schemes with dynamic space criteria (AIM schemes, flash algorithms based on the thermodynamic state of each cell) is a source of unbalance that cannot statically be resolved;simulation result storages on irregular data structures, such as unstructured grids, multilateral smart wells and perforated cells, lead to store an important amount of information during the simulation. With the variety of IO subsystems found on Linux clusters the simulator must be able to adapt its IO strategy to the underlying IO library/file system and hardware. In this paper, we present different approaches to overcome these kinds of problems. We discuss technical choices such like: advanced scalable linear solver algorithm;load balancing issue with different domain decomposition strategies;dynamic space criteria, mesh partitioner strategy and parallel solver performance management;flexible IO strategy from simple file system to more complex parallel file system or database. We have developed and benchmarked these different solutions on published reference large scale problems and actual case studies with several tens millions of cells. We analyze the results and discuss the efficiency of each solution to overcome the scalability difficulties and performance limitations due to load unbalance.

Sparse Matrix Vector multiplication (SpMV) is an important kernel in both traditional high performance computing and emerging data-intensive applications. By far, SpMV libraries are optimized by either application-specific or architecture-specific approaches, making the libraries become too complicated to be used extensively in real applications. In this work we develop a Sparse Matrix-vector multiplication Auto-Tuning system (SMAT) to bridge the gap between specific optimizations and general-purpose usage. SMAT provides users with a unified programming interface in compressed sparse row (CSR) format and automatically determines the optimal format and implementation for any input sparse matrix at runtime. For this purpose, SMAT leverages a learning model, which is generated in an off-line stage by a machine learning method with a training set of more than 2000 matrices from the UF sparse matrix collection, to quickly predict the best combination of the matrix feature parameters. Our experiments show that SMAT achieves impressive performance of up to 51GFLOPS in single-precision and 37GFLOPS in double-precision on mainstream x86 multi-core processors, which are both more than 3 times faster than the Intel MKL library. We also demonstrate its adaptability in an algebraic multigrid solver from Hypre library with above 20% performance improvement reported.