MIQR: A Multilevel Incomplete QR Preconditioner for Large Sparse Least‐Squares Problems

A grid-based multilevel incomplete LU factorization preconditioning technique for general sparse matrices

Standard preconditioners, like incomplete factorizations, perform well when the coefficient matrix is diagonally dominant, but often fail on general sparse matrices. We experiment with nonsymmetric permutations and scalings aimed at placing large entries on the diagonal in the context of preconditioning for general sparse matrices. The permutations and scalings are those developed by Olschowka and Neumaier [Linear Algebra Appl., 240 (1996), pp. 131--151] and by Duff and Koster [SIAM J. Matrix Anal. Appl., 20 (1999), pp. 889--901; Tech. report Ral-Tr-99-030, Rutherford Appleton Laboratory, Chilton, UK, 1999]. We target highly indefinite, nonsymmetric problems that cause difficulties for preconditioned iterative solvers. Our numerical experiments indicate that the reliability and performance of preconditioned iterative solvers are greatly enhanced by such preprocessing.

This paper presents a new fine-grained parallel algorithm for computing an incomplete LU factorization. All nonzeros in the incomplete factors can be computed in parallel and asynchronously, using one or more sweeps that iteratively improve the accuracy of the factorization. Unlike existing parallel algorithms, the amount of parallelism is large irrespective of the ordering of the matrix, and matrix ordering can be used to enhance the accuracy of the factorization rather than to increase parallelism. Numerical tests show that very few sweeps are needed to construct a factorization that is an effective preconditioner.

Numerical study on incomplete orthogonal factorization preconditioners

Robust linear and nonlinear strategies for solution of the transonic Euler equations

We investigate the scalability of the hypergraph-based sparse matrix partitioning methods with respect to the increasing sizes of matrices and number of nonzeros. We propose a method to rowwise partition the matrices that correspond to the discretization of two-dimensional domains with the five-point stencil. The proposed method obtains perfect load balance and achieves very good total communication volume. We investigate the behaviour of the hypergraph-based rowwise partitioning method with respect to the proposed method, in an attempt to understand how scalable the former method is. In another set of experiments, we work on general sparse matrices under different scenarios to understand the scalability of various hypergraph-based one- and two-dimensional matrix partitioning methods.

The discussion of some special algorithms within the general k-stage scheme will begin with the simplest example; Gaussian elimination (GE). If GE is used in the solution of linear algebraic equations with general sparse matrices, then the pivotal strategy plays a very important role. The pivotal strategy is a powerful tool that can efficiently be used during the efforts to preserve as well as possible the sparsity of the original matrix and, at the same time, to keep the rounding errors as small as possible. Normally these two requirements (preserving both sparsity and stability) cannot be satisfied similtaneously. Therefore a compromise is needed in order to obtain a balance between sparsity and accuracy. Some simple examples, where the lack of such a balance leads to catastrophic results, will be demonstrated in this chapter. Several pivotal strategies will be described and tested on many systems with sparse matrices. Some recommendations concerning the choice of a pivotal strategy for a sparse matrix code will be given.

Considering matrices obtained by the application of a five-point stencil on a 2D rectangular grid, we analyse a preconditioning method introduced by Axelsson and Eijkhout, and by Brand and Heinemann. In this method, one performs a (modified) incomplete factorization with respect to a so-called ‘repeated’ or ‘recursive’ red–black ordering of the unknowns while fill-in is accepted provided that the red unknowns in a same level remain uncoupled.Considering discrete second order elliptic PDEs with isotropic coefficients, we show that the condition number is bounded by [MATHEMATICAL SCRIPT CAPITAL O](nmath image) where n is the total number of unknowns (½ log2(√(5) − 1) = 0.153), and thus, that the total arithmetic work for the solution is bounded by [MATHEMATICAL SCRIPT CAPITAL O](n1.077). Our condition number estimate, which turns out to be better than standard [MATHEMATICAL SCRIPT CAPITAL O](log2n) estimates for any realistic problem size, is purely algebraic and holds in the presence of Neumann boundary conditions and/or discontinuities in the PDE coefficients.Numerical tests are reported, displaying the efficiency of the method and the relevance of our analysis. © 1997 John Wiley & Sons, Ltd.

Iterative linear equation solvers depend on high quality preconditioners to achieve fast convergence. For sparse symmetric systems arising from large power grid analysis problems, however, preconditioners generated by traditional incomplete Cholesky factorizations are usually of low quality, resulting in slow convergence. On the other hand, although preconditioners generated by random walks are quite effective to reduce the number of iterations, it takes considerable amount of time to compute them in a stochastic manner. In this paper, we propose a new preconditioning technique for power grid analysis, named deterministic random walk, that combines the advantages of the above two approaches. Our proposed algorithm computes the preconditioners in a deterministic manner to reduce computation time, while achieving similar quality as stochastic random walk preconditioning by modifying fill-ins to compensate dropped entries. We have proved that for such compensation scheme, our algorithm will not fail for certain matrix orderings, which otherwise cannot be guaranteed by traditional incomplete factorizations. We demonstrate that by incorporating our proposed preconditioner, a conjugate gradient solver is able to outperform a state-of-the-art algebraic multigrid preconditioned solver on public IBM power grid benchmarks for DC power grid analysis, while potentially remaining very efficient for transient analysis.

We consider the problem of vectorizing the recursive calculations found in modified incomplete factorizations and SSOR preconditioners for the conjugate gradient method. We examine matrix problems derived from partial differential equations which are discretized on regular 2-D and 3-D grids, where the grid nodes are ordered in the natural ordering. By performing data dependency analyses of the computations, we show that there is concurrency in both the factorization and the forward and backsolves. The computations may be performed with an average vector length of $O(n)$ on an $n^2 $ or $n^3 $ grid in two and three dimensions. Numerical studies on four model problems show that the conjugate gradient method using these vectorized implementations of the modified incomplete factorizations and SSOR preconditioners achieves overall speeds approaching 100 megaflops on a Cray X-MP/24 vector computer. Furthermore, these methods require considerably less overall execution time than diagonal scaling and no-fill red-black incomplete factorization preconditioners, both of which allow full vectorization but are not as convergent.

In this article, we present two new algorithms for solving given triangular systems in parallel on a shared memory architecture. Multilevel incomplete LU factorization based preconditioners, which have been very successful for solving linear systems iteratively, require these triangular solves. Hence, the algorithms presented here can be seen as parallelizing the application of these preconditioners. The first algorithm solves the triangular matrix by block anti-diagonals. The drawback of this approach is that it can be difficult to choose an appropriate block structure. On the other hand, if a good block partition can be found, this algorithm can be quite effective. The second algorithm takes a hybrid approach by solving the triangular system by block columns and anti-diagonals. It is usually as effective as the first algorithm, but the block structure can be chosen in a nearly optimal manner. Although numerical results indicate that the speed-up can be fairly good, systems with matrices having a strong diagonal structure or narrow bandwidth cannot be solved effectively in parallel. Hence, for these matrices, the results are disappointing. On the other hand, the results are better for matrices having a more uniform distribution of non-zero elements. Although not discussed in this article, these algorithms can possibly be adapted for distributed memory architectures.

Boundary element discretizations of exterior Maxwell problems lead to dense complex non-Hermitian systems of linear equations that are difficult to solve from a linear algebra point of view. We show that the recently developed class of inverse-based multilevel incomplete LU factorization has very good potential to precondition these systems effectively. This family of algorithms can produce numerically stable factorizations and exploits efficiently the possible symmetry of the underlying integral formulation. The results are highlighted by calculating the radar-cross-section of a full aircraft, and by a numerical comparison against other standard preconditioners.

New results concerning a recently introduced parallel preconditioner for the solution of large nonconforming FE linear systems are presented. The studied algorithm is based on the modified incomplete Cholesky factorization MIC(0) applied to a locally constructed approximation of the original stiffness matrix. The overlapping of communications and computations is possible due to a suitable reordering of the computations applied in the MPI code. The obtained improvement of the real performance is illustrated by numerical tests on a Beowulf-type Linux cluster and on a Sun Fire symmetric multiprocessor.

In this paper, we study a class of modified incomplete LDLT preconditioners followed by the conjugate orthogonal conjugate gradient (COCG) method for solving the large complex symmetric system with linear equations Ax = b, arising from the scattering problems. The preconditioner is derived from the incomplete Cholesky factorization with diagonal compensation. With an ordering scheme, our factorized preconditioners can reduce the memory requirement as well as the COCG iteration numbers. Numerical tests on harmonic analysis for plane wave scattering from a metallic plate and a metallic sphere coated by a lossy dielectric layer show the efficiency of our method.

Age-at-onset is one of the critical traits in cohort studies of age-related diseases. Large-scale genome-wide association studies (GWAS) of age-at-onset traits can provide more insights into genetic effects on disease progression and transitions between stages. Moreover, proportional hazards (or Cox) regression models can achieve higher statistical power in a cohort study than a case-control trait using logistic regression. Although mixed-effects models are widely used in GWAS to correct for sample dependence, application of Cox mixed-effects models (CMEMs) to large-scale GWAS is so far hindered by intractable computational cost. In this work, we propose COXMEG, an efficient R package for conducting GWAS of age-at-onset traits using CMEMs. COXMEG introduces fast estimation algorithms for general sparse relatedness matrices including, but not limited to, block-diagonal pedigree-based matrices. COXMEG also introduces a fast and powerful score test for dense relatedness matrices, accounting for both population stratification and family structure. In addition, COXMEG generalizes existing algorithms to support positive semidefinite relatedness matrices, which are common in twin and family studies. Our simulation studies suggest that COXMEG, depending on the structure of the relatedness matrix, is orders of magnitude computationally more efficient than coxme and coxph with frailty for GWAS. We found that using sparse approximation of relatedness matrices yielded highly comparable results in controlling false-positive rate and retaining statistical power for an ethnically homogeneous family-based sample. By applying COXMEG to a study of Alzheimer's disease (AD) with a Late-Onset Alzheimer's Disease Family Study from the National Institute on Aging sample comprising 3456 non-Hispanic whites and 287 African Americans, we identified the APOE ε4 variant with strong statistical power (P = 1e-101), far more significant than that reported in a previous study using a transformed variable and a marginal Cox model. Furthermore, we identified novel SNP rs36051450 (P = 2e-9) near GRAMD1B, the minor allele of which significantly reduced the hazards of AD in both genders. These results demonstrated that COXMEG greatly facilitates the application of CMEMs in GWAS of age-at-onset traits.