A robust inner–outer hierarchically semi‐separable preconditioner

We present a distributed memory algorithm for the approximate hierarchical of symmetric positive definite (SPD) matrices. Our method is based on the distributed memory GOFMM, an algorithm that appeared in SC18 (doi:10.1109/SC.2018.00018). GOFMM constructs a hierarchical matrix approximation of an arbitrary SPD matrix that compresses the matrix by creating low-rank approximations of the off-diagonal blocks. GOFMM method has no guarantees of success for arbitrary SPD matrices. (This is similar to the SVD; not every matrix admits a good low-rank approximation.) But for many SPD matrices, GOFMM does enable compression that results in fast matrix-vector multiplication that can reach N logN time—as opposed to N2 required for a dense matrix. GOFMM supports shared and distributed memory parallelism. In this paper, we build an approximate based on the Hierarchically Semi-Separable (HSS) compression of the GOFMM. This requires O(N) work (given the compressed matrix) and O(N=p) + O(log p) time on p MPI processes (assuming a hypercube topology). The previous state-of-the-art required O(N logN) work. We present the algorithm, discuss its complexity, and present weak and strong scaling results for the factorization and phases of our algorithm. We also discuss the performance of the inexact ULV as a preconditioner for a few exemplary large dense linear systems. In our largest run, we were able to factorize a 67M-by-67M matrix in less than one second; and solve a system with 64 right-hand sides in less than one-tenth of a second. This run was on 6,144 Intel Skylake cores on the SKX partition of the Stampede2 system at the Texas Advanced Computing Center.

Rank structures provide an opportunity to develop new efficient numerical methods for practical problems, when the off-diagonal blocks of certain dense intermediate matrices have small (numerical) ranks. In this work, we present a framework of structured direct factorizations for general sparse matrices, including discretized PDEs on general meshes, based on the multifrontal method and hierarchically semiseparable (HSS) matrices. We prove the idea of replacing certain complex structured operations by fast simple ones performed on compact reduced matrix forms. Such forms result from the hierarchical factorization of a tree-structured HSS matrix in a ULV-type scheme, so that the tree structure is reduced into a single node, the root of the original tree. This idea is shown to be very useful in the partial ULV factorization of an HSS matrix (for quickly computing Schur complements) as well as the solution stage. These techniques are then built into the multifrontal method for sparse factorizations after nested dissection, so as to convert the intermediate dense factorizations into fast structured ones. This method keeps certain Schur complements dense so as to avoid complicated data assembly, and is much simpler and more general than some existing methods. In particular, if the matrix arises from the discretization of certain PDEs, the factorization costs roughly $O(n)$ flops in two dimensions, and roughly $O(n^{4/3})$ flops or less in three dimensions. The solution cost and memory are nearly $O(n)$ in both cases. These counts are obtained with an idea of rank relaxation, so that this method is more generally applicable to problems where the intermediate off-diagonal ranks are not small. We demonstrate the performance of the method with two- and three-dimensional discretized equations, as well as various examples from a sparse matrix collection. The ideas here are also useful in future developments of fast structured solvers.

In this paper, in order to improve the convergence rates of iterative method solving the linear system, the improving modified Gauss-Seidel (IMGS) iterative method with a preconditioner is proposed. Some convergence and comparison results are given when is a symmetric definite matrix are provided.

In this paper, a modified Hermitian and skew-Hermitian splitting (MHSS) iteration method for solving the complex linear matrix equation $AXB = C$ has been presented. As the theoretical analysis shows, the MHSS iteration method will converge under certain conditions. Each iteration in this method requires the solution of four linear matrix equations with real symmetric positive definite coefficient matrices, although the original coefficient matrices are complex and non-Hermitian. In addition, the optimal parameter of the new iteration method is proposed. Numerical results show that MHSS iteration method is efficient and robust.

In this paper, we introduce and analyze an accelerated preconditioning modification of the Hermitian and skew-Hermitian splitting (APMHSS) iteration method for solving a broad class of complex symmetric linear systems. This accelerated PMHSS algorithm involves two iteration parameters ź,β and two preconditioned matrices whose special choices can recover the known PMHSS (preconditioned modification of the Hermitian and skew-Hermitian splitting) iteration method which includes the MHSS method, as well as yield new ones. The convergence theory of this class of APMHSS iteration methods is established under suitable conditions. Each iteration of this method requires the solution of two linear systems with real symmetric positive definite coefficient matrices. Theoretical analyses show that the upper bound ź1(ź,β) of the asymptotic convergence rate of the APMHSS method is smaller than that of the PMHSS iteration method. This implies that the APMHSS method may converge faster than the PMHSS method. Numerical experiments on a few model problems are presented to illustrate the theoretical results and examine the numerical effectiveness of the new method.

Computer Solution of Large Linear Systems

A generalization of preconditioned MHSS iteration method for complex symmetric indefinite linear systems

SUMMARY A generalized skew-Hermitian triangular splitting iteration method is presented for solving non-Hermitian linear systems with strong skew-Hermitian parts. We study the convergence of the generalized skew-Hermitian triangular splitting iteration methods for non-Hermitian positive definite linear systems, as well as spectrum distribution of the preconditioned matrix with respect to the preconditioner induced from the generalized skew-Hermitian triangular splitting. Then the generalized skew-Hermitian triangular splitting iteration method is applied to non-Hermitian positive semidefinite saddle-point linear systems, and we prove its convergence under suitable restrictions on the iteration parameters. By specially choosing the values of the iteration parameters, we obtain a few of the existing iteration methods in the literature. Numerical results show that the generalized skew-Hermitian triangular splitting iteration methods are effective for solving non-Hermitian saddle-point linear systems with strong skew-Hermitian parts. Copyright © 2013 John Wiley & Sons, Ltd.

A derivation of the Levinson algorithm for solving linear systems with symmetric positive definite Toeplitz matrix

In this paper we will adapt a known method for diagonal scaling of symmetric positive definite tridiagonal matrices towards the semiseparable case. Based on the fact that a symmetric, positive definite tridiagonal matrix $$T$$ satisfies property A, one can easily construct a diagonal matrix $$\hat{D}$$ such that $$\hat{D}T\hat{D}$$ has the lowest condition number over all matrices $$DTD$$ , for any choice of diagonal matrix $$D$$ . Knowing that semiseparable matrices are the inverses of tridiagonal matrices, one can derive similar properties for semiseparable matrices. Here, we will construct the optimal diagonal scaling of a semiseparable matrix, based on a new inversion formula for semiseparable matrices. Some numerical experiments are performed. In a first experiment we compare the condition numbers of the semiseparable matrices before and after the scaling. In a second numerical experiment we compare the scalability of matrices coming from the reduction to semiseparable form and matrices coming from the reduction to tridiagonal form.

A Schur-based algorithm for computing bounds to the smallest eigenvalue of a symmetric positive definite Toeplitz matrix

Symmetric Positive Definite (SPD) matrices have been used in many fields of medical data analysis. Many Riemannian metrics have been defined on this manifold but the choice of the Riemannian structure lacks a set of principles that could lead one to choose properly the metric. This drives us to introduce the principle of balanced metrics that relate the affine-invariant metric with the Euclidean and inverse-Euclidean metric, or the Bogoliubov-Kubo-Mori metric with the Euclidean and log-Euclidean metrics. We introduce two new families of balanced metrics, the mixed-power-Euclidean and the mixed-power-affine metrics and we discuss the relation between this new principle of balanced metrics and the concept of dual connections in information geometry.

The geometry of mixed-Euclidean metrics on symmetric positive definite matrices

In order to compute the log-likelihood for high dimensional Gaussian models, it is necessary to compute the determinant of the large, sparse, symmetric positive definite precision matrix. Traditional methods for evaluating the log-likelihood, which are typically based on Cholesky factorisations, are not feasible for very large models due to the massive memory requirements. We present a novel approach for evaluating such likelihoods that only requires the computation of matrix-vector products. In this approach we utilise matrix functions, Krylov subspaces, and probing vectors to construct an iterative numerical method for computing the log-likelihood.

In this paper, a class of two-step modulus-based matrix splitting (TMMS) iteration methods are proposed to solve the quasi-complementarity problems. In the new TMMS iteration method, we first use the modulus technique to equivalently transform the quasi-complementarity problem into a fixed point equation, and then construct a class of two-step matrix splitting iteration methods to further accelerate the convergence rates. Theoretical results show that the proposed TMMS iteration methods are global convergent under certain conditions when the system matrix is either a positive definite matrix or an $$H_+$$-matrix. Finally, two numerical examples are presented to demonstrate that the TMMS iteration methods are superior to some existing modulus-based iteration methods studied recently for solving the quasi-complementarity problems.