Geometry-oblivious FMM for compressing dense SPD matrices

We present a distributed memory algorithm for the approximate hierarchical of symmetric positive definite (SPD) matrices. Our method is based on the distributed memory GOFMM, an algorithm that appeared in SC18 (doi:10.1109/SC.2018.00018). GOFMM constructs a hierarchical matrix approximation of an arbitrary SPD matrix that compresses the matrix by creating low-rank approximations of the off-diagonal blocks. GOFMM method has no guarantees of success for arbitrary SPD matrices. (This is similar to the SVD; not every matrix admits a good low-rank approximation.) But for many SPD matrices, GOFMM does enable compression that results in fast matrix-vector multiplication that can reach N logN time—as opposed to N2 required for a dense matrix. GOFMM supports shared and distributed memory parallelism. In this paper, we build an approximate based on the Hierarchically Semi-Separable (HSS) compression of the GOFMM. This requires O(N) work (given the compressed matrix) and O(N=p) + O(log p) time on p MPI processes (assuming a hypercube topology). The previous state-of-the-art required O(N logN) work. We present the algorithm, discuss its complexity, and present weak and strong scaling results for the factorization and phases of our algorithm. We also discuss the performance of the inexact ULV as a preconditioner for a few exemplary large dense linear systems. In our largest run, we were able to factorize a 67M-by-67M matrix in less than one second; and solve a system with 64 right-hand sides in less than one-tenth of a second. This run was on 6,144 Intel Skylake cores on the SKX partition of the Stampede2 system at the Texas Advanced Computing Center.

Symmetric positive definite (SPD) matrices used as feature descriptors in image recognition are usually high dimensional. Traditional manifold learning is only applicable for reducing the dimension of high-dimensional vector-form data.For high-dimensional SPD matrices, directly using manifold learning algorithms to reduce the dimension of matrix-form data is impossible. The SPD matrix must first be transformed into a long vector, and then the dimension of this vector must be reduced. However, this approach breaks the spatial structure of the SPD matrix space. To overcome this limitation, we propose a new dimension reduction algorithm on SPD matrix space to transform high-dimensional SPD matrices into low-dimensional SPD matrices. Our work is based on the fact that the set of all SPD matrices with the same size has a Lie group structure, and we aim to transform the manifold learning to the SPD matrix Lie group. We use the basic idea of the manifold learning algorithm called locality preserving projection (LPP) to construct the corresponding Laplacian matrix on the SPD matrix Lie group. Thus, we call our approach Lie-LPP to emphasize its Lie group character. We present a detailed algorithm analysis and show through experiments that Lie-LPP achieves effective results on human action recognition and human face recognition.

<p><b>Metric Learning has proved valuable in information retrieval and classification problems, with many algorithms using statistical formulations to their advantage. Optimal Transport, a powerful framework for computing distances between probability distributions, has seen significant uptake in Machine Learning and some applications in Metric Learning. The Bures-Wasserstein distance, a particular formulation of Optimal Transport, has seen limited application in Metric Learning despite being well suited to the field. In this thesis Linear Metric Learning algorithms incorporating the Bures-Wasserstein distance are developed, utilising its novel properties. The first set of algorithms we contribute use the Bures-Wasserstein distance for learning linear metrics using Cyclic Projections, Riemannian barycenters and gradient-based algorithms. The convergence properties and classification performance of these algorithms are then compared to benchmark Linear Metric Learning algorithms. Our algorithms achieve competitive classification performance on feature and image datasets and converge reliability.</b></p> <p>The second set of contributions extend Linear Metric Learning with the Bures-Wasserstein distance to a low-rank context to address issues related to scalability. An extension of the previous Cyclic Projections algorithm and a Riemannian optimisation method are developed. Convergence and classification experiments analogous to the full-rank case are conducted to examine how the reduced rank affects both of these performance metrics. Both algorithms converge reliably under a variety of synthetic conditions. Reduction in rank generally decreased the training time for algorithms until effects caused by the size of the training data dominated. Reducing rank either left classification error unchanged or increased it. With respect to other low-rank Linear Metric Learning algorithms our algorithms again performed competitively with respect to classification error. </p> <p>The last set of contributions consider Linear Metric Learning on Symmetric Positive Definite (SPD) matrices. Some datasets are expressed as or are suited to a SPD representation. As such, Metric Learning algorithms that respect the structure of these matrices have found success when adapted to this kind of data. Another extension of the Cyclic Projections algorithm is developed to use the Generalised Bures-Wasserstein distance on SPD matrices. Two other methods based on a ground metric formulation of the Bures-Wasserstein distance for learning linear metrics on SPD data are also introduced. One ground metric formulation uses an approximation (inspired by the Sinkhorn distance) to increase its scalability. Convergence experiments on synthetic problems adapted to SPD matrices are conducted on the ground metric algorithms to establish their convergence properties. Classification experiments are also conducted with respect to several types of SPD data. While the Cyclic Projections method is competitive, the ground metric algorithms have poor performance. Despite their poor classification performance, the approximation method does evaluate faster than the Generalised Bures-Wasserstein distance, potentially offering some scalability improvements.</p>

Eddy current nondestructive evaluation (NDE) of airframe structures involves the detection of electromagnetic field irregularities due to non-conducting inhomogeneities in an electrically conducting material. Usually, the eddy current NDE problem can be formulated by the boundary integral equations (BIE) and discretized into matrix equations by the method of moments (MoM) or the boundary element method (BEM). The fast multipole method (FMM) is a well-established and effective method for accelerating numerical solutions of the matrix equations. Accelerated by the FMM, the BIE method can now solve large-scale electromagnetic wave propagation and diffusion problems. The traditional BIE method requires O(N) operations to compute the system of equations and another O(N) operations to solve the system using direct solvers, with N being the number of unknowns; in contrast, the BIE method accelerated by the two-level FMM can potentially reduce the operations and memory requirement to O(N). Moreover, several approaches have been proposed for the field calculation in the presence of flaws in three dimensional NDE; however, seldom work has been done in applying efficient methods to seek rapid solution in eddy current NDE simulation. As elaborated in the dissertation, we introduce a fast multipole BIE method for two-dimensional diffusive scalar problem and an efficient BIE method for three-dimensional eddy current NDE. Firstly, we work with the two-dimensional Helmholtz equation with a complex wave number for non-trivial boundary geometry. We describe the FMM acceleration procedure of the BIE method and its features briefly, explaining that the FMM is not only efficient in meshing complicated geometries, accurate for solving singular fields or

Representing images and videos with Symmetric Positive Definite (SPD) matrices, and considering the Riemannian geometry of the resulting space, has been shown to yield high discriminative power in many visual recognition tasks. Unfortunately, computation on the Riemannian manifold of SPD matrices -especially of high-dimensional ones- comes at a high cost that limits the applicability of existing techniques. In this paper, we introduce algorithms able to handle high-dimensional SPD matrices by constructing a lower-dimensional SPD manifold. To this end, we propose to model the mapping from the high-dimensional SPD manifold to the low-dimensional one with an orthonormal projection. This lets us formulate dimensionality reduction as the problem of finding a projection that yields a low-dimensional manifold either with maximum discriminative power in the supervised scenario, or with maximum variance of the data in the unsupervised one. We show that learning can be expressed as an optimization problem on a Grassmann manifold and discuss fast solutions for special cases. Our evaluation on several classification tasks evidences that our approach leads to a significant accuracy gain over state-of-the-art methods.

Though computational molecular dynamics is an effective tool for nano-scale phenomenon analysis, computational costs associated with its computer simulation are extremely high. There are two major computational steps associated with computer simulation of dynamics of molecular structures. They are calculation of interatomic forces and formation and solution of the equations of motion. Currently, these two computational steps are treated separately in most commonly-used methods. For example, Fast Multipole Method (FMM) and Cell Multipole Method (CMM) have been used for calculation of interatomic forces, and Cartesian Coordinate Method (CCM) and Internal Coordinate Molecular Dynamics Method (ICMD) have been created for the formation and solution of equations of motion of an atomistic molecular system. In this paper, a new procedure is presented through a proper integration between multibody molecular algorithms (MMA) and fast multipole methods to improve computational efficiency for computer simulation of the dynamical behaviors of multibody molecular structures in polymers and biopolymers. For the computational costs associated with interatomic forces, a fast multipole method is used to calculate the interatomic forces due to the potentials. For the computational costs associated with formation and solution of equations of motion, a multibody molecular algorithm developed by the author in his previous work will be utilized to integrate with fast multipole methods. The algorithm significantly improves computational efficiency when comparing with its counterpart procedures. The fast multipole method begins by scaling all atoms into a box with coordinate ranges to ensure numerical stability of subsequent operations. The parent box is then divided into half in the direction of each Cartesian axis and each child box is then subdivided to form a computational family tree. The flow of calculations is carried out along the tree structure with five passes. The fast multipole method has been improved and modified to achieve better effectiveness and higher efficiency since it was created. The multibody molecular algorithm starts with numbering subsets, forming bond graph, and developing three computing passes along the tree structure of an atomistic molecular system. Computing data flows in the fast multipole method and the multibody molecular algorithm will properly line up with the parent-child recursive relationship along the configuration of the tree structure due to linear recursive natures of both fast multipole method and multibody molecular algorithm. Then the time spent on the recursive simulation passes in the fast multipole method for computing forces may overlap with the time spent on the three recursive computational passes in the multibody molecular algorithm for forming and solving equations of motion.

We consider the solution of elliptic partial differential equations with random input data. In particular, we are interested in the mean and the correlation of the solution of these partial differential equations. Once the correlation is available, the variance can be computed from its diagonal. If the dependence of the solution on the random input data is linear and the mean of the input data is available, the mean of the solution can be computed whenever the corresponding partial differential equation can be solved. When the correlation of the input data is available, the correlation of the solution is given as the solution of a higher dimensional partial differential equation in the product domain. A Galerkin discretization yields a matrix equation with a typically densely populated right-hand side. This and the higher dimension of the problem makes the equation prohibitively expensive to solve. Since existing approaches to these correlation equations are known to struggle for roughly correlated input data, we use hierarchical matrices and their corresponding arithmetic to represent and solve the matrix equation. The regularity assumptions required for hierarchical matrices can be justified on sufficiently smooth domains. The feasibility of the approach is illustrated for finite element and boundary element discretizations and several specialities of hierarchical matrices for finite element discretizations are discussed. An almost linear scaling with respect to the dimension of the used finite element spaces is verified. The nonlinear dependence of the solution on the random input data is exemplarily discussed for the case of random domains. Extending previous perturbation approaches, we can linearize the problem and can compute the mean and the correlation up to third order accuracy in almost linear time. We discuss that these spaces can become even fourth order accurate under certain circumstances. A full convergence analysis is presented, which shows that higher order boundary elements are required for the solution of the problem. Therefore, we develop a fast multipole method for higher order boundary element methods on parametric surfaces.

Abstract: In this paper, a half space adaptive fast multipole boundary facemethod (FMBFM) is presented for solving the three-dimensional half space ex-terior acoustic problems. In the presented method, the Burton-Miller equationbased on the conventional boundary integral equation (CBIE) and its hyper-singularboundary integral equation (HBIE) is used to deal with the fictitious eigenfrequen-cies problem. The half space Green’s function is employed, thus the tree structurein the fast multipole method can be used only for the real domain. The higher orderelements and an adaptive tree structure are used to improve the efficiency of theFMBFM. This half space adaptive FMBFM for half space acoustic problems is anextension of the adaptive FMBFM for full space acoustic problems developed bythe authors. Numerical examples for half space acoustic problems in this paperdemonstrate the efficiency and validity of this method.Keywords: fast multipole boundary face method, Burton-Miller equation, acous-tic problems, half space, modified hyper-singular boundary integral equation1 IntroductionAcoustic problems in half space are a major subject in engineering. Many researchworks have been done on this aspect (Seybert et al., 1988; Secgin et al., 2010; Caoet al., 2010). The half space Green’s function (Seybert et al., 1988) can be appliedby adding the source solution at the image point to the original full-space Green’sfunction for dealing with half space problems. Sound source localization and iden-tification of a refrigerator were accomplished by using the half space boundaryelement method (BEM) (Secgin et al., 2010). Using the half space Green’s func-tion, the discretization of the rigid infinite plane is removed and only the boundaries

The computation of the acoustic scattering from objects in the ocean is useful to analyze the characteristics of the targets, which can be numerically modelled by acoustic boundary element method (BEM). The BEM has a defect of fictitious frequency problem, Burton-Miller method is studied to overcome the singular problems. In this paper, the Hadamard finite-part integral method based on collocation method is studied to overcome the singularity. It is found that, the near singularity is also quite significant and deteriorates largely the accuracy of BEM. A method based on polar transformation is proposed in this paper to overcome such a defect. Once the problems of singularity and near-singularity are solved, the conventional BEM model is established to serve as a foundation for fast BEM.It is well known that conventional BEM yields a full matrix, which is of O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) memory complexity. Furthermore, the time complexity could be up to O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ), if Gaussian elimination method is adopted to solve the linear equation of BEM. Many fast methods are proposed to solve the problems, such as the GMRES iterative solver, fast multipole method and H-matrix (Hierarchical Matrices) method. H-matrix method is applied to overcome the faults of conventional BEM, by using the low-rank approximations to compress blocks of the matrix, which is a pure algebraic method. The ACA (Adaptive Cross Approximation) method is proposed to compute the outer-product form of low-rank matrices, instead of SVD (Singular Value Decomposition), which consumes high CPU time and is of no usability. ACA-BEM is used to predict the acoustic scattering from objects in the ocean, especially the large-scale problems. The directivities of a rigid sphere and a rigid ellipsoid are computed by ACA BEM, the time consumed is very short. For a problem of more than 20,000 DOFs, it can be solved within 2 minutes. Therefore, ACA BEM can be used to solve large-scale acoustic scattering problems very efficiently.

Fast multipole singular boundary method for Stokes flow problems

Standard numerical models in acoustics rely on the isentropic Helmholtz equation. Its derivation assumes adiabatic and reversible, i.e., dissipation-free, wave propagation. Sound waves in fluids are, however, subject to viscous and thermal losses. These losses originate from viscous friction and heat conduction, leading to the formation of acoustic boundary layers. Considering these effects becomes significant in setups with acoustic cavities of similar dimension as the boundary layers. Recently, boundary element methods (BEM) accounting for the viscothermal dissipation have been proposed. These methods limit the discretization to the surface of the fluid domain and require significantly fewer degrees of freedom than comparable finite element models. However, the BEM coefficient matrices are fully populated, resulting in high computational costs and storage requirements. This study develops a new BEM formulation for acoustics in viscothermal fluids that uses the fast multipole method - a low-rank approximation technique based on a hierarchical subdivision of the computational domain - to alleviate this shortcoming. It is shown that the fast viscothermal BEM improves the algorithmic complexity over the conventional formulation, reducing both the memory requirements and solution time. The results indicate good scalability of the method making it feasible for larger applications such as acoustic metamaterials.

Symmetric Positive Definite (SPD) matrices have been used in many fields of medical data analysis. Many Riemannian metrics have been defined on this manifold but the choice of the Riemannian structure lacks a set of principles that could lead one to choose properly the metric. This drives us to introduce the principle of balanced metrics that relate the affine-invariant metric with the Euclidean and inverse-Euclidean metric, or the Bogoliubov-Kubo-Mori metric with the Euclidean and log-Euclidean metrics. We introduce two new families of balanced metrics, the mixed-power-Euclidean and the mixed-power-affine metrics and we discuss the relation between this new principle of balanced metrics and the concept of dual connections in information geometry.

The geometry of mixed-Euclidean metrics on symmetric positive definite matrices

The region covariance descriptors have recently become a popular method for detection and tracking of objects in an image. However, these descriptors are not suitable for classification of images with heterogeneous contents. In this paper, we present an image-level descriptor obtained using an affine-invariant geodesic mean of region covariance descriptors on the Riemannian manifold of symmetric positive definite (SPD) matrices. The resulting image descriptors are also SPD matrices, lending themselves to tractable geodesic distance based k-nearest neighbour classification using efficient kernels. We show that the proposed descriptor yields high classification accuracy on a challenging problem of nuclear pleomorphism scoring in breast cancer histology images.

Efficient parallel factorization and solution of structured and unstructured linear systems