Storage algorithm for wavelet Galerkin method

ABSTRACTAn accurate and efficient 3D finite‐difference (FD) forward algorithm for DC resistivity modelling is developed. In general, the most time‐consuming part of FD calculation is to solve large sets of linear equations: Ax=b, where A is a large sparse band symmetric matrix. The direct method using complete Choleski decomposition is quite slow and requires much more computer storage. We have introduced a row‐indexed sparse storage mode to store the coefficient matrix A and an incomplete Choleski conjugate‐gradient (ICCG) method to solve the large linear systems. By taking advantage of the matrix symmetry and sparsity, the ICCG method converges much more quickly and requires much less computer storage. It takes approximately 15 s on a 533 MHz Pentium computer for a grid with 46 020 nodes, which is approximately 700 times faster than the direct method and 2.5 times faster than the symmetric successive over‐relaxation (SSOR) conjugate‐gradient method. Compared with 3D finite‐element resistivity modelling with the improved ICCG solver, our algorithm is more efficient in terms of number of iterations and computer time. In addition, we solve for the secondary potential in 3D DC resistivity modelling by a simple manipulation of the FD equations. Two numerical examples of a two‐layered model and a vertical contact show that the method can achieve much higher accuracy than solving for the total potential directly with the same grid nodes. In addition, a 3D cubic body is simulated, for which the dipole–dipole apparent resistivities agree well with the results obtained with the finite‐element and integral‐equation methods. In conclusion, the combination of several techniques provides a rapid and accurate 3D FD forward modelling method which is fundamental to 3D resistivity inversion.

Multicore processors are marking the beginning of a new era of computing where massive parallelism is available and necessary. Slightly slower but easy to parallelize kernels are becoming more valuable than sequentially faster kernels that are unscalable when parallelized. In this paper, we focus on the multiplication of sparse matrices (SpGEMM). We first present the issues with existing sparse matrix representations and multiplication algorithms that make them unscalable to thousands of processors. Then, we develop and analyze two new algorithms that overcome these limitations. We consider our algorithms first as the sequential kernel of a scalable parallel sparse matrix multiplication algorithm and second as part of a polyalgorithm for SpGEMM that would execute different kernels depending on the sparsity of the input matrices. Such a sequential kernel requires a new data structure that exploits the hypersparsity of the individual submatrices owned by a single processor after the 2D partitioning. We experimentally evaluate the performance and characteristics of our algorithms and show that they scale significantly better than existing kernels.

Recent studies in Learning to Rank have shown the possibility to effectively\ndistill a neural network from an ensemble of regression trees. This result\nleads neural networks to become a natural competitor of tree-based ensembles on\nthe ranking task. Nevertheless, ensembles of regression trees outperform neural\nmodels both in terms of efficiency and effectiveness, particularly when scoring\non CPU. In this paper, we propose an approach for speeding up neural scoring\ntime by applying a combination of Distillation, Pruning and Fast Matrix\nmultiplication. We employ knowledge distillation to learn shallow neural\nnetworks from an ensemble of regression trees. Then, we exploit an\nefficiency-oriented pruning technique that performs a sparsification of the\nmost computationally-intensive layers of the neural network that is then scored\nwith optimized sparse matrix multiplication. Moreover, by studying both dense\nand sparse high performance matrix multiplication, we develop a scoring time\nprediction model which helps in devising neural network architectures that\nmatch the desired efficiency requirements. Comprehensive experiments on two\npublic learning-to-rank datasets show that neural networks produced with our\nnovel approach are competitive at any point of the effectiveness-efficiency\ntrade-off when compared with tree-based ensembles, providing up to 4x scoring\ntime speed-up without affecting the ranking quality.\n

Computer Solution of Large Linear Systems

Sparse matrix-sparse matrix multiplication (spMspM) is at the heart of a wide range of scientific and machine learning applications. spMspM is inefficient on general-purpose architectures, making accelerators attractive. However, prior spMspM accelerators use inner- or outer-product dataflows that suffer poor input or output reuse, leading to high traffic and poor performance. These prior accelerators have not explored Gustavson's algorithm, an alternative spMspM dataflow that does not suffer from these problems but features irregular memory access patterns that prior accelerators do not support.

Load-Balanced Sparse Matrix–Vector Multiplication on Parallel Computers

Linear-scaling density functional theory (DFT) is an efficient method to describe the electronic structures of molecules, semiconductors, and insulators to avoid the high cubic-scaling cost in conventional DFT calculations. Here, we present a parallel implementation of linear-scaling density matrix trace correcting (TC) purification algorithm to solve the Kohn–Sham (KS) equations with the numerical atomic orbitals in the HONPAS package. Such a linear-scaling density matrix purification algorithm is based on the Kohn's nearsightedness principle, resulting in a sparse Hamiltonian matrix with localized basis sets in the DFT calculations. Therefore, sparse matrix multiplication is the most time-consuming step in the density matrix purification algorithm for linear-scaling DFT calculations. We propose to use the MPI_Allgather function for parallel programming to deal with the sparse matrix multiplication within the compressed sparse row (CSR) format, which can scale up to hundreds of processing cores on modern heterogeneous supercomputers. We demonstrate the computational accuracy and efficiency of this parallel density matrix purification algorithm by performing large-scale DFT calculations on boron nitrogen nanotubes containing tens of thousands of atoms.

A sparse matrix multiplication scheme with multiatom blocks is reported, a tool that can be very useful for developing linear-scaling methods with atom-centered basis functions. Compared to conventional element-by-element sparse matrix multiplication schemes, efficiency is gained by the use of the highly optimized basic linear algebra subroutines (BLAS). However, some sparsity is lost in the multiatom blocking scheme because these matrix blocks will in general contain negligible elements. As a result, an optimal block size that minimizes the CPU time by balancing these two effects is recovered. In calculations on linear alkanes, polyglycines, estane polymers, and water clusters the optimal block size is found to be between 40 and 100 basis functions, where about 55-75% of the machine peak performance was achieved on an IBM RS6000 workstation. In these calculations, the blocked sparse matrix multiplications can be 10 times faster than a standard element-by-element sparse matrix package.

Parallel direct method for linear equations

PARALLEL AND LARGE SCALE MATRIX COMPUTATIONS IN CONTROL: SOME IDEAS

<abstract><p>In this paper, fast numerical methods for solving the real quasi-symmetric Toeplitz linear system are studied in two stages. First, based on an order-reduction algorithm and the factorization of Toeplitz matrix inversion, a sequence of linear systems with a constant symmetric Toeplitz matrix are solved. Second, two new fast algorithms are employed to solve the real quasi-symmetric Toeplitz linear system. Furthermore, we show a fast algorithm for quasi-symmetric Toeplitz matrix-vector multiplication. In addition, the stability analysis of the splitting symmetric Toeplitz inversion is discussed. In mathematical or engineering problems, the proposed algorithms are extraordinarily effective for solving a sequence of linear systems with a constant symmetric Toeplitz matrix. Fast matrix-vector multiplication and a quasi-symmetric Toeplitz linear solver are proven to be suitable for image encryption and decryption.</p></abstract>

Memory access is known to be the main bottleneck for shared-memory parallel graph applications especially for large and irregular graphs. Propagation blocking (PB) idea was proposed recently to improve the parallel performance of PageRank and sparse matrix and vector multiplication operations. The idea is based on separating parallel computation into two phases, binning and accumulation, such that random memory accesses are replaced with contiguous accesses. In this paper, we propose an algorithm that allows execution of these two phases concurrently. We propose several improvements to increase parallel throughput, reduce memory overhead, and improve work efficiency. Our experimental results show that our proposed algorithms improve shared-memory parallel throughput by a factor of up to 2× compared to the original PB algorithms. We also show that the memory overhead can be reduced significantly (from 170 percent down to less than 5 percent) without significant degradation of performance. Finally, we demonstrate that our concurrent execution model allows asynchronous parallel execution, leading to significant work efficiency in addition to throughput improvements.

In this paper, we present an object-oriented concept of sparse matrix and iterative linear solver for large scale parallel and sequential finite element analysis of multi-field problems. With the present concept, the partitioning and parallel solving of linear equation systems can be easily realized, and the memory usage to solve coupled multi-field problems is optimized. For the parallel computing, the present objects are tailored to the domain decomposition approach for both equation assembly and linear solver. With such approach, the assembly of a global equation system is thoroughly avoided. Parallelization is realized in the sparse matrix object by the means of (1) enable the constructor of the sparse matrix class to use domain decomposition data to establish the local domain sparse pattern and (2) introduce MPI calls into the member function of matrix-vector multiplication to collect local results and form global solutions. The performance of these objects in C++ is demonstrated by a geotechnical application of 3D thermal, hydraulic and mechanical (THM) coupled problem in parallel manner.

The problem of solving linear systems is one of the most fundamental problems in computer science, where given a satisfiable linear system (A,b), for A ∈ ℝ^{n×n} and b ∈ ℝⁿ, we wish to find a vector x ∈ ℝⁿ such that Ax = b. The current best algorithms for solving dense linear systems reduce the problem to matrix multiplication, and run in time O(n^ω). We consider the problem of finding ε-approximate solutions to linear systems with respect to the L₂-norm, that is, given a satisfiable linear system (A ∈ ℝ^{n×n}, b ∈ ℝⁿ), find an x ∈ ℝⁿ such that ||Ax - b||₂ ≤ ε||b||₂. Our main result is a fine-grained reduction from computing the rank of a matrix to finding ε-approximate solutions to linear systems. In particular, if the best known Õ(n^ω) time algorithm for computing the rank of n × O(n) matrices is optimal (which we conjecture is true), then finding an ε-approximate solution to a dense linear system also requires Ω̃(n^ω) time, even for ε as large as (1 - 1/poly(n)). We also prove (under some modified conjectures for the rank-finding problem) optimal hardness of approximation for sparse linear systems, linear systems over positive semidefinite matrices and well-conditioned linear systems. At the heart of our results is a novel reduction from the rank problem to a decision version of the approximate linear systems problem. This reduction preserves properties such as matrix sparsity and bit complexity.

For solving the Poisson equation it is usually possible to discretize it into solving the corresponding linear system $Ax=b$. Variational quantum algorithms (VQAs) for \textcolor{red}{the discreted} Poisson equation have been studied before. We give a VQA based on the banded Toeplitz systems for solving the Poisson equation with respect to the structural features of matrix $A$. In detail, we decompose the matrix $A$ and $A^2$ into a linear combination of the corresponding banded Toeplitz matrix and sparse matrices with only a few non-zero elements. For the one-dimensional Poisson equation with different boundary conditions and the $d$-dimensional Poisson equation with Dirichlet boundary conditions, \textcolor{red}{the number of decomposition terms is less than the work in [Phys. Rev. A108, 032418 (2023)]}. Based on the decomposition of the matrix, we design quantum circuits that \textcolor{red}{evaluate efficiently} the cost function. \textcolor{red}{Additionally, numerical simulation verifies the feasibility of the proposed algorithm. In the end,} the VQAs for linear systems of equations and matrix–vector multiplications with $K$-banded Teoplitz matrix $T_n^K$ \textcolor{red}{are} given, where $T_n^K\in R^{n\times n}$ and $K\in O(ploy\log n)$.