Multigrid accelerated simulations for Twisted Mass fermions

The bidomain equations are widely used for the simulation of electrical activity in cardiac tissue but are computationally expensive, limiting the size of the problem which can be modeled. The purpose of this study is to determine more efficient ways to solve the elliptic portion of the bidomain equations, the most computationally expensive part of the computation. Specifically, we assessed the performance of a parallel multigrid (MG) preconditioner for a conjugate gradient solver. We employed an operator splitting technique, dividing the computation in a parabolic equation, an elliptical equation, and a nonlinear system of ordinary differential equations at each time step. The elliptic equation was solved by the preconditioned conjugate gradient method, and the traditional block incomplete LU parallel preconditioner (ILU) was compared to MG. Execution time was minimized for each preconditioner by adjusting the fill-in factor for ILU, and by choosing the optimal number of levels for MG. The parallel implementation was based on the PETSc library and we report results for up to 16 nodes on a distributed cluster, for two and three dimensional simulations. A direct solver was also available to compare results for single processor runs. MG was found to solve the system in one third of the time required by ILU but required about 40% more memory. Thus, MG offered an attractive tradeoff between memory usage and speed, since its performance lay between those of the classic iterative methods (slow and low memory consumption) and direct methods (fast and high memory consumption). Results suggest the MG preconditioner is well suited for quickly and accurately solving the bidomain equations.

HYBRID MONTE CARLO ALGORITHM ON THE CONNECTION MACHINE

We present preliminary results on the light, charmed and bottom baryon spectra using overlap valence quarks on the background of 2+1+1 flavours HISQ gauge configurations of the MILC collaboration. These calculations are performed on three different gauge ensembles at three lattice spacings (a ∼ 0.12 fm, 0.09 fm and 0.06 fm) and for physical strange, charm and bottom quark masses. The SU(2) heavy baryon chiral perturbation theory is used to extrapolate baryon masses to the physical pion mass and the continuum limit extrapolations are also performed. Our results are consistent with the well measured charmed baryons. We predict the masses of many other states which are yet to be discovered.

This paper presents a computational performance analysis of a parallel implementation of a conjugate gradient (CG) solver using domain decomposition and distributed memory computers, applied to a 3-D finite element method problem. The results show a superlinear speedup, which is not usually expected. The analysis shows why and how it can happen.

OpenMP is an architecture-independent language for programming in the shared memory model. OpenMP is designed to be simple and powerful in terms of programming abstractions. Unfortunately, the architecture-independent abstractions sometimes come with the price of low parallel performance. This is especially true for applications with an unstructured data access pattern running on distributed shared memory systems (DSM). Here, proper data distribution and algorithmic optimizations play a vital role for performance. In this article, we have investigated ways of improving the performance of an industrial class conjugate gradient (CG) solver, implemented in OpenMP running on two types of shared memory systems. We have evaluated bandwidth minimization, graph partitioning and reformulations of the original algorithm reducing global barriers. By a detailed analysis of barrier time and memory system performance, we found that bandwidth minimization is the most important optimization reducing both L2 misses and remote memory accesses. On a uniform memory system, we get perfect scaling. On a NUMA system, the performance is significantly improved with the algorithmic optimizations leaving the system dependent global reduction operations as a bottleneck.

An FFT based chemo-mechanical framework with fracture: Application to mesoscopic electrode degradation

While the advances in synchrotron light sources, together with the development of focusing optics and detectors, allow nanoscale ptychographic imaging of materials and biological specimens, the corresponding experiments can yield terabyte-scale volumes of data that can impose a heavy burden on the computing platform. Although graphics processing units (GPUs) provide high performance for such large-scale ptychography datasets, a single GPU is typically insufficient for analysis and reconstruction. Several works have considered leveraging multiple GPUs to accelerate the ptychographic reconstruction. However, most of these works utilize only the Message Passing Interface to handle the communications between GPUs. This approach poses inefficiency for a hardware configuration that has multiple GPUs in a single node, especially while reconstructing a single large projection, since it provides no optimizations to handle the heterogeneous GPU interconnections containing both low-speed (e.g., PCIe) and high-speed links (e.g., NVLink). In this paper, we provide an optimized intranode multi-GPU implementation that can efficiently solve large-scale ptychographic reconstruction problems. We focus on the maximum likelihood reconstruction problem using a conjugate gradient (CG) method for the solution and propose a novel hybrid parallelization model to address the performance bottlenecks in the CG solver. Accordingly, we have developed a tool, called PtyGer (Ptychographic GPU(multiple)-based reconstruction), implementing our hybrid parallelization model design. A comprehensive evaluation verifies that PtyGer can fully preserve the original algorithm’s accuracy while achieving outstanding intranode GPU scalability.

Assembly free FEM bypasses the assembly step and solves the system of linear equations at the element level using Conjugate Gradient (CG) type iterative solver. The smaller dense Matrix-vector Products (MvPs) are encapsulated within the CG solver and are computed either at element level or degree of freedom (DoF) level. Both these strategies exploit the computing power of GPU effectively, but the performance is lagging due to the uncoalesced global memory access on GPU. This paper proposes an improved MvP strategy in assembly free FEM, which improves the performance by coalesced global memory access using on-chip faster shared memory and using the texture cache memory on GPU. Since GPU has limited shared memory (in few KBs), the proposed technique suffers from a problem known as low occupancy. Despite the low occupancy issue, the proposed strategy outperforms both element based and DoF based MvP strategies on GPU. Numerical experiments compared with element level and DoF level strategies on GPU and found that, GPU instance of proposed MvP outperforms both strategies approximately by factor of 7 and 1.5 respectively.

An effective wavelet based multigrid preconditioned conjugate gradient method is developed to solve electromagnetic large matrix problem for millimeter wave scattering application. By using wavelet transformation we restrict the large matrix equation to a relative smaller matrix and which can be solved rapidly. The solution is prolonged as the new improvement for the conjugate gradient (CG) method. Numerical results show that our developed wavelet based multigrid preconditioned CG method can reach large improvement of computational complexity. Due to the automaticity of wavelet transformation, this method is potential to be a block box solver without physical background.

Hadron spectroscopy is predominantly the study of resonances that decay via the strong interaction into a multitude of stable hadrons, such as the pion. The vast majority of resonances decay via an intermediate hadron with non-zero intrinsic spin. In this thesis, I will present the results of scattering calculations featuring mesons with non-zero intrinsic spin. Before doing so, I will first give a brief introduction to QCD and review the framework necessary to perform lattice QCD calculations in Chapters 1 and 2. In Chapter 3, I present the first lattice calculation of ?? scattering in isospin-2. Here, ?? features in dynamically-coupled 3S1 and 3D1 partial-waves with JP=1+. No resonance enhancement is anticipated in the flavour exotic isospin-2 channel and as such it provides an ideal testing ground for this first calculation. I work at heavier than physical quark masses at the SU(3)F point where the up, down and strange quarks are mass degenerate. Finite-volume spectra are calculated and, utilising the relationship between the discrete energy spectrum and the infinite-volume scattering amplitudes, partial-wave amplitudes with J?3 and the degree of dynamical mixing between the coupled 3S1 and 3D1 channels are determined. In Chapter 4, I investigate ?? in isospin-1 where the a1 axial-vector resonance is expected to feature. Here, I present a discussion on G-parity and Bose-symmetry at the SU(3)F point. Working at heavier than physical quark masses, the resulting finite volume spectrum suggests that the a1 is a bound-state and that the 3S1- and 3D1-wave, ?? scattering amplitudes are similar to those in isospin-2. I present the first calculation of coupled ?? and ?? scattering in Chapter 5 where resonant enhancement is seen experimentally in the JP=1+ channel. Working at a somewhat lighter pion mass than in previous chapters, the finite-volume spectra are determined and the scattering amplitudes are calculated. Analytically continuing the amplitudes into the complex energy plane, a resonance pole is found, interpreted as the analogue of the b1 axial-vector, which couples dominantly to 3S1-wave ??, with a much-suppressed coupling to 3D1-wave ??, and a negligible coupling to ??. In Chapter 6, the exotic JPC=1?+ channel is studied. These quantum numbers are not allowed in the quark model but can be obtained, for example, through a gluonic excitation coupled to a quark-antiquark pair. In this exploratory calculation, performed at the SU(3)F point, the finite-volume spectra and coupled-channel scattering amplitudes are presented. A single resonance pole is found, interpreted as the exotic ?1, and couplings to meson-meson channels, including for example ??{1P1}, ???{1P1} and ??{3P1}, are calculated for the first time in lattice QCD. In order to minimally present the contents of a unitary n-channel scattering matrix, I introduce, in Chapter 7, an n-channel generalisation of the traditional two-channel Stapp parameterisation.

Nucleon-nucleon interactions, both bare and effective, play an important role in our understanding of the non-perturbative strong interaction, as well as nuclear structure and reactions. In recent years, tremendous efforts have been seen in the lattice QCD community to derive nucleon-nucleon interactions from first principles. Because of the daunting computing resources needed, most of such simulations were still performed with larger than physical light quark masses. In the present work, employing the recently proposed covariant chiral effective field theory (ChEFT), we study the light quark mass dependence of the nucleon-nucleon interaction extracted by the HALQCD group. It is shown that the pion-full version of the ChEFT can describe the lattice QCD data with mπ=469 MeV and their experimental counterpart reasonably well, while the pion-less version can describe the lattice QCD data with mπ=672,837,1015,1171 MeV, for both the S01 and S13-D13 channels. The slightly better description of the single channel than the triplet channel indicates that higher order studies are necessary for the latter. Our results confirmed previous studies that the nucleon-nucleon interaction becomes more attractive for both the singlet and triplet channels as the pion mass decreases towards its physical value. It is shown that the virtual bound state in the S01 channel remains virtual down to the chiral limit, while the deuteron only appears for a pion mass smaller than about 400 MeV. It seems that proper chiral extrapolations of nucleon-nucleon interaction are possible for pion masses smaller than 500 MeV, similar to the mesonic and one-baryon sectors.

We present a comparison of a number of iterative solvers of linear systems of equations for obtaining the fermion propagator in lattice QCD. In particular, we consider chirally invariant overlap and chirally improved Wilson (maximally) twisted mass fermions. The comparison of both formulations of lattice QCD is performed at four fixed values of the pion mass between 230MeV and 720MeV. For overlap fermions we address adaptive precision and low mode preconditioning while for twisted mass fermions we discuss even/odd preconditioning. Taking the best available algorithms in each case we find that calculations with the overlap operator are by a factor of 30-120 more expensive than with the twisted mass operator.

When fluid flow problems are solved by the SIMPLE-like sequential procedures, most of the computing time is spent on handling the pressure-correction equation. As a result, the overall efficiency of those sequential procedures is strongly dependent on how good is the matrix solver for the pressure correction equation. In this article, a modified conjugate gradient solver (MCGS) is presented that is specially tuned to solve the pressure-correction equations arising from both two- and three-dimensional fluid flow problems. A new solver described here lakes the advantages of conjugate gradient solver (CGS) and strongly implicit procedure (SIP) but eliminates the disadvantages of each. In contrast to the SIP, the present MCGS adopts the partial cancellation of zeroth order and is insensitive to variation of the cancellation parameter. In general, MCGS can be two or three times faster than CGS but is an order of magnitude faster than SIP and block-corrected ADI in treating large-scale problems.

The A-phi formulation, which is widely used in electromagnetic analysis, leads to a redundant linear system of equations that includes a substantial number of redundant degrees of freedom (DOF). We can derive a redundancy-reduced linear system of equations by eliminating the redundant DOF, thereby decreasing the computation costs per iteration for iterative solvers, such as the incomplete Cholesky conjugate gradient (ICCG) solver. This does not, however, result in a reduction in total computation time, due to significant convergence deterioration. In this paper, we present a solution to this problem in the form of folded preconditioners. First, the theorem presented reveals that, for any preconditioned Krylov subspace method for the original redundant linear systems, we can derive the equivalent Krylov subspace method for the redundancy-reduced linear systems by using the corresponding folded preconditioner. As an uncomplicated example, the standard ICCG solver for the original redundant systems has exactly the same convergence property as the CG solver for the redundancy-reduced systems using the folded variant of the IC preconditioner (the folded IC preconditioner). Furthermore, we discuss efficient computational procedures for the folded preconditioners and the design of Krylov subspace algorithms using the preconditioners. A sample full-wave analysis demonstrates the good performance of a newly developed solver, the conjugate orthogonal conjugate gradient (COCG) method with the folded IC preconditioner. The new solver not only lowers the computation costs per iteration by reducing the number of DOF, but also completely avoids the convergence deterioration.

<p style='text-indent:20px;'>The ensemble method has been developed for accelerating a sequence of numerical simulations of evolution problems. Its main idea is, by manipulating the time stepping and grouping discrete problems, to make all members in the same group share a common coefficient matrix. Thus, at each time step, instead of solving a sequence of linear systems each of which contains only one right-hand-side vector, the ensemble method simultaneously solves a single linear system with multiple right-hand-side vectors for each group. Such a system could be solved efficiently by using direct linear solvers when the problems are of small scale, as the same LU factorization would work for the entire group members. However, for large-scale problems, iterative linear solvers often have to be used and then this appealing advantage becomes not obvious. In this paper we systematically investigate numerical performance of the ensemble method with block iterative solvers for two typical evolution problems: the heat equation and the incompressible Navier-Stokes equations. In particular, the block conjugate gradient (CG) solver is considered for the former and the block generalized minimal residual (GMRES) solver for the latter. Our numerical results demonstrate the effectiveness and efficiency of the ensemble method when working together with these block iterative solvers.