An Efficient High Performance Parallelization of a Discontinuous Galerkin Spectral Element Method

En esta tesis se presentan varias técnicas de solución para métodos de Galerkin discontinuo (DG, p.s.s.e.i.1) de alto orden. Si bien el objetivo principal es el desarrollo de técnicas precisas y computacionalmente eficientes para resolver las ecuaciones de Navier-Stokes de flujo compresible en 3D por medio del método Discontinuous Galerkin Spectral Element Method (DGSEM), todos los métodos desarrollados en esta tesis se pueden aplicar a otras ecuaciones de conservación no lineales y a otros esquemas DG. Las técnicas de solución acá presentadas pueden ser clasificadas en dos grupos: (i) las que están orientadas a acelarar la integración temporal, y (ii) las que mejoran la discretización espacial con adaptación-p local. El primer grupo de técnicas disminuye el tiempo de cálculo dada una discretización espacial con un número fijo de grados de libertad (NDOF, p.s.s.e.i.) mediante la mejora de los algoritmos de solución. Por ese motivo, se presta especial atención al desarrollo de algoritmos eficientes para integración temporal implícita y de métodos multimalla. También se detalla una metodología para obtener el Jacobiano analítico DG para ecuaciónes de advección-difusión. Además, se demuestra que la discretización implícita del DGSEM en nodos de Gauss-Lobatto puede ser resuelta eficientemente mediante el método static condensation. Lo anterior permite resolver el problema con sistemas lineales más pequeños y mejor condicionados, lo cual disminuye los tiempos de cálculo para órdenes moderados. El segundo grupo de técnicas consiste en modificar localmente la discretización espacial para reducir NDOF sin deteriorar la precisión del método. Para lograr este obetivo, en esta tesis se desarrolla un nuevo estimador de error para el método DGSEM p-anisitrópico basado en el método _-estimation, el cual es más preciso y requiere menos recursos computacionales que enfoques previos. Posteriormente, este estimador se usa con éxito para realizar adaptación-p anisotrópica en casos estacionaros y no estacionarios, y se demuestra que éste se puede acoplar con métodos multimalla para acelerar más las simulaciones. Adicionalmente, se desarrollan dos técnicas de adaptación-p para casos no estacionarios: un algoritmo dinámico y un algoritmo estático, que es computacionalmente más eficiente en algunos casos estadísticamente estacionarios. ----------ABSTRACT---------- In this thesis, we develop space and time solution techniques for high-order Discontinuous Galerkin (DG) methods. Even though the main focus is the accurate and computationally efficient solution of the 3D compressible Navier-Stokes equations using the Discontinuous Galerkin Spectral Element Method (DGSEM), the methods developed here can be applied to other systems of nonlinear conservation laws and other DG methods. The solution techniques can be classified into two groups: (i) improved solution techniques for temporal discretizations, and (ii) local p-adaptation techniques that enhance the spatial discretization. The first group of techniques decreases the computational time for a given spatial discretization with a fixed number of degrees of freedom (NDOF) by improving the linear and nonlinear solver algorithms. In that regard, special attention is paid to the development of efficient implicit time-integration solvers and to multigrid methods (both explicit and implicit in time). We derive a methodology to obtain the analytical Jacobian of DG discretizations of nonlinear advectiondiffusion equations. Furthermore, we show that time-implicit DGSEM discretizations on Gauss-Lobatto points can be formulated as Schur complement problems and solved using the static-condensation method. This enables one to solve smaller, better-conditioned problems, which yields shorter computation times for moderate polynomial orders. The second group of techniques modifies the spatial discretization by reducing the NDOF while maintaining similar accuracy. To do that, we develop a novel truncation error estimator that uses the _-estimation method for the p-anisotropic DGSEM. The new method is more accurate and computationally cheaper than previous approaches. The error estimator is then successfully applied to perform local anisotropic p-adaptation for steady and unsteady flows. We show that the truncation error estimator can be readily coupled with multigrid techniques to obtain further speed-ups. Moreover, we develop two p-adaptation strategies for unsteady flows. One of them is a dynamic _-based anisotropic p-adaptation method. The other one is a static _-based anisotropic p-adaptation method, which may be computationally cheaper in statistically steady flows.

This work is focused on the entropy analysis of a semi-discrete nodal discontinuous Galerkin spectral element method (DGSEM) on moving meshes for hyperbolic conservation laws. The DGSEM is constructed with a local tensor-product Lagrange-polynomial basis computed from Legendre–Gauss–Lobatto points. Furthermore, the collocation of interpolation and quadrature nodes is used in the spatial discretization. This approach leads to discrete derivative approximations in space that are summation-by-parts (SBP) operators. On a static mesh, the SBP property and suitable two-point flux functions, which satisfy the entropy condition from Tadmor, allow to mimic results from the continuous entropy analysis, if it is ensured that properties such as positivity preservation (of the water height, density or pressure) are satisfied on the discrete level. In this paper, Tadmor’s condition is extended to the moving mesh framework. We show that the volume terms in the semi-discrete moving mesh DGSEM do not contribute to the discrete entropy evolution when a two-point flux function that satisfies the moving mesh entropy condition is applied in the split form DG framework. The discrete entropy behavior then depends solely on the interface contributions and on the domain boundary contribution. The interface contributions are directly controlled by proper choice of the numerical element interface fluxes. If an entropy conserving two-point flux is chosen, the interface contributions vanish. To increase the robustness of the discretization we use so-called entropy stable two-point fluxes at the interfaces that are guaranteed entropy dissipative and thus give a bound on the interface contributions in the discrete entropy balance. The remaining boundary condition contributions depend on the type of the considered boundary condition. E.g. for periodic boundary conditions that are of entropy conserving type, our methodology with the entropy conserving interface fluxes is fully entropy conservative and with the entropy stable interface fluxes is guaranteed entropy stable. The presented proof does not require any exactness of quadrature in the spatial integrals of the variational forms. As it is the case for static meshes, these results rely on the assumption that additional properties like positivity preservation are satisfied on the discrete level. Besides the entropy stability, the time discretization of the moving mesh DGSEM will be investigated and it will be proven that the moving mesh DGSEM satisfies the free stream preservation property for an arbitrary s-stage Runge–Kutta method, when periodic boundary conditions are used. The theoretical properties of the moving mesh DGSEM will be validated by numerical experiments for the compressible Euler equations with periodic boundary conditions.

The discontinuous Galerkin Spectral Element Method (DG-SEM) is highly attractive for both DNS and LES of turbulent flows due to its low dispersion and dissipation errors, but also because of its good parallel scaling property. We show that especially for underresolved simulations the method has highly beneficial properties for LES, as well as for the direct treatment of the acoustic propagation. We also discuss different approaches for non-reflecting boundary conditions for DG-SEM and show the behavior of the methods for airfoil flows. Our main intend is to directly simulate trailing edge noise for airfoil flows at medium Reynolds numbers.

In this paper, we study the discontinuous Galerkin spectral element method for solving the population balance differential equation. We use the Legendre polynomials of order k as test functions on each element. Calculate the matrices used by Gauss Quadrature integration and then compare the numerical results with the other methods.

An entropy stable high-order discontinuous Galerkin spectral element method for the Baer-Nunziato two-phase flow model

A collocated-grid spectral difference method for compressible flows

We use the behavior of the L_{2} norm of the solutions of linear hyperbolic equations with discontinuous coefficient matrices as a surrogate to infer stability of discontinuous Galerkin spectral element methods (DGSEM). Although the L_{2} norm is not bounded in terms of the initial data for homogeneous and dissipative boundary conditions for such systems, the L_{2} norm is easier to work with than a norm that discounts growth due to the discontinuities. We show that the DGSEM with an upwind numerical flux that satisfies the Rankine–Hugoniot (or conservation) condition has the same energy bound as the partial differential equation does in the L_{2} norm, plus an added dissipation that depends on how much the approximate solution fails to satisfy the Rankine–Hugoniot jump.

Explicit discontinuous Galerkin methods for unsteady problems

We discuss two approaches for the formulation and implementation of space-time discontinuous Galerkin spectral element methods (DG-SEM). In one, time is treated as an additional coordinate direction and a Galerkin procedure is applied to the entire problem. In the other, the method of lines is used with DG-SEM in space and the fully implicit Runge–Kutta method Lobatto IIIC in time. The two approaches are mathematically equivalent in the sense that they lead to the same discrete solution. However, in practice they differ in several important respects, including the terminology used to describe them, the structure of the resulting software, and the interaction with nonlinear solvers. Challenges and merits of the two approaches are discussed with the goal of providing the practitioner with sufficient consideration to choose which path to follow. Additionally, implementations of the two methods are provided as a starting point for further development. Numerical experiments validate the theoretical accuracy of these codes and demonstrate their utility, even for 4D problems.

A high-order fluid–structure interaction framework with application to shock-wave/turbulent boundary-layer interaction over an elastic panel

In the second part of this three-paper series, high-fidelity simulations of the transonic cascade TEAMAero at the aerodynamic design point with Rein=1.35×106 and Main=1.21 are presented. A high-order discontinuous Galerkin spectral element method with finite-volume subcell shock capturing is employed to simulate the flow based on an implicit large eddy simulation (LES) scheme and advanced over several buffeting cycles to reliably capture the shock unsteadiness. A study on the spanwise domain size shows that the shock oscillation amplitude decreases with increasing span, although its frequency and mean location remain fixed through the simulations. By comparing high- and low-resolution LES results, it is further presented that deviations from under-resolution are mostly limited to the separated region past the shock, where the high-fidelity results match experimental results more closely. In addition to the LES, low-fidelity unsteady Reynolds-averaged Navier–Stokes is shown to capture the shock unsteadiness correctly, but at a reduced amplitude and fails to match the force distributions on the blade surface. Through examination of instantaneous flow features, space–time relations and spectral proper orthogonal decomposition, a basic analysis of the shock–boundary layer interaction is presented and indicates that velocity perturbations travel upstream through the subsonic boundary layer and periodically cause oblique shock waves, transporting the information from the boundary layer into the passage.

The development of any atmosphere or ocean model warrants a suite of test cases (TCs) to verify its spatial and temporal discretizations, order of accuracy, stability, reproducibility, portability, scalability, etc. In this paper, we present a suite of shallow water TCs designed to verify the barotropic solver of atmosphere and ocean models. These include the non‐dispersive coastal Kelvin wave; the dispersive inertia‐gravity wave; the dispersive planetary and topographic Rossby waves; the barotropic tide; and a non‐linear manufactured solution. These TCs check the implementation of the linear pressure gradient term; the linear constant or variable‐coefficient Coriolis and bathymetry terms; and the non‐linear advection terms. Simulation results are presented for a variety of time‐stepping methods as well as two spatial discretizations: a mimetic finite volume method based on the TRiSK scheme, and a high‐order discontinuous Galerkin spectral element method. The experimental procedure for conducting these numerical experiments is detailed. It underscores several key considerations that vary depending on the chosen spatial discretization method. Finally, convergence studies of every TC are conducted with refinement in both space and time, only in space, and only in time. The convergence slopes match the expected theoretical predictions.

In this paper, we show that diagonal-norm summation by parts (SBP) discretizations of general non-conservative systems of hyperbolic balance laws can be rewritten as a finite-volume-type formula, also known as flux-differencing formula, if the non-conservative terms can be written as the product of a local and a symmetric contribution. Furthermore, we show that the existence of a flux-differencing formula enables the use of recent subcell limiting strategies to improve the robustness of the high-order discretizations. The methods are valid on unstructured curvilinear grids using tensor-product basis functions.To demonstrate the utility of the novel flux-differencing formula, we construct hybrid schemes that combine high-order SBP methods (the discontinuous Galerkin spectral element method and a high-order SBP finite difference method) with a compatible low-order finite volume (FV) scheme at the subcell level. We apply the hybrid schemes to solve challenging magnetohydrodynamics (MHD) problems featuring strong shocks.

This paper summarizes our progress in the application of a high-order discontinuous Galerkin (DG) method for scale resolving fluid dynamics simulations on the Cray XC40 Hazel Hen cluster at HLRS. We present the large eddy simulation (LES) of flow around a wall mounted cylinder, a LES of flow around an airfoil at realistic Reynolds number using a recently introduced kinetic energy preserving flux formulation and a simulation of transitional flow in a low pressure turbine. Furthermore, it provides an overview over the parallel efficiency reached by our code when using up to 49, 152 CPUs and the latest developments of our DG framework.

A relaxation model for the non-isothermal Navier-Stokes-Korteweg equations in confined domains