Reply on RC2

In this paper, the application of the multilevel Monte Carlo (MLMC) method to numerical simulations of turbulent flows with uncertain parameters is investigated. Several strategies for setting up the MLMC method are presented, and the advantages and disadvantages of each strategy are also discussed. A numerical experiment is carried out using an idealized model for the Antarctic Circumpolar Current (ACC) with uncertain, small-scale bottom topographic features. It is demonstrated that unlike the pointwise solutions, the averaged volume transports are correlated across grid resolutions, and the MLMC method can increase simulation efficiency without losing accuracy in uncertainty assessments.

This documents describes several studies undertaken to assess the applicability of MultiLevel Monte Carlo (MLMC) methods to problems of interest; namely in turbulent fluid flow over civil engineering structures. Several numerical experiments are presented wherein the convergence of quantities of interest with mesh parameters are studied at different Reynolds’ numbers and geometries. It was found that MLMC methods could be used successfully for low Reynolds’ number flows when combined with appropriate Adaptive Mesh Refinement (AMR) strategies. However, the hypotheses for optimal MLMC performance were found to not be satisfied at higher turbulent Reynolds’ numbers despite the use of AMR strategies. Recommendations are made for future research directions based on these studies. A tentative outline for an MLMC algorithm with adapted meshes is made, as well as recommendations for alternatives to MLMC methods for cases where the underlying assumptions for optimal MLMC performance are not satisfied.

In this report, we study the use of Multi-Level Monte Carlo (MLMC) methods for time dependent problems. It was found that the usability of MLMC methods depends strongly on whether or not the underlying time dependent problem is chaotic in nature. Numerical experiments are conducted on both simple problems, as well as fluid flow problems of practical interest to the ExaQUte project, to demonstrate this. For the non-chaotic cases, the hypotheses that enable the use of MLMC methods were found to be satisfied. For the chaotic cases, especially the case of high Reynolds’ number fluid flow, the hypotheses were not satisfied. However, it was found that correlations between the different levels were high enough to merit the use of multi-fidelity or control-variate approaches. It was also noted that MLMC methods could work for chaotic problems if the time window of analysis were chosen to be small enough. Future studies are proposed to examine this possibility.

The focus of this work is the introduction of some computable a posteriori error control to the popular multilevel Monte Carlo sampling for PDE with stochastic data. We are especially interested in applications where some quantity of interest should be estimated accurately. Based on a spatial discretization by the finite element method, a goal functional is defined which encodes the quantity of interest. The devised goal-oriented a posteriori error estimator enables one to determine guaranteed path-wise a posteriori error bounds for this quantity. An adaptive algorithm is proposed which employs the computed error estimates and adaptive meshes to control the approximation error. Moreover, the stochastic error is controlled such that the determined bounds are guaranteed in probability. The approach allows for the adaptive refinement of the mesh hierarchy used in the multilevel Monte Carlo simulation which is used for a problem-dependent construction of discretization levels. Numerical experiments illustrate the performance of the adaptive method for a posteriori error control in Monte Carlo and multilevel Monte Carlo methods with respect to localized goals. Moreover, the computational efficiency of the multilevel and classical Monte Carlo approaches are compared. It is illustrated that adaptively refined meshes can yield significant benefits when combined with Monte Carlo methods. In particular, with a problem-adapted mesh hierarchy, the efficiency gains of the multilevel Monte Carlo method in terms of computational cost can also be exploited in the context of error control.

Multilevel Monte Carlo method for ergodic SDEs without contractivity

Reservoir heterogeneity can be detrimental to the success of chemical enhanced oil recovery (EOR) processes. Therefore, it is important to evaluate the effect of uncertainty in reservoir heterogeneity on the performance of chemical EOR. Usually, a Monte Carlo sampling approach is followed were a number of stochastic reservoir model realizations are generated and then numerical simulation is performed to obtain a certain objective function, such as the recovery factor; however, Monte Carlo simulation (MCS) has a slow convergence and requires a large number of samples to produce accurate results. This is computationally expensive when using reservoir simulators. This study applies an extension to MCS using a multi-scale approach. The applied method is known as the multilevel Monte Carlo (MLMC) method and has been only recently applied to problems of flow in porous media. This method is based on running a small number of expensive simulations on the finer scale model and a large number of less expensive simulations on coarser scale models — these are upscaled models of the fine scale model — and then combining the results to produce the quantities of interest. The purpose of this method is to reduce computational cost while maintaining the accuracy of the finer scale model. The results of this approach are compared with reference MCS, assuming a large number of simulations on the fine scale model. This study used MLMC to efficiently quantify the effect of uncertainty in heterogeneity on the recovery factor of different chemical EOR processes. The permeability field was assumed to be the random input. This approach was implemented by writing a MATLAB code to generate the stochastic realizations for the permeability field and also performing the coarsening processes. The code is then coupled with ECLIPSE, which was used as the numerical simulator for the chemical EOR processes to obtain the recovery factor. The code then combines the results obtained from the different scale models to produce the statistical moments for the recovery factor, such as the mean and variance. This method was applied for two-dimensional (2D) and three-dimensional (3D) stylized reservoir models using Gaussian randomly generated permeability fields. Different coarsening algorithms were used and compared, such as the renormalization and pressure solver methods, and polymer and surfactant-polymer (SP) flooding processes where the chemical EOR processes were considered. The results were compared with running Monte Carlo for the fine scale model while equating the computational cost for the multilevel Monte Carlo method. Both of these results were then compared with the reference case, which uses a large number of runs of the fine scale model. The results show that it is possible to robustly quantify spatial uncertainty for chemical EOR processes while greatly reducing the computational requirement, up to two orders of magnitude compared to traditional Monte Carlo. The method can be easily extendable to other EOR processes to quantify spatial uncertainty such as carbon dioxide (CO2) EOR.

This thesis examines the quantification of uncertainty in real-world engineering systems using the multilevel Monte Carlo method. It is often infeasible to use the traditional Monte Carlo method to investigate the impact of uncertainty because computationally it can be prohibitively expensive for complex systems. Therefore, the newer multilevel method is investigated and the cost of this method is analysed in the finite element framework. The Monte Carlo and multilevel Monte Carlo methods are compared for two prototypical examples: structural vibrations and buoyancy driven flows through porous media. In the first example, the impact of random mass density is quantified for structural vibration problems in several dimensions using the multilevel Monte Carlo method. Comparable eigenvalues and energy density approximations are found for the traditional Monte Carlo method and the multilevel Monte Carlo method, but for certain problems the expectation and variance of the quantities of interest can be computed over 100 times faster using the multilevel Monte Carlo method. It is also tractable to use the multilevel method for three dimensional structures, where the traditional Monte Carlo method is often prohibitively expensive. In the second example, the impact of uncertainty in buoyancy driven flows through porous media is quantified using the multilevel Monte Carlo method. Again, comparable results are obtained from the two methods for diffusion dominated flows and the multilevel method is orders of magnitude cheaper. The finite element models for this investigation are formulated carefully to ensure that spurious numerical artefacts are not added to the solution and are compared to an analytical model describing the long term sequestration of CO2 in the presence of a background flow. Additional cost reductions are achieved by solving the individual independent samples in parallel using the new podS library. This library schedules the Monte Carlo and multilevel Monte Carlo methods in parallel across different computer architectures for the two examples considered in this thesis. Nearly linear cost reductions are obtained as the number of processes is increased.

Groundwater contaminant transport modeling is a vitally important topic. Since modeled processes include uncertainties, Monte Carlo methods are adopted to obtain some statistics. However, accurate models have a substantial computational cost. This drawback can be overcome by employing the multilevel Monte Carlo method (MLMC) or approximating the original model using a meta-model. We combined both of these approaches. A stochastic model is substituted with a deep learning meta-model that consists of a graph convolutional neural network and a feed-forward neural network. This meta-model can approximate models solved on unstructured meshes. The meta-model within the standard Monte Carlo method can bring significant computational cost savings. Nevertheless, the meta-model must be highly accurate to obtain similar errors as when using the original model. Proposed MLMC with the new lowest-accurate level of meta-models can reduce total computational costs, and the accuracy of the meta-model does not have to be so high. The size of the computational cost savings depends on the cost distribution across MLMC levels. Our approach is especially efficacious when the dominant computational cost is on the lowest-accuracy MLMC level. Depending on the number of estimated moments, we can reduce computational costs by up to ca. 25% while maintaining the accuracy of estimates.

We formulate standard and multilevel Monte Carlo methods for the kth moment Mεk[ξ] of a Banach space valued random variable ξ:Ω→E, interpreted as an element of the k-fold injective tensor product space ⊗εkE. For the standard Monte Carlo estimator of Mεk[ξ], we prove the k-independent convergence rate 1−1p in the Lq(Ω;⊗εkE)-norm, provided that (i) ξ∈Lkq(Ω;E) and (ii) q∈[p,∞), where p∈[1,2] is the Rademacher type of E. By using the fact that Rademacher averages are dominated by Gaussian sums combined with a version of Slepian's inequality for Gaussian processes due to Fernique, we moreover derive corresponding results for multilevel Monte Carlo methods, including a rigorous error estimate in the Lq(Ω;⊗εkE)-norm and the optimization of the computational cost for a given accuracy. Whenever the type of the Banach space E is p=2, our findings coincide with known results for Hilbert space valued random variables.We illustrate the abstract results by three model problems: second-order elliptic PDEs with random forcing or random coefficient, and stochastic evolution equations. In these cases, the solution processes naturally take values in non-Hilbertian Banach spaces. Further applications, where physical modeling constraints impose a setting in Banach spaces of type p<2, are indicated.

Abstract: Uncertainty quantification for multi‐physics problems like fluid‐structure interaction (FSI) is challenging in terms of the required computational effort and complexity of physics involved. A Multilevel Monte Carlo (MLMC) approach for stochastic analysis of FSI problems is presented in the current study. An overall reduction in computational cost is achieved by variance reduction in MLMC method. The efficacy of MLMC algorithm for expensive problems like FSI is compared with classical Monte Carlo(MC) method and MLMC is found to achieve the required accuracy at a significantly reduced computational cost. An FSI benchmark test case with uncertain inputs is presented here. The possibility of uncertainty quantification for FSI systems with a plausible computational cost using MLMC is demonstrated.

Research on multi-fidelity aerodynamic optimization methods

Space-time multilevel Monte Carlo methods and their application to cardiac electrophysiology

Monte Carlo methods are a very general and useful approach for the estimation of expectations arising from stochastic simulation. However, they can be computationally expensive, particularly when the cost of generating individual stochastic samples is very high, as in the case of stochastic PDEs. Multilevel Monte Carlo is a recently developed approach which greatly reduces the computational cost by performing most simulations with low accuracy at a correspondingly low cost, with relatively few simulations being performed at high accuracy and a high cost.In this article, we review the ideas behind the multilevel Monte Carlo method, and various recent generalizations and extensions, and discuss a number of applications which illustrate the flexibility and generality of the approach and the challenges in developing more efficient implementations with a faster rate of convergence of the multilevel correction variance.

• Differentiable multi-fidelity inference: BMFIA enables gradient-based Bayesian inference for high-dimensional, non-differentiable multi-physics problems using low-fidelity model gradients. • Efficiency with minimal HF data: Accurate posterior estimates are achieved from only 100–300 HF–LF simulations with orders-of-magnitude speed-ups over HF-only inference. • Accuracy despite crude LF models: Robust posterior reconstruction is maintained even with weakly correlated or simplified low-fidelity models. High-dimensional Bayesian inverse analysis ( dim ≫ 100 ) is mostly unfeasible for computationally demanding, nonlinear physics-based high-fidelity (HF) models. Usually, the use of more efficient gradient-based inference schemes is impeded if the multi-physics models are provided by complex legacy codes. Adjoint-based derivatives are either exceedingly cumbersome to derive or nonexistent for practically relevant large-scale nonlinear and coupled multi-physics problems. Similarly, holistic automated differentiation w. r. t. primary variables of multi-physics codes is usually not yet an option and requires extensive code restructuring if not considered from the outset in the software design. This absence of differentiability further exacerbates the already present computational challenges. To overcome the existing limitations, we propose a novel inference approach called Bayesian multi-fidelity inverse analysis (BMFIA) , which leverages simpler and computationally cheaper lower-fidelity (LF) models that are designed to provide model derivatives. BMFIA learns a simple, probabilistic dependence of the LF and HF models, which is then employed in an altered likelihood formulation to statistically correct the inaccurate LF response. From a Bayesian viewpoint, this dependence represents a multi-fidelity (MF) conditional density (discriminative model). We demonstrate how this MF conditional density can be learned robustly in the small data regime from only a few HF and LF simulations (50 to 300), which would not be sufficient for naive surrogate approaches. The formulation is fully differentiable and allows the flexible design of a wide range of LF models. We demonstrate that BMFIA solves Bayesian inverse problems for scenarios that used to be prohibitive, such as finely-resolved and hence high-dimensional spatial reconstruction problems in two-dimensional Euclidean domains with static posteriors, given nonlinear and transient coupled poro-elastic media physics. We show that the resulting static MF posteriors are in excellent agreement with the (usually inaccessible) HF posteriors or ground-truth data and note that extending the framework to arbitrary three-dimensional domains is a natural and important direction for future work.

Scaling Properties of Multi-Fidelity Shape Optimization Algorithms