Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems

The Bayesian optimal experimental design (OED) problem seeks to identify data, sensor configurations, or experiments which can optimally reduce uncertainty. The goal of OED is to find an experiment that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge about expected data. Therefore, within the context of seismic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types, and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize the ability to locate seismic events from arrival time data of detected seismic phases. In order to do utilize Bayesian OED we must develop four elements:1. A likelihood function that describes the uncertainty of detection and travel times; 2. A Bayesian solver that takes a prior and likelihood to identify the posterior; 3. An algorithm to compute EIG; and, 4. An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we can explore many relevant questions to monitoring such as: how and what multiphenomenology data can be used to optimally reduce uncertainty, how to trade off sensor fidelity and earth model uncertainty, and how sensor types, number, and locations influence uncertainty

Bayesian optimal experimental design (OED) plays an important role in minimizing model uncertainty with limited experimental data in a Bayesian framework. In many applications, rather than minimizing the uncertainty in the inference of model parameters, one seeks to minimize the uncertainty of a model-dependent quantity of interest (QoI). This is known as goal-oriented OED (GOOED). Here, we consider GOOED for linear Bayesian inverse problems governed by large-scale models represented by partial differential equations (PDE) that are computationally expensive to solve. In particular, we consider optimal sensor placement by maximizing an expected information gain (EIG) for the QoI. We develop an efficient method to solve such problems by deriving a new formulation of the goal-oriented EIG. Based on this formulation we propose an offline-online decomposition scheme that achieves significant computational reduction by computing all of the PDE-dependent quantities in an offline stage just once, and optimizing the sensor locations in an online stage without solving any PDEs. Moreover, in the offline stage we need only to compute low-rank approximations of two Hessian-related operators. The computational cost of these low-rank approximations, measured by the number of PDE solves, does not depend on the parameter or data dimensions for a large class of elliptic, parabolic, and sufficiently dissipative hyperbolic inverse problem that exhibit dimension-independent rapid spectra decay. We carry out detailed error analysis for the approximate goal-oriented EIG due to the low-rank approximations of the two operators. Furthermore, in the online stage we extend a swapping greedy method to optimize the sensor locations developed in our recent work that is demonstrated to be more efficient than a standard greedy method. We conduct a numerical experiment for a contaminant transport inverse problem with an infinite-dimensional parameter field to demonstrate the efficiency, accuracy, and both data- and parameter-dimension independence of the proposed algorithm.

Optimal experimental design is an important methodology for most efficiently allocating resources in an experiment to best achieve some goal. Bayesian experimental design considers the potential impact that various choices of the controllable variables has on the posterior distribution of the unknowns. Optimal Bayesian design involves optimising an expected utility function, which is an analytically intractable integral over the prior predictive distribution. These integrals are typically estimated via standard Monte Carlo methods. In this paper, we demonstrate that the use of randomised quasi-Monte Carlo can bring significant reductions to the variance of the estimated expected utility. This variance reduction will then lead to a more efficient optimisation algorithm for maximising the expected utility.

In high-risk, time-pressure domains, the ability to get only salient information is of paramount importance. Missing or superfluous information in these domains can detract from a decision maker’s ability to make correct judgments. Consequently, decision support systems are being developed to facilitate expert decision-making by modifying the information presented to the operator. In this paper we introduce a method for presenting decision makers with the most environment-relevant information for a given decision task. This study explores a statistical method, Bayesian Optimal Experimental Design (OED), as a means of acquiring and restricting information to improve the probability of selecting the correct decision. We use probability gain theory to acquire the most useful piece of information to present to the decision maker, and we extend this to create a probability loss theory that restricts information that does not aid (probabilistically) or aids the least in the decision-making process.

We consider the problem of learning structures and parameters of continuous-time Bayesian networks (CTBNs) from time-course data under minimal experimental resources. In practice, the cost of generating experimental data poses a bottleneck, especially in the natural and social sciences. A popular approach to overcome this is Bayesian optimal experimental design (BOED). However, BOED becomes infeasible in high-dimensional settings, as it involves integration over all possible experimental outcomes. We propose a novel criterion for experimental design based on a variational approximation of the expected information gain. We show that for CTBNs, a semi-analytical expression for this criterion can be calculated for structure and parameter learning. By doing so, we can replace sampling over experimental outcomes by solving the CTBNs master-equation, for which scalable approximations exist. This alleviates the computational burden of integrating over possible experimental outcomes in high-dimensions. We employ this framework in order to recommend interventional sequences. In this context, we extend the CTBN model to conditional CTBNs in order to incorporate interventions. We demonstrate the performance of our criterion on synthetic and real-world data.

Earthquakes arise from tectonic loading of complex fault systems consisting of heterogeneous material parameters, geometry, rheology, and prestress. All of those are subject to uncertainties, and their interactions and sensitivities for the dynamic rupture problem are incompletely understood. Here, we apply Graph Neural Networks (GNNs) to approximate the behavior learned from more computationally intensive, physics‐based (“high‐fidelity”) computations to build a GNN‐based simulator (GNS) for earthquake rupture dynamics. Given only a minimum input –the hypocenter location– our GNS can reproduce rate‐weakening friction governed dynamic rupture behavior, from nucleation to propagation and termination. Outside the training set, the GNS can generalize well to different hypocenter locations, fault sizes, and pre‐stress state levels while achieving a factor ∼20–40 per‐time‐step computational speedup. This may allow for more efficient estimates of the mapping from pre‐earthquake state, as might be inferred from geodesy, to expected rupture dynamics, for example. By extracting a coarse‐grained version of the underlying dynamics, the GNS provides new perspectives to explore the physics of rupture. Further development of GNS may enable new kinds of parameter space exploration and provide surrogates for Bayesian model inference, uncertainty quantification, and optimal experimental design.

Bayesian optimal experimental design for constitutive model calibration

Surrogate-based sequential Bayesian experimental design using non-stationary Gaussian Processes

We consider the problem of A-optimal design of experiment under a Bayesian probabilistic model with both categorical and continuous response variables. The utility function of the local design problem is derived by applying Bayesian experimental design framework. We also develop an efficient optimization algorithm to obtain the local optimal design by combining the particle swarm optimization and the blocked coordinate descent methods. In addition, we discuss two different ways of constructing the global optimal design based on the algorithm for local optimal design. Simulation studies are presented to illustrate the efficiency of our approach.

Bayesian OED for Sensor Placement: Analysis and Optimization of Seismo-Acoustic Monitoring Networks with Bayesian Optimal Experimental Design

Efficient Sampling of Climate Models using Bayesian Optimal Experimental Design

Derivative-informed projected neural networks for high-dimensional parametric maps governed by PDEs

Summary When Shannon entropy is used as a criterion in the optimal design of experiments, advantage can be taken of the classical identity representing the joint entropy of parameters and observations as the sum of the marginal entropy of the observations and the preposterior conditional entropy of the parameters. Following previous work in which this idea was used in spatial sampling, the method is applied to standard parameterized Bayesian optimal experimental design. Under suitable conditions, which include non-linear as well as linear regression models, it is shown in a few steps that maximizing the marginal entropy of the sample is equivalent to minimizing the preposterior entropy, the usual Bayesian criterion, thus avoiding the use of conditional distributions. It is shown using this marginal formulation that under normality assumptions every standard model which has a two-point prior distribution on the parameters gives an optimal design supported on a single point. Other results include a new asymptotic formula which applies as the error variance is large and bounds on support size.

We propose and analyze a Stein variational reduced basis method (SVRB) to solve large-scale PDE-constrained Bayesian inverse problems. To address the computational challenge of drawing numerous samples requiring expensive PDE solves from the posterior distribution, we integrate an adaptive and goal-oriented model reduction technique with an optimization-based Stein variational gradient descent method. We present detailed analyses for the reduced basis approximation errors of the potential and its gradient, the induced errors of the posterior distribution measured by Kullback--Leibler divergence, as well as the induced errors of the Stein variational samples. To demonstrate the computational accuracy and efficiency of SVRB, we report results of numerical experiments on a Bayesian inverse problem governed by a diffusion PDE with random parameters with both uniform and Gaussian prior distributions. Over 100X speedups can be achieved while the accuracy of the approximation of the potential and its gradient is preserved.

Score-based diffusion models (SDMs) have emerged as a powerful tool for sampling from the posterior distribution in Bayesian inverse problems. However, existing methods often require multiple evaluations of the forward mapping to generate a single sample, resulting in significant computational costs for large-scale inverse problems. To address this, we propose an unconditional representation of the conditional score function (UCoS) tailored to linear inverse problems, which avoids forward model evaluations during sampling by shifting com putational effort to an offline training phase. In this phase, a task-dependent score function is learned based on the linear forward operator. Crucially, we show that the conditional score can be derived exactly from a trained (unconditional) score using affine transformations, eliminating the need for conditional score approximations. Our approach is formulated in infinite-dimensional function spaces, making it inherently discretization-invariant. We support this formulation with a rigorous convergence analysis that justifies UCoS beyond any specific discretization. Finally we validate UCoS through high-dimensional computed tomography (CT) and image deblurring experiments, demonstrating both scalability and accuracy.