Accelerating Pipeline Corrosion Modeling via Bayesian Active Learning

Ab initio protein docking represents a major challenge for optimizing a noisy and costly "black box"-like function in a high-dimensional space. Despite progress in this field, there is a lack of rigorous uncertainty quantification (UQ). To fill the gap, we introduce a novel algorithm, Bayesian active learning (BAL), for optimization and UQ of such black-box functions with applications to flexible protein docking. BAL directly models the posterior distribution of the global optimum (i.e., native structures) with active sampling and posterior estimation iteratively feeding each other. Furthermore, it uses complex normal modes to span a homogeneous, Euclidean conformation space suitable for high-dimensional optimization and constructs funnel-like energy models for quality estimation of encounter complexes. Over a protein-docking benchmark set and a CAPRI set including homology docking, we establish that BAL significantly improves against starting points from rigid docking and refinements by particle swarm optimization, providing a top-3 near-native prediction for one third targets. Quality assessment empowered with UQ leads to tight quality intervals with half range around 25% of the actual interface root-mean-square deviation and confidence level at 85%. BAL's estimated probability of a prediction being near-native achieves binary classification AUROC at 0.93 and area under the precision recall curve over 0.60 (compared to 0.50 and 0.14, respectively, by chance), which also improves ranking predictions. This study represents the first UQ solution for protein docking, with rigorous theoretical frameworks and comprehensive empirical assessments.

Computational Fluid Dynamics (CFD) is regarded as an important tool for analyzing the flow field, thermal environment, and air quality around the built environment. However, for built environment applications, the high computational cost of CFD hinders large-scale scenario simulation and efficient design optimization. In the field of built environment research, surrogate modeling has become a key technology to connect the needs of high-fidelity CFD simulation and rapid prediction, whereas the low-dimensional nature of traditional surrogate models is unable to match the physical complexity and prediction needs of built flow fields. Therefore, combining machine learning (ML) with CFD to predict flow fields in built environments offers a promising way to increase simulation speed while maintaining reasonable accuracy. This review briefly reviews traditional surrogate models and focuses on ML-based surrogate models, especially the specific application of neural network architectures in rapidly predicting flow fields in the built environment. The review indicates that ML accelerates the three core aspects of CFD, namely mesh preprocessing, numerical solving, and post-processing visualization, in order to achieve efficient coupled CFD simulation. Although ML surrogate models still face challenges such as data availability, multi-physics field coupling, and generalization capability, the emergence of physical information-driven data enhancement techniques effectively alleviates the above problems. Meanwhile, the integration of traditional methods with ML can further enhance the comprehensive performance of surrogate models. Notably, the online ministry of trained ML models using transfer learning strategies deserves further research. These advances will provide an important basis for advancing efficient and accurate operational solutions in sustainable building design and operation.

Running detailed, physics-based numerical simulations of subsurface transport is often computationally expensive. This becomes a challenge when calibrating models against observed data using methods that require a large number of model runs, such as Bayesian inference. To address this challenge, surrogate models are frequently used to approximate simulation outputs. Surrogates are trained using input-output pairs generated by the physics-based model. Traditional approaches typically rely on space-filling designs that uniformly cover the entire parameter space. However, for high-dimensional problems, this becomes impractical and tends to waste computational resources on parameter regions that are either physically irrelevant or contradict available measurement data.To overcome these limitations, we utilize a Bayesian Active Learning (BAL) framework that iteratively selects training points most informative for Bayesian inference given available measurements. We employ Gaussian Processes and Bayesian-Polynomial Chaos Expansions as surrogates, which provide probability distributions for their predictions. Our approach takes advantage of these predictive distributions to evaluate candidate training points using information-theoretic criteria. To account for measurement uncertainty and prevent the algorithm from over-sampling local likelihood maxima, we investigate different strategies for representing observations within the selection process. These criteria are integrated into a multi-objective scoring function that balances global exploration (reducing surrogate uncertainty) with targeted exploitation (refining high-likelihood regions). Additionally, we demonstrate how observations from early time steps can iteratively guide the selection of training points to improve predictive accuracy for later, critical periods of the transport process.We test this method on analytical benchmarks and on subsurface transport models. The framework is evaluated in terms of convergence speed and posterior accuracy relative to existing active learning strategies and reference solutions derived from the full physics-based model. Overall, the proposed goal-oriented strategy aims to reduce the number of expensive model evaluations required for surrogate training, improving the efficiency of subsurface characterization, model calibration and predictive modeling.

Gaussian process regression is a well-established Bayesian machine learning method. We propose a new approach to Gaussian process regression using quantum kernels based on parameterized quantum circuits. By employing a hardware-efficient feature map and careful regularization of the Gram matrix, we demonstrate that the variance information of the resulting quantum Gaussian process can be preserved. We also show that quantum Gaussian processes can be used as a surrogate model for Bayesian optimization, a task that critically relies on the variance of the surrogate model. To demonstrate the performance of this quantum Bayesian optimization algorithm, we apply it to the hyperparameter optimization of a machine learning model which performs regression on a real-world dataset. We benchmark the quantum Bayesian optimization against its classical counterpart and show that quantum version can match its performance.

Chapter 10 - Bayesian Active Learning on Graphs

A surrogate model assisted with a subdomain model for permanent-magnet machine

Study of a hull form optimization system based on a Gaussian process regression algorithm and an adaptive sampling strategy, Part I: Single-objective optimization

To avoid redundant trial and error experiments in hope of achieving acceptable surface roughness, reliable predictive models must be engineered to anticipate surface characteristics based on process parameter inputs. In the present study, two rigorously tested supervised machine-learning based models are proposed to predict the arithmetic average (Ra) of profile deviation for the surface of Ti-6Al-4V alloy manufactured via selective laser melting (SLM). Firstly, a Gaussian Process Regression (GPR) model with Rational Quadratic kernel function is constructed after ten-fold cross validation of the training data. Secondly, using the same training data and the same ten-fold cross validation, a feed-forward narrow neural network (NN) is employed. Primary input parameters of SLM process, namely laser power, scanning speed, hatch spacing, layer thickness and volumetric energy density are mined from literature investigating as-built surface characteristics of SLMed Ti-6Al-4V alloy. To further test the developed machine-learning models, ten 8 × 8 × 8 mm Ti-6Al-4V cubes are manufactured and a comparative between two non-parametric (GPR and NN) models is performed by predicting the surface roughness (Ra) of the ten samples. It is discovered that the NN model underperforms with a root mean squared error (RMSE) of 2.76 μm, as opposed to its counterpart GPR model, exhibiting RMSE of 0.82 μm. Additionally, analyses of the surface characteristics of the fabricated samples, the surface profiles, and impact of such a predictive model are provided.

Wave episode based Gaussian process regression for extreme event statistics in ship dynamics: Between the Scylla of Karhunen–Loève convergence and the Charybdis of transient features

Bi-objective optimization of traffic assignment with air quality consideration via CFD-based surrogate model

The credibility of Computational Fluid Dynamics (CFD) has been a topic of debate due to the significant uncertainties inherent in its modeling processes and numerical implementations. Uncertainty Quantification (UQ) offers a scientific framework for quantitatively assessing and mitigating uncertainties in CFD simulations. However, this procedure typically requires numerous CFD simulations and considerable manual effort for both simulation management and data analysis. To overcome these challenges, this work develops a platform called UQ4CFD, a browser–server software that provides automated and customized uncertainty quantification capabilities for CFD studies. The UQ4CFD platform integrates different kinds of methodologies to perform comprehensive uncertainty analysis, including uncertainty propagation, sensitivity analysis, surrogate modeling, numerical discretization uncertainty analysis, model validation, model calibration, etc. A tightly coupled CFD-UQ workflow is built to automate the complete analytical process, encompassing parameter sampling, simulation execution, and results analysis, which significantly improves computational efficiency while reducing risks associated with data processing errors. Comprehensive validation employing both analytical benchmark functions and practical CFD cases has been conducted to demonstrate the platform’s effectiveness and adaptability in diverse UQ scenarios.

Surrogate model based uncertainty quantification of CFD simulations of the viscous flow around a ship advancing in shallow water

In this work, we present the development of robust surrogate models for Ion Cyclotron Range of Frequencies (ICRF) and High-Harmonic Fast Wave (HHFW) heating predictions in fusion plasmas. Building upon our previous efforts to achieve real-time capable models, we identify the cause of the outliers found using TORIC in certain HHFW heating scenarios. The outliers are observed to be spurious ion Bernstein wave (IBW)-like modes caused by a wavelength control algorithm designed to address challenging scenarios with high perpendicular wavenumbers. The effect arises from the modulation in the perpendicular susceptibility, which can induce sign reversal and IBW-like propagation for scenarios featuring normalized ion Larmor radius λi≫1. We use TORIC with this algorithm disabled to generate a novel HHFW-NSTX database that is free of outliers. Surrogate models trained on this database, including Random Forest Regressor (RFR), Multi-Layer Perceptrons, and Gaussian Process Regressors (GPR), demonstrate the ability to accurately predict HHFW heating profiles, with regression scores of R2∈[0.93−0.99]. Additionally we demonstrate that it is possible to generalize predictions beyond training data by the use of both RFR and GPR models, enabling the prediction of scenarios previously limited to the original model. GPR models also provide uncertainty quantification, offering insights into model confidence. This work introduces a comprehensive Verification, Validation, and Uncertainty Quantification methodology for surrogate modeling, applicable not only to ICRF heating but also to other RF heating challenges and fusion physics problems. Beyond accelerated inference, these models show effective extrapolation capabilities, providing an alternative for addressing numerical challenges.

Surrogate models are statistical or conceptual approximations for more complex simulation models. In this context, it is crucial to propagate the uncertainty induced by limited simulation budget and surrogate approximation error to predictions, inference, and subsequent decision-relevant quantities. However, quantifying and then propagating the uncertainty of surrogates is usually limited to special analytic cases or is otherwise computationally very expensive. In this paper, we propose a framework enabling a scalable, Bayesian approach to surrogate modeling with thorough uncertainty quantification, propagation, and validation. Specifically, we present three methods for Bayesian inference with surrogate models given measurement data. This is a task where the propagation of surrogate uncertainty is especially relevant, because failing to account for it may lead to biased and/or overconfident estimates of the parameters of interest. We showcase our approach in three detailed case studies for linear and nonlinear real-world modeling scenarios. Uncertainty propagation in surrogate models enables more reliable and safe approximation of expensive simulators and will therefore be useful in various fields of applications.

We propose the Bayesian Active Learning by Disagreement (BALD) model for keyword spotting in handwritten documents. In the context of keyword spotting in handwritten documents, the background text is all regions in the document that do not contain the keywords. The model tries to learn certain characteristics of the keyword and background text in an active learning framework. It takes into account the local character level scores and global word level scores to distinguish keywords from non-keywords. We propose to apply the bayesian active learning strategy to identify the regions of sample space from which more meaningful labeled samples of keywords and non-keywords can be extracted. This work is an extension to our previous work which used a variational dynamic background model to model the large variations of background text. The approach has been tested on IAM dataset for English. The results show that a decent background model can be learned in a more quicker and efficient manner using the BALD framework. The approach outperforms our prior work and other state of the art approaches.