Bayesian Methodology for Uncertainty Quantification in Complex Engineering Systems

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning and the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but underrated area is the uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture, and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of the UQ of ML. We elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling are discussed, demonstrated, and compared. We also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian processes, and conformal prediction. Finally, we discuss the need for building a verification, validation, and UQ framework to establish ML credibility.

Recently, there has been much attention in the use of machine learning methods, particularly deep learning for stock price prediction. A major limitation of conventional deep learning is uncertainty quantification in predictions which affect investor confidence. Bayesian neural networks feature Bayesian inference for providing inference (training) of model parameters that provides a rigorous methodology for uncertainty quantification in predictions. Markov Chain Monte Carlo (MCMC) sampling methods have been prominent in implementing inference of Bayesian neural networks; however certain limitations existed due to a large number of parameters and the need for better computational resources. Recently, there has been much progress in the area of Bayesian neural networks given the use of Langevin gradients with parallel tempering MCMC that can be implemented in a parallel computing environment. The COVID-19 pandemic had a drastic impact in the world economy and stock markets given different levels of lockdowns due to rise and fall of daily infections. It is important to investigate the performance of related forecasting models during the COVID-19 pandemic given the volatility in stock markets. In this paper, we use novel Bayesian neural networks for multi-step-ahead stock price forecasting before and during COVID-19. We also investigate if the pre-COVID-19 datasets are useful of modelling stock price forecasting during COVID-19. Our results indicate due to high volatility in the stock-price during COVID-19, it is more challenging to provide forecasting. However, we found that Bayesian neural networks could provide reasonable predictions with uncertainty quantification despite high market volatility during the first peak of the COVID-19 pandemic.

Rule-based weight of evidence approaches to ecological risk assessment may not account for uncertainties and generally lack probabilistic integration of lines of evidence. Bayesian networks allow causal inferences to be made from evidence by including causal knowledge about the problem, using this knowledge with probabilistic calculus to combine multiple lines of evidence, and minimizing biases in predicting or diagnosing causal relationships. Too often, sources of uncertainty in conventional weight of evidence approaches are ignored that can be accounted for with Bayesian networks. Specifying and propagating uncertainties improve the ability of models to incorporate strength of the evidence in the risk management phase of an assessment. Probabilistic inference from a Bayesian network allows evaluation of changes in uncertainty for variables from the evidence. The network structure and probabilistic framework of a Bayesian approach provide advantages over qualitative approaches in weight of evidence for capturing the impacts of multiple sources of quantifiable uncertainty on predictions of ecological risk. Bayesian networks can facilitate the development of evidence-based policy under conditions of uncertainty by incorporating analytical inaccuracies or the implications of imperfect information, structuring and communicating causal issues through qualitative directed graph formulations, and quantitatively comparing the causal power of multiple stressors on valued ecological resources. These aspects are demonstrated through hypothetical problem scenarios that explore some major benefits of using Bayesian networks for reasoning and making inferences in evidence-based policy.

Resource allocation methodology based on object-oriented discrete event simulation: A production logistics system case study

A high-level state model of the national airspace system (NAS) has been developed and refined. That model provides guidance on the effect and interactions of sources of uncertainty in the NAS. The model also provides guidance on the possible effect of NextGen concepts and technologies on the NAS. The model is being coded as a high-level, abstract simulation that, when provided with specific uncertainty distributions, can also identify the emergent effect of those changes in the uncertainty distributions. The results of that analysis and simulation provide guidance for prioritizing the development and adoption of NextGen concepts and technologies for the NAS. I. Introduction he National Airspace System (NAS) is a highly complex, socio-technical system. At its lowest level, it consists of many thousands of entities, including aircraft and other sophisticated technology, passengers, pilots, and air traffic controllers. Modeling and simulation of such a system is difficult due to the large number of entities to be modeled, the infinite (or nearly so) number of degrees of freedom within the system, and the high degree of possible interactivity in the system. As a result, existing models/simulations must simplify the system by eliminating from consideration certain aspects, and therefore cannot be completely accurate in determining the effect of uncertainties on the performance of the NAS. To try to fill this gap in capability, a model of the NAS was developed, with the purpose of determining the effect of the various sources of uncertainty on the system performance of the NAS. This model is state-based, and models the system from a high level (Landry and Archer, 2013). The model is believed to be complete. Given this model, one can reason about the effects of uncertainty on the NAS, as well as the interactions of these sources of uncertainty (Landry and Archer, 2013). In addition, if one characterizes the effect of different operational concepts and technologies being considered under NextGen in terms of their effect on those sources of uncertainty, one can then reason about the relative benefits of those different concepts and technologies. In this work, we focus on the effect of concepts and technologies on the performance of the NAS. Following a brief discussion of the model, an enumeration of the technologies and concepts under consideration for NextGen will be presented, including their anticipated degree of effect on various sources of uncertainty. Lastly, the results of simulation exercises to test the model will be presented. II. Model of the NAS: Framework

Improved spatiotemporal monitoring of soil salinity using filtered kriging with measurement errors: An application to the West Urmia Lake, Iran

Within the Best Estimate Plus Uncertainty framework for the safety analysis of Nuclear Power Plants, the quantification of the uncertainties affecting the Thermal-Hydraulics (T-H) codes used is crucial. For this, Inverse Uncertainty Quantification (IUQ) methodologies are being developed for determining the probability density functions of relevant T-H codes input parameters, based on experimental data from Separate Effect Tests (SETs) experimental facilities. In practice, IUQ is challenged by the large range of variability of the experimental data in terms of Initial and Boundary Conditions (ICs & BCs), because the experimental campaigns are designed to cover the widest possible domain of conditions with the smallest number of experiments, so that same or similar ICs and BCs are seldomly repeated. To address this issue, we propose to use global sensitivity analysis, to tailor the IUQ on specific sub-regions described by segmented ICs & BCs domains. The methodology proposed is exemplified on two SETs, namely Sozzi-Sutherland and Super Moby Dick, whose experimental databases have been made available in the ATRIUM (Application Tests for Realization of Inverse Uncertainty quantification and validation Methodologies in thermal hydraulics) project promoted by the OECD/NEA/CSNI. The results obtained are superior to those of traditional IUQ methodologies for models highly sensitive to ICs & BCs.

The accurate prediction of shear wave velocity (Vs) is critical for earthquake engineering applications. However, the prediction is inevitably influenced by geotechnical variability and various sources of uncertainty. This paper investigates the effectiveness of integrating explainable machine learning (ML) model and Bayesian generalized linear model (GLM) to enhance both predictive accuracy and uncertainty quantification in Vs prediction. The study utilizes an Extreme Gradient Boosting (XGBoost) algorithm coupled with Shapley Additive Explanations (SHAPs) and partial dependency analysis to identify key geotechnical parameters influencing Vs predictions. Additionally, a Bayesian GLM is developed to explicitly account for uncertainties arising from geotechnical variability. The effectiveness and predictive performance of the proposed models were validated through comparison with real case scenarios. The results highlight the unique advantages of each model. The XGBoost model demonstrates good predictive performance, achieving high coefficient of determination (R2), index of agreement (IA), Kling–Gupta efficiency (KGE) values, and low error values while effectively explaining the impact of input parameters on Vs. In contrast, the Bayesian GLM provides probabilistic predictions with 95% credible intervals, capturing the uncertainty associated with the predictions. The integration of these two approaches creates a comprehensive framework that combines the strengths of high-accuracy ML predictions with the uncertainty quantification of Bayesian inference. This hybrid methodology offers a powerful and interpretable tool for Vs prediction, providing engineers with the confidence to make informed decisions.

Spatiotemporal variation of soil organic matter (SOM) contents was significant to research on global warming, greenhouse effect, and ecosystem health and quality. However, the spatiotemporal modeling for soil properties in most studies focused on a discrete-time calibration and validation, and faced the problems of missing observations. Integrated Nested Laplace Approximation with the Stochastic Partial Differential Equation (INLA-SPDE) model that is robust to missing data and unbalanced sampling design was proposed as a potential model for spatiotemporal soil modeling. This study presented an application of INLA-SPDE for spatiotemporal modeling of SOM (2006, 2010, 2018) using 924 samples and 8 environmental covariates in Lishu County, Northeast China. The results demonstrated that the INLA-SPDE model incorporating spatiotemporal information generally outperformed the two-phase methods based on Cubist and Random Forest, particularly in years requiring greater temporal extrapolation (2006 and 2018), while achieving comparable performance in 2010. This superiority can be attributed to its comprehensive consideration of various sources of uncertainty. Furthermore, the posterior distributions derived from the model provided valuable insights into the effects of environmental covariates on SOM spatiotemporal variation, with clay content showing the strongest positive influence and annual precipitation exhibiting a notable negative effect. The spatial pattern of SOM consistently exhibited higher values in the east and lower values in the west. After an overall decline from 2006 to 2010, mean SOM content increased from 17.97 g kg-1 to 20.85 g kg-1 between 2010 and 2018 (a total increase of 2.88 g kg-1 at an annual rate of 0.36 g kg-1 yr-1), with notable recovery in central and eastern areas, likely associated with the implementation of straw returning practices. In addition to prediction accuracy, computational complexity, and uncertainty analysis, the study evaluated the model from new perspectives, including covariate interpretability and flexibility. This research provides a promising spatiotemporal modeling framework for digital soil mapping.

Machine learning using bayesian networks to predict response in patients with newly diagnosed Acute Myeloid Leukemia

To address the environmental concern and improve the economic efficiency, the wind power is rapidly integrated into smart grids. However, the inherent uncertainty of wind energy raises operational challenges. To ensure the cost-efficient, reliable and robust operation, it is critically important to find the optimal decision that can correctly and rigorously hedge against all sources of uncertainty. In this paper, we propose data-driven stochastic unit commitment (SUC) to guide the power grids scheduling. Specifically, given the finite historical data, the posterior predictive distribution is developed to quantify the wind power prediction uncertainty accounting for both inherent stochastic uncertainty of wind power generation and input model estimation error. For complex power grid systems, a finite number of scenarios is used to estimate the expected cost in the planning horizon. To further control the impact of finite sampling error induced by using the sample average approximation (SAA), we propose a parallel computing based optimization solution methodology, which can quickly find the reliable optimal unit commitment decision hedging against various sources of uncertainty. The empirical study over six-bus and 118-bus systems demonstrates that our approach can provide more cost-efficient and robust performance than the existing deterministic and stochastic unit commitment approaches.

Shape Moment Invariants as a New Methodology for Uncertainty Quantification in Microstructures

A methodology for uncertainty quantification and sensitivity analysis for responses subject to Monte Carlo uncertainty with application to fuel plate characteristics in the ATRC

<div class="section abstract"><div class="htmlview paragraph">Self-piercing riveting (SPR) is a key joining method in multi/thin-material automotive structures, yet accurately predicting the mechanical strength of SPR joints remains challenging due to numerous influencing factors. Empirical engineering equations [<span class="xref">1</span>] provide a foundation for estimating lap-shear and cross-tension strength but require several geometric parameters that are often unavailable in the design phase.</div><div class="htmlview paragraph">To address this limitation, we extract and leverage the core physical relationships embedded in these formulas. By reformulating the dependence of joint strength on the yield strength and total thickness of the sheet stack as practical regression models, we enable strength prediction using only commonly available material properties.</div><div class="htmlview paragraph">Furthermore, a Bayesian convolutional neural network (BCNN) model is developed to incorporate additional material features, offering improved prediction accuracy and uncertainty quantification.</div></div>

In recent years, the demand to provide best-estimate predictions with confident bounds is increasing for the nuclear reactor performance and safety analysis. The Organisation for Economic Co-operation and Development Nuclear Energy Agency has been developing an international benchmark of the light water reactor (LWR) uncertainty analysis in modeling (UAM) for the examination of uncertainty quantification and propagation methodologies with various modeling and simulation code systems. The objective of the present work is to develop an uncertainty propagation mechanism based on the stochastic sampling method by taking into account the uncertainties of both basic nuclear data and fuel modeling parameters in the simulation of pressurized water reactors (PWRs) that can be incorporated in the conventional LWR simulation approach. More specifically, the Three Mile Island Unit 1–related exercises from the LWR-UAM benchmark have been modeled using the coupled TRACE/PARCS code system in the three-dimensional core representation. The input uncertainties of the neutronics simulation include few-group cross sections and kinetics parameters generated using the Sampler/Polaris sequence of SCALE 6.2.1. Several heat transfer–related variables for the fuel modeling were considered as sources of input uncertainty of the thermal-hydraulics simulations, including the thermal conductivity of fuel and cladding, fuel heat capacity, and the gap conductance. Dakota was used to sample input parameters of the coupled code system and to perform the uncertainty analysis. Two types of simulations were conducted: steady-state calculation at hot full-power condition and transient scenario initiated by the spatially asymmetric rod ejection accident. Quantities of interest for the steady-state calculation, including core multiplication factor and power peaking factors, were calculated with associated uncertainties. For transient calculations, best-estimate plus uncertainty results of the time evolution of core reactivity, core power, and peak fuel temperature were generated and analyzed. The Wilks’ formula was used to determine the necessary sample size to achieve a 95% confidence of the 95% limit of output quantities of interest. Although the uncertainty propagation and quantification method presented in this paper was developed for PWRs, it could be in general applicable to the multiphysics uncertainty quantification of other types of LWR cores.