Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation

Groundwater models always involve high‐dimensional parameters, which makes computationally tractable data assimilation using surrogate models very challenging. To address this issue, one common practice is to employ dimension reduction (DR) techniques. Nevertheless, traditional DR methods are usually implemented based on prior parameter statistics, that is, without considering the inherent system dynamics. Here, we show that when significant difference in parameter sensitivity exists, further efficiency can be achieved by adopting a supervised DR method, that is, the active subspace (AS) method. To avoid non‐trivial efforts in calculating the gradient information needed in the standard AS method, a cluster‐based gradient‐free AS (GFAS) method is developed in this study. By combining GFAS with Gaussian process regression, a surrogate model for the CPU‐demanding groundwater model can be adaptively constructed to accelerate data assimilation. Furthermore, a compensation scheme is proposed to cope with uncertainty underestimation caused by DR. The developed approach is tested with numerical experiments and field cases, which illustrated that the new approach is more efficient than the previously developed unsupervised ones by incorporating sensitivity information. Although an iterative ensemble smoother is employed in this study, the proposed method can also be used in other data assimilation approaches, such as Markov chain Monte Carlo and ensemble Kalman filter.

In gas turbine engineering, film cooling is critical for protecting engine components from high temperatures that can compromise safety and performance. However, the process is fraught with uncertainties due to complex inflow conditions and geometrical configurations, significantly affecting cooling efficiency, and it’s important to pinpoint the dominant factors in a quantified way. Traditional methods, such as the Monte Carlo (MC) approach, struggle with the ‘curse of dimensionality’, rendering them computationally intensive as variable counts increase. This study addresses these challenges by implementing a deep learning strategy using a Convolutional Neural Network (CNN) model to efficiently predict film cooling effectiveness, reducing computational loads compared to traditional Computational Fluid Dynamics (CFD) simulations, which can surrogate the CFD model and also provide the gradient information utilizing by subsequent works. Additionally, the study employs Active Subspace (AS) analysis for dimensionality reduction, identifying dominant parameters based on the trained CNN model. This approach not only enhances the speed of simulations, but also provide an effective way to analyse dominant parameters and carry out optimizations. Results show a strong correlation between the CNN predictions and detailed CFD simulations, validating the model’s efficacy in capturing complex cooling dynamics. And the dominant analysis based on AS method gives the most important factors: blowing ratio, mainstream Mach number. Subsequent optimizations are done by an inverse network which treated with low-dimensional active inputs, and the results show an improvement compared to the maximum results in all samples.

The recently introduced basis adaptation method for homogeneous (Wiener) chaos expansions is explored in a new context where the rotation/projection matrices are computed by discovering the active subspace (AS) where the random input exhibits most of its variability. In the case where a One-dimensional (1D) AS exists, the methodology can be applicable to generalized polynomial chaos expansions (PCE), thus enabling the projection of a high-dimensional input to a single input variable and the efficient estimation of a univariate chaos expansion. Attractive features of this approach, such as the significant computational savings and the high accuracy in computing statistics of interest are investigated.

Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

This paper describes a real-time data collection framework and an adaptive machining learning method for constructing a real-time energy prediction model for a machine tool. To effectively establish the energy consumption pattern of a machine tool over time, the energy prediction model is continuously updated with new measurement data to account for time-varying effects of the machine tool, such as tool wear and machine tool deterioration. In this work, a real-time data collection and processing framework is developed to retrieve raw data from a milling machine tool and its sensors and convert them into relevant input features. The extracted input features are then used to construct the energy prediction model using Gaussian Process (GP) regression. To update the GP regression model with real-time streaming data, we investigate the use of sparse representation of the covariance matrix to reduce the computational and storage demands of the GP regression. We compare computational efficiency of sparse GP to that of full GP regression model and show the effectiveness of the sparse GP regression model for tracking the variation in the energy consumption pattern of the target machine.

The development of high-performance MEMS devices necessitates efficient design optimization and yield analysis in the presence of unavoidable process variations. The existing algorithms are not able to integrate the different requirements of optimization and uncertainty quantification (UQ), highlighting the necessity for a comprehensive algorithmic framework. To this end, a novel algorithm framework is proposed for the optimization and UQ of MEMS devices. The deep neural network (DNN) and polynomial chaos expansion (PCE) are respectively leveraged to build the global surrogate model for the nondominated sorting genetic algorithm II (NSGA-II) and the local surrogate model for the UQ analysis. The DNN is refined by the bi-fidelity data training scheme to reduce the training data demanded, and the active subspace dimension reduction (AS) is employed to fully utilize the information within the global surrogate model, thereby minimizing the computational resources needed for the local surrogate model construction. This algorithm is applied to a kind of MEMS thermal flow sensor. The proposed algorithmic simultaneously increases sensor sensitivity by 14.77% while reducing computational expenditure by 44.4% during the optimization phase. Besides, the yield analysis is also derived accurately, with computational cost reductions of 73.0%. These results indicate that this novel algorithm supplies an effective, accurate, and efficient technique for the optimization and yield analysis for MEMS devices. [2025-0231]

BackgroundThe analysis of gene expression from time series underpins many biological studies. Two basic forms of analysis recur for data of this type: removing inactive (quiet) genes from the study and determining which genes are differentially expressed. Often these analysis stages are applied disregarding the fact that the data is drawn from a time series. In this paper we propose a simple model for accounting for the underlying temporal nature of the data based on a Gaussian process.ResultsWe review Gaussian process (GP) regression for estimating the continuous trajectories underlying in gene expression time-series. We present a simple approach which can be used to filter quiet genes, or for the case of time series in the form of expression ratios, quantify differential expression. We assess via ROC curves the rankings produced by our regression framework and compare them to a recently proposed hierarchical Bayesian model for the analysis of gene expression time-series (BATS). We compare on both simulated and experimental data showing that the proposed approach considerably outperforms the current state of the art.ConclusionsGaussian processes offer an attractive trade-off between efficiency and usability for the analysis of microarray time series. The Gaussian process framework offers a natural way of handling biological replicates and missing values and provides confidence intervals along the estimated curves of gene expression. Therefore, we believe Gaussian processes should be a standard tool in the analysis of gene expression time series.

Kepler-78b is a transiting planet that is 1.2 times the radius of Earth and orbits a young, active K dwarf every 8 hours. The mass of Kepler-78b has been independently reported by two teams based on radial velocity measurements using the HIRES and HARPS-N spectrographs. Due to the active nature of the host star, a stellar activity model is required to distinguish and isolate the planetary signal in radial velocity data. Whereas previous studies tested parametric stellar activity models, we modeled this system using nonparametric Gaussian process (GP) regression. We produced a GP regression of relevant Kepler photometry. We then use the posterior parameter distribution for our photometric fit as a prior for our simultaneous GP + Keplerian orbit models of the radial velocity datasets. We tested three simple kernel functions for our GP regressions. Based on a Bayesian likelihood analysis, we selected a quasi-periodic kernel model with GP hyperparameters coupled between the two RV datasets, giving a Doppler amplitude of 1.86 $\pm$ 0.25 m s$^{-1}$ and supporting our belief that the correlated noise we are modeling is astrophysical. The corresponding mass of 1.87 $^{+0.27}_{-0.26}$ M$_{\oplus}$ is consistent with that measured in previous studies, and more robust due to our nonparametric signal estimation. Based on our mass and the radius measurement from transit photometry, Kepler-78b has a bulk density of 6.0$^{+1.9}_{-1.4}$ g cm$^{-3}$. We estimate that Kepler-78b is 32$\pm$26% iron using a two-component rock-iron model. This is consistent with an Earth-like composition, with uncertainty spanning Moon-like to Mercury-like compositions.

Gaussian process (GP) regression is a nonparametric regression method that can be used to predict continuous quantities. Here, we show that the same technique can be applied to a class of phase imaging techniques based on measurements of intensity at multiple propagation distances, i.e. the transport of intensity equation (TIE). In this paper, we demonstrate how to apply GP regression to estimate the first intensity derivative along the direction of propagation and incorporate non-uniform propagation distance sampling. The low-frequency artifacts that often occur in phase recovery using traditional methods can be significantly suppressed by the proposed GP TIE method. The method is shown to be stable with moderate amounts of Gaussian noise. We validate the method experimentally by recovering the phase of human cheek cells in a bright field microscope and show better performance as compared to other TIE reconstruction methods.

We present a novel computational approach for extracting localized signals from smooth background distributions. We focus on datasets that can be naturally presented as binned integer counts, demonstrating our procedure on the CERN open dataset with the Higgs boson signature, from the ATLAS collaboration at the Large Hadron Collider. Our approach is based on Gaussian Process (GP) regression — a powerful and flexible machine learning technique which has allowed us to model the background without specifying its functional form explicitly and separately measure the background and signal contributions in a robust and reproducible manner. Unlike functional fits, our GP-regression-based approach does not need to be constantly updated as more data becomes available. We discuss how to select the GP kernel type, considering trade-offs between kernel complexity and its ability to capture the features of the background distribution. We show that our GP framework can be used to detect the Higgs boson resonance in the data with more statistical significance than a polynomial fit specifically tailored to the dataset. Finally, we use Markov Chain Monte Carlo (MCMC) sampling to confirm the statistical significance of the extracted Higgs signature.

Robust topology optimization for multi-material structures considering material uncertainties

The Expected Value of Perfect Partial Information (EVPPI) is a decision‐theoretic measure of the ‘cost’ of parametric uncertainty in decision making used principally in health economic decision making. Despite this decision‐theoretic grounding, the uptake of EVPPI calculations in practice has been slow. This is in part due to the prohibitive computational time required to estimate the EVPPI via Monte Carlo simulations. However, recent developments have demonstrated that the EVPPI can be estimated by non‐parametric regression methods, which have significantly decreased the computation time required to approximate the EVPPI. Under certain circumstances, high‐dimensional Gaussian Process (GP) regression is suggested, but this can still be prohibitively expensive. Applying fast computation methods developed in spatial statistics using Integrated Nested Laplace Approximations (INLA) and projecting from a high‐dimensional into a low‐dimensional input space allows us to decrease the computation time for fitting these high‐dimensional GP, often substantially. We demonstrate that the EVPPI calculated using our method for GP regression is in line with the standard GP regression method and that despite the apparent methodological complexity of this new method, R functions are available in the package BCEA to implement it simply and efficiently. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Nonnegativity-enforced Gaussian process regression

Human motion intent learning based motion assistance control for a wearable exoskeleton