Variable Fidelity Regression Using Low Fidelity Function Blackbox and Sparsification

Surrogate models or metamodels are commonly used to exploit expensive computational simulations within a design optimization framework. The application of multi-fidelity surrogate modeling approaches has recently been gaining ground due to the potential for further reductions in simulation effort over single fidelity approaches. However, given a black box problem when exactly should a designer select a multi-fidelity approach over a single fidelity approach and vice versa? Using a series of analytical test functions and engineering design examples from the literature, the following paper illustrates the potential pitfalls of choosing one technique over the other without a careful consideration of the optimization problem at hand. These examples are then used to define and validate a set of guidelines for the creation of a multi-fidelity Kriging model. The resulting guidelines state that the different fidelity functions should be well correlated, that the amount of low fidelity data in the model should be greater than the amount of high fidelity data and that more than 10 % and less than 80 % of the total simulation budget should be spent on low fidelity simulations in order for the resulting multi-fidelity model to perform better than the equivalent costing high fidelity model.

Gaussian process regression for multivariate spectroscopic calibration

A novel multiobjective adaptive surrogate modeling-based optimization (MO-ASMO) framework is proposed to utilize a minimal number of training samples efficiently for sequential model updates. All the sample points are enforced to be feasible, and to provide coverage of sparsely explored sparse design regions using a new optimization subproblem. The MO-ASMO method only evaluates high-fidelity functions at feasible sample points. During an exploitation sample phase, samples are selected to enhance solution accuracy rather than the global exploration. Sampling tasks are especially challenging for multiobjective optimization; for an n-dimensional design space, a strategy is required for generating model update sample points near an (n − 1)-dimensional hypersurface corresponding to the Pareto set in the design space. This is addressed here using a force-directed layout algorithm, adapted from graph visualization strategies, to distribute feasible sample points evenly near the estimated Pareto set. Model validation samples are chosen uniformly on the Pareto set hypersurface, and surrogate model estimates at these points are compared to high-fidelity model responses. All high-fidelity model evaluations are stored for later use to train an updated surrogate model. The MO-ASMO algorithm, along with the set of new sampling strategies, are tested using two mathematical and one realistic engineering problems. The second mathematical test problems is specifically designed to test the limits of this algorithm to cope with very narrow, non-convex feasible domains. It involves oscillatory objective functions, giving rise to a discontinuous set of Pareto-optimal solutions. Also, the third test problem demonstrates that the MO-ASMO algorithm can handle a practical engineering problem with more than 10 design variables and black-box simulations. The efficiency of the MO-ASMO algorithm is demonstrated by comparing the result of two mathematical problems to the results of the NSGA-II algorithm in terms of the number of high fidelity function evaluations, and is shown to reduce total function evaluations by several orders of magnitude when converging to the same Pareto sets.

Multi-fidelity surrogate (MFS) modeling technology, which efficiently constructs surrogate models using low-fidelity (LF) and high-fidelity (HF) data, has been studied to enhance the predictive capability of engineering performances. In addition, several neural network (NN) structures for MFS modeling have been introduced, benefiting from recent developments in deep learning research. However, existing multi-fidelity (MF) NNs have been developed assuming identical sets of input variables for LF and HF data, a condition that is often not met in practical engineering systems. Therefore, this study proposes a new structure of composite NN designed for MF data with different input variables. The proposed network structure includes an input mapping network that connects the LF and HF data's input variables. Even when the physical relationship between these variables is unknown, the input mapping network can be concurrently trained during the process of training the whole network model. Customized loss functions and activation variables are suggested in this study to facilitate forward and backward propagation for the proposed NN structures when training MF data with different inputs. The effectiveness of the proposed method, in terms of prediction accuracy, is demonstrated through mathematical examples and practical engineering problems related to tire performances. The results confirm that the proposed method offers better accuracy than existing surrogate models in most problems. Moreover, the proposed method proves advantageous for surrogate modeling of nonlinear or discrete functions, a characteristic feature of NN-based methods.

Accurate prognostication of carbon allowance valuation movements is essential for informing environmental policy formulation and enhancing the efficacy of market-driven regulatory instruments. Sophisticated predictive methodologies enable policymakers to calibrate carbon tax structures more effectively, optimize Emissions Trading Scheme (ETS) functionality, and channel capital toward sustainable, low-emission projects with increased confidence. This research focuses on the Tianjin Emissions Trading Scheme (TJTS) — an early provincial-level carbon exchange established as a component of China’s comprehensive national decarbonization agenda. We introduce a novel prediction framework utilizing Gaussian process regression (GPR), wherein model hyperparameters are determined via Bayesian optimization. This approach facilitates the estimator’s dynamic adaptation to unobserved market dynamics and latent trading patterns. Our empirical analysis employs daily settlement prices for Tianjin Emission Allowances (TJEA) spanning December 26, 2013, to March 23, 2021. This period encompasses significant regulatory reforms, distinct phases of market maturation, and evolving participant behavior within China’s progressively integrated national carbon pricing system. Model performance was rigorously evaluated using an out-of-sample testing window from June 8, 2020, to March 23, 2021. The results demonstrate exceptional predictive accuracy, quantified by a relative root-mean-square error (RRMSE) of 0.3818%, root-mean-square error (RMSE) of 0.0971, mean absolute error (MAE) of 0.0628, and a correlation coefficient (CC) reaching 99.838%. To our knowledge, this constitutes the inaugural application of Gaussian process regression within the context of China’s carbon trading exchanges. Beyond advancing theoretical insights into price discovery mechanisms within nascent emissions markets, the developed methodology offers a versatile analytical template readily transferable to comparable cap-and-trade systems globally.

Modern performance earthquake engineering practices frequently require a large number of time‐consuming non‐linear time‐history simulations to appropriately address excitation and structural uncertainties when estimating engineering demand parameter (EDP) distributions. Surrogate modeling techniques have emerged as an attractive tool for alleviating such high computational burden in similar engineering problems. A key challenge for the application of surrogate models in earthquake engineering context relates to the aleatoric variability associated with the seismic hazard. This variability is typically expressed as high‐dimensional or non‐parametric uncertainty, and so cannot be easily incorporated within standard surrogate modeling frameworks. Rather, a surrogate modeling approach that can directly approximate the full distribution of the response output is warranted for this application. This approach needs to additionally address the fact that the response variability may change as input parameter changes, yielding a heteroscedastic behavior. Stochastic emulation techniques have emerged as a viable solution to accurately capture aleatoric uncertainties in similar contexts, and recent work by the second author has established a framework to accommodate this for earthquake engineering applications, using Gaussian Process (GP) regression to predict the EDP response distribution. The established formulation requires for a portion of the training samples the replication of simulations for different descriptions of the aleatoric uncertainty. In particular, the replicated samples are used to build a secondary GP model to predict the heteroscedastic characteristics, and these predictions are then used to formulate the primary GP that produces the full EDP distribution. This practice, however, has two downsides: it always requires minimum replications when training the secondary GP, and the information from the non‐replicated samples is utilized only for the primary GP. This research adopts an alternative stochastic GP formulation that can address both limitations. To this end, the secondary GP is trained by measuring the square of sample deviations from the mean instead of the crude sample variances. To establish the primitive mean estimates, another auxiliary GP is introduced. This way, information from all replicated and non‐replicated samples is fully leveraged for estimating both the EDP distribution and the underlying heteroscedastic behavior, while formulation accommodates an implementation using no replications. The case study examples using three different stochastic ground motion models demonstrate that the proposed approach can address both aforementioned challenges.

The growing demand for biopharmaceutical products reflects their effectiveness in medical treatments. However, developing new biopharmaceuticals remains a major bottleneck, often taking up to a decade before market approval. Machine learning (ML) models have the potential to accelerate this process, but their success depends on access to large and diverse data sets for training. Multi-fidelity ML techniques offer a promising solution by integrating abundant, low-cost, and less accurate low-fidelity (LF) data with limited, expensive, and more accurate high-fidelity (HF) data. In this framework, LF data capture global system trends, while HF data refine and align model predictions with the available ground truth. Such integration can substantially reduce development costs and timelines by minimizing the need to acquire HF data, for example, through extensive experimental campaigns. This work reviews developments in surrogate modeling within the biopharmaceutical context, including Gaussian processes, neural networks, and physics-informed approaches. It also provides practical recommendations for identifying appropriate LF and HF data. Existing research has primarily focused on upstream processing and drug discovery, highlighting opportunities to extend these methods to other stages, like downstream processing. While Gaussian processes and neural networks remain the most frequently used models, emerging architectures such as Transformer and diffusion models present promising directions for future research.

Machine Learning-Assisted probabilistic fatigue evaluation of Rib-to-Deck joints in orthotropic steel decks

Multi-fidelity surrogate modeling, by integrating high-fidelity (HF) and low-fidelity (LF) data, provides an efficient approximation framework for complex fluid dynamics problems, particularly those driven by numerical resolution discrepancies. However, under limited computational budgets, efficiently selecting HF samples remains a critical challenge. This study proposes a Nonlinear-Guided Active Learning Multi-Fidelity Surrogate (NG-AL-MFS) modeling method. The proposed framework employs a recursive Co-Kriging model combined with an adaptive sampling strategy to achieve hierarchical data fusion. In this approach, the adaptive sampling strategy introduces a local nonlinearity indicator derived from LF responses to identify complex regions and integrates a distance-based exploration criterion to balance exploitation and exploration during potential HF sampling. Throughout the process, NG-AL-MFS quantitatively extracts local nonlinear features from LF responses to iteratively and adaptively update HF samples, thereby constructing a multi-fidelity surrogate that achieves a superior trade-off between modeling accuracy and computational cost. The proposed framework is validated using three benchmark functions and two fluid engineering applications. Results demonstrate that NG-AL-MFS outperforms existing approaches in terms of prediction accuracy, convergence efficiency, and statistical robustness, offering notable advantages in multivariate nonlinear scenarios. Overall, NG-AL-MFS provides a data-efficient and reliable solution for budget-constrained surrogate modeling in fluid dynamics and other computation-intensive domains.

Abstarct Engineering design problems typically require optimizing a quality measure by finding the right combination of controllable input parameters. In Additive Manufacturing (AM), the output characteristics of the process can often be non-stationary functions of the process parameters. Bayesian Optimization (BO) is a methodology to optimize such “black-box” functions, i.e., the input–output relationship is unknown and expensive to compute. Optimization tasks involving “black-box” functions widely use BO with Gaussian Process (GP) regression surrogate model. Using GPs with standard kernels is insufficient for modeling non-stationary functions, while GPs with non-stationary kernels are typically over-parameterized. On the other hand, a Deep Gaussian Process (DGP) can overcome GPs’ shortcomings by considering a composition of multiple GPs. Inference in a DGP is challenging due to its structure resulting in a non-Gaussian posterior, and using DGP as a surrogate model for BO is not straightforward. Stochastic Imputation (SI)-based inference is promising in speed and accuracy for BO. This work proposes a bootstrap aggregation-based procedure to effectively utilize the SI-based inference for BO with a DGP surrogate model. The proposed BO algorithm DGP-SI-BO is faster and empirically better than the state-of-the-art BO method in optimizing non-stationary functions. Several analytical test functions and a case study in metal AM simulation demonstrate the applicability of the proposed method.

Design approach for tilt propellers of UAM/eVTOLs for cruise and hover considering aerodynamic and aeroacoustic characteristics via a multi-fidelity model

Research on multi-fidelity aerodynamic optimization methods

A stepwise optimization approach based on Gaussian process (GP) surrogate model is proposed to determine the process parameters and improve the quality control for injection molding. In order to improve the global performance in this optimization, an enhanced probability of improvement criterion is also introduced. Firstly, GP surrogate model is constructed with the initial samples which are obtained from an optimal design of experiment method. GP is capable of giving both a prediction and an estimate of the confidence for the prediction simultaneously. Secondly, an enhanced probability of improvement criterion is used to find the direction of adding training samples and optimize the surrogate model. Since the global optimal region of the model become accurate efficiently after steps of optimizing the surrogate model, the proposed enhanced probability of improvement criterion can switch more swiftly to global optima compared with other improvement criterion. Finally, an auto front grille molding process is taken as an example to illustrate the method. The results show that the proposed optimization method can effectively decrease the warpage of injection-molded parts.

Dent-gouges as a result of the mechanical damage have serious implications for the burst capacity of oil and gas pipelines. The burst capacity of pipelines containing dent-gouges is lower than that of the same plain dented pipelines without gouges and that of the same gouged pipelines without dents. The well-known burst capacity prediction model adopted by the European Pipeline Research Group, i.e. the EPRG model, results in predictions of the burst capacity with high variability. In this study, a machine learning tool is employed to improve the predictive accuracy of the EPRG model for pipelines containing dent-gouges. To this end, a relatively large number of full-scale burst tests of pipe specimens containing dent-gouges are collected from the literature. The Gaussian process regression (GPR) technique, which is a class of non-parametric Bayesian model widely used in the machine learning, is employed to improve the EPRG model based on the collected full-scale burst test data. The full-scale burst tests are used to evaluate the hyper-parameters involved in the GPR analysis and validate the predictive accuracy of the improved EPRG model after the application of GPR. To facilitate the practical application of the improved EPRG model, a computer program with a graphic user interface (GUI) is further developed to compute the burst capacity of pipelines containing dent-gouges by inputting key parameters such as the pipe geometry and material properties as well as sizes of the dent and gouge through a GUI. This research will improve the fitness-for-service assessment of pipelines containing dent-gouges.

There are many parameters, which affect the dynamic response of structures, such as the type of structure, type of foundation, soil characteristics, etc. In this paper, soil-structure interaction in the analysis and design of reinforced concrete framed buildings has been investigated using SAP2000 vs16. Base shear, bending moment, deflection and natural period of multi-storey buildings with raft foundations, having different height to width ratios, resting on different types of soils, and subjected to a seismic ground motion were studied using Gaussian process regression (GPR) and least-square support vector machine (LSSVM). The ground motions were selected from the data of the Bhuj earthquake of 2001. A comparison of the performance of the GPR model and LSSVM model has been done based on Probability Distribution Function (PDF) and Cumulative Distribution Function (CDF).The results showed that the developed bending moment model was found more efficient compared to base shear and deflection models.