High-Performance Kriging Surrogate Models Using PyTorch: Accelerating Engineering Analysis and Optimization

Study of a hull form optimization system based on a Gaussian process regression algorithm and an adaptive sampling strategy, Part I: Single-objective optimization

Wave episode based Gaussian process regression for extreme event statistics in ship dynamics: Between the Scylla of Karhunen–Loève convergence and the Charybdis of transient features

In this work, we present the development of robust surrogate models for Ion Cyclotron Range of Frequencies (ICRF) and High-Harmonic Fast Wave (HHFW) heating predictions in fusion plasmas. Building upon our previous efforts to achieve real-time capable models, we identify the cause of the outliers found using TORIC in certain HHFW heating scenarios. The outliers are observed to be spurious ion Bernstein wave (IBW)-like modes caused by a wavelength control algorithm designed to address challenging scenarios with high perpendicular wavenumbers. The effect arises from the modulation in the perpendicular susceptibility, which can induce sign reversal and IBW-like propagation for scenarios featuring normalized ion Larmor radius λi≫1. We use TORIC with this algorithm disabled to generate a novel HHFW-NSTX database that is free of outliers. Surrogate models trained on this database, including Random Forest Regressor (RFR), Multi-Layer Perceptrons, and Gaussian Process Regressors (GPR), demonstrate the ability to accurately predict HHFW heating profiles, with regression scores of R2∈[0.93−0.99]. Additionally we demonstrate that it is possible to generalize predictions beyond training data by the use of both RFR and GPR models, enabling the prediction of scenarios previously limited to the original model. GPR models also provide uncertainty quantification, offering insights into model confidence. This work introduces a comprehensive Verification, Validation, and Uncertainty Quantification methodology for surrogate modeling, applicable not only to ICRF heating but also to other RF heating challenges and fusion physics problems. Beyond accelerated inference, these models show effective extrapolation capabilities, providing an alternative for addressing numerical challenges.

Pipeline corrosion poses significant challenges and risks to the energy industry and its mitigation requires extensive and reliable predictive modeling. Corrosion models based on computational fluid dynamics (CFD) stands as a desirable candidate for its detailed physical characterization and modeling flexibility, but its applications in practical industrial settings is limited by the high computational cost and laborious manual operation in the modeling and sampling process. To address these challenges, we propose a Bayesian active learning method. The method consists of a surrogate model formulated using Gaussian process regression (GPR) to provide rapid model prediction as well as uncertainty quantification, and an adaptive sampling scheme to automate and accelerate the data collection process. Careful dimension reduction guided by both physics and data is also carried out to significantly simplify the sampling space. The capability of the overall method for efficient and automated sampling and surrogate modeling is demonstrated on an example case of corrosion predictive modeling and can be leveraged in industrial applications at a much larger scale.

This paper elaborates on the large number of Uncertainty Quantification (UQ) techniques that have been proposed to enhance the reliability and the fidelity of Digital Twins that are increasingly finding applications in domains like Robotics and Smart Manufacturing. Digital twins are virtual duplicates or virtual models of a physical asset; they use advanced techniques such as data analytics and simulation-driven methods. However, the development and use of these advanced systems are plagued by a host of uncertainties, which are mainly introduced from sensor noise, intermittent connectivity, biases from data processing, and model abstractions and simulation stochasticity. Such uncertainties can be quantified by methods such as frequentist statistics, interval analysis, Bayesian inference, and random sampling. The mapping is important in gaining insights into these UQ methods and their associated advantages and limitations and the mitigation guidelines are to be used throughout the Digital Twin pipeline. UQ at its core involves real-time adaptive control in dynamically changing environments that leverage state awareness towards responsive action within predictive control models and feedback systems. In addition, machine learning algorithms support the ability to make better decisions from the identification of patterns in historical data to make plans for responsive trajectories of robots. UQ further allows the collaboration of human and machine, giving early warnings on anomalies and risks that enhance visibility which further fosters coordination and communication during disruptive situations. Robust development of digital twins for robotics and manufacturing relies on integrated UQ practices. The current review provides best practices, insights, and guidelines on the application of UQ across modeling, control strategies, and collaborative workflows aimed at delivering actionable and reliable insights from digital twin simulations, analytics, and decision support.

A stock price index measures the change in several share prices, which can describe the market and assist investors in deciding on a specific investment. Thus, foreseeing the stock price index benefits investors in creating a better investment strategy. However, forecasting the stock price index can be challenging due to its non-linearity, non-stationary and high uncertainty. Gaussian process regression (GPR) is an attractive and powerful approach for prediction, especially when the data fluctuates over time with fewer restrictions. Besides, the GPR gains advantages over other forecasting techniques as it can offer predictions with uncertainty to provide margin errors. In this study, we evaluate the use of GPR to predict the stock price of Thailand (SET). The SET data are divided into 2 datasets; the data in the year 2015 - 2020 and the data in the year 2020 due to the massive change during the COVID-19 pandemic. The prediction results from the GPR are then compared to the machine learning approaches, artificial neural network (ANN) and recurrent neural network (RNN) using evaluation scores; the root mean square error (RMSE), the mean absolute error (MAE), the mean absolute percentage error (MAPE) and the Nash-Sutcliffe efficiency (NSE). The results indicate that the GPR is superior to the ANN and RNN for both datasets as it provides a high prediction accuracy. Moreover, the results suggest that the GPR is less sensitive to the number of input lags in the model. Therefore, the GPR is more favorable for the prediction of SET than the ANN and RNN.
 HIGHLIGHTS
 
 The Gaussian process regression (GPR) was applied to predict the stock price index of Thailand (SET)
 The predictive performance of the GPR was compared to artificial neural networks (ANNs) and recurrent neural networks (RNNs)
 The results indicate that GPR outperformed the other methods as it provided a high prediction accuracy along with prediction intervals
 
 GRAPHICAL ABSTRACT

Uncertainty quantification of two-phase flow in porous media via the Coupled-TgNN surrogate model

Abstract. Synergies between neural networks and traditional surrogate modelling techniques have emerged as the forefront of data-driven engineering. Neural network-based surrogate models, trained on carefully selected experimental data or high-fidelity simulations, can predict behaviours of complex systems with remarkable speed and accuracy. This review examines the current state and recent developments in neural surrogate technologies, highlighting their expanding roles in engineering design optimisation and generation. It also covers various feature engineering methods for representing 3D geometries, the principles of neural surrogate modelling, and the potential of emerging AI-driven design tools. While feature engineering remains a challenge, especially in parameterising complex designs for machine learning, recent advancements in code/language-based representations offer promising solutions for digitalising various design scenarios. Moreover, the emergence of AI-driven design tools, including text-to-CAD models powered by large language models, enables engineers to rapidly generate and evaluate innovative design concepts. Neural surrogate modelling has the potential to transform engineering workflows. Continued research into geometric feature engineering, along with the integration of AI-driven design tools, will speed up the use of neural surrogate models in engineering designs.

Recently, machine learning (ML) techniques have gained widespread diffusion, since they have been successfully applied in several research fields. This paper investigates the effectiveness of advanced ML regressions in two EMC applications. Specifically, support vector machine, least-squares support vector machine and Gaussian process regressions are adopted to construct accurate and fast-to-evaluate surrogate models able to predict the output variable of interest as a function of the system parameters. The resulting surrogates, built from a limited set of training samples, can be suitably adopted for both uncertainty quantification and optimization purposes. The accuracy and the key features of each of the considered machine learning techniques are investigated by comparing their predictions with the ones provided by either circuital simulations or measurements.

A surrogate model assisted with a subdomain model for permanent-magnet machine

A data-driven optimization framework that integrates machine learning surrogate models, finite element analysis (FEA), and a multi-objective optimization algorithm is used in this study for developing thermoplastic elastomer (TPE) parts for aerospace applications. By using FEA simulations and experiments, a database of input design parameters (e.g., geometry and structural shape modifier) is generated. Afterwards, we train surrogate models (e.g., Gaussian Process Regression, neural networks) to approximate mappings from design space to performance space. Finally, we propose Pareto-optimal TPE designs using the surrogate embedded in a multi-objective optimization loop (such as NSGA-II or gradient-based methods). The novelty of this approach is demonstrated by employing highly simplified surrogate models, including an artificial neural network (ANN) with 10 hidden neurons trained on analytically generated synthetic data. The proposed methodology has been validated using an aerospace-related case study: a vibration-damping plate. Compared with the baseline configuration, Pareto-optimal designs identified by the proposed framework achieved a reduction in maximum deflection of 23%–28% and a reduction in von Mises stress of 18%–24%, depending on the selected trade-off solution, as the number of full FEA simulations required for optimization was reduced from 500 to 50. This framework enables faster design of TPE components for aerospace systems. Validation against high-fidelity ANSYS simulations showed a mean error of ~1.18% and a maximum deviation of ~2.6%.

Multi-fidelity neural network for uncertainty quantification of chemical reaction models

Segregation of granular materials is a critical challenge in many industries, often aimed at being controlled or minimised. The discrete element method (DEM) offers valuable insights into this phenomenon. However, calibrating DEM models is a crucial, albeit time-consuming, step. Recently, using machine learning (ML)-based surrogate models (SMs) in the calibration process has emerged as a promising solution. Nevertheless, developing such SMs is challenging due to the high number of DEM simulations required for training. Additionally, choosing a suitable ML model is not trivial. This study aims to develop SMs that effectively link particle-particle and particle-wall DEM interaction parameters to segregation of a multi-component mixture. We evaluate several ML models, ranging from artificial neural networks to ensemble learning, that are trained on a very cost-effective dataset, employing Bayesian optimisation with cross-validation to tune their hyperparameters. Next, we introduce a novel transfer learning (TL)-based approach that leverages knowledge from a few scenarios to handle new “unseen” ones. This method enables the construction of adaptive SMs for unseen scenarios, such as a new initial configuration (IC) of granular mixtures, without the need for a full-sized dataset. Our findings indicate that Gaussian process regression (GPR) efficiently builds accurate SMs on a very small dataset. We also demonstrate that only a few samples are required to build an accurate SM for the unseen IC, which significantly reduces the data preparation burden. By incorporating one and five samples from unseen scenarios to update the TL-GPR-based surrogate model, the SM’s performance (based on ) on unseen scenarios improves by 17 and 47%, respectively. The insights and methodology presented in this study will facilitate and accelerate the development of accurate SMs for DEM calibration, assisting in developing reliable DEM models in a shorter timeframe.

Investigating adaptive sampling strategies for optimal building energy performance using artificial neural networks and kriging surrogate models

Two-dimensional design and analysis issues on the meridional surface, which is important in the preliminary design procedure of compressors, are highly dependent on the accuracy of empirical models, such as the prediction of total pressure loss model and turning flow angle. Most of the widely used models are derived or improved from experimental data of some specific cascades with low-loading and low-speed airfoil types. These models may work for most conventional compressors but are incapable of accurately estimating the performance for some specific cases like transonic compressors. The errors made by these models may mislead the final design results. Therefore, surrogate models are developed in this work to reduce the errors and replace the conventional empirical models in the through-flow calculation procedure. A group of experimental data considering a two-stage transonic compressor is used to generate the airfoils database for training the surrogate models. Sensitivity analysis is applied to select the most influential features. Two supervised learning approaches including support vector regression (SVR) and Gaussian process regression (GPR) are used to train the models with a Bayesian optimization algorithm to obtain the optimal hyper parameters. The trained models are integrated into the through-flow code based on streamline curvature method (SLC) to predict the overall performance and internal flow field of the transonic compressor on five rotational speed lines for validation. The predictions are compared with the experimental data and the results of conventional empirical models. The comparison shows that SVR and GPR respectively reduce the predicted error of empirical models by 62.2% and 55.2% for the total pressure ratio and 48.4% and 50.1% for adiabatic efficiency on average. This suggests that the surrogate models constitute an alternative way to predict the performance of airfoils in through-flow calculation where empirical models are inefficient.