Efficient Training of Deep-Learning-Based Surrogate Models for Subsurface Flow Simulation Using Multifidelity Data and Physics Constraints

In subsurface flow simulation, data-driven deep learning surrogate models have emerged as a promising alternative to traditional simulation methods. However, a major challenge is the large amount of high-fidelity simulation data required for training, and purely data-driven flow surrogate models often lack theoretical support, limiting their applicability to engineering problems. In this study, we propose an efficient surrogate model construction method that integrates physical constraints and multi-fidelity training data. The process begins with pre-training a deep learning model using a large amount of low-cost, low-fidelity data to effectively initialize network parameters. The pre-trained model is then fine-tuned with a smaller amount of high-fidelity data. Throughout the training, physical losses constrain the model to adhere to physical principles. For the case considered here, this approach reduces the need for high-fidelity data by approximately 83% compared to traditional training methods without compromising predictive accuracy. Additionally, incorporating physical loss during model training e ffectively enhances the surrogate model's predictive accuracy, with physical loss reduced by nearly 70% compared to purely data-driven training. The method employs the Jacobian matrix to aid in model training, significantly reducing the computational cost of physical losses. Overall, this surrogate model construction method addresses the challenges of obtaining training data and the lack of physical support in surrogate models, providing reliable support for complex reservoir development decisions.

Multi-fidelity surrogate (MFS) modeling technology, which efficiently constructs surrogate models using low-fidelity (LF) and high-fidelity (HF) data, has been studied to enhance the predictive capability of engineering performances. In addition, several neural network (NN) structures for MFS modeling have been introduced, benefiting from recent developments in deep learning research. However, existing multi-fidelity (MF) NNs have been developed assuming identical sets of input variables for LF and HF data, a condition that is often not met in practical engineering systems. Therefore, this study proposes a new structure of composite NN designed for MF data with different input variables. The proposed network structure includes an input mapping network that connects the LF and HF data's input variables. Even when the physical relationship between these variables is unknown, the input mapping network can be concurrently trained during the process of training the whole network model. Customized loss functions and activation variables are suggested in this study to facilitate forward and backward propagation for the proposed NN structures when training MF data with different inputs. The effectiveness of the proposed method, in terms of prediction accuracy, is demonstrated through mathematical examples and practical engineering problems related to tire performances. The results confirm that the proposed method offers better accuracy than existing surrogate models in most problems. Moreover, the proposed method proves advantageous for surrogate modeling of nonlinear or discrete functions, a characteristic feature of NN-based methods.

The next generation low-cost modular unmanned combat aerial vehicles (UCAVs) provide the opportunity to implement innovative solutions to complex tasks, while also bringing new challenges in design, production, and certification subjects. Solving these problems with tools that provide fast modeling in line with the digital twin concept is possible. In this work, we develop an artificial intelligence (AI) based multifidelity surrogate model to determine performance parameters of innovative modular UCAVs. First, we develop a data generation algorithm that includes a high-fidelity model based on computational fluid dynamics methods and a low-fidelity model based on computational aerodynamic approaches. In the next step, a new transfer learning-based surrogate model is generated using multifidelity data. Thanks to this approach, the developed AI model more accurately predicted the flow conditions that were missing in the high-fidelity data with the data obtained from the low-order model. The performance of the proposed AI-based surrogate model is to be investigated in terms of accuracy, robustness, and computational cost using a generic modular UCAV configuration.

Design approach for tilt propellers of UAM/eVTOLs for cruise and hover considering aerodynamic and aeroacoustic characteristics via a multi-fidelity model

The growing demand for biopharmaceutical products reflects their effectiveness in medical treatments. However, developing new biopharmaceuticals remains a major bottleneck, often taking up to a decade before market approval. Machine learning (ML) models have the potential to accelerate this process, but their success depends on access to large and diverse data sets for training. Multi-fidelity ML techniques offer a promising solution by integrating abundant, low-cost, and less accurate low-fidelity (LF) data with limited, expensive, and more accurate high-fidelity (HF) data. In this framework, LF data capture global system trends, while HF data refine and align model predictions with the available ground truth. Such integration can substantially reduce development costs and timelines by minimizing the need to acquire HF data, for example, through extensive experimental campaigns. This work reviews developments in surrogate modeling within the biopharmaceutical context, including Gaussian processes, neural networks, and physics-informed approaches. It also provides practical recommendations for identifying appropriate LF and HF data. Existing research has primarily focused on upstream processing and drug discovery, highlighting opportunities to extend these methods to other stages, like downstream processing. While Gaussian processes and neural networks remain the most frequently used models, emerging architectures such as Transformer and diffusion models present promising directions for future research.

This study presents a method that combines two deep learning models—the surrogate model and the bandgap optimization model—to design 1-D Phononic Crystals. The surrogate model is trained to predict the bandgaps of Phononic Crystals, while the bandgap optimization model is used to determine their geometric parameters. The goal is to leverage deep learning’s ability to capture complex input-output relationships and exploit the inherent optimization capabilities of these models. To improve design accuracy and account for manufacturing constraints, a user-defined activation function is incorporated into the optimization model. Various loss functions are introduced to address different design objectives, such as maximizing the bandgap at a lower frequency or maximizing while aligning the center frequency of the bandgap with a specified value. The 1-D Phononic Crystal designed for bandgap maximization at lower frequencies using this method outperformed results predicted by a Genetic Algorithm. Unlike previous studies that often rely on deep learning only as a predictive tool or require external optimizers, our method uses a deep learning model directly as an optimization engine. The integration of customizable loss functions, user-defined physical constraints, and an unsupervised training approach enables versatile and efficient design without the need for labeled optimization datasets. The approach is readily extensible to 2-D and 3-D configurations, offering a versatile solution for advanced Phononic Crystal design.

Multi-fidelity graph neural networks for efficient power flow analysis under high-dimensional demand and renewable generation uncertainty

Surrogate models have been used as a substitute for traditional engineering simulation to get target simulation results faster. These surrogate models usually need sufficient data to fulfill their optimum performance. However, the acquisition of high quality datasets for cases like welding residual stress is both difficult and costly. This work developed a workflow that can rapidly develop machine learning based surrogate models to predict post-weld residual stresses using finite element (FE) simulation data. The workflow comprises five steps: (i) baseline FE model creation, (ii) FE model processing (iii) scripting (iv) data generation and (v) surrogate model development. The workflow presented in the paper automates all the steps involved in generating, calculating, and extracting results from FE simulations, with the aim of reducing the time and cost required for data acquisition. In addition, to minimize the possibility of FE analyses not completing successfully during the automated data generation process and to ensure that the training data can be obtained without any manual intervention, the FE model was simplified without significant loss of accuracy. A single weld bead-on-plate structure, taken from the NeT project Task Group 1, was used as a case study to validate the developed workflow. The overall aim of this work was to validate the feasibility of developing a surrogate model for specific welding conditions with no experimental dataset and with the aid of simulations, and to investigate the predictive performance of an Artificial Neural Network based surrogate model. Four welding parameters, arc advance speed, net energy input rate, arc travel length and position, were considered as the input for the surrogate model to predict the longitudinal stress along the line running across the sample thickness in the mid-length of the specimen. The relative root mean square error of the predictive surrogate model for the test dataset was 0.0024. Residual stress prediction based on the NeT-TG1 benchmark welding case met the R6 validation criteria and the developed model generates predictions that are in line with other methods used by contributors to the NeT-TG1 round robin.

Presented on Tuesday 21 May: Session 6 We introduce an innovative inversion approach for deducing subsurface fractures through observations of ground surface tilt. We have constructed, evaluated, and applied a surrogate forward model, crafted using conditional Generative Adversarial Networks (cGAN), to forecast the tilts (displacement gradients) at the ground surface caused by subsurface fractures under pressure. Our findings indicate that this surrogate forward model accurately estimates the tilt vector at the surface resulting from the specified pressurised fracture. Even in complex scenarios involving multiple fractures at various depths, the model, which was initially trained on scenarios with single fractures at a fixed depth, performed well. Subsequently, we employed a Bayesian inversion algorithm to derive the optimised solution (the pressurised fracture) for a given set of surface tilt data, leveraging the surrogate forward model. The outcomes demonstrate that the inversion process with the surrogate model is both effective and significantly faster compared to the traditional finite element model that generated the training data. To access the Oral Presentation click the link on the right. To read the full paper click here

Multi-fidelity reinforcement learning with control variates

Seepage criteria based optimal design of water retaining structures with reliability quantification utilizing surrogate model linked simulation-optimization approach

Multi-fidelity surrogate modeling, by integrating high-fidelity (HF) and low-fidelity (LF) data, provides an efficient approximation framework for complex fluid dynamics problems, particularly those driven by numerical resolution discrepancies. However, under limited computational budgets, efficiently selecting HF samples remains a critical challenge. This study proposes a Nonlinear-Guided Active Learning Multi-Fidelity Surrogate (NG-AL-MFS) modeling method. The proposed framework employs a recursive Co-Kriging model combined with an adaptive sampling strategy to achieve hierarchical data fusion. In this approach, the adaptive sampling strategy introduces a local nonlinearity indicator derived from LF responses to identify complex regions and integrates a distance-based exploration criterion to balance exploitation and exploration during potential HF sampling. Throughout the process, NG-AL-MFS quantitatively extracts local nonlinear features from LF responses to iteratively and adaptively update HF samples, thereby constructing a multi-fidelity surrogate that achieves a superior trade-off between modeling accuracy and computational cost. The proposed framework is validated using three benchmark functions and two fluid engineering applications. Results demonstrate that NG-AL-MFS outperforms existing approaches in terms of prediction accuracy, convergence efficiency, and statistical robustness, offering notable advantages in multivariate nonlinear scenarios. Overall, NG-AL-MFS provides a data-efficient and reliable solution for budget-constrained surrogate modeling in fluid dynamics and other computation-intensive domains.

With ever-increasing complexity in Upstream project planning, to ensure decision quality, the dynamics of subsurface resources need to be embedded into concept screening to maintain consistency between the production forecast and development plan. We developed a decision-driven subsurface surrogate model that encapsulates key reservoir dynamics into the machine augmented mathematical technologies for holistic decision recommendation in concept selection and development planning under uncertainties. The surrogate model replicates the essential subsurface dynamics by using a hybrid-approach that takes into accounts both reservoir simulation data and physical first principle. In addition to standalone usage on production forecast for rapid profile screening under resource uncertainties, the subsurface surrogate model is incorporated into mathematical optimization models that simultaneously consider surface network, commercial obligation and project economics etc. to provide alternative concepts under various uncertainties. Our subsurface surrogate model has been applied for decision making on gas gathering system design, field development optimization, field-management timing and sequencing, and field tie-back study etc. Results not only show the capability of surrogate models to enable large scale rapid decision screening, but also bear a close resemblance between the predicted production profiles and the reservoir simulation results when fed with the field operating strategy recommended by our decision models with surrogate dynamics. The study demonstrates the reliability of our surrogate modeling technology on ensuring decision quality and helps build business’ confidence on technology adoption. By further incorporating subsurface uncertainties into surrogate models, the decision makers are provided with: probabilistic analysis of the outcomes, value of information analysis, cost of optionality and flexibility, and holistic project outlook etc. Our decision-driven surrogate modeling technology incorporated in mathematical decision models is the first of its kind for holistic decision support in concept selection and development planning in oil and gas industry, and has full potentials in a variety of asset lines with reliable subsurface performance prediction under uncertainties.

In this study, high-Fidelity Large eddy simulation (LES) data was used to create a surrogate turbulence model to enhance the estimation of turbulent heat flux in a Reynolds averaged Navier Stokes (RANS) solver for film cooling applications. The LES simulation (A flat plate film cooling application with a Reynolds number of 17,382, a blowing ratio of 1 and a density ratio of 2) was validated by comparison with experimental data. A correlation analysis was performed to identify the most influential parameters, and temperature gradient, velocity gradient, turbulent dissipation rate, shear strain rate, and turbulent viscosity ratio were selected as the most effective parameters. A second model was developed to create a Galilean-invariant model that remains unaffected by changes in the coordinate system. Neural network was applied to the LES training data to propose a modified surrogate dynamic turbulent Prandtl number to be applied to the k − ε realizable RANS simulation. The first surrogate model was implemented to two different geometries, one geometry was a flat plate film cooling similar to the training data but with different flow conditions, and another one was a more complex geometry with pressure gradient. The second model was only implemented on the second geometry and produced similar results. The surrogate models predicted a more accurate thermal field near the cooling wall (up-to 58 % error reduction) with a computational time complexity similar to the base RANS solver.

Computational Fluid Dynamics (CFD) is regarded as an important tool for analyzing the flow field, thermal environment, and air quality around the built environment. However, for built environment applications, the high computational cost of CFD hinders large-scale scenario simulation and efficient design optimization. In the field of built environment research, surrogate modeling has become a key technology to connect the needs of high-fidelity CFD simulation and rapid prediction, whereas the low-dimensional nature of traditional surrogate models is unable to match the physical complexity and prediction needs of built flow fields. Therefore, combining machine learning (ML) with CFD to predict flow fields in built environments offers a promising way to increase simulation speed while maintaining reasonable accuracy. This review briefly reviews traditional surrogate models and focuses on ML-based surrogate models, especially the specific application of neural network architectures in rapidly predicting flow fields in the built environment. The review indicates that ML accelerates the three core aspects of CFD, namely mesh preprocessing, numerical solving, and post-processing visualization, in order to achieve efficient coupled CFD simulation. Although ML surrogate models still face challenges such as data availability, multi-physics field coupling, and generalization capability, the emergence of physical information-driven data enhancement techniques effectively alleviates the above problems. Meanwhile, the integration of traditional methods with ML can further enhance the comprehensive performance of surrogate models. Notably, the online ministry of trained ML models using transfer learning strategies deserves further research. These advances will provide an important basis for advancing efficient and accurate operational solutions in sustainable building design and operation.