Model Rake: A Defense Against Stealing Attacks in Split Learning

Summary In the field of subsurface flow simulation, deep-learning-based surrogate modeling is shown as a promising approach to significantly reduce the computational cost associated with full-physics reservoir simulations. However, the successful construction of highly accurate deep-learning-based models often requires a large number of training simulations, which can be very time-consuming to generate by itself for large-scale systems. The incorporation of physics constraints during the training process was shown to be an effective approach to reduce the training cost and to improve model accuracy. However, it remains challenging to reimplement the physics constraints correctly for modeling training when it comes to reservoir models with complex physics or gridding methods. In this study, we propose a physics-constrained surrogate (PCS) model training approach for production optimization that integrates multifidelity training data, together with a new way of implementing the physics constraints that leverages an existing mature reservoir simulator. The proposed approach starts with model pretraining on a relatively large number of low-cost, low-fidelity training data, followed by model fine-tuning with a smaller number of high-fidelity training samples and the inclusion of physics constraints. The physics constraints are implemented by adding the residuals of discretized governing equations into the loss function. Training with multifidelity data and incorporating physics constraints allows for reduced reliance on high-fidelity data while enhancing physical consistency. Systematic comparison studies were performed for both 2D and 3D cases, and it was shown that the proposed approach can reduce the computational costs associated with training data generation by about 80%, while achieving a similar level of prediction accuracy of the surrogate model. In addition, under a small number of training simulations (i.e., 50 equivalent high-fidelity runs), our proposed PCS model with multifidelity data can reduce the prediction error by 90%, in comparison with the model trained with only high-fidelity data. Finally, the trained surrogate model was applied to a well-control optimization problem. In comparison with the use of full-order simulations, the total computational time can be reduced by 97.7%.

Seepage criteria based optimal design of water retaining structures with reliability quantification utilizing surrogate model linked simulation-optimization approach

In subsurface flow simulation, data-driven deep learning surrogate models have emerged as a promising alternative to traditional simulation methods. However, a major challenge is the large amount of high-fidelity simulation data required for training, and purely data-driven flow surrogate models often lack theoretical support, limiting their applicability to engineering problems. In this study, we propose an efficient surrogate model construction method that integrates physical constraints and multi-fidelity training data. The process begins with pre-training a deep learning model using a large amount of low-cost, low-fidelity data to effectively initialize network parameters. The pre-trained model is then fine-tuned with a smaller amount of high-fidelity data. Throughout the training, physical losses constrain the model to adhere to physical principles. For the case considered here, this approach reduces the need for high-fidelity data by approximately 83% compared to traditional training methods without compromising predictive accuracy. Additionally, incorporating physical loss during model training e ffectively enhances the surrogate model's predictive accuracy, with physical loss reduced by nearly 70% compared to purely data-driven training. The method employs the Jacobian matrix to aid in model training, significantly reducing the computational cost of physical losses. Overall, this surrogate model construction method addresses the challenges of obtaining training data and the lack of physical support in surrogate models, providing reliable support for complex reservoir development decisions.

Presented on Tuesday 21 May: Session 6 We introduce an innovative inversion approach for deducing subsurface fractures through observations of ground surface tilt. We have constructed, evaluated, and applied a surrogate forward model, crafted using conditional Generative Adversarial Networks (cGAN), to forecast the tilts (displacement gradients) at the ground surface caused by subsurface fractures under pressure. Our findings indicate that this surrogate forward model accurately estimates the tilt vector at the surface resulting from the specified pressurised fracture. Even in complex scenarios involving multiple fractures at various depths, the model, which was initially trained on scenarios with single fractures at a fixed depth, performed well. Subsequently, we employed a Bayesian inversion algorithm to derive the optimised solution (the pressurised fracture) for a given set of surface tilt data, leveraging the surrogate forward model. The outcomes demonstrate that the inversion process with the surrogate model is both effective and significantly faster compared to the traditional finite element model that generated the training data. To access the Oral Presentation click the link on the right. To read the full paper click here

Computational Fluid Dynamics (CFD) is regarded as an important tool for analyzing the flow field, thermal environment, and air quality around the built environment. However, for built environment applications, the high computational cost of CFD hinders large-scale scenario simulation and efficient design optimization. In the field of built environment research, surrogate modeling has become a key technology to connect the needs of high-fidelity CFD simulation and rapid prediction, whereas the low-dimensional nature of traditional surrogate models is unable to match the physical complexity and prediction needs of built flow fields. Therefore, combining machine learning (ML) with CFD to predict flow fields in built environments offers a promising way to increase simulation speed while maintaining reasonable accuracy. This review briefly reviews traditional surrogate models and focuses on ML-based surrogate models, especially the specific application of neural network architectures in rapidly predicting flow fields in the built environment. The review indicates that ML accelerates the three core aspects of CFD, namely mesh preprocessing, numerical solving, and post-processing visualization, in order to achieve efficient coupled CFD simulation. Although ML surrogate models still face challenges such as data availability, multi-physics field coupling, and generalization capability, the emergence of physical information-driven data enhancement techniques effectively alleviates the above problems. Meanwhile, the integration of traditional methods with ML can further enhance the comprehensive performance of surrogate models. Notably, the online ministry of trained ML models using transfer learning strategies deserves further research. These advances will provide an important basis for advancing efficient and accurate operational solutions in sustainable building design and operation.

Transport of contaminant species undergoing chemical reactions in groundwater aquifers is a complex physical and biochemical process. Simulating this transport process involves solving complex nonlinear equations and requires huge computational time for a given aquifer study area. Development of optimal remediation strategies in aquifers may require repeated solution of such complex numerical simulation models. To overcome this computational limitation and improve the computational feasibility of large number of repeated simulations, Genetic Programming based trained surrogate models are developed to approximately simulate such complex transport processes. Transport process of acid mine drainage, a hazardous pollutant is first simulated using a numerical simulated model: HYDROGEOCHEM 5.0 for a study area resembling a mine site. Simulation model solution results for an illustrative contaminated aquifer site is then approximated by training and testing a Genetic Programming (GP) based surrogate model. To decrease the total number of GP formulations, the coordinates of observation locations are implemented as input data in the surrogate models. Comparison of the surrogate models and numerical simulation results show that the surrogate models can provide acceptable approximations of this complex transport process in contaminated groundwater aquifers.

The complex and iterative nature of plasma scenario optimization in fusion devices necessitates the use of reduced models in early stages of the design process to filter through a large parameter space in an efficient manner. Ray-tracing codes, such as TORAY, offer considerable advantages in run time for electron cyclotron (EC) heating and current drive (H/CD) cases over full-wave codes while maintaining a high degree of fidelity. We deploy the Fusion Reactor Design and Assessment (FREDA)–TokDesigner workflow to enable training of a surrogate model for EC H/CD radial profiles based on the TORAY ray-tracing code, coupled to the Integrated Plasma Simulator (IPS)–FASTRAN framework. The surrogate model is trained to predict key H/CD profile characteristics for EC cases based on a subset of plasma and EC launcher parameters for a Compact Advanced Tokamak (CAT) design point. The CAT was selected as a baseline to assess the performance of the surrogate model trained in the fusion pilot plant regime. The surrogate model is able to accomplish this an order of magnitude faster than TORAY coupled to IPS-FASTRAN while still maintaining a high level of accuracy. The surrogate model demonstrates invertibility, being able to solve the inverse problem to generate an accurate parameter space for a set of desired H/CD profile characteristics.

Two-dimensional design and analysis issues on the meridional surface, which is important in the preliminary design procedure of compressors, are highly dependent on the accuracy of empirical models, such as the prediction of total pressure loss model and turning flow angle. Most of the widely used models are derived or improved from experimental data of some specific cascades with low-loading and low-speed airfoil types. These models may work for most conventional compressors but are incapable of accurately estimating the performance for some specific cases like transonic compressors. The errors made by these models may mislead the final design results. Therefore, surrogate models are developed in this work to reduce the errors and replace the conventional empirical models in the through-flow calculation procedure. A group of experimental data considering a two-stage transonic compressor is used to generate the airfoils database for training the surrogate models. Sensitivity analysis is applied to select the most influential features. Two supervised learning approaches including support vector regression (SVR) and Gaussian process regression (GPR) are used to train the models with a Bayesian optimization algorithm to obtain the optimal hyper parameters. The trained models are integrated into the through-flow code based on streamline curvature method (SLC) to predict the overall performance and internal flow field of the transonic compressor on five rotational speed lines for validation. The predictions are compared with the experimental data and the results of conventional empirical models. The comparison shows that SVR and GPR respectively reduce the predicted error of empirical models by 62.2% and 55.2% for the total pressure ratio and 48.4% and 50.1% for adiabatic efficiency on average. This suggests that the surrogate models constitute an alternative way to predict the performance of airfoils in through-flow calculation where empirical models are inefficient.

Surrogate models have been used as a substitute for traditional engineering simulation to get target simulation results faster. These surrogate models usually need sufficient data to fulfill their optimum performance. However, the acquisition of high quality datasets for cases like welding residual stress is both difficult and costly. This work developed a workflow that can rapidly develop machine learning based surrogate models to predict post-weld residual stresses using finite element (FE) simulation data. The workflow comprises five steps: (i) baseline FE model creation, (ii) FE model processing (iii) scripting (iv) data generation and (v) surrogate model development. The workflow presented in the paper automates all the steps involved in generating, calculating, and extracting results from FE simulations, with the aim of reducing the time and cost required for data acquisition. In addition, to minimize the possibility of FE analyses not completing successfully during the automated data generation process and to ensure that the training data can be obtained without any manual intervention, the FE model was simplified without significant loss of accuracy. A single weld bead-on-plate structure, taken from the NeT project Task Group 1, was used as a case study to validate the developed workflow. The overall aim of this work was to validate the feasibility of developing a surrogate model for specific welding conditions with no experimental dataset and with the aid of simulations, and to investigate the predictive performance of an Artificial Neural Network based surrogate model. Four welding parameters, arc advance speed, net energy input rate, arc travel length and position, were considered as the input for the surrogate model to predict the longitudinal stress along the line running across the sample thickness in the mid-length of the specimen. The relative root mean square error of the predictive surrogate model for the test dataset was 0.0024. Residual stress prediction based on the NeT-TG1 benchmark welding case met the R6 validation criteria and the developed model generates predictions that are in line with other methods used by contributors to the NeT-TG1 round robin.

Surrogate modeling of phase equilibrium calculations using adaptive sampling

In axial compressor blade design optimization research, the surrogate model serves as a critical tool for performance prediction, but its construction often incurs high computational costs. To reduce surrogate model construction costs while ensuring prediction accuracy, this study proposes a primary-auxiliary composite neural network surrogate model based on flow field extraction. During model construction, extensive internal flow field data are extracted and employed to support model training, which enables efficient global optimization. To validate the proposed surrogate model, this study selects the Darmstadt transonic compressor (DTC) as the test case, sets the rotor blade sweep as the optimization variable, and targets increases in both the compressor stable operating range and the efficiency at the peak-efficiency point as the optimization objectives. The study employed the primary-auxiliary composite surrogate model with the non-dominated sorting genetic algorithm II (NSGA-II) to explore the Pareto frontier and execute aerodynamic performance optimization design. Results demonstrate that, under few-shot conditions, the prediction accuracy achieved using the primary-auxiliary composite surrogate model is 61.2% higher compared to directly constructed surrogate models. Optimization of the rotor blade sweep led to a reduction in tip leakage flow intensity, a downstream shift, and weakening of boundary layer separation at the lower blade span. While the design point efficiency experienced a minor increase, the stable operating range of the compressor expanded by 17.7%. This study verifies the applicability of the primary-auxiliary composite surrogate model to compressor blade profile optimization and provides a novel means to boost prediction accuracy under low-cost surrogate model constraints.

ABSTRACTLinked simulation-optimisation (S–O) models need to simulate the physical processes either by using a rigorous numerical model, or a trained surrogate model approximating the physical processes. A methodology is proposed to evolve Pareto optimal management strategies for a multi-layered coastal aquifer system using a trained and tested Multivariate Adaptive Regression Spline (MARS) surrogate model linked to a multi-objective saltwater intrusion management model. Performance of the developed methodology is evaluated using an illustrative multi-layered coastal aquifer system. Solution results indicate that MARS is capable of approximately replacing the more rigorous numerical simulation model within the linked S–O model to ensure computational efficiency and feasibility in applying such linked S–O models for coastal aquifer management problems. Furthermore, the ability of MARS to recognise the most relevant input variables in predicting the responses as outputs enables the construction of an efficient and robust surrogate model. Integration of parallel processing capabilities within the optimisation model is shown to improve computational efficiency and feasibly of solving such large scale multi-objective problems. Therefore, the developed methodology utilising the MARS based surrogate model is potentially applicable for developing optimal groundwater extraction strategies for sustainable regional scale management of coastal aquifers.

Recently, the wafers used in solar cells have been increasing in size, leading to larger module sizes and weights. The increased weight can cause deflection of photovoltaic (PV) module, which may lead to decreased cell efficiency. In this study, we developed a deep neural network (DNN)-based finite element (FE) surrogate model to obtain the optimal frame design factors that can improve deflection in large-scale bifacial PV module. Initially, an FE model was constructed for large-scale bifacial PV module. Based on this, the FE surrogate model was trained using 243 FEA datasets generated within the proposed range of factors. Furthermore, it was improved through Bayesian optimization and k-fold validation. As a result, the final loss value was 3.743×10-4\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.743 \ imes 10^{-4}$$\\end{document}, and the average mean absolute percentage error (MAPE) and coefficient of determination (R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^2$$\\end{document}) values for deflection and weight were 0.0017, 0.9972 for the training set, and 0.0020, 0.9962 for the test set, respectively. This indicates that the trained FE surrogate model possesses significant accuracy. After generating 1 million datasets within the range of frame design factors, the trained model was used to obtain predictions. Based on this data, the frame design factors that minimize both deflection and weight were identified as about a = 1.5, b = 13.7, c = 1.5, d = 3.0, e = 4.3. At this point, the deflection was 11.1 mm, and the weight was 3.6 kg. After altering the frame shape with the derived factors, FEA was conducted. The results matched for both deflection and weight, with almost no error. At this point, the weight increased by approximately 12.8% compared to the existing, while the deflection decreased by about 9.6%. Additionally, we analyzed the relationship between deflection and weight for each factor and secured the basis for the derived results. Consequently, our FE surrogate model accurately predicted the FEA results and quickly identified the optimal factors that minimize deflection and weight.

Nowadays, Computational Fluid Dynamics (CFD) simulations play an increasingly important role for turbine airfoil design. This high-fidelity approach is capable to provide accurate information of flow fields. Meanwhile, the calculation accuracy is always gained at the expense of numerical cost. This gap limits opportunities for design space exploration. To address this problem, surrogate models (also known as metamodels) are introduced to approximate high-fidelity CFD models. However, traditional surrogate models, such as Kriging or Radial Basis Function, construct response surface on a design space with limited dimensions. This prevents users from predicting the flow fields directly from the geometry and performing interactive design of airfoil. In the present work, we propose a Convolutional Neural Network (CNN) based surrogate model to predict flow properties on turbine vane airfoil surface from 3D airfoil profile defined by point cloud. The proposed CNN architecture adopts a symmetric expanding path that is similar to the so-called U-Net. The geometries in the training and testing dataset are generated via varying the parameters defined by the Free-Form Deformation approach. The corresponding flow fields are obtained through high-fidelity CFD simulations performed in a finite volume context. Furthermore, a gaussian process based Bayesian optimization technique is utilized to tune automatically the hyperparameters of the network. In this work, we trained the CNN based surrogate model with static pressure and temperature on the mean section of turbine vane airfoil surface. The trained model is able to predict in a reliable and efficient way the corresponding property directly from the 3D geometry, which allows engineers to agilely adjust their airfoil design.

Data-driven Artificial Intelligence (AI)/Machine Learning (ML)-based models of CPS (also called Digital Twins) are becoming important in the design and control of modern CPS. CPS are a unique class of intelligent system, where the governing process models for the system (or its parts) are available to inform decision making. The governing physics or chemistry models provide an additional source of information, potentially improving the accuracy of CPS modeling, but also increasing the complexity of the information fusion process. The resource constraints of CPS, however, makes it hard to perform the information fusion within the CPS itself. Moreover, building a large, monolithic model of the complete CPS is also infeasible due to model size and training complexity. The hierarchical and compositional structure of CPS calls for distributed model training and composition. Resource constraints and real-time control needs call for rapid and low-cost training of models, which makes surrogate modeling a promising approach. However, when surrogate models are composed, their approximate nature may lead to larger errors. Further, data-driven models are prone to adversarial attacks, whose impacts may be even more pronounced in a federated model. This position paper highlights the key research challenges in this realm and need for new research.