Comparative study of surrogate approaches while optimizing computationally expensive reaction networks

Implementation of online optimization and control of complex processes near impossible in given time frame owing to large computational time of the models. Fast and efficient surrogate models such as ANN offer a credible solution to this problem. Optimization of the complex model can proceed with a data based surrogate model, thereby making it much faster than the conventional run. However, the process of surrogate model construction involves certain number of parameters that are usually fixed based on some heuristic, thereby constructing an inefficient model. This paper aims at presenting a novel parameter free surrogate building algorithm specifically focusing on artificial neural networks (ANN) for emulating the physics based models. The proposed algorithm intelligently designs the configuration of the network along with simultaneous determination of sample size required for training it to predict results with maximum accuracy. The novel sample size determination algorithm is based on a potential concept of hypercube sampling technique, which assures convergence to results in very less computational time. The multi-objective optimization framework ensures parsimonious behavior of the network. The proposed algorithm is utilized to build optimal ANN models for highly nonlinear polyvinyl acetate polymer reaction network model. Optimization of this complicated reaction network with the ANN surrogate in place is achieved nearly 12 times faster than the first principle based model. All this being possible with a surrogate building algorithm, which just takes the simulation model as input.

Despite showing immense potential as an optimization technique for solving complex industrial problems, the use of evolutionary algorithms, especially genetic algorithms (GAs), is restricted to offline applications in many industrial cases due to their computationally expensive nature. This problem becomes even more severe when the underlying function as well as constraint evaluation is computationally expensive. To reduce the overall application time under this kind of scenario, a combined usage of the original expensive model and a relatively less expensive surrogate model built around the data provided by the original model in the course of optimization has been proposed in this work. Use of surrogates provides the quickness in the application, thereby saving the execution time, and the use of original model allows the optimization tool to be in the right path of the search process. Switching to the surrogate model happens if predictability of the model is of acceptable accuracy (to be decided by the decision maker), and thereby the optimization time is saved without compromising the solution quality. This concept of successive use of surrogate (artificial neural network [ANN]) and original expensive model is applied on an industrial two-layer sintering process where optimization decides the individual thickness and coke content of each layer to maximize sinter quality and minimize coke consumption simultaneously. The use of surrogate could reduce the execution time by 60% and thereby improve the decision support system utilization without compromising the solution quality.

Significant progress in the oil and gas industry advances the application of pipeline into an intelligent era, which poses rigorous requirements on pipeline safety, reliability, and maintainability, especially when crossing seismic zones. In general, strike-slip faults are prone to induce large deformation leading to local buckling and global rupture eventually. To evaluate the performance and safety of pipelines in this situation, numerical simulations are proved to be a relatively accurate and reliable technique based on the built-in physical models and advanced grid technology. However, the computational cost is prohibitive, so one has to wait for a long time to attain a calculation result for complex large-scale pipelines. In this manuscript, an efficient and accurate surrogate model based on machine learning is proposed for strain demand prediction of buried X80 pipelines subjected to strike-slip faults. Specifically, the support vector regression model serves as a surrogate model to learn the high-dimensionally nonlinear relationship which maps multiple input variables, including pipe geometries, internal pressures, and strike-slip displacements, to output variables (namely tensile strains and compressive strains). The effectiveness and efficiency of the proposed method are validated by numerical studies considering different effects caused by structural sizes, internal pressure, and strike-slip movements.

Surrogate-based method can dramatically reduce the number of expensive function evaluations in real-world multi- objective optimization problems (MOP). When the number of objectives is small, using surrogate models combined with expected hypervolume improvement (EHVI) infill sampling criterion (ISC) has been proved to be efficient to provide a set of solutions with good diversity and good proximity to the Pareto front (PF) in aerodynamic shape optimization. However, traditional hypervolume-based infilling strategies use only one kind of ISC to generate one or multiple sample points, the advantages of various kinds of ISC cannot be comprehensively utilized and the parallelization is not easy to implement. This paper proposes a combined multi-point infilling strategy based on Kriging models and develops an efficient global multi-objective constrained optimization method (EGMOCO) to solve multi- objective aerodynamic shape optimization with complex constraints. Multiple sample points are generated by using four ISC considering hypervolume at each iteration and then evaluated in parallel. Firstly, the performance of EGMOCO is compared with that of single criterion EHVI strategy on six benchmarks within the same computational budget to prove its effectiveness, and then EGMOCO is implemented in an aerodynamic shape optimization problem with complex constraints. The result shows that EGMOCO has good performance in balancing local exploitation and global exploration with faster convergence rate and high robustness, the whole PF can be fully explored in limited evaluations and the constraint handling is effective especially for real-world problems with complex and nonlinear constraints, the comprehensive aerodynamic performance of the airfoil is greatly improved. It can be confirmed that Kriging-based multi-objective optimization method combined with multi-point infilling strategy performs better than single infilling criterion EHVI, since different sample infilling criteria can complement with each other, both local exploitation and global exploration can be considered and well balanced.

Surrogate-assisted evolutionary framework with an ensemble of teaching-learning and differential evolution for expensive optimization

This paper investigates the development of a kriging surrogate model for storm surge prediction utilizing an existing database of high-fidelity, synthetic storms. This surrogate model (metamodel) provides a fast-to-compute mathematical approximation to the input/output relationship of the computationally expensive simulation model that created this database. The implementation is considered over a large coastal region composed of nearshore nodes (locations where storm surge is predicted) and further examines the ability to provide time-series forecasting. This setting creates a high-dimensional output (over a few thousand surge responses) for the surrogate model with anticipated high spatial/temporal correlation. Kriging is considered as a surrogate model, and special attention is given to the appropriate parameterization of the synthetic storms, based on the characteristics of the given database, to determine the input for the metamodel formulation. Principal component analysis (PCA) is integrated in this formulation as a dimension reduction technique to improve computational efficiency, as well as to provide accurate and continuous predictions for time-dependent outputs without the need to introduce time averaging in the time-series forecasting. This is established by leveraging the aforementioned correlation characteristics within the initial database. A range of different implementation choices is examined within the integrated kriging/PCA setting, such as the development of single or multiple metamodels for the different outputs. The metamodel accuracy for inland nodes that have remained dry in some of the storms in the initial database is also examined. The performance of the surrogate modeling approach is evaluated through a case study, utilizing a database of 446 synthetic storms for the Gulf of Mexico (Louisiana coast). The output considered includes time histories for 30 locations over a period of 45.5 h with 92 uniform time steps, as well as peak responses over a grid of 545,635 nearshore nodes. High accuracy and computational efficiency are observed for the proposed implementation, whereas including the prediction error statistics provides estimations with significant safety margins.

In subsurface flow simulation, data-driven deep learning surrogate models have emerged as a promising alternative to traditional simulation methods. However, a major challenge is the large amount of high-fidelity simulation data required for training, and purely data-driven flow surrogate models often lack theoretical support, limiting their applicability to engineering problems. In this study, we propose an efficient surrogate model construction method that integrates physical constraints and multi-fidelity training data. The process begins with pre-training a deep learning model using a large amount of low-cost, low-fidelity data to effectively initialize network parameters. The pre-trained model is then fine-tuned with a smaller amount of high-fidelity data. Throughout the training, physical losses constrain the model to adhere to physical principles. For the case considered here, this approach reduces the need for high-fidelity data by approximately 83% compared to traditional training methods without compromising predictive accuracy. Additionally, incorporating physical loss during model training e ffectively enhances the surrogate model's predictive accuracy, with physical loss reduced by nearly 70% compared to purely data-driven training. The method employs the Jacobian matrix to aid in model training, significantly reducing the computational cost of physical losses. Overall, this surrogate model construction method addresses the challenges of obtaining training data and the lack of physical support in surrogate models, providing reliable support for complex reservoir development decisions.

Optimizing conjunctive use of surface water and groundwater for irrigation to address human-nature water conflicts: A surrogate modeling approach

Surrogate models have been used to replace computationally expensive analysis models in engineering design problems. However, time-dependent variables and historical data are usually ignored in the surrogate modeling process. For instance, in a dam network design, using hydraulic simulations to estimate the water flow is computationally expensive, and the data is in the form of time series. So, we need time-dependent surrogate models to replace these simulations and manage this computational complexity. In this paper, we describe surrogate models to predict the amount of water flow into a reservoir. The challenge is that the flow is a time-dependent variable, and we need to incorporate time series into surrogate models. Thus, there are three contributions: (1) using surrogate modeling to predict flow for dam network design, (2) incorporating time series analysis in surrogate models for water network design, (3) using an ensemble of surrogates to increase the accuracy of prediction. We also demonstrate how to integrate surrogate models and machine learning with time series analysis for more accurate and faster prediction. Due to the availability of data, we use the Buffalo Reservoir in the Red River Basin as an example. Based on the time series data for flow, evaporation, precipitation, and maximum and minimum temperature, three surrogate models are used to examine the impact of integrating time series into surrogate models. These are multivariate autoregressive integrated moving average (MARIMA), a classic time series analysis method; artificial neural network (ANN), and random forest (RF) methods, two machine learning surrogate models. We use seven different time lags as features within an RF model, as an ensemble of surrogate models, and predict the flow for seven-time steps ahead. We successfully incorporate the time series data and particularly the concept of the time lag within surrogate models. We show that RF as the ensemble of surrogates provides more accurate predictions than the other two surrogate models. Although this method has been demonstrated for the Red River Basin, it could also be applied to designing anything in which time-dependent flow is an issue, for example, in biomedical applications, the management of manufacturing processes and product sales as well as any products in which fluid flow is an issue.

Computational speed is a major limiting factor for performing design sensitivity and optimization studies of total knee replacements. Much of this limitation arises from extensive geometry calculations required by contact analyses. This study presents a novel surrogate contact modeling approach to address this limitation. The approach involves fitting contact forces from a computationally expensive contact model (e.g., a finite element model) as a function of the relative pose between the contacting bodies. Because contact forces are much more sensitive to displacements in some directions than others, standard surrogate sampling and modeling techniques do not work well, necessitating the development of special techniques for contact problems. We present a computational evaluation and practical application of the approach using dynamic wear simulation of a total knee replacement constrained to planar motion in a Stanmore machine. The sample points needed for surrogate model fitting were generated by an elastic foundation (EF) contact model. For the computational evaluation, we performed nine different dynamic wear simulations with both the surrogate contact model and the EF contact model. In all cases, the surrogate contact model accurately reproduced the contact force, motion, and wear volume results from the EF model, with computation time being reduced from 13 min to 13 s. For the practical application, we performed a series of Monte Carlo analyses to determine the sensitivity of predicted wear volume to Stanmore machine setup issues. Wear volume was highly sensitive to small variations in motion and load inputs, especially femoral flexion angle, but not to small variations in component placements. Computational speed was reduced from an estimated 230 h to 4 h per analysis. Surrogate contact modeling can significantly improve the computational speed of dynamic contact and wear simulations of total knee replacements and is appropriate for use in design sensitivity and optimization studies.

Multivariate conditional regression using a Bayesian estimation approach for the development of a surrogate dissolution model as part of a comprehensive Process Analytical Technology (PAT) release strategy is the subject of this chapter. An appropriate experimental design provides the basis for constructing a process model in the first step, relating dissolution variables as the response to the critical process parameters (CPPs). Dissolution variables are either: Case 1: in vitro release measurements at one or several specific dissolution time points, or Case 2: the three parameters of the Weibull function describing the full profile at the tablet or batch or run average level. The dissolution variables are augmented with the measured API Content-by-NIR of the tablet to form the multivariate response vector in the process model. Then a conditional regression of the dissolution variables given API Content-by-NIR content and process parameters settings in the second step yields a predictive surrogate model of the dissolution variables. In the first case, the surrogate model directly yields predictions of the in vitro release measurement at the specific time point(s). In the second case, Weibull parameters are predicted, yielding a mean dissolution prediction at any time point on the dissolution profile. The Bayesian estimation approach to the conditional regression model permits simulations that can characterize the analytical performance of the surrogate model with respect to accuracy and precision for model qualification purposes. Two examples are given which show the statistical modeling details of both cases.

An accurate and efficient surrogate model plays a significant role in the optimal design of electrical motors. This paper presents an accurate and efficient surrogate modeling method for a flux reversal permanent magnet arc motor (FR-PMAM) based on machine learning algorithm. Specifically, the motor topology and the optimized design of the FR-PMAM are firstly illustrated. Then, the PM magnetic motive force-permeance model is developed to select structural parameters of FR-PMAM. After that, a finite-element analysis (FEA) model of FR-PMAM is calculated to obtain sample data. A machine learning algorithm random forest is introduced into the field of motor optimal design for the first time to evaluate the contribution of each parameter to the design objectives. Based on the sample data, an efficient machine learning algorithm CatBoost is innovatively employed to establish the accurate and efficient surrogate model of the FR-PMAM that can capture the complex function relationship between output objectives and input parameters. Subsequently, an improved genetic algorithm with the elitist reservation strategy is adopted to obtain the optimal combination of the structural parameters of FR-PMAM. Finally, a prototype of the FR-PMAM is manufactured. The FEA and experimental results validate the effectiveness of the investigated surrogate modeling method.

A hybrid finite element and surrogate modelling approach for simulation and monitoring supported TBM steering

This paper presents an engine sizing and cycle selection study of ultra high bypass ratio engines applied to a subsonic commercial aircraft in the N+2 (2025) timeframe. NASA has created the Environmentally Responsible Aviation (ERA) project to serve as a technology transition bridge between fundamental research (TRL 1–4) and potential commercial application (TRL 7). Specifically, ERA is focused on subsonic transport technologies that could reach TRL 6 by 2020 and can be integrated into an advanced vehicle concept to simultaneously meet the ERA project metrics for noise, emissions, and fuel burn. An important variable in exploring the technology trade space is the selection of the optimal engine cycle for use on the advanced aircraft. Previous literature demonstrated the cycle optimization using a design of experiments (DOE) to explore the engine cycle design space for a pre-defined technology package. However, since the optimal engine cycle is dependent upon the specific technology package, this process would have to be repeated to ensure optimal performance for each technology package. With more than 80 technologies to be analyzed, the combinatorial space of technology packages is enormous. As a result, executing a DOE to find the optimum engine cycle for each technology package is infeasible. To address this issue, it is proposed to use surrogate models that encompass the engine cycle and technology design space to enable fast and accurate optimization of the engine cycle for any given technology package. This paper describes the generation and analysis of surrogate models used for technology assessment and cycle optimization of an ultra high bypass geared turbofan engine architecture. The first study in the paper shows that a single surrogate model can be used to accurately simulate both a technology and cycle design space. To demonstrate the proposed surrogate modeling approach, the cycle design space for three different technology packages was analyzed. This study demonstrated that when an optimal cycle is found within the constrained interior of a design space, the surrogate modeling approach is quite accurate. The study also established that the surrogate models can also be used to assess potential cycles at the boundaries or even outside of the region for which they were trained.

Preliminary design of a complex system often involves exploring a large design space. This may require repeated use of computationally expensive simulations. To ease the computational burden, surrogate models are built to provide rapid approximations of more expensive models. However, the surrogate models themselves are often expensive to build because they are based on repeated experiments with computationally expensive simulations. An alternative approach is to replace the detailed simulations with simplified approximate simulations, thereby sacrificing accuracy for reduced computational time. Naturally, surrogate models built from these approximate simulations will also be imprecise. A strategy is needed for improving the precision of surrogate models based on approximate simulations without significantly increasing computational time. In this paper, a new approach is taken to integrate data from approximate and detailed simulations to build a surrogate model to describe the relationship between output and input parameters. Experimental results from approximate simulations form the bulk of the data, and they are used to build a model based on a Gaussian process. The fitted model is then ‘adjusted’ by incorporating small amounts of data from detailed simulations to obtain a more accurate prediction model. The effectiveness of this approach is demonstrated with a design application for a cellular material that is used to cool a microprocessor. The emphasis is on the method and not on the results per se.