A Bayesian Approach to Sequential Optimization based on Computer Experiments

View Video Presentation: https://doi.org/10.2514/6.2023-3313.vid In this study, an optimization approach based solely on open-source libraries is employed to optimize a centrifugal blower (both impeller and housing). Salome, OpenFoam, and Dakota are used for geometry and mesh generation, Computational Fluid Dynamics (CFD) simulation, and optimization, respectively. For the numerical analysis, Reynolds-averaged Navier-Stokes equations with the k-�� SST turbulence model are used. The Latin Hypercube Sampling (LHS) method is used to determine the design points while eleven design variables related to both the impeller and housing are chosen. Then different metamodels, including Artificial Neural Networks (ANN), Efficient Global Optimization (EGO), Gaussian Process (GP), and Multivariate Adaptive Regression Spline (MARS) are built to obtain optimal design. The findings show that GP proposes the best design among metamodels, with maximum efficiency (57.62%) and negligible error (0.12%) when compared to CFD calculation. In order to validate the optimal design from GP, a centrifugal blower is finally fabricated, demonstrating an acceptable agreement between experimental and numerical results

A recent paper in the journal by Jones and Johnson1 describes the use of designed experiments with computer models for product design and development activities. In recent years we have seen a dramatic increase in the use of computer models for engineering design activities. Some typical applications include computational fluid dynamics models to study turbulent flow, finite element analysis models to investigate the distribution of stresses in systems, and computer design tools for electrical circuits and components. Computer models are also widely used in weather forecasting and the study of hurricanes. These applications are continuing to grow and the applications environments are expanding. Researchers can use these models as substitutes for the physical systems that they represent because if they are properly calibrated and validated, experiments on the model can provide basically the same information as experiments on the real physical system at a fraction of the time and cost. Many of these models are deterministic; that is, they are systems of complex equations without any random components; hence, many of the traditional principles of designing experiments such as randomization, replication, and blocking do not directly apply. In many cases the traditional low-order polynomial models that we typically associate with the designed experiments are not appropriate either, because the underlying system is very complex. The Gaussian process model, a spatial correlation model, is often used to fit the data from a computer experiment. Space-filling designs such as the Latin hypercube are used instead of the familiar factorial and response surface designs. This is a dynamic area of research and development in the experimental design field; interaction between engineers, scientists, and experimental design experts will produce new methods and techniques. There is an another class of computer models not discussed in the Jones and Johnson paper. These are stochastic simulations, that is, computer simulations driven by streams of random numbers producing output responses that can be viewed as random variables. Discrete-event simulations, typical of these models, have been widely used since the 1960s to study factory scheduling problems, hospital operating room activities, and supply chain operations, just to name a few applications. They are also being used increasingly to study many types of transactional and service operations. These computer models also present challenges to the experimental designer. Often they are very high-fidelity models with many input factors; hence, traditional experimental approaches to factor screening and optimization may not be directly applicable. The output may be a time series or some type of functional response; hence, traditional low-order polynomials may not be appropriate. Sometimes these models include both controllable factors and uncontrollable or ‘noise’ factors, hence, some type of robust design or process robustness analysis may be required. Designing experiments for both types of computer model present significant challenges. Quality and Reliability Engineering International welcomes papers, including applications and case studies, on all aspects of these problems, illustrating how computer model and designed experiments can be effectively deployed to solve a significant practical problem, as well as methodological papers on aspects of design.

Regression and Kriging metamodels with their experimental designs in simulation: A review

Designing and understanding the mechanism of non-stoichiometric materials with enhanced properties is challenging, both experimentally and even computationally, due to the large number of chemical spaces and their distributions through the material. In the current work, it is proposed a Machine Learning approach coupled with the Efficient Global Optimization (EGO) method-an Adaptive Design (AD)-to model local defects in materials from first-principle calculations. Our method takes into account the smallest sample set as possible, envisioning the material defect structure relationship with target properties for new insights. As an example, the AD framework allows us to study the stability and the structure of the modified goethite (Fe0.875Al0.125OOH) by considering a proper defect distribution, from first-principle calculations. The chemical space search for the modified goethite was evaluated by starting from different sizes and configurations of the samples as well as different surrogate models (ANN and Gaussian Process; GP), acquisition functions, and descriptors. Our results show that the same local solution of several defect arrangements in Fe0.875Al0.125OOH is found regardless of the initial sample and regression model. This indicates the efficiency of our search method. We also discuss the role of the descriptors in the accelerated global search for defects in material modeling. We conclude that the AD method applied in material defects is a successful approach in automating the search within huge chemical spaces from first-principle calculations by considering small samples. This method can be applied to mechanistic elucidation of non-stoichiometric materials, solid solutions, alloys, and Schottky and Frenkel defects, essential for material design and discovery. Graphical abstract.

As engineering applications become increasingly complex, they are often characterized by implicit response functions that are both expensive to evaluate and nonlinear in their behavior. Reliability assessment given this type of response is dicult with available methods. Current reliability methods focus on the discovery of a single most probable point of failure, and then build a low-order approximation to the limit state at this point. This creates inaccuracies when applied to engineering applications for which the limit state has a higher degree of nonlinearity or is multimodal. Sampling methods, on the other hand, do not rely on an approximation to the shape of the limit state and are therefore generally more accurate when applied to problems with nonlinear limit states. However, sampling methods typically require a large number of response function evaluations, which can make their application infeasible for computationally expensive problems. This paper describes the application of ecient global optimization to reliability assessment to provide a method that eciently characterizes the limit state throughout the uncertain space. The method begins with a Gaussian process model built from a very small number of samples, and then intelligently chooses where to generate subsequent samples to ensure the model is accurate in the vicinity of the limit state. The resulting Gaussian process model is then sampled using multimodal adaptive importance sampling to calculate the probability of exceeding (or failing to exceed) the response level of interest. By locating multiple points on or near the limit state, more complex limit states can be modeled, leading to more accurate probability integration. By concentrating the samples in the area where accuracy is important (i.e. in the vicinity of the limit state), only a small number of true function evaluations are required to build a quality surrogate model. The resulting method is both accurate for any arbitrarily shaped limit state and computationally ecient even for expensive response functions. This new method is applied to a collection of example problems that currently available methods have diculty

The Kriging method based on machine learning is an attractive tool. In this work, a sequential Kriging method assisted by trust region strategy (SKM-TRS) is proposed to solve unconstrained black-box problems. In this SKM-TRS, the complex and expensive objective function is approximated by Kriging model. And then, a sub-optimization problem, which is constructed by Kriging and a distance factor, is minimized by the improved trust region strategy to determine next update point during each iteration cycle. The proposed method is verified by ten well-known benchmark optimization problems and a proxy cache size optimization of the streaming media video data due to fragment popularity distribution. The final test results demonstrate the efficiency and robustness of the SKM-TRS in contrast with Efficient Global Optimization (EGO), Trust Region Implementation in Kriging-based optimization with Expected improvement (TRIKE) and an Adaptive Metamodel based Global Optimization algorithm (AMGO).

The present article addresses the problem of inference in a multiscale computational model of pulmonary arterial and venous blood circulation. The model is a computationally expensive simulator which, given specific parameter values, solves a system of nonlinear partial differential equations and returns predicted pressure and flow values at different locations in the arterial and venous blood vessels. The standard approach in parameter calibration for computer code is to emulate the simulator using a Gaussian Process prior. In the present work, we take a different approach and emulate the objective function itself, i.e. the residual sum of squares between the simulations and the observed data. The Efficient Global Optimization (EGO) algorithm [2] is used to minimize the residual sum of squares. A generalization of the EGO algorithm that can handle hidden constraints is described. We demonstrate that this modified emulator achieves a reduction in the computational costs of inference by two orders of magnitude.

An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization

Due to the increasing demand for high performance and cost reduction within the framework of complex system design, numerical optimization of computationally costly problems is an increasingly popular topic in most engineering fields. In this paper, several variants of the Efficient Global Optimization algorithm for costly constrained problems depending simultaneously on continuous decision variables as well as on quantitative and/or qualitative discrete design parameters are proposed. The adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product between the standard continuous kernel and a second kernel representing the covariance between the discrete variable values. Several parameterizations of this discrete kernel, with their respective strengths and weaknesses, are discussed in this paper. The novel algorithms are tested on a number of analytical test-cases and an aerospace related design problem, and it is shown that they require fewer function evaluations in order to converge towards the neighborhoods of the problem optima when compared to more commonly used optimization algorithms.

We present two new sequential optimization strategies, a sequential subspace optimization method (SSOM) and an improved sequential optimization method (SOM) with orthogonal experimental design technique, to deal with optimization design problems of electromagnetic devices in this work. To implement the proposed methods, we first divide the whole design factors into three sets, namely highly-significant, significant, and nonsignificant factors. Then the whole design space can be correspondingly divided into three subspaces with these three sets of factors. Thereafter, SSOM is presented to sequentially optimize those subspaces. In the subspace, we present an improved SOM based on orthogonal experimental design technique to get optimal solutions. Finally, by investigating TEAM benchmark problem 22, we can see that the sampling efficiency can be improved significantly and the computational cost of finite element analysis can be saved remarkably by the proposed methods.

This study compared hierarchical Bayesian, mixed-effects Gaussian process regression, and random forest models for predicting height to crown base (HCB) in Qinghai spruce (Picea crassifolia Kom.) forests using LiDAR-derived data. Both modeling approaches were applied to a dataset of 510 trees from 16 plots in northern China. The models incorporated tree-level variables (height, diameter at breast height, crown projection area) and plot-level spatial competition indices. Model performance was evaluated using leave-one-plot-out cross-validation. The Gaussian mixed-effects process model (with an RMSE of 1.59 and MAE of 1.25) slightly outperformed the hierarchical Bayesian model and the random forest model. Both models identified LiDAR-derived tree height, DBH, and LiDAR-derived crown projection area as primary factors influencing HCB. The spatial competition index (SCI) emerged as the most effective random effect, with the lowest AIC and BIC values, highlighting the importance of local competition dynamics in HCB formation. Uncertainty analysis revealed consistent patterns across the predicted values, with an average relative uncertainty of 33.89% for the Gaussian process model. These findings provide valuable insights for forest management and suggest that incorporating spatial competition indices can enhance HCB predictions.

Gaussian processes are the most commonly used statistical model for spatial and spatio-temporal processes that vary continuously. They are broadly applicable in the physical sciences and engineering and are also frequently used to approximate the output of complex computer models, deterministic or stochastic. We undertook research related to theory, computation, and applications of Gaussian processes as well as some work on estimating extremes of distributions for which a Gaussian process assumption might be inappropriate. Our theoretical contributions include the development of new classes of spatial-temporal covariance functions with desirable properties and new results showing that certain covariance models lead to predictions with undesirable properties. To understand how Gaussian process models behave when applied to deterministic computer models, we derived what we believe to be the first significant results on the large sample properties of estimators of parameters of Gaussian processes when the actual process is a simple deterministic function. Finally, we investigated some theoretical issues related to maxima of observations with varying upper bounds and found that, depending on the circumstances, standard large sample results for maxima may or may not hold. Our computational innovations include methods for analyzing large spatial datasets when observations fall on a partially observed grid and methods for estimating parameters of a Gaussian process model from observations taken by a polar-orbiting satellite. In our application of Gaussian process models to deterministic computer experiments, we carried out some matrix computations that would have been infeasible using even extended precision arithmetic by focusing on special cases in which all elements of the matrices under study are rational and using exact arithmetic. The applications we studied include total column ozone as measured from a polar-orbiting satellite, sea surface temperatures over the Pacific Ocean, and annual temperature extremes at a site in New York City. In each of these applications, our theoretical and computational innovations were directly motivated by the challenges posed by analyzing these and similar types of data.

Sequential multiobjective optimization method based on radial basis function model is presented in this work to deal with the multiobjective design and optimization problems of engineering electromagnetic devices. Firstly, the initial sample set can be generated by using the sequential optimization method (SOM). SOM can greatly reduce the sample size by integrating the advantages of optimization algorithms and approximate models. Secondly, the optimal Pareto solutions of the multiobjective design problems can be obtained by updating the samples and approximate models sequentially. Thereafter, we calculate the root mean square error for each objective to determine whether it has achieved the default value. Finally, to illustrate the performance of the new method, a classic mathematic test function and a design example of permanent magnet synchronous machine are investigated. It can be found that the proposed method combined the advantages of effectiveness of sequential optimization strategy and lower computation cost of approximate models. The obtained solutions are satisfactory while the computation cost of finite element analysis needed by the new method is less than 1/5 compare with that of direct optimization algorithm.

One comparative analysis method of English comparative literature based on big data Bayesian method has been proposed to improve the calculation accuracy of comparative analysis process of English comparative literature. Firstly, one multilayer Bayesian classification recognition algorithm mechanism has been proposed for the problem of low accuracy and poor calculation efficiency of Bayesian classification algorithm under the big data environment; Gabor multilayer feature extraction algorithm has been proposed for multilayer Bayesian algorithm, Gabor multilayer feature extraction Bayesian algorithm has been designed and realized. Secondly, the algorithm process of comparative analysis system of English comparative literature based on this improved algorithm has been designed and the success rate of comparative analysis of English comparative literature has been improved effectively based on the features of comparative analysis of English comparative literature. Finally, the effectiveness of algorithm has been verified through experimental simulation.

Sequential equi-marginal optimization method for ranking strategies for thermal building renovation