Pareto-Based Multi-output Metamodeling with Active Learning

The use of computer simulations has become a viable alternative for real-life controlled experiments. Due to the computational cost of these high fidelity simulations, surrogate models are often employed as an approximation for the original simulator. Because of their compact formulation and negligible evaluation time, global surrogate models are very useful tools for exploring the design space, optimization, visualization and sensitivity analysis. Additionally, multiple surrogate models can be chained together to model large scale systems where the direct use of the original expensive simulators would be too cumbersome. Many surrogate model types, such as neural networks, support vector machines and rational models have been proposed, and many more techniques have been developed to minimize the number of expensive simulations required to train a sufficiently accurate surrogate model. In this chapter, we present a fully automated and integrated global surrogate modeling methodology for regression modeling and active learning that readily enables the adoption of advanced global surrogate modeling methods by application scientists. The work brings together insights from distributed systems, artificial intelligence, and modeling & simulation, and has applications in a very wide range of fields. The merits of this approach are illustrated with several examples, and several surrogate model types.

In machine learning,active learningrefers to algorithms that autonomously select the data points from which they will learn. There are many data mining applications in which large amounts of unlabeled data are readily available, but labels (e.g., human annotations or results coming from complex experiments) are costly to obtain. In such scenarios, an active learning algorithm aims at identifying data points that, if labeled and used for training, would most improve the learned model. Labels are then obtained only for the most promising data points. This speeds up learning and reduces labeling costs. Support vector machine (SVM) classifiers are particularly well‐suited for active learning due to their convenient mathematical properties. They perform linear classification, typically in a kernel‐induced feature space, which makes expressing the distance of a data point from the decision boundary straightforward. Furthermore, heuristics can efficiently help estimate how strongly learning from a data point influences the current model. This information can be used to actively select training samples. After a brief introduction to the active learning problem, we discuss different query strategies for selecting informative data points and review how these strategies give rise to different variants of active learning withSVMs.This article is categorized under:Technologies > Machine Learning

A multi-objective evolutionary approach to training set selection for support vector machine

In electronic circuit design, preliminary analyses of the circuit performances are generally carried out using time-consuming simulations. These analyses should be performed as fast as possible because of the strict temporal constraints on the industrial sector time to market. On the other hand, there is the need of precision and reliability of the analyses. For these reasons, there is more and more interest toward surrogate models able to approximate the behavior of a device with a high precision making use of a limited set of samples. Using suitable surrogate models instead of simulations, it is possible to perform a reliable analysis in less time. In this work, we are going to analyze how the surrogate models given by the support vector machine (SVM) perform when they are used to approximate the behavior of industrial circuits that will be employed in consumer electronics. The SVM is also compared to the surrogate models given by the response surface methodology using a commercial software currently adopted for this kind of applications.

Surrogate modeling immersed probability density evolution method for structural reliability analysis in high dimensions

Traditional reliability-based design optimization(RBDO)is either computational intensive or not accurate enough.In this work,a new RBDO method based on Support Vector Machines(SVM)is proposed.For reliability analysis,SVM is used to create a surrogate model of the limit-state function at the Most Probable Point(MPP).The uniqueness of the new method is the use of the gradient of the limit-state function at the MPP.This guarantees that the surrogate model not only passes through the MPP but also is tangent to the limit-state function at the MPP.Then Importance Sampling(IS)is used to calculate the probability of failure based on the surrogate model.This treatment significantly improves the accuracy of reliability analysis.For optimization,the Sequential Optimization and Reliability Assessment(SORA)is employed,which decouples deterministic optimization from the SVM reliability analysis.The decoupling makes RBDO more efficient.The two examples show that the new method is more accurate with a moderately increased computational cost.

The paper introduces a novel modality to efficiently tune the convolutional layers of a deep neural network (CNN) and an approach to also rank the importance of the involved hyperparameters. Evolutionary algorithms (EA) offer a flexible solution to this twofold issue, while the expensive simulations of the deep learner with the generated configurations are resolved by surrogate modelling. Three models have been used and evaluated as surrogates: random forests (RF), support vector machines (SVM) and Kriging. Sample convolutional configurations are generated by Latin hypercube sampling and have attached computed accuracy outcomes from real CNN runs. For the hyperparameter estimation task, the fitness of an individual from the EA associated with a surrogate model is subsequently derived from the CNN accuracy estimation on those variable values. With respect to the ranking and variable selection task, RF includes implicit variable selection, the SVM can be straightforwardly supported by a second EA, and Kriging offers a ranking based on the corresponding θ values. The estimated accuracy of the found hyperparameter values is compared with the true validation accuracy, and they are next used for the prediction on the test cases. The ranking of the variables for each of the three surrogate models is compared, and their influence is also revealed by response surface methodology. The experimental testing of the proposed EA–surrogate approaches is conducted on a real-world scenario of histopathological image interpretation in colorectal cancer diagnosis.

Surrogate modeling of high-dimensional problems via data-driven polynomial chaos expansions and sparse partial least square

Recent progress on machine learning techniques for flow field optimization toward performance and commercial scale proton exchange membrane fuel cell: A systematic review

The demand for fast and accurate electromagnetic solutions to support current and emerging technologies has fueled the rapid development of various machine learning techniques for applications such as antenna design and optimization, microwave imaging, device diagnostics, and more. Multi-fidelity (MF) surrogate modeling methods have shown great promise in significantly reducing computational costs associated with surrogate modeling while maintaining high model accuracy. This work offers a comprehensive review of the available MF surrogate modeling methods in electromagnetics, focusing on specific methodologies, related challenges, and the generation of variable-fidelity datasets. The article is structured around the two main types of electromagnetic problems: forward and inverse. It begins by summarizing key machine learning concepts and limitations. This transitions to discussing multi-fidelity surrogate model architectures and low-fidelity data techniques for the forward problem. Subsequently, the unique challenges of the inverse problem are presented, along with traditional solutions and their limitations. Following this, the review examines MF surrogate modeling approaches tailored to the inverse problem. In conclusion, the review outlines promising future directions in MF modeling for electromagnetics, aiming to provide fundamental insights into understanding these developing methods.

Uncertainty quantification has proven to be an indispensable study for enhancing reliability and robustness of engineering systems in the early design phase. Single and multi-fidelity surrogate modelling methods have been used to replace the expensive high fidelity analyses which must be repeated many times for uncertainty quantification. However, since the number of analyses required to build an accurate surrogate model increases exponentially with the number of random input variables, most surrogate modelling methods suffer from the curse of dimensionality. As an alternative approach, the Low-Rank Approximation method can be applied to high-dimensional uncertainty quantification studies with a low computational cost, where the number of coefficients for building the surrogate model increases only linearly with the number of random input variables. In this study, the Low-Rank Approximation method is implemented for multi-fidelity applications with additive and multiplicative correction approaches to make the high-dimensional uncertainty quantification analysis more efficient and accurate. The developed uncertainty quantification methodology is tested on supersonic aircraft design problems and its predictions are compared with the results of single- and multi-fidelity Polynomial Chaos Expansion and Monte Carlo methods. For the same computational cost, the Low-Rank Approximation method outperformed both in surrogate modeling and uncertainty quantification cases for all the benchmarks and real-world engineering problems addressed in the present study.

The paper shows the feasibility of building closed-form and fast-to-evaluate surrogate models via supervised kernel-based machine learning regressions behaving as digital twins for computationally expensive multibody simulations. The aforementioned surrogate models are adopted to predict the railway vehicle dynamics safety indexes defined in the international standards, depending on the wheel-rail forces, directly from the results of longitudinal train dynamics simulations. The digital twin models are trained with the outputs of Simpack multibody simulations of a reference freight wagon, to which the in-train forces calculated by an in-house MATLAB longitudinal train dynamics simulator are applied. Two machine learning techniques are considered: the support vector machine and the least-squares support vector machine regressions. Both techniques ensure a good accuracy even with a limited number of training samples. The derivation of the surrogate models can strongly enhance the modelling capabilities of common longitudinal train dynamics simulators, that cannot evaluate the wheel-rail contact forces. At the same time, the method shown in the paper allows to strongly reduce the total computational times, as the evaluation of the closed-form surrogate models allows to effectively estimate the safety indexes with no need for computationally demanding multibody simulations.

<p>The Indian subcontinent is prone to tropical cyclones that originate in the Indian Ocean and cause widespread destruction to life and property. Accurate prediction of cyclone track, landfall, wind, and precipitation are critical in minimizing damage. The Weather Research and Forecast (WRF) model is widely used to predict tropical cyclones. The accuracy of the model prediction depends on initial conditions, physics schemes, and model parameters. The parameter values are selected empirically by scheme developers using the trial and error method, implying that the parameter values are sensitive to climatological conditions and regions. The number of tunable parameters in the WRF model is about several hundred, and calibrating all of them is highly impossible since it requires thousands of simulations. Therefore, sensitivity analysis is critical to screen out the parameters that significantly impact the meteorological variables. The Sobol’ sensitivity analysis method is used to identify the sensitive WRF model parameters. As this method requires a considerable amount of samples to evaluate the sensitivity adequately, machine learning algorithms are used to construct surrogate models trained using a limited number of samples. They could help generate a vast number of required pseudo-samples. Five machine learning algorithms, namely, Gaussian Process Regression (GPR), Support Vector Machine, Regression Tree, Random Forest, and K-Nearest Neighbor, are considered in this study. Ten-fold cross-validation is used to evaluate the surrogate models constructed using the five algorithms and identify the robust surrogate model among them. The samples generated from this surrogate model are then used by the Sobol’ method to evaluate the WRF model parameter sensitivity.</p>

In this paper, an evolutionary algorithm with spatially distributed surrogates (EASDS) for multiobjective optimization is presented. The algorithm performs actual analysis for the initial population and periodically every few generations. An external archive of the unique solutions evaluated using the actual analysis is maintained to train the surrogate models. The data points in the archive are split into multiple partitions using k-Means clustering. A Radial Basis Function (RBF) network surrogate model is built for each partition using a fraction of the points in that partition. The rest of the points in the partition are used as a validation data to decide the prediction accuracy of the surrogate model. Prediction of a new candidate solution is done by the surrogate model with the least prediction error in the neighborhood of that point. Five multiobjective test problems are presented in this study and a comparison with Nondominated Sorting Genetic Algorithm II (NSGA-II) is included to highlight the benefits offered by our approach. EASDS algorithm consistently reported better nondominated solutions for all the test cases for the same number of actual evaluations as compared to a single global surrogate model and NSGA-II.

This effort develops a surrogate modeling approach for predicting the effects of manufacturing variations on blade unsteadiness and performance of a transonic turbine. CFD results from a set of 105 as-manufactured turbine blade geometries are used to train and validate the surrogate models. Blade geometry variation is characterized with point clouds created from a structured light optical measurement system and as-measured CFD grids are generated through mesh morphing of the nominal design grid data. Results from a Reynolds-averaged Navier-Stokes flow solver with the two-equation Wilcox turbulence model are used as training and validation data. Principal Component Analysis (PCA) of the measured airfoil geometry variations is used to create reduced basis of independent surrogate model parameters. Results of the surrogate are compared to the CFD results. It is shown that the surrogate model typically captures between 60% and 80% of the full CFD predicted variance. Three new approaches are introduced to improve the accuracy of the surrogate. A zonal PCA approach is defined which improves surrogate accuracy by focusing on key regions of the airfoil. A training point reduction strategy is proposed that is based on the kd-tree nearest neighbor search algorithm and reduces the required training points by 25%. A second reduction approach uses k-means clustering to effectively select training points from the 105 blade population and is used to reduce the required training points by up to 66%.