Mixed-Variable Global Sensitivity Analysis for Knowledge Discovery and Efficient Combinatorial Materials Design

Global Sensitivity Analysis (GSA) is the study of the influence of any given inputs on the outputs of a model. In the context of engineering design, GSA has been widely used to understand both individual and collective contributions of design variables on the design objectives. So far, global sensitivity studies have often been limited to design spaces with only quantitative (numerical) design variables. However, many engineering systems also contain, if not only, qualitative (categorical) design variables in addition to quantitative design variables. In this paper, we integrate the novel Latent Variable Gaussian Process (LVGP) with Sobol’ analysis to develop the first metamodel-based mixed-variable GSA method. Through two analytical case studies, we first validate and demonstrate the effectiveness of our proposed method for mixed-variable problems. Furthermore, while the new metamodel-based mixed-variable GSA method can benefit various engineering design applications, we employ our method with multi-objective Bayesian optimization (BO) to accelerate the Pareto front design exploration in many-level combinatorial design spaces. Specifically, we implement a sensitivity-aware design framework for metal-organic framework (MOF) materials that are constructed only from qualitative design variables and show the benefits of our method for expediting the exploration of novel MOF candidates from a many-level large combinatorial design space.

High variability in tree growth and shape and horticultural performance is very common in guava (Psidium guajava L.) orchards in Zulia state, Venezuela. In consequence, morphological characterization of elite genotypes of guava seedlings was completed. Guava variants with white or red mesocarp and endocarp were chosen. The research was performed at the Centro Fruticola de Zulia, Mara County (10°48'00 LN, 71°48'00 LW), and at RFA and La Rosaida orchards, Baralt county (09°36'02 LN, 70°58'33 LW), Zulia State, Venezuela. The recorded quantitative and qualitative variables were: stem surface (SS), perimeter (PS), internode length (LI) and phyllotaxy (F), leaf shape (LS), leaf margin (ML), foliar area (AF), apex (FA) and leaf base (FB) shapes, and flowers (flower diameter (DF). Frequency tables and analysis of variance were utilized to evaluate the qualitative and quantitative variables, respectively. Multivariate analysis through principal components analysis (PCA) was accomplished to determine the typical traits of each guava variant. Regarding the qualitative variables, the acute FB, sinuate ML, opposite F and smooth with few flaky SS predominated in white and red guavas. The oblong LS and obtuse FA was present in white guavas, while elliptical LS and acute FA dominated in red guavas. The quantitative variables AF, LI and PS were significantly different (P 0.05) were determined for DF. The first three PCAs expressed 78% of the variability among guava variants. The variables that allowed differentiating guava variants were: AF, LI, FB, SS and PS.

Integrating experimental data with computational methods enables multicriteria design of nanocomposites using quantitative and qualitative design variables.

In multi-objective Bayesian optimization, an infill criterion is an important part, as it is the indicator to evaluate how much good a new set of solutions is, compared to a Pareto-front approximation set. This paper presents a deterministic algorithm for computing the Expected R2 Indicator for bi-objective problems and studies its use as an infill criterion in Bayesian Global Optimization. The R2-Indicator was introduced in 1998 by M. Hansen and A. Jaszkiewicz for performance assessment in multi-objective optimization and is more recently also used in indicator-based multi-criterion evolutionary algorithms (IBEAs). In Bayesian Global Optimization, we propose the Expected R2-indicator Improvement (ER2I) as an infill criterion. It is defined as the expected decrease of the R2 indicator by a point that is sampled from a predictive Gaussian distribution. The ER2I can also be used as a pre-selection criterion in surrogate-assisted IBEAs. It provides an alternative to the Expected Hypervolume-Indicator Improvement (EHVI) that requires a reference point, bounding the Pareto front from above. In contrast, the ER2I works with a utopian reference point that bounds the Pareto front from below. In addition, the ER2I supports preference modelling with utility functions and its computation time grows only linearly with the number of considered weight combinations. It is straightforward to approximate the ER2I by Monte Carlo Integration, but so far a deterministic algorithm to solve the non-linear integral remained unknown. We outline a deterministic algorithm for the computation of the bi-objective ER2I with Chebychev utility functions. Moreover, we study monotonicity properties of the ER2I w.r.t. parameters of the predictive distribution and numerical simulations demonstrate fast convergence to Pareto fronts of different shapes and the ability of the ER2I Bayesian optimization to fill gaps in the Pareto front approximation.

We propose a surrogate-assisted reference vector adaptation (SRVA) method to\nsolve expensive multi- and many-objective optimization problems with various\nPareto front shapes. SRVA is coupled with a multi-objective Bayesian\noptimization (MBO) algorithm using reference vectors for scalarization of\nobjective functions. The Kriging surrogate models for MBO is used to estimate\nthe Pareto front shape and generate adaptive reference vectors uniformly\ndistributed on the estimated Pareto front. We combine SRVA with expected\nimprovement of penalty-based boundary intersection as an infill criterion for\nMBO. The proposed algorithm is compared with two other MBO algorithms by\napplying them to benchmark problems with various Pareto front shapes.\nExperimental results show that the proposed algorithm outperforms the other two\nin the problems whose objective functions are reasonably approximated by the\nKriging models. SRVA improves diversity of non-dominated solutions for these\nproblems with continuous, discontinuous, and degenerated Pareto fronts.\nBesides, the proposed algorithm obtains much better solutions from early stages\nof optimization especially in many-objective problems.\n

(1) Background: In the preoperative planning stage of scoliosis surgery, it is routine to use radiographs obtained with and without traction to observe the curve flexibility in order to estimate curve correction, but its association with mechanical complications is not completely understood. (2) Methods: Retrospective cohort study of all patients undergoing infantile, congenital, neuromuscular or idiopathic adolescent scoliosis correction surgery at a single institution between 2015 and 2019, with a minimum follow-up of 24 months. Associations between qualitative variables were tested with the chi-square test. The association between qualitative and quantitative variables were tested with the Mann–Whitney test, and correlations between quantitative variables was tested with Spearman’s correlation. (3) Results: A total of 330 patients, 88 males and 242 females, with a mean age of 16.98 years at surgery, were included. The mean value of preoperative main curves, its flexibility and postoperative value were 54.44 degrees, 21.73 degrees and 18.08 degrees, respectively. (4) Conclusions: Preoperative spinal X-ray examination with traction or bending films is a reasonable option for assessing scoliotic curve flexibility, and patients with neuromuscular scoliosis who are not ambulatory can be informed of the increased risks of late mechanical complications.

Gaussian processes (GPs) are ubiquitously used in sciences and engineering as metamodels. Standard GPs, however, can only handle numerical or quantitative variables. In this paper, we introduce latent map Gaussian processes (LMGPs) that inherit the attractive properties of GPs and are also applicable to mixed data which have both quantitative and qualitative inputs. The core idea behind LMGPs is to learn a continuous, low-dimensional latent space or manifold which encodes all qualitative inputs. To learn this manifold, we first assign a unique prior vector representation to each combination of qualitative inputs. We then use a low-rank linear map to project these priors on a manifold that characterizes the posterior representations. As the posteriors are quantitative, they can be directly used in any standard correlation function such as the Gaussian or Matern. Hence, the optimal map and the corresponding manifold, along with other hyperparameters of the correlation function, can be systematically learned via maximum likelihood estimation. Through a wide range of analytic and real-world examples, we demonstrate the advantages of LMGPs over state-of-the-art methods in terms of accuracy and versatility. In particular, we show that LMGPs can handle variable-length inputs, have an explainable neural network interpretation, and provide insights into how qualitative inputs affect the response or interact with each other. We also employ LMGPs in Bayesian optimization and illustrate that they can discover optimal compound compositions more efficiently than conventional methods that convert compositions to qualitative variables via manual featurization.

Bayesian optimization (BO) methods solve problems with several black-box objectives and constraints.Each black-box is expensive to evaluate and lacks a closed-form.They use a model of each black-box to guide the search for the problem's solution.Sometimes, however, the black-boxes may be evaluated at different fidelity levels.A lower fidelity is simply a cheap proxy of the corresponding black-box.Thus, lower fidelities that correlate with the actual black-box can be used to reduce the optimization cost.We propose Joint Entropy Search for Multi-Fidelity and Multi-objective Bayesian Optimization with Constraints (MF-JESMOC), a BO method for solving the aforementioned problems.It chooses the next point and fidelity level at which to evaluate the black-boxes as the one that is expected to reduce the most the joint entropy of the Pareto set and the Pareto front, normalized by the fidelity's cost.Deep Gaussian processes are used to model each black-box and dependencies between fidelities.In our experiments, MF-JESMOC outperforms other state-of-the-art methods for multi-objective BO with constraints and different fidelity levels.

In this paper, we present an optimal shape design method for a dielectric microlens which is used to focus an incoming infrared plane wave in wideband, by exploiting the finite difference time domain (FDTD) technique and the topology optimization technique. Topology optimization is a scheme to search an optimal shape by adjusting the material properties, which are design variables, within the design space. And by introducing the adjoint variable method, we can effectively calculate a derivative of the objective function with respect to the design variable. To verify the proposed method, a shape design problem of a dielectric microlens is tested when illuminated by a transverse electric (TE)-polarized infrared plane wave. In this problem, the design variable is the dielectric constant within the design space of a dielectric microlens. The design objective is to maximally focus the incoming magnetic field at a specific point in wideband.

An experimental comparison of several approaches to the discriminant problem with both qualitative and quantitative variables

Regression analysis with dummy variables has interesting and useful relationship among qualitative and quantitative variables under study. Qualitative variables are used to classify the data into different groups. The aim of this paper is to suggest a method to estimate two variable linear model, three variable linear model and three variable non linear model in a single regression using dummy variables with a numerical example.

Multicriterial design methodology with safety as one of the design objectives is presented. The aim of the paper is to analyze the influence of safety based design objectives on generated nondominated designs on the Pareto frontier. Possible improvements in nondominated designs are investigated by comparison to ones obtained with the standard design procedure when safety criteria are used as design constraints only. It is assumed that safety based objectives and targets act as attractors, driving nondominated designs along the constant cost/weight contours in design space towards its safer regions. Global safety objectives (for hogging/sagging modes), are based on the maximization of ultimate longitudinal strength in vertical bending calculated via the extended IACS incremental-iterative method. Applied compound safety measures for gross-panel (stiffened panel with associated girders) are based upon 34 failure modes, belonging to serviceability/collapse subsets. Objectives based on the maximization of safety measures are applied together with standard design objectives such as minimization of initial cost and weight. The following problems were solved with different sets of objectives: (a) minimize cost and weight objectives subject to safety constraints (used for reference), (b) only the maximization of local safety measures is added to (a) as additional objectives, (c) only the maximization of global safety measures is added to (a) as additional objectives, (d) maximization of safety measures ad (b) and (c) are added to (a) as additional objectives. For each of the problems (a–d) the developed design procedure is executed. It contains two basic tasks for structural design of realistic (non-academic) problems: (1) multicriterial optimization with topology / geometry design variables; (2) multicriterial optimization of gross-panels with scantling / material design variables. Design procedure steps are executed using a fast and balanced collection of analysis and synthesis modules/methods of the OCTOPUS design system: • Determination of design load sets; • MOGA / MOPSO based generation of nondominated designs for the selected ship structure; • For each design the following analysis blocks are executed: – calculation of ship’s primary and racking response fields, – calculation of ship’s ultimate longitudinal strength, – calculation of serviceability and collapse safety criteria on the gross-panel level. Comparisons of results, based on generated Pareto hyper-surfaces and on subset of preferred designs, are given for problems (a–d). Insights into the results of optimization process, using 5-D graphics for design and attribute spaces, are also presented. Design problems of modern RoPax and SWATH structures are used in case studies.

Multi-Objective Bayesian Global Optimization using expected hypervolume improvement gradient

Due to the adversarial limitations between the output solutions in the multi-objective optimization stage, the energy efficiency optimization of large pumping stations has the disadvantage of difficult coordination between power consumption and efficiency. Therefore, a multi-objective Bayesian hyperparameter optimization algorithm design for energy efficiency optimization of large pumping stations has been carried out. A multi-objective optimization model for energy efficiency of large pumping stations was constructed by combining the input power of the main motor, transmission energy loss, transformer loss, and energy consumption of auxiliary equipment, with efficiency as the constraint. We extract sequence features that affect pump water demand, and use feature labels as hyperparameters for the multi-objective optimization model of energy efficiency in large pumping stations. We transform the multi-objective optimization problem into a hyperparameter optimization problem, avoiding adversarial output solutions. We iterate the query points collected by EI, so the output model reaches a convergent optimal solution. The test results show that the designed algorithm can improve the pumping efficiency of the pump station while controlling the power consumption of the pump station, achieving collaborative optimization of efficiency.

In the present study, the effective parameters of water-Al2O3 nanofluid flowing in flat tubes are investigated using the EFAST Sensitivity Analysis (SA) method. The SA is performed using GMDH type artificial neural networks (ANN) which are based on validated numerical data of two phase modeling of nanofluid flow in flat tubes. There are five design variables namely: tube flattening (H), flow rate (Q), wall heat flux ( ), nanoparticle diameter (dp) and nanoparticle volume fraction (φ) and there are two objective functions namely: pressure drop (∆P) and heat transfer coefficient (h). The results show that among design variables, the tube flattening has the highest effect on variations of pressure drop (74%) and heat transfer coefficient (40%). Except tube flattening, the flow rate and the nanoparticle volume fraction has the highest effect on pressure drop (24%) and heat transfer coefficient (25%) respectively. The effects of all of the design variables on objective functions are shown in the results.