Mixed-Variable Global Sensitivity Analysis With Applications to Data-Driven Combinatorial Materials Design

Global Sensitivity Analysis (GSA) is the study of the influence of any given input on the outputs of a model. In the context of engineering design, GSA has been widely used to understand both individual and collective contributions of design variables on the design objectives. So far, global sensitivity studies have often been limited to design spaces with only quantitative (numerical) design variables. However, many engineering systems also contain, if not only, qualitative (categorical) design variables in addition to quantitative design variables. In this paper, we integrate Latent Variable Gaussian Process (LVGP) with Sobol’ analysis to develop the first metamodel-based mixed-variable GSA method. Through numerical case studies, we validate and demonstrate the effectiveness of our proposed method for mixed-variable problems. Furthermore, while the proposed GSA method is general enough to benefit various engineering design applications, we integrate it with multi-objective Bayesian optimization (BO) to create a sensitivity-aware design framework in accelerating the Pareto front design exploration for metal-organic framework (MOF) materials with many-level combinatorial design spaces. Although MOFs are constructed only from qualitative variables that are notoriously difficult to design, our method can utilize sensitivity analysis to navigate the optimization in the many-level large combinatorial design space, greatly expediting the exploration of novel MOF candidates.

In multi-objective Bayesian optimization, an infill criterion is an important part, as it is the indicator to evaluate how much good a new set of solutions is, compared to a Pareto-front approximation set. This paper presents a deterministic algorithm for computing the Expected R2 Indicator for bi-objective problems and studies its use as an infill criterion in Bayesian Global Optimization. The R2-Indicator was introduced in 1998 by M. Hansen and A. Jaszkiewicz for performance assessment in multi-objective optimization and is more recently also used in indicator-based multi-criterion evolutionary algorithms (IBEAs). In Bayesian Global Optimization, we propose the Expected R2-indicator Improvement (ER2I) as an infill criterion. It is defined as the expected decrease of the R2 indicator by a point that is sampled from a predictive Gaussian distribution. The ER2I can also be used as a pre-selection criterion in surrogate-assisted IBEAs. It provides an alternative to the Expected Hypervolume-Indicator Improvement (EHVI) that requires a reference point, bounding the Pareto front from above. In contrast, the ER2I works with a utopian reference point that bounds the Pareto front from below. In addition, the ER2I supports preference modelling with utility functions and its computation time grows only linearly with the number of considered weight combinations. It is straightforward to approximate the ER2I by Monte Carlo Integration, but so far a deterministic algorithm to solve the non-linear integral remained unknown. We outline a deterministic algorithm for the computation of the bi-objective ER2I with Chebychev utility functions. Moreover, we study monotonicity properties of the ER2I w.r.t. parameters of the predictive distribution and numerical simulations demonstrate fast convergence to Pareto fronts of different shapes and the ability of the ER2I Bayesian optimization to fill gaps in the Pareto front approximation.

Gaussian processes (GPs) are ubiquitously used in sciences and engineering as metamodels. Standard GPs, however, can only handle numerical or quantitative variables. In this paper, we introduce latent map Gaussian processes (LMGPs) that inherit the attractive properties of GPs and are also applicable to mixed data which have both quantitative and qualitative inputs. The core idea behind LMGPs is to learn a continuous, low-dimensional latent space or manifold which encodes all qualitative inputs. To learn this manifold, we first assign a unique prior vector representation to each combination of qualitative inputs. We then use a low-rank linear map to project these priors on a manifold that characterizes the posterior representations. As the posteriors are quantitative, they can be directly used in any standard correlation function such as the Gaussian or Matern. Hence, the optimal map and the corresponding manifold, along with other hyperparameters of the correlation function, can be systematically learned via maximum likelihood estimation. Through a wide range of analytic and real-world examples, we demonstrate the advantages of LMGPs over state-of-the-art methods in terms of accuracy and versatility. In particular, we show that LMGPs can handle variable-length inputs, have an explainable neural network interpretation, and provide insights into how qualitative inputs affect the response or interact with each other. We also employ LMGPs in Bayesian optimization and illustrate that they can discover optimal compound compositions more efficiently than conventional methods that convert compositions to qualitative variables via manual featurization.

Integrating experimental data with computational methods enables multicriteria design of nanocomposites using quantitative and qualitative design variables.

We propose a surrogate-assisted reference vector adaptation (SRVA) method to\nsolve expensive multi- and many-objective optimization problems with various\nPareto front shapes. SRVA is coupled with a multi-objective Bayesian\noptimization (MBO) algorithm using reference vectors for scalarization of\nobjective functions. The Kriging surrogate models for MBO is used to estimate\nthe Pareto front shape and generate adaptive reference vectors uniformly\ndistributed on the estimated Pareto front. We combine SRVA with expected\nimprovement of penalty-based boundary intersection as an infill criterion for\nMBO. The proposed algorithm is compared with two other MBO algorithms by\napplying them to benchmark problems with various Pareto front shapes.\nExperimental results show that the proposed algorithm outperforms the other two\nin the problems whose objective functions are reasonably approximated by the\nKriging models. SRVA improves diversity of non-dominated solutions for these\nproblems with continuous, discontinuous, and degenerated Pareto fronts.\nBesides, the proposed algorithm obtains much better solutions from early stages\nof optimization especially in many-objective problems.\n

In this paper, we present an optimal shape design method for a dielectric microlens which is used to focus an incoming infrared plane wave in wideband, by exploiting the finite difference time domain (FDTD) technique and the topology optimization technique. Topology optimization is a scheme to search an optimal shape by adjusting the material properties, which are design variables, within the design space. And by introducing the adjoint variable method, we can effectively calculate a derivative of the objective function with respect to the design variable. To verify the proposed method, a shape design problem of a dielectric microlens is tested when illuminated by a transverse electric (TE)-polarized infrared plane wave. In this problem, the design variable is the dielectric constant within the design space of a dielectric microlens. The design objective is to maximally focus the incoming magnetic field at a specific point in wideband.

Bayesian optimization (BO) methods solve problems with several black-box objectives and constraints.Each black-box is expensive to evaluate and lacks a closed-form.They use a model of each black-box to guide the search for the problem's solution.Sometimes, however, the black-boxes may be evaluated at different fidelity levels.A lower fidelity is simply a cheap proxy of the corresponding black-box.Thus, lower fidelities that correlate with the actual black-box can be used to reduce the optimization cost.We propose Joint Entropy Search for Multi-Fidelity and Multi-objective Bayesian Optimization with Constraints (MF-JESMOC), a BO method for solving the aforementioned problems.It chooses the next point and fidelity level at which to evaluate the black-boxes as the one that is expected to reduce the most the joint entropy of the Pareto set and the Pareto front, normalized by the fidelity's cost.Deep Gaussian processes are used to model each black-box and dependencies between fidelities.In our experiments, MF-JESMOC outperforms other state-of-the-art methods for multi-objective BO with constraints and different fidelity levels.

Multicriterial design methodology with safety as one of the design objectives is presented. The aim of the paper is to analyze the influence of safety based design objectives on generated nondominated designs on the Pareto frontier. Possible improvements in nondominated designs are investigated by comparison to ones obtained with the standard design procedure when safety criteria are used as design constraints only. It is assumed that safety based objectives and targets act as attractors, driving nondominated designs along the constant cost/weight contours in design space towards its safer regions. Global safety objectives (for hogging/sagging modes), are based on the maximization of ultimate longitudinal strength in vertical bending calculated via the extended IACS incremental-iterative method. Applied compound safety measures for gross-panel (stiffened panel with associated girders) are based upon 34 failure modes, belonging to serviceability/collapse subsets. Objectives based on the maximization of safety measures are applied together with standard design objectives such as minimization of initial cost and weight. The following problems were solved with different sets of objectives: (a) minimize cost and weight objectives subject to safety constraints (used for reference), (b) only the maximization of local safety measures is added to (a) as additional objectives, (c) only the maximization of global safety measures is added to (a) as additional objectives, (d) maximization of safety measures ad (b) and (c) are added to (a) as additional objectives. For each of the problems (a–d) the developed design procedure is executed. It contains two basic tasks for structural design of realistic (non-academic) problems: (1) multicriterial optimization with topology / geometry design variables; (2) multicriterial optimization of gross-panels with scantling / material design variables. Design procedure steps are executed using a fast and balanced collection of analysis and synthesis modules/methods of the OCTOPUS design system: • Determination of design load sets; • MOGA / MOPSO based generation of nondominated designs for the selected ship structure; • For each design the following analysis blocks are executed: – calculation of ship’s primary and racking response fields, – calculation of ship’s ultimate longitudinal strength, – calculation of serviceability and collapse safety criteria on the gross-panel level. Comparisons of results, based on generated Pareto hyper-surfaces and on subset of preferred designs, are given for problems (a–d). Insights into the results of optimization process, using 5-D graphics for design and attribute spaces, are also presented. Design problems of modern RoPax and SWATH structures are used in case studies.

High variability in tree growth and shape and horticultural performance is very common in guava (Psidium guajava L.) orchards in Zulia state, Venezuela. In consequence, morphological characterization of elite genotypes of guava seedlings was completed. Guava variants with white or red mesocarp and endocarp were chosen. The research was performed at the Centro Fruticola de Zulia, Mara County (10°48'00 LN, 71°48'00 LW), and at RFA and La Rosaida orchards, Baralt county (09°36'02 LN, 70°58'33 LW), Zulia State, Venezuela. The recorded quantitative and qualitative variables were: stem surface (SS), perimeter (PS), internode length (LI) and phyllotaxy (F), leaf shape (LS), leaf margin (ML), foliar area (AF), apex (FA) and leaf base (FB) shapes, and flowers (flower diameter (DF). Frequency tables and analysis of variance were utilized to evaluate the qualitative and quantitative variables, respectively. Multivariate analysis through principal components analysis (PCA) was accomplished to determine the typical traits of each guava variant. Regarding the qualitative variables, the acute FB, sinuate ML, opposite F and smooth with few flaky SS predominated in white and red guavas. The oblong LS and obtuse FA was present in white guavas, while elliptical LS and acute FA dominated in red guavas. The quantitative variables AF, LI and PS were significantly different (P 0.05) were determined for DF. The first three PCAs expressed 78% of the variability among guava variants. The variables that allowed differentiating guava variants were: AF, LI, FB, SS and PS.

Multi-Objective Bayesian Global Optimization using expected hypervolume improvement gradient

Due to the adversarial limitations between the output solutions in the multi-objective optimization stage, the energy efficiency optimization of large pumping stations has the disadvantage of difficult coordination between power consumption and efficiency. Therefore, a multi-objective Bayesian hyperparameter optimization algorithm design for energy efficiency optimization of large pumping stations has been carried out. A multi-objective optimization model for energy efficiency of large pumping stations was constructed by combining the input power of the main motor, transmission energy loss, transformer loss, and energy consumption of auxiliary equipment, with efficiency as the constraint. We extract sequence features that affect pump water demand, and use feature labels as hyperparameters for the multi-objective optimization model of energy efficiency in large pumping stations. We transform the multi-objective optimization problem into a hyperparameter optimization problem, avoiding adversarial output solutions. We iterate the query points collected by EI, so the output model reaches a convergent optimal solution. The test results show that the designed algorithm can improve the pumping efficiency of the pump station while controlling the power consumption of the pump station, achieving collaborative optimization of efficiency.

Multi-disciplinary optimization of railway wheels

We report a human-in-the-loop implementation of the multi-objective experimental design via a Bayesian optimization platform (EDBO+) towards the optimization of butylpyridinium bromide synthesis under continuous flow conditions. The algorithm simultaneously optimized reaction yield and production rate (or space-time yield) and generated a well defined Pareto front. The versatility of EDBO+ was demonstrated by expanding the reaction space mid-campaign by increasing the upper temperature limit. Incorporation of continuous flow techniques enabled improved control over reaction parameters compared to common batch chemistry processes, while providing a route towards future automated syntheses and improved scalability. To that end, we applied the open-source Python module, nmrglue, for semi-automated nuclear magnetic resonance (NMR) spectroscopy analysis, and compared the acquired outputs against those obtained through manual processing methods from spectra collected on both low-field (60 MHz) and high-field (400 MHz) NMR spectrometers. The EDBO+ based model was retrained with these four different datasets and the resulting Pareto front predictions provided insight into the effect of data analysis on model predictions. Finally, quaternization of poly(4-vinylpyridine) with bromobutane illustrated the extension of continuous flow chemistry to synthesize functional materials.

Additive manufacturing is a modern technique to produce parts with a complex geometry. However, the choice of the printing parameters is a time-consuming and costly process. In this study, the parameter optimization for the laser powder bed fusion process was investigated. Using state-of-the art multi-objective Bayesian optimization, the set of the most-promising process parameters (laser power, scanning speed, hatch distance, etc.), which would yield parts with the desired hardness and porosity, was established. The Gaussian process surrogate model was built on 57 empirical data points, and through efficient sampling in the design space, we were able to obtain three points in the Pareto front in just over six iterations. The produced parts had a hardness ranging from 224–235 HV and a porosity in the range of 0.2–0.37%. The trained model recommended using the following parameters for high-quality parts: 58 W, 257 mm/s, 45 µm, with a scan rotation angle of 131 degrees. The proposed methodology greatly reduces the number of experiments, thus saving time and resources. The candidate process parameters prescribed by the model were experimentally validated and tested.

Multifunctional materials that balance mechanical resilience and fluid dynamic efficiency are critical in engineering applications, yet their synergistic optimization remains challenging due to inherent trade-offs, computational expense, and high-dimensional design spaces. Inspired by the skeleton of the deep-sea sponge Euplectella aspergillum, this work presents an automated framework integrating Finite Element Analysis for mechanics, Computational Fluid Dynamics for flow behavior, and multi-objective Bayesian optimization. Leveraging high-performance computing, the framework efficiently explores complex design spaces to identify Pareto-optimal solutions. Optimized lattices achieve an average 140% increase in critical buckling load across a range of volume fractions relative to baseline designs, while simultaneously reducing drag, lift, and vortex shedding at porosities as low as 5%. We fabricate selected designs via stereolithography and validate them through compression experiments and particle image velocimetry, showing agreement with simulations. By jointly optimizing mechanics and fluidics, this work establishes a scalable methodology for designing lightweight, high-performance architected materials.