Best practices for multi-fidelity Bayesian optimization in materials and molecular research.

<div class="section abstract"><div class="htmlview paragraph">The multi-fidelity Bayesian optimization of large models with multiple parameters can be computationally very expensive and thereby the proper choice of the infilling strategy is crucial to minimize the required convergence time. If, on one hand, nested infilling guarantees a better performance of the multi-fidelity Gaussian algorithm, on the other hand the new samples added at every iteration and at all levels represent a non-negligible cost factor. In this paper the two alternative infilling approaches, nested and non-nested sampling, are analyzed for a family of data sets, each one characterized by a different correlation factor. The aim is to establish for each case the best option that minimizes the computational time and thus a connection between correlation level of the data and infilling strategy. Three data sets are investigated: low, medium and highly correlated data in accordance with their correlation matrix calculated ex ante knowing the exact solutions in the entire domain. The total process time is given by the summation for each iteration of the time required for the single acquisition and this variable depends not only on the infilling strategy, single or multilevel, but also on the single level acquisition itself assuming that adding points on the high-fidelity level is more expensive than on the low-fidelity level. For this reason a weighted selection of the acquisition function (AQF) is defined. The exact data correlation factor (DCF) is in real cases unknown but the partial data correlation factor based on the available data can already provide useful information. The present study has general validity but it has been especially designed as preliminary investigation for the optimization of a laser welding process.</div></div>

Nonlinear state-space models are ubiquitous in modeling real-world dynamical systems. Sequential Monte Carlo (SMC) techniques, also known as particle methods, are a well-known class of parameter estimation methods for this general class of state-space models. Existing SMC-based techniques rely on excessive sampling of the parameter space, which makes their computation intractable for large systems or tall data sets. Bayesian optimization techniques have been used for fast inference in state-space models with intractable likelihoods. These techniques aim to find the maximum of the likelihood function by sequential sampling of the parameter space through a single SMC approximator. Various SMC approximators with different fidelities and computational costs are often available for sample-based likelihood approximation. In this paper, we propose a multi-fidelity Bayesian optimization algorithm for the inference of general nonlinear state-space models (MFBO-SSM), which enables simultaneous sequential selection of parameters and approximators. The accuracy and speed of the algorithm are demonstrated by numerical experiments using synthetic gene expression data from a gene regulatory network model and real data from the VIX stock price index.

Learning cardiac activation maps from 12-lead ECG with multi-fidelity Bayesian optimization on manifolds

• An active learning Kriging methodology for Bayesian updating is presented. • The BUS method recasts the updating problem into an equivalent reliability problem, solved with a Kriging metamodel. • A multi-fidelity approach is employed in the active learning Kriging model, where models of different accuracy are adopted. • Multi-fidelity Bayesian optimization is used for calculating the constant c of the BUS method. • An improved version of the Quantified Active learning Subset Simulation method is proposed to define the posterior samples. The paper presents a multi-fidelity framework for efficient Bayesian updating using active learning Kriging models. The updating problem is solved as an equivalent probabilistic problem using the Bayesian updating with Structural reliability (BUS) method. The proposed approach leverages an ensemble surrogate modeling strategy that combines two Kriging models of multiple fidelities, thereby enhancing predictive capacity while reducing computational cost by exploiting information from models of different fidelity levels. A key contribution is the integration of multi-fidelity Bayesian optimization in order to determine the optimal BUS constant, ensuring accurate transformation of the posterior into a reliability problem. Furthermore, an improved version of the Quantified Active Learning Subset Simulation (qAK-SuS) method, previously proposed by the authors, is proposed in order to efficiently estimate small failure probabilities and obtain the posterior distribution. The proposed framework, called qAK-BUS, offers a robust solution for Bayesian inference in engineering systems by smartly balancing model fidelity, prediction variance, and learning efficiency. Three numerical examples and an engineering application that involves the updating of a reinforced concrete bridge are examined for demonstrating and validating the proposed framework.

We employ multi-fidelity Bayesian optimization to search a large candidate set of covalent organic frameworks (COFs) for the one with the largest [simulated] equilibrium adsorptive selectivity for xenon (Xe) over krypton (Kr) at room temperature.

Today, debugging failed software upgrades is a long and tedious activity, as developers may have to consider large sections of code to locate the bug. We argue that failed upgrade debugging can be simplified by exploiting the characteristics of upgrade problems to prioritize the set of routines to consider. In particular, previous work has shown that differences between the computing environment in the developer's and users' sites cause most upgrade problems. Based on this observation, we design and implement Sahara, a system that identifies the aspects of the environment that are most likely the culprits of the misbehavior, finds the subset of routines that relate to those aspects, and selects an even smaller subset of routines to debug first. We evaluate Sahara for three real upgrade problems with the OpenSSH suite, one synthetic problem with the SQLite database, and one synthetic problem with the uServer Web server. Our results show that the system produces accurate recommendations comprising only a small number of routines.

Optimizing multiple, non-preferential objectives for mixed-variable, expensive black-box problems is important in many areas of engineering and science. The expensive, noisy, black-box nature of these problems makes them ideal candidates for Bayesian optimization (BO). Mixed-variable and multi-objective problems, however, are a challenge due to BO's underlying smooth Gaussian process surrogate model. Current multi-objective BO algorithms cannot deal with mixed-variable problems. We present MixMOBO, the first mixed-variable, multi-objective Bayesian optimization framework for such problems. Using MixMOBO, optimal Pareto-fronts for multi-objective, mixed-variable design spaces can be found efficiently while ensuring diverse solutions. The method is sufficiently flexible to incorporate different kernels and acquisition functions, including those that were developed for mixed-variable or multi-objective problems by other authors. We also present HedgeMO, a modified Hedge strategy that uses a portfolio of acquisition functions for multi-objective problems. We present a new acquisition function, SMC. Our results show that MixMOBO performs well against other mixed-variable algorithms on synthetic problems. We apply MixMOBO to the real-world design of an architected material and show that our optimal design, which was experimentally fabricated and validated, has a normalized strain energy density $10^4$ times greater than existing structures.

Accelerating the discovery of new molecules and materials, as well as developing green and sustainable ways to synthesize them, will help to address global challenges in energy, sustainability and healthcare. The recent growth of data science and automated experimentation techniques has resulted in the advent of self-driving labs (SDLs) via the integration of machine learning, lab automation and robotics. An SDL is a machine-learning-assisted modular experimental platform that iteratively operates a series of experiments selected by the machine learning algorithm to achieve a user-defined objective. These intelligent robotic assistants help researchers to accelerate the pace of fundamental and applied research through rapid exploration of the chemical space. In this Review, we introduce SDLs and provide a roadmap for their implementation by non-expert scientists. We present the status quo of successful SDL implementations in the field and discuss their current limitations and future opportunities to accelerate finding solutions for societal needs. Self-driving labs (SDLs) combine machine learning with automated experimental platforms, enabling rapid exploration of the chemical space and accelerating the pace of materials and molecular discovery. In this Review, the application of SDLs, their limitations and future opportunities are discussed, and a roadmap is provided for their implementation by non-expert scientists.

A general variational framework for image approximation and segmentation is introduced in which the boundary function has a simple explicit form in terms of the approximation function. At the same time, this variational framework is general enough to include the most commonly used objective functions. Since the optimal boundary function, that minimizes the associated objective functional for a given approximation function, can be found explicitly, the objective functional can be expressed in a reduced form that depends only on the approximating function. From this a partial differential equation descent method, aimed at minimizing the objective functional, is derived. The method is fast and produces excellent results as illustrated by a number of real and synthetic image problems.

A new branch-and-filter exact algorithm for binary constraint satisfaction problems

Modeling has come of age as a research tool in the basic sciences, especially the exact sciences. Specifically, in the discipline of chemistry, it has been of great utility. Its use dates back to the 17th Century and includes such wide areas as computational chemistry, chemoinformatics, molecular mechanics, chemical dynamics, molecular dynamics, molecular graphics and algorithms. Modeling has been employed extensively in discovering newer chemical compounds and in explaining some chemical properties hitherto unexplained by laboratory experimental procedures. In modeling, some ways of determining the total energy to predict molecular structures include the ab initio method, density functional method and the semi-empirical/empirical methods. Several computer softwares have been employed in chemistrybased modeling including the HUMO-LUMO, ATMOL, GAUSSIAN, IBMOL, and POLYAYTOM,

The Knowledge Environment for Collaborative Science (KnECS) is an open‐source informatics toolkit designed to enable knowledge Grids that interconnect science communities, unique facilities, data, and tools. KnECS features a Web portal with team and data collaboration tools, lightweight federation of data, provenance tracking, and multi‐level support for application integration. We identify the capabilities of KnECS and discuss extensions from the Collaboratory for Multi‐Scale Chemical Sciences (CMCS) which enable diverse combustion science communities to create and share verified, documented data sets and reference data, thereby demonstrating new methods of community interaction and data interoperability required by systems science approaches. Finally, we summarize the challenges we encountered and foresee for knowledge environments. Copyright © 2007 John Wiley & Sons, Ltd.

This study develops an approach referred to as SA‐MF using simulated annealing (SA) and a three‐dimensional groundwater flow model (MODFLOW‐2000) to determine the pumping source information, namely the pumping source location, the pumping rate and the pumping period. Eight scenarios, each with various cases for aquifers with homogeneous or heterogeneous conductivities, are used to test the applicability and accuracy of SA‐MF in determining the pumping source information. The results show that at least five measured heads should be used to analyse the pumping source identification problem and at least three observation wells are required to effectively determine the pumping source information. The SA‐MF gives good estimated results in both synthetic and real problems even if the measured heads contain measurement errors. Copyright © 2008 John Wiley & Sons, Ltd.

Unlike constructive approaches in which a partial assignment to the variables is incrementally extended, repair approaches start with an inconsistent assignment (e.g. old solution) and search through the space of possible repairs. In this paper, we propose an Extended Partial-order Dynamic Backtracking (EPDB) algorithm for dynamically changed environments. The EPDB allows dynamic CSPs to be dealt efficiently according to based-repair heuristic approaches. A past solution can be repaired using retroactive data structures: safety conditions and no goods, saved previously in Partial-order Dynamic Backtracking process. We evaluate our algorithm on synthetic and real problems, and experimental results show that the proposed algorithm outperforms the original algorithm PDB, in terms of run-time and constraints checks.

Game theory finds nowadays a broad range of applications in engineering and machine learning. However, in a derivative-free, expensive black-box context, very few algorithmic solutions are available to find game equilibria. Here, we propose a novel Gaussian-process based approach for solving games in this context. We follow a classical Bayesian optimization framework, with sequential sampling decisions based on acquisition functions. Two strategies are proposed, based either on the probability of achieving equilibrium or on the stepwise uncertainty reduction paradigm. Practical and numerical aspects are discussed in order to enhance the scalability and reduce computation time. Our approach is evaluated on several synthetic game problems with varying number of players and decision space dimensions. We show that equilibria can be found reliably for a fraction of the cost (in terms of black-box evaluations) compared to classical, derivative-based algorithms. The method is available in the R package GPGame available on CRAN at https://cran.r-project.org/package=GPGame .