A multiscale Bayesian optimization framework for process and material codesign

Bayesian optimization for congestion pricing problems: A general framework and its instability

Estimation of internal states in a Li-ion battery using BiLSTM with Bayesian hyperparameter optimization

pBO-2GP-3B: A batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics

Designing preference elicitation (PE) methodologies that can quickly ascertain a user’s top item preferences in a cold-start setting is a key challenge for building effective and personalized conversational recommendation (ConvRec) systems. While large language models (LLMs) enable fully natural language (NL) PE dialogues, we hypothesize that monolithic LLM NL-PE approaches lack the multi-turn, decision-theoretic reasoning required to effectively balance the exploration and exploitation of user preferences towards an arbitrary item set. In contrast, traditional Bayesian optimization PE methods define theoretically optimal PE strategies, but cannot generate arbitrary NL queries or reason over content in NL item descriptions – requiring users to express preferences via ratings or comparisons of unfamiliar items. To overcome the limitations of both approaches, we formulate NL-PE in a Bayesian Optimization (BO) framework that seeks to actively elicit NL feedback to identify the best recommendation. Key challenges in generalizing BO to deal with natural language feedback include determining: (a) how to leverage LLMs to model the likelihood of NL preference feedback as a function of item utilities, and (b) how to design an acquisition function for NL BO that can elicit preferences in the infinite space of language. We demonstrate our framework in a novel NL-PE algorithm, PEBOL, which uses: 1) Natural Language Inference (NLI) between user preference utterances and NL item descriptions to maintain Bayesian preference beliefs, and 2) BO strategies such as Thompson Sampling (TS) and Upper Confidence Bound (UCB) to guide LLM query generation. We numerically evaluate our methods in controlled simulations, finding that after 10 turns of dialogue, PEBOL can achieve an MRR@10 of up to 0.27 compared to the best monolithic LLM baseline’s MRR@10 of 0.17, despite relying on earlier and smaller LLMs.1

Given the increasing importance of machine learning (ML) in our lives, several algorithmic fairness techniques have been proposed to mitigate biases in the outcomes of the ML models. However, most of these techniques are specialized to cater to a single family of ML models and a specific definition of fairness, limiting their adaptibility in practice. We introduce a general constrained Bayesian optimization (BO) framework to optimize the performance of any ML model while enforcing one or multiple fairness constraints. BO is a model-agnostic optimization method that has been successfully applied to automatically tune the hyperparameters of ML models. We apply BO with fairness constraints to a range of popular models, including random forests, gradient boosting, and neural networks, showing that we can obtain accurate and fair solutions by acting solely on the hyperparameters. We also show empirically that our approach is competitive with specialized techniques that enforce model-specific fairness constraints, and outperforms preprocessing methods that learn fair representations of the input data. Moreover, our method can be used in synergy with such specialized fairness techniques to tune their hyperparameters. Finally, we study the relationship between fairness and the hyperparameters selected by BO. We observe a correlation between regularization and unbiased models, explaining why acting on the hyperparameters leads to ML models that generalize well and are fair.

Liquid organic hydrogen carriers (LOHCs) are a promising option for hydrogen storage, but a high efficiency of the LOHC cycle is essential in order to serve as an attractive technology in the context of decarbonization. This paper presents different methods to optimize steady‐state operating points of a LOHC dehydrogenation (DH) reactor. For this purpose, an analytical model of a DH reactor is described, which is extended to a hybrid model (HM) to achieve sufficient model accuracy. The model quality is subsequently validated by measurements. For the optimization of the steady‐state operation point, a Bayesian optimization framework is presented and compared to classical model‐based optimization. The methods are evaluated with the HM as well as with experimental results.

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.

Manufacturing is a promising technique for producing complex and custom-made parts with a high degree of precision. It can also provide us with desired materials and products with specified properties. To achieve that, it is crucial to find out the optimum point of process parameters that have a significant impact on the properties and quality of the final product. Unfortunately, optimizing these parameters can be challenging due to the complex and nonlinear nature of the underlying process, which becomes more complicated when there are conflicting objectives, sometimes with multiple goals. Furthermore, experiments are usually costly, time-consuming, and require expensive materials, man, and machine hours. So, each experiment is valuable and it's critical to determine the optimal experiment location to gain the most comprehensive understanding of the process. Sequential learning is a promising approach to actively learn from the ongoing experiments, iteratively update the underlying optimization routine, and adapt the data collection process on the go. This thesis presents a multi-objective Bayesian optimization framework to find out the optimum processing conditions for a manufacturing setup. It uses an acquisition function to collect data points sequentially and iteratively update its understanding of the underlying design space utilizing a Gaussian Process-based surrogate model. In manufacturing processes, the focus is often on obtaining a rough understanding of the design space using minimal experimentation, rather than finding the optimal parameters. This falls under the category of "approximating the underlying function" rather than design optimization. This approach can provide material scientists or manufacturing engineers with a comprehensive view of the entire design space, increasing the likelihood of making discoveries or making robust decisions. However, a precise and reliable prediction model is necessary for a good approximation. To meet this requirement, this thesis proposes an epsilon-greedy sequential prediction framework that is distinct from the optimization framework. The data acquisition strategy has been refined to balance exploration and exploitation, and a threshold has been established to determine when to switch between the two. The performance of this proposed optimization and prediction framework is evaluated using real-life datasets against the traditional design of experiments. The proposed frameworks have generated effective optimization and prediction results using fewer experiments.

The parameters of a transportation simulation model need to pass through a careful calibration process to ensure that the model’s output is as close as possible to the actual system. Owing to the computationally expensive and black-box nature of a simulation model, there is a need for robust and efficient calibration algorithms. This paper proposes a Bayesian optimization framework for the high-dimensional calibration problem of transportation simulation models. Bayesian optimization uses acquisition functions to determine more promising values for future evaluation, instead of relying on local gradient approximations. It guarantees convergence to the global optimum with a reduced number of evaluations, therefore is very computationally efficient. The proposed algorithm is applied to the calibration of a simulation network coded in simulation of urban mobility (SUMO), an open-source microscopic transportation simulation platform, and compared with a well-known method named simultaneous perturbation stochastic approximation (SPSA). To assess the calibration accuracy, speed distributions obtained from the two models calibrated using these two different methods are compared with the observation. For both the Bayesian optimization and SPSA results, the simulated and observed distributions are validated to be from the same distribution at a 95% confidence level for multiple sensor locations. Thus, the calibration accuracy of the two approaches are both acceptable for a stochastic transportation simulation model. However, Bayesian optimization shows a better convergence and a higher computational efficiency than SPSA. In addition, the comparative results of multiple implementations validate its robustness for a noisy objective function, unlike SPSA which may sometimes get stuck in a local optimum and fail to converge in a global solution.

Due to the growing complexity and numerous manufacturing variation in safety-critical analog and mixed-signal (AMS) circuit design, rare failure detection in the high-dimensional variational space is one of the major challenges in AMS verification. Efficient AMS failure detection is very demanding with limited samples on account of high simulation and manufacturing cost. In this work, we combine a reversible network and a gating architecture to identify essential features from datasets and reduce feature dimension for fast failure detection. While reversible residual networks (RevNets) have been actively studied for its restoration ability from output to input without the loss of information, the gating network facilitates the RevNet to aim at effective dimension reduction. We incorporate the proposed reversible gating architecture into Bayesian optimization (BO) framework to reduce the dimensionality of BO embedding important features clarified by gating fusion weights so that the failure points can be efficiently located. Furthermore, we propose a conditional density estimation of important and non-important features to extract high-dimensional original input features from the low-dimension important features, improving the efficiency of the proposed methods. The improvements of our proposed approach on rare failure detection is demonstrated in AMS data under the high-dimensional process variations.

Multi-objective materials bayesian optimization with active learning of design constraints: Design of ductile refractory multi-principal-element alloys

This paper reports a Drug Formulation System to Generics with Optimization-Based Dosage Calibration to hasten the creation of bioequivalent generic drugs. The methodology uses constrained multi-objective Bayesian Optimization (BO) to maximize probability of bioequivalence (P(BE)) and at the same time meet the critical quality attributes e.g., dissolution, friability and hardness of the tablet. The system uses BoTorch (including Ax), a PyTorch-based optimization library, to combine mechanistic modeling with surrogate learning to minimize the need to rely on trial-and-error experimentation. It has been demonstrated that the BO framework can reach P(BE) ≥ 0.90 in much fewer optimization cycles than both the conventional response surface methodology and genetic algorithms and also delivers better constraint satisfaction and computational efficiency. The proposed system can provide an uncertainty-sensitive formulation design solution that offers a regulatory-conformant avenue to decrease development time, decrease the cost of experiments, and improve reliability in generic drug approvals.

To maintain a sufficient chlorine residual in water distribution systems (WDSs), chlorine dosage needs to be regulated. The majority of previous studies that aimed to optimize chlorine dosage in WDSs considered single-species water quality (WQ) models featuring chlorine decay with simple reaction kinetics. Recent efforts have proposed using multi-species water quality (MS-WQ) models to account for chlorine interactions with various chemical and microbiological species, thus providing a comprehensive and accurate evaluation of the WQ within WDSs. Nevertheless, the key challenge of implementing MS-WQ models within optimization frameworks is their high computational cost and poor scalability for larger WDSs. Furthermore, previous optimization studies generally relied on evolutionary algorithms (EAs), which require conducting a significant number of WQ simulations. Bayesian optimization (BO) has been recently suggested as an efficient alternative to EAs for the optimization of computationally expensive functions. This study aims to present a systematic comparison between BO and other widely used EAs for the optimization of MS-WQ in WDSs. A case study featuring a real-life, midsized benchmark WDS was implemented to comprehensively evaluate all three optimization techniques. The results revealed that BO is notably more computationally efficient and less sensitive to changes in the constraints than EAs.

Bayesian optimization has proven effective for optimizing expensive-to-evaluate functions in Chemical Engineering. However, valuable physical insights from domain experts are often overlooked. This article introduces a collaborative Bayesian optimization approach that re-integrates human input into the data-driven decision-making process. By combining high-throughput Bayesian optimization with discrete decision theory, experts can influence the selection of experiments via a discrete choice. We propose a multi-objective approach togenerate a set of high-utility and distinct solutions, from which the expert selects the desired solution for evaluation at each iteration. Our methodology maintains the advantages of Bayesian optimization while incorporating expert knowledge and improving accountability. The approach is demonstrated across various case studies, including bioprocess optimization and reactor geometry design, demonstrating that even with an uninformed practitioner, the algorithm recovers the regret of standard Bayesian optimization. By including continuous expert opinion, the proposed method enables faster convergence and improved accountability for Bayesian optimization in engineering systems.

Cocrystallization can be used to tune key drug properties, such as aqueous solubility, without altering molecular structure; however, the space of possible coformers is enormous and design rules are empirical. We present a Bayesian optimization framework that couples Gaussian process (GP) classification and regression that can accelerate cocrystal discovery and solubility enhancement. Starting from 6338 literature-derived binary coformer pairs, vector fingerprints that combine 2D structural information (fragment and MQN fingerprints) with low-cost shape and polarity descriptors were engineered for cocrystal prediction. A GP classifier, trained on an actively constructed training set of ∼1000 coformer pairs selected by uncertainty sampling achieves up to 94% accuracy and Matthews correlation coefficient of 0.79 on a test set of >5000 unseen pairs. Property-driven coformer selection was formulated as a Bayesian optimization problem, using a machine learning model as a surrogate for aqueous solubility and Tanimoto-similarity to guide campaigns across several discovery scenarios. In simulations, the framework rapidly identifies highly soluble cocrystals, typically recovering top-5 candidates after fewer than 10 evaluations. Finally, we validate the workflow experimentally with 12 pharmaceutical and pharmaceutical-like compounds, discovering two new cocrystals, resveratrol + praziquantel and purin-6-amine + thiazole-4-carboxylic acid, with markedly enhanced aqueous solubility. These results demonstrate a practical, data-efficient route to Bayesian optimization for cocrystal design.