Exploration of the Global Minimum and Conical Intersection with Bayesian Optimization.

This paper incorporates Bayesian estimation and optimization into a portfolio selection framework, particularly for high-dimensional portfolios in which the number of assets is larger than the number of observations. We leverage a constrained minimization approach, called the linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account the time-dependent nature of financial problems and (2) extending the commonly used expected improvement acquisition function to include a tunable trade-off with the improvement’s variance. Allowing a general case of noisy observations, we theoretically derive the sublinear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO results in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with a significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperforms the original proposal of LPO, as well as the benchmark equally weighted and plugin strategies.

The ground-, (1)(pipi*)-, and (3)(pipi*)-state potential-energy surfaces of 1,2-cyclononadiene and isomeric C(9)H(14) species, as well as 1-methyl-1,2-cyclononadiene and isomeric C(10)H(16) species were all mapped using CASSCF and the 6-31G(d) basis set. Theoretical results were found to be in good agreement with the available experimental observations for both 1,2-cyclononadiene and 1-methyl-1,2-cyclononadiene isomerization reactions under singlet and triplet direct or sensitized irradiation. Extremely efficient decay occurs from the first singlet excited state to the ground state through at least three different conical intersections (surface crossings). The first of these crossing points is accessed by a one-bond ring closure. From this conical intersection point (CI-A or CI-C), some possible subsequent ground-state reaction paths have been identified: 1) intramolecular C--H bond insertion to form the bicyclic photoproduct and 2) intramolecular C--H bond insertion to form tricyclic photoproducts. An excited state [1,3]-sigmatropic shift leads to the second conical intersection (CI-B or CI-E), which can give a three-bond cyclononyne species. Besides these, in the singlet photochemical reactions of 1-methyl-1,2-cyclononadiene, excited-state, one allenic C--H bond insertion leads to a third conical intersection (CI-D). Possible ground-state reaction pathways from this structure lead to the formation of a diene photoproduct or to transannular insertion photoproducts. Moreover, in the case of triplet 1,2-cyclononadiene and 1-methyl-1,2-cyclononadiene photoisomerization reactions, both chemical reactions will adopt a 1,3-biradical (T(1)/S(0)-1, T(1)/S(0)-2, and T(1)/S(0)-3), which may undergo intersystem crossings leading to the formation of tricyclic or bicyclic photoproducts. The results obtained allow a number of predictions to be made.

ABSTRACTBayesian optimization is a popular and versatile approach that is well suited to solve challenging optimization problems. Their popularity comes from their effective minimization of expensive function evaluations, their capability to leverage gradients, and their efficient use of noisy data. Bayesian optimizers have commonly been applied to global unconstrained problems, with limited development for many other classes of problems. In this article, two alternative methods are developed that enable rapid and deep convergence of nonlinearly‐constrained local optimization problems using a Bayesian optimizer. The first method uses an exact augmented Lagrangian and the second augments the minimization of the acquisition function to contain additional constraints. Both of these methods can be applied to nonlinear equality constraints, unlike most previous methods developed for constrained Bayesian optimizers. The new methods are applied with a gradient‐enhanced Bayesian optimizer and enable deeper convergence for three nonlinearly‐constrained unimodal optimization problems than previously developed methods for constrained Bayesian optimization. In addition, both new methods enable the Bayesian optimizer to reach a desired tolerance with fewer function evaluations than popular quasi‐Newton optimizers from SciPy and MATLAB for unimodal problems with 2 to 30 variables. The Bayesian optimizer had similar results using both methods. It is recommended that users first try using the second method, which adds constraints to the acquisition function minimization, since its parameters are more intuitive to tune for new problems.

Materials exploration requires the optimization of a multidimensional space including the chemical composition and synthesis parameters such as temperature and pressure. Bayesian optimization has attracted attention as a method for efficient multidimensional optimization. Appropriate choices of the acquisition function and initial values of the hyperparameters of the kernel functions are essential for the Bayesian optimization of synthesis conditions in a small number of experiments. However, to date, there has been little discussion on how to tune Bayesian optimization for materials exploration, and no guidelines have been provided for materials scientists. In this study, we investigated the optimum initial values of the hyperparameters in Bayesian optimization using one-dimensional model functions that mimic actual materials syntheses. The optimal lengthscale and variance for different process windows of materials synthesis were investigated. It was shown that the use of an appropriate acquisition function and suitable initial values of the hyperparameters of the kernel functions enable the optimization of synthesis conditions in a small number of trials. These results provide insight for enabling fully automated and autonomous materials synthesis using Bayesian optimization and robotics for materials exploration.

Bayesian Optimization for Field-Scale Geological Carbon Storage

Rather than optimizing expensive objective functions such as complex engineering simulations directly, Bayesian optimization methodologies fit a surrogate model (typically Kriging or a Gaussian Process) on evaluations of the objective function(s). To determine the next evaluation, an acquisition function is optimized (also referred to as infill criterion or sampling policy) which incorporates the model prediction and uncertainty and balances exploration and exploitation. Therefore, Bayesian optimization methodologies replace a single optimization of the objective function by a sequence of optimization problems: this makes sense as the acquisition function is cheap-to-evaluate whereas the objective is not. Depending on the goal different acquisition functions are available: multi-objective acquisition functions are relatively new and this chapter gives a state-of-the-art overview and illustrates some approaches based on hypervolume improvement. It is shown that the quality of the model is crucial for the performance of Bayesian optimization and illustrate this by using the more flexible Student-t processes as surrogate models.

Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model in a utility measure for each point to be evaluated. Considering such aspect, selection and design of acquisition functions are one of the most popular research topics in BO. Since no single acquisition function was proved to have better performance in all tasks, a well-established approach consists of selecting different acquisition functions along the iterations of a BO execution. In such an approach, the GP-Hedge algorithm is a widely used option given its simplicity and good performance. Despite its success in various applications, GP-Hedge shows an undesirable characteristic of accounting on all past performance measures of each acquisition function to select the next function to be used. In this case, good or bad values obtained in an initial iteration may impact the choice of the acquisition function for the rest of the algorithm. This fact may induce a dominant behavior of an acquisition function and impact the final performance of the method. Aiming to overcome such limitation, in this work we propose a variant of GP-Hedge, named No-PASt-BO, that reduce the influence of far past evaluations. Moreover, our method presents a built-in normalization that avoids the functions in the portfolio to have similar probabilities, thus improving the exploration. The obtained results on both synthetic and real-world optimization tasks indicate that No-PASt-BO presents competitive performance and always outperforms GP-Hedge.

Driven by increased integration of renewable energy sources, the widespread decarbonization of power systems has led to energy price fluctuations that require greater adaptability and flexibility from grid users in order to maximize profits. Industrial loads equipped with flexible resources can optimize energy consumption rather than merely reacting to immediate events, thereby capitalizing on volatile energy prices. However, the absence of sufficient measured data in industrial processes limits the ability to fully harness this flexibility. To address this challenge, we presents a black-box optimization model for optimizing the energy consumption of cooling systems in the aquaculture industry using Extreme Gradient Boosting (XGBoost) and Bayesian Optimization (BO). XGBoost is employed to establish a nonlinear relationship between cooling system power consumption and available measured data. Based on this model, Bayesian Optimization with the Lower Confidence Bound (LCB) acquisition function is used to determine the optimal discharge temperature of water into breeding pools, minimizing day-ahead electricity costs. The proposed approach is validated using real-world data from a case study at the Port of Hirtshals, Denmark based on measurements from 2023. Our findings illustrate that leveraging the inherent flexibility of industrial processes can yield financial benefits while providing valuable signals for grid operators to adjust consumption behaviors through appropriate price mechanisms. Furthermore, machine learning techniques prove effective in optimizing energy consumption for industries with limited measured data, delivering accurate and practical estimations.

Bayesian optimization seeks the global optimum of a black-box, objective function f (x), in the fewest possible iterations. Recent work applied knowledge of the true value of the optimum to the Gaussian Process probabilistic model typically used in Bayesian optimization. This, together with a new acquisition function called Confidence Bound Minimization, resulted in a Gaussian probabilistic posterior in which the predictions were no greater than the known maximum (and no less than for minimum). Our novel work applies Confidence Bound Minimization to Bayesian optimization with Student's-t Processes, a probabilistic alternative which addresses known weaknesses in Gaussian Processes - outliers' probability and the calculation of posterior covariance. The new model is applied to the problem of hyperparameter tuning for an XGBoost classifier. Experiments show superior regret minimization and predictive accuracy, versus the popular Expected Improvement acquisition function. Combining Confidence Bound Minimization with a transformed Student's-t Process probabilistic model and known optima produces superior training regret minimization and posterior predictions for the Six-Hump Camel(2D) and Levy(4D) benchmark problems, which do not fall below true minima.

Gaussian Process based Bayesian Optimization is largely adopted for solving problems where the inputs are in Euclidean spaces. In this paper we associate the inputs to discrete probability distributions which are elements of the probability simplex. To search in the new design space, we need a distance between distributions. The optimal transport distance (aka Wasserstein distance) is chosen due to its mathematical structure and the computational strategies enabled by it. Both the GP and the acquisition function is generalized to an acquisition functional over the probability simplex. To optimize this functional two methods are proposed, one based on auto differentiation and the other based on proximal-point algorithm and the gradient flow. Finally, we report a preliminary set of computational results on a class of problems whose dimension ranges from 5 to 100. These results show that embedding the Bayesian optimization process in the probability simplex enables an effective algorithm whose performance over standard Bayesian optimization improves with the increase of problem dimensionality.

Optimizing multiple, non-preferential objectives for mixed-variable, expensive black-box problems is important in many areas of engineering and science. The expensive, noisy, black-box nature of these problems makes them ideal candidates for Bayesian optimization (BO). Mixed-variable and multi-objective problems, however, are a challenge due to BO's underlying smooth Gaussian process surrogate model. Current multi-objective BO algorithms cannot deal with mixed-variable problems. We present MixMOBO, the first mixed-variable, multi-objective Bayesian optimization framework for such problems. Using MixMOBO, optimal Pareto-fronts for multi-objective, mixed-variable design spaces can be found efficiently while ensuring diverse solutions. The method is sufficiently flexible to incorporate different kernels and acquisition functions, including those that were developed for mixed-variable or multi-objective problems by other authors. We also present HedgeMO, a modified Hedge strategy that uses a portfolio of acquisition functions for multi-objective problems. We present a new acquisition function, SMC. Our results show that MixMOBO performs well against other mixed-variable algorithms on synthetic problems. We apply MixMOBO to the real-world design of an architected material and show that our optimal design, which was experimentally fabricated and validated, has a normalized strain energy density $10^4$ times greater than existing structures.

We propose a numerically simple and straightforward, yet accurate and efficient neural networks-based fitting strategy to construct coupled potential energy surfaces (PESs) in a quasi-diabatic representation. The fundamental invariants are incorporated to account for the complete nuclear permutation inversion symmetry. Instead of derivative couplings or interstate couplings, a so-called modified derivative coupling term is fitted by neural networks, resulting in accurate description of near degeneracy points, such as the conical intersections. The adiabatic energies, energy gradients, and derivative couplings are well reproduced, and the vanishing of derivative couplings as well as the isotropic topography of adiabatic and diabatic energies in asymptotic regions are automatically satisfied. All of these features of the coupled global PESs are requisite for accurate dynamics simulations. Our approach is expected to be very useful in developing highly accurate coupled PESs in a quasi-diabatic representation in an efficient machine learning-based way.

Conformational search is a key part of reaction modeling, molecular docking, and other fields of computational chemistry, where it is important to take molecules' flexibility into account. However, modern conformational search approaches provide no guarantee that they did not miss any important conformation. Thus, identifying missing conformations from an existing ensemble is of broad importance for computational chemistry. In this paper, we introduce a Bayesian optimization algorithm for conformational ensemble augmentation, that is, locating missing conformers in an existing ensemble, which employs Bayesian optimization with physics-informed torsion-potential-based kernel function and a novel acquisition function that prioritizes potential energy surface exploration for increased conformer diversity. The devised method demonstrates high efficiency on a test set of biologically relevant molecules.

In the Born-Oppenheimer approximation for molecular dynamics as generalized by Born and Huang, nuclei move on multiple potential-energy surfaces corresponding to different electronic states. These surfaces may intersect at a point in the nuclear coordinates with the topology of a double cone. These conical intersections have important consequences for the dynamics. When an adiabatic electronic wave function is transported around a closed loop in nuclear coordinate space that encloses a conical intersection point, it acquires an additional geometric, or Berry, phase. The Schr\"odinger equation for nuclear motion must be modified accordingly. A conical intersection also permits efficient nonadiabatic transitions between potential-energy surfaces. Most examples of the geometric phase in molecular dynamics have been in situations in which a molecular point-group symmetry required the electronic degeneracy and the consequent conical intersection. Similarly, it has been commonly assumed that the conical intersections facilitating nonadiabatic transitions were largely symmetry driven. However, conical intersections also occur in the absence of any symmetry considerations. This review discusses computational tools for finding and characterizing the conical intersections in such systems. Because these purely accidental intersections are difficult to anticipate, they may occur more frequently than previously thought and in unexpected situations, making the geometric phase effect and the occurrence of efficient nonadiabatic transitions more commonplace phenomena. [S0034-6861(96)00404-7]

A computer simulation procedure is developed for modeling intramolecular dynamics in polyatomic molecules. Electronic-vibrational excitation by ultrashort laser pulses (20 fs divided by 1 ps) is treated explicitly using quantum theory in harmonic approximation. MD simulation is used for studying the excited state dynamics. Stilbene photoinduced isomerization is modeled. Model potential energy surfaces (PES) for the ground and first excited singlet states are obtained using experimental absorption spectra in supersonic jet. Using a symmetrical along the torsional coordinate PES, it is shown that cis-stilbene undergoes the first stage of the isomerization reaction, i.e. transition to the twisted configuration, much faster than trans- stilbene, only due to the specific conformational properties.