Optimizing Power Consumption in Aquaculture Cooling Systems: A Bayesian Optimization and XGBoost Approach Under Limited Data

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

The pursuit of chiral nanographenes with robust chiral stability and good chiroptical responses is of great interest for material-based applications. However, the most reported preparation processes involve intricate synthetic pathways and harsh conditions, resulting in readily epimerization due to their low epimerization barriers. In this study, we present a streamlined one-pot electrochemical synthesis for a novel double oxaza[7]dehydrohelicene, characterized by a substantial epimerization barrier (33.8 kcal mol−1) and notable chiroptical responses (|glum| = 1.5 × 10−3). To optimize the electrochemical conditions efficiently, we applied a Bayesian optimization (BO) approach with expected improvement (EI) or lower confidence bound (LCB) as an acquisition function, aiming to maximize exploration and exploitation while minimizing the number of experiments needed to identify global maxima. Additionally, the structural and optical features of this molecule have been studied using X-ray crystallographic analysis, and the absorption and emission behaviors were rationalized based on DFT calculations.

This paper incorporates Bayesian estimation and optimization into a portfolio selection framework, particularly for high-dimensional portfolios in which the number of assets is larger than the number of observations. We leverage a constrained minimization approach, called the linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account the time-dependent nature of financial problems and (2) extending the commonly used expected improvement acquisition function to include a tunable trade-off with the improvement’s variance. Allowing a general case of noisy observations, we theoretically derive the sublinear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO results in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with a significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperforms the original proposal of LPO, as well as the benchmark equally weighted and plugin strategies.

ABSTRACTBayesian optimization is a popular and versatile approach that is well suited to solve challenging optimization problems. Their popularity comes from their effective minimization of expensive function evaluations, their capability to leverage gradients, and their efficient use of noisy data. Bayesian optimizers have commonly been applied to global unconstrained problems, with limited development for many other classes of problems. In this article, two alternative methods are developed that enable rapid and deep convergence of nonlinearly‐constrained local optimization problems using a Bayesian optimizer. The first method uses an exact augmented Lagrangian and the second augments the minimization of the acquisition function to contain additional constraints. Both of these methods can be applied to nonlinear equality constraints, unlike most previous methods developed for constrained Bayesian optimizers. The new methods are applied with a gradient‐enhanced Bayesian optimizer and enable deeper convergence for three nonlinearly‐constrained unimodal optimization problems than previously developed methods for constrained Bayesian optimization. In addition, both new methods enable the Bayesian optimizer to reach a desired tolerance with fewer function evaluations than popular quasi‐Newton optimizers from SciPy and MATLAB for unimodal problems with 2 to 30 variables. The Bayesian optimizer had similar results using both methods. It is recommended that users first try using the second method, which adds constraints to the acquisition function minimization, since its parameters are more intuitive to tune for new problems.

With the sustainable development of the construction industry, recycled aggregate (RA) has been widely used in concrete preparation to reduce the environmental impact of construction waste. Compressive strength is an essential measure of the performance of recycled aggregate concrete (RAC). In order to understand the correspondence between relevant factors and the compressive strength of recycled concrete and accurately predict the compressive strength of RAC, this paper establishes a model for predicting the compressive strength of RAC using machine learning and hyperparameter optimization techniques. RAC experimental data from published literature as the dataset, extreme gradient boosting (XGBoost), random forest (RF), K-nearest neighbour (KNN), support vector machine regression Support Vector Regression (SVR), and gradient boosted decision tree (GBDT) RAC compressive strength prediction models were developed. The models were validated and compared using correlation coefficients (R2), Root Mean Square Error (RMSE), mean absolute error (MAE), and the gap between the experimental results of the predicted outcomes. In particular, The effects of different hyperparameter optimization techniques (Grid search, Random search, Bayesian optimization-Tree-structured Parzen Estimator, Bayesian optimization- Gaussian Process Regression) on model prediction efficiency and prediction accuracy were investigated. The results show that the optimal combination of hyperparameters can be searched in the shortest time using the Bayesian optimization algorithm based on TPE (Tree-structured Parzen Estimator); the BO-TPE-GBDT RAC compressive strength prediction model has higher prediction accuracy and generalisation ability. This high-performance compressive strength prediction model provides a basis for RAC’s research and practice and a new way to predict the performance of RAC.

Materials exploration requires the optimization of a multidimensional space including the chemical composition and synthesis parameters such as temperature and pressure. Bayesian optimization has attracted attention as a method for efficient multidimensional optimization. Appropriate choices of the acquisition function and initial values of the hyperparameters of the kernel functions are essential for the Bayesian optimization of synthesis conditions in a small number of experiments. However, to date, there has been little discussion on how to tune Bayesian optimization for materials exploration, and no guidelines have been provided for materials scientists. In this study, we investigated the optimum initial values of the hyperparameters in Bayesian optimization using one-dimensional model functions that mimic actual materials syntheses. The optimal lengthscale and variance for different process windows of materials synthesis were investigated. It was shown that the use of an appropriate acquisition function and suitable initial values of the hyperparameters of the kernel functions enable the optimization of synthesis conditions in a small number of trials. These results provide insight for enabling fully automated and autonomous materials synthesis using Bayesian optimization and robotics for materials exploration.

Bayesian Optimization for Field-Scale Geological Carbon Storage

Rather than optimizing expensive objective functions such as complex engineering simulations directly, Bayesian optimization methodologies fit a surrogate model (typically Kriging or a Gaussian Process) on evaluations of the objective function(s). To determine the next evaluation, an acquisition function is optimized (also referred to as infill criterion or sampling policy) which incorporates the model prediction and uncertainty and balances exploration and exploitation. Therefore, Bayesian optimization methodologies replace a single optimization of the objective function by a sequence of optimization problems: this makes sense as the acquisition function is cheap-to-evaluate whereas the objective is not. Depending on the goal different acquisition functions are available: multi-objective acquisition functions are relatively new and this chapter gives a state-of-the-art overview and illustrates some approaches based on hypervolume improvement. It is shown that the quality of the model is crucial for the performance of Bayesian optimization and illustrate this by using the more flexible Student-t processes as surrogate models.

Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model in a utility measure for each point to be evaluated. Considering such aspect, selection and design of acquisition functions are one of the most popular research topics in BO. Since no single acquisition function was proved to have better performance in all tasks, a well-established approach consists of selecting different acquisition functions along the iterations of a BO execution. In such an approach, the GP-Hedge algorithm is a widely used option given its simplicity and good performance. Despite its success in various applications, GP-Hedge shows an undesirable characteristic of accounting on all past performance measures of each acquisition function to select the next function to be used. In this case, good or bad values obtained in an initial iteration may impact the choice of the acquisition function for the rest of the algorithm. This fact may induce a dominant behavior of an acquisition function and impact the final performance of the method. Aiming to overcome such limitation, in this work we propose a variant of GP-Hedge, named No-PASt-BO, that reduce the influence of far past evaluations. Moreover, our method presents a built-in normalization that avoids the functions in the portfolio to have similar probabilities, thus improving the exploration. The obtained results on both synthetic and real-world optimization tasks indicate that No-PASt-BO presents competitive performance and always outperforms GP-Hedge.

The stable operation of strip rolling mill is the key factor to ensure the stability of product quality. The design capability of existing domestic imported and self-developed strip rolling mills cannot be fully developed, and the frequent occurrence of mill vibration and operation instability problems seriously restrict the equipment capacity and the production of high-end strip products. The vibration prediction analysis method for hot strip mill based on eXtreme gradient boosting (XGBoost) and Bayesian optimization (BO) is proposed. First, an XGBoost prediction model is developed based on a self-built data set to construct a complex functional relationship between process parameters and rolling mill vibration. Second, the important hyperparameters and parameters of XGBoost are optimized using Bayesian optimization algorithm to improve the prediction accuracy, computational efficiency, and stability of the model. Third, a comprehensive comparison is made between the prediction model in this paper and other well-known machine learning benchmark models. Finally, the prediction results of the model are interpreted using the SHapley Additive exPlanations (SHAP) method. The proposed model outperforms existing models in terms of prediction accuracy, computational speed and stability. At the same time, the degree of influence of each feature on rolling mill vibration is also obtained.

The modeling and optimization of electrospinning parameters are essential for controlling the fiber diameter and material properties. This study uses machine learning to examine the effects of multiple electrospinning parameters on fiber diameter. Ten regression models were evaluated, with hyperparameter optimization performed using grid search cross-validation and Bayesian optimization with multiple fold configurations. The Random Forest model demonstrated superior performance (root mean square error = 129.308, coefficient of determination = 0.542, mean absolute error = 104.014, mean absolute percentage error = 0.371). Further improvement was achieved through Bayesian optimization (root mean square error = 127.400, coefficient of determination = 0.555, mean absolute percentage error = 0.360). Extreme Gradient Boosting and Gradient Boosting also showed high accuracy, while linear models performed poorly. The Shapley Additive Explanations analysis identified rotational speed as the most influential parameter (value = 0.473), followed by flow rate (0.36), porosity (0.32) and needle diameter (0.27), all positively affecting fiber diameter. In contrast, voltage (-0.24), temperature (-0.19), towing (-0.14), and humidity (-0.13) showed negative impacts. Experimentally, Polycaprolactone (Molecular number = 80,000) nanofibers were manufactured at three rotation speeds (150, 450 and 750 revolutions per minute), resulting in fiber diameters of 100.09, 154.0, and 175.45 nanometers, respectively. These findings reveal complex interactions between the electrospinning parameters and the fiber morphology, demonstrating the potential of machine learning to optimize nanofiber production.

Cost estimation is a key component of project plans, yet it is challenging to provide reliable and efficient estimations using conventional methods in the conceptual phase of infrastructure projects. This study proposes a framework that integrates feature selection, extreme gradient boosting (XGBoost), Bayesian optimization (BO), and SHapley Additive exPlanations (SHAP) to provide conceptual cost estimations and explain the results for early decision-making. Correlation analysis and forward search are combined to select the key features. XGBoost is developed as the estimator and enhanced by BO in accuracy and efficiency. Model explanations were presented using SHAP. The framework is demonstrated through a case study of electric substations containing 605 samples. The results show that the proposed framework can provide satisfactory performance on conceptual cost estimations, where BO-XGBoost outperforms the benchmark models (with R 2 ~0.9567, adjusted R 2 ~0.9549, RMSE ~ 0.8690, and MAE ~ 0.4875). SHAP reveals how the features contribute to the cost based on both global and local explanations. The framework provides a guideline for more accurate, efficient, and explainable cost estimations in the conceptual phase of infrastructure projects. It can support the government and project planners in early decision-making, including reliable project budget and plan alternatives selection.

Prediction of the jump height of transmission lines after ice-shedding based on XGBoost and Bayesian optimization

Predicting the adhesive force between steel reinforcement and concrete is crucial as it influences stress distribution and the overall mechanical behavior of reinforced concrete. This study proposes a novel approach to enhance bond strength prediction using machine learning (ML) models optimized through Bayesian optimization (BO). A dataset comprising 401 beam tests with six key factors was used to train three distinct ML algorithms—Support Vector Regression (SVR), Random Forest (RF), and Extreme Gradient Boosting (XGBoost). The prediction models were first trained on the full dataset, with BO applied to fine-tune hyperparameters and improve accuracy. Among these models, the BO-XGBoost achieved the best performance, with an R2 of 0.74, MAE of 1.412 MPa, and RMSE of 1.516 MPa on the test set, and R2 = 0.80, MAE = 0.950 MPa, RMSE = 1.200 MPa on the training set. In addition, a simplified model was developed, incorporating only three critical variables—rebar thickness, reinforcement tensile strength, and concrete compressive capacity—to make the model more applicable in real-world engineering scenarios. To further interpret the model’s predictions, Shapley additive explanations (SHAP) were employed, revealing the specific influence of each variable on bond strength. This study demonstrates that the integration of ML with Bayesian optimization can significantly improve the accuracy of bond strength predictions, offering valuable insights for structural design optimization.

A novel prediction model for the resilient modulus of unbound aggregate bases using eXtreme Gradient Boosting (XGBoost) and Bayesian optimisation (BO) is presented in this study. First, a local dataset containing 260 samples of some common tests, including sieve analysis, Atterberg limits tests, compaction tests, and repeated load triaxial tests of unbound aggregate materials, is collected. Then, the proposed BO-XGBoost model is trained on the collected dataset. Multiple evaluation metrics, including Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and coefficient of determination (R2), were employed to evaluate the performance of the BO-XGBoost model. The model's accuracy and generalisation capabilities were evaluated by computing these metrics for both the training and testing datasets. The findings of this study, clearly showed that the combined BO-XGBoost model outperformed other state-of-the-art machine learning or even deep learning models, in predicting the resilient modulus of unbound aggregate base. Finally, the permutation method was used to conduct a feature importance analysis and determine the impact of different input features on the resilient modulus of the unbound aggregate base. This latter was highly influenced by stress, moisture content, and liquidity limit, as demonstrated by the results of the feature importance analysis.