Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios

Bayesian optimization (BO) is a versatile and robust global optimization method under uncertainty. However, most of the BO algorithms were developed for problems with only continuous variables. For practical engineering optimization, discrete variables are also prevalent. BO methods based on Gaussian process (GP) surrogates also suffers from the curse-of-dimensionality problem. To address these challenges, in this paper, a Bayesian discrete optimization algorithm is introduced to solve permutation-based combinatorial problems. A new kernel function is developed based on position distances for permutation. To improve the efficiency and scalability of the algorithm, a sparse GP model based on inducing points is further developed, where the simulated annealing algorithm is applied to select inducing points. The new algorithm is demonstrated and tested with the production scheduling problem for additive manufacturing. Experimental results show that the proposed algorithm can find a better solution with limited evaluations than state-of-the-art algorithms.

To address the problems of local optima and insufficient convergence accuracy in parameter identification of primary frequency regulation (PFR) for steam turbines, this paper proposed a hybrid identification method that integrated an Improved Bayesian Optimization (IBO) algorithm and an Improved Whale Optimization Algorithm (IWOA). By initializing the Bayesian parameter population using Tent chaotic mapping and the reverse learning strategy, employing a radial basis kernel function hyperparameter training mechanism based on the Adam optimizer and optimizing the Expected Improvement (EI) function using the Limited-memory Broyden–Fletcher– Goldfarb–Shanno with Bounds (L-BFGS-B) method, IBO was proposed to obtain the optimal candidate set with the smallest objective function value. By introducing a nonlinear convergence factor and the adaptive Levy flight perturbation strategy, IWOA was proposed to obtain locally optimized optimal solutions. By using the reverse-guided optimization mechanism and employing a fitness-oriented selection strategy, the optimal solution was chosen to complete the closed-loop process of reverse learning feedback. Nine standard test functions and the Proportional Integral Derivative (PID) parameter identification of the electro-hydraulic servo system in a 330 MW steam turbine were presented as examples. Compared with Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), Bayesian Optimization (BO) and Particle Swarm Optimization-Grey Wolf Optimizer (PSO-GWO), the Improved Bayesian Optimization-Whale Optimization Algorithm (IBO-WOA) proposed in this paper has been validated to effectively avoid the problem of getting stuck in local optima during complex optimization and has high parameter recognition accuracy. Meanwhile, an Out-Of-Distribution (OOD) Test based on noise injection had demonstrated that IBO-WOA had good robustness. The time constant identification of the steam turbine were carried out using IBO-WOA under two experimental conditions, and the identification results were input into the PFR model. The simulated power curve can track the experimental measured curve well, proving that the parameter identification results obtained by IBO-WOA have high accuracy and can be used for the modeling and response characteristic analysis of the steam turbine PFR.

Financial experts seek to predict the variability of financial markets to ensure investors’ successful investments. However, there has been a big trend in finance in the last few years, which are the ESG (Economic, Social and Governance) criteria, due to the growing importance of investments being socially responsible, and because of the financial impact companies suffer when not complying with them. Consequently, creating a stock portfolio should consider not only its financial performance but compliance with ESG criteria. Portfolio optimization (PO) techniques previously applied to ESG portfolios, are all closed-form analytical ones. But the real world is rather a black box with unknown analytical expressions. Thus, in this paper we use Bayesian optimization (BO), a sequential state-of-the-art design strategy to optimize black-boxes with unknown analytical and costly-to-compute expressions, to maximize the performance of a stock portfolio under the presence of ESG criteria soft constraints incorporated into the objective function. And we compare it to two other black-box techniques widely applied for the optimization of ‘conventional portfolios’ (non-ESG ones): the metaheuristics Genetic algorithm (GA) and Simulated Annealing (SA). Although BO has many theoretical advantages over GA and SA, it has never been applied to PO. Thus, this paper investigates whether BO can be used in the ESG PO framework as an alternative and compares it with GA and SA. This is the research gap to which this paper responds. To show the empirical performance of BO, we carry out four illustrative experiments and find evidence of BO outperforming the baselines. Thus we add another different optimization approach to the world of ESG investing: a black-box non-heuristic optimization approach through BO. Our study is the first paper that leverages BO and ESG scores into a PO technique. This paper opens the door to many new research lines in (ESG) portfolio optimization.

Electrochemical machining is a promising non-traditional manufacturing process to make high-quality parts. The benefits of minimal thermally and mechanically induced stresses, free of burr, and a low surface roughness are appealing for industry and research institutes. However, the combined chemical reaction, electric field, fluid mechanics, and material properties involve a significant number of independent parameters which are difficult to analyze in order to draw comprehensive conclusions. To our current knowledge, process responses such as the material removal rate, optimal feed rate, and cutting profile cannot be represented accurately by analytical solutions. In recent years, deep learning has had tremendous success in analyzing sophisticated systems. The improved computation efficiency and reduced size of the training dataset required for deep learning have enabled various prediction models in the manufacturing industry. In this paper, a new approach is developed using the deep convolutional network with the Bayesian optimization algorithm to predict the diameters of the drilled hole from an electrochemical machining process. The Keras application programming interface (API) was used to build the deep convolutional network; the feed rate, pulse-on time, and voltage were used as input parameters to provide a fair comparison with a neural network from previous research. Random dropout layers were added to prevent overfitting of the network. Instead of tuning the network parameter by trial and error, the Bayesian parameter optimization algorithm was implemented to find the optimal set of parameters of the deep convolutional network that yields the minimum mean square error. The proposed algorithm was compared with a previously developed neural network with partially embedded physical knowledge. Improved training speed and accuracy were observed in comparison with the traditional neural network. The prediction model using the proposed deep learning algorithm demonstrated better prediction accuracy and provided a more systematic way to select the hyperparameter for the deep convolutional network.

In this study, an efficient optimization framework was developed to determine the parameters of through‐silicon vias and the layout of heat sources in three‐dimensional integrated circuits (3D ICs), employing an artificial neural network (ANN) reduced‐order model in conjunction with a Bayesian optimization (BO) algorithm. The proposed method effectively predicts the temperature distribution in 3D ICs and refines their thermal parameters, offering solutions to thermal management challenges. Latin hypercube sampling was utilized for data sampling, enhancing the previously established rapid thermal analysis method through parameterization of heat source locations. The temperature distribution data for varying hotspot locations in 3D ICs were fitted using an appropriately defined objective function, leading to the development of a reduced‐order ANN model that accelerates temperature prediction. The computational results demonstrate that the neural network model exhibits a deviation in predicted values of less than 2%, and the coefficient of determination R2 approximately 0.93, underscoring high predictive accuracy. Additionally, the optimization outcomes and the efficiency of the selected BO algorithm were thoroughly evaluated. Notably, the BO algorithm achieved the global optimum in just 4.07 s across 250 iterations, demonstrating an effective power distribution strategy for the 3D ICs model.

This study proposes an investment portfolio optimization system based on the AutoML framework, which integrates Bayesian hyperparameter optimization and machine learning techniques to achieve accurate prediction of financial market volatility and dynamic asset allocation. The system first extracts multi-scale features (including short-term, medium-term, and long-term returns, volatility, trading volume, and momentum indicators) based on historical stock price data, and uses a Bayesian optimized support vector regression (SVR) model to predict the volatility for the next 20 days; At the asset allocation level, the system dynamically adjusts the weights of the investment portfolio based on the predicted volatility, controlling the target volatility at 20% while constraining the leverage ratio within the range of 0-150%. The empirical results show that: 1) The volatility prediction model has excellent performance.

We propose a highly efficient searching algorithm in crystal structure prediction. The searching algorithm is a hybrid of the evolutionary algorithm and Bayesian optimization. The evolutionary algorithm is widely used in crystal structure prediction, and the Bayesian optimization is one of the selection-type algorithms we have developed. We have performed simulations of crystal structure prediction to compare the success rates of the random search, evolutionary algorithm, Bayesian optimization, and hybrid algorithm for up to ternary systems such as Si, Y2Co17, Al2O3, and CuGaS2, using the CrySPY code. These results demonstrate that the evolutionary algorithm can generate structures more efficiently than random structure generation, and the Bayesian optimization can efficiently select potential candidates from a large number of structures. Moreover, the hybrid algorithm, which has the advantages of both, is proved to be the most efficient searching algorithm among them.

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this article attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization that are mainly based on Gaussian processes and identify challenging open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.

<div>With the rapid development of autonomous driving technology, unmanned ground vehicles (UGVs) are gradually replacing humans to perform tasks such as reconnaissance, target tracking, and search in special scenarios. Omnidirectional mobility based on rapid adjustment of vehicle heading posture enhances the applicability of UGVs in specialized scenarios. Omnidirectional mobility signifies the capability for rapid adjustments to the vehicle’s heading angle, longitudinal velocity, and lateral velocity. Traditional vehicles are constrained by the limitations of under-actuation, which prevents active regulation of lateral movement. Instead, they rely on the coordinated regulation of longitudinal and yaw movements, failing to meet the requirements for omnidirectional mobility. Distributed vehicles featuring steering distributed between the front/rear axles and four-wheel independent drive leverage the over-actuation advantages provided by multi-actuator coordinated control, making them particularly suitable for omnidirectional mobility at large sideslip angles. This feature enables the UGVs to achieve rapid adjustment of vehicle heading posture. However, existing control strategies centered on stabilizing yaw rate and suppressing sideslip angles cannot adapt to the decoupling control requirements of such platforms. Additionally, the strong coupling characteristics between actuator subsystems further exacerbate control difficulties. To this end, this article proposes a full-state decoupling motion control strategy, the nonlinear model is locally linearized at each equilibrium point of the vehicle, and a set of equilibrium state models is derived. The validity of this local linearization method is verified through phase diagram analysis and modal analysis. The Bayesian optimization (BO) algorithm is then employed to optimize and identify the cornering stiffness of the front/rear axles at each equilibrium point in these locally linearized models, thereby enhancing the characterization ability of the linear model for the nonlinear dynamic model at the corresponding equilibrium points. Subsequently, a full-state decoupling motion controller is designed by integrating the model predictive control (MPC) algorithm. Finally, the controller presented in this article is employed on the distributed vehicle experiment platform (DVEP). The experimental results demonstrate that in two drift-like scenarios with different sideslip angles, compared with the baseline controller, the path tracking error of this method is reduced by more than 13%, and the sideslip angle tracking error is reduced by more than 12%.</div>

Load forecasting impacts directly financial returns and information in electrical systems planning. A promising approach to load forecasting is the Echo State Network (ESN), a recurrent neural network for the processing of temporal dependencies. The low computational cost and powerful performance of ESN make it widely used in a range of applications including forecasting tasks and nonlinear modeling. This paper presents a Bayesian optimization algorithm (BOA) of ESN hyperparameters in load forecasting with its main contributions including helping the selection of optimization algorithms for tuning ESN to solve real-world forecasting problems, as well as the evaluation of the performance of Bayesian optimization with different acquisition function settings. For this purpose, the ESN hyperparameters were set as variables to be optimized. Then, the adopted BOA employs a probabilist model using Gaussian process to find the best set of ESN hyperparameters using three different options of acquisition function and a surrogate utility function. Finally, the optimized hyperparameters are used by the ESN for predictions. Two datasets have been used to test the effectiveness of the proposed forecasting ESN model using BOA approaches, one from Poland and another from Brazil. The results of optimization statistics, convergence curves, execution time profile, and the hyperparameters’ best solution frequencies indicate that each problem requires a different setting for the BOA. Simulation results are promising in terms of short-term load forecasting quality and low error predictions may be achieved, given the correct options settings are used. Furthermore, since there is not an optimal global optimization solution known for real-world problems, correlations among certain values of hyperparameters are useful to guide the selection of such a solution.

In the dynamic landscape of financial markets, the pivotal role of adept portfolio construction is more prominent than ever. By enabling investors to form combinations of assets, it not only aids in accomplishing specified investment objectives but also in diversifying potential risks. This process employs a systematic investment approach, fostering informed and judicious decision-making strategies in the financial domain. This research seeks to explore the application of Bayesian statistics in parameter estimation and portfolio optimization, particularly focusing on the volatility of the technology sector. Utilizing a conjugate prior to obtaining the posterior and employing Maximum Likelihood Estimation (MLE) to calculate the expected return and covariance matrix, the study constructed portfolio weights based on Mean-Variance Optimization. This portfolio was then compared with weights derived from historical data analysis and a Monte Carlo simulation optimized by the Sharpe ratio. Remarkably, upon empirical testing on an out-of-sample dataset, the Bayesian approach exhibited a superior performance, showcasing robust resilience to potential risks. The study also acknowledges several limitations, suggesting directions for future research to refine the precision and reliability of portfolio optimization in the face of market volatility.

In recent decades, various software effort estimation (SEE) algorithms have been suggested. Unfortunately, generating high-precision accuracy is still a major challenge in the context of SEE. The use of traditional techniques and parametric approaches is largely inaccurate because they produce biased and subjective accuracy. Meanwhile, none of the machine learning methods performed well. This study applies the AdaBoost ensemble learning method and random forest (RF), on the other hand the Bayesian optimization method is applied to determine the hyperparameters of this model. The PROMISE repository and the ISBSG dataset were used to build the SEE model. The developed model was comprehensively compared with four machine learning methods (classification and regression tree, k-nearest neighbor, multilayer perceptron, and support vector regression) under 3-fold cross validation (CV). It can be seen that the RF method based on AdaBoost ensemble learning and bayesian optimization outperforms this approach. In addition, the AdaBoost-based model assigns a feature importance rating, which makes it a promising tool in software effort prediction.

The paper presents a novel approach to applying Bayesian Optimization (BO) in predicting an unknown constraint boundary, also representing the discontinuity of an unknown function, for a feasibility check on the design space, thereby representing a classification tool to discern between a feasible and infeasible region. Bayesian optimization is a low-cost black-box global optimization tool in the Sequential Design Methods where one learns and updates knowledge from prior evaluated designs, and proceeds to the selection of new designs for future evaluation. However, BO is best suited to problems with the assumption of a continuous objective function and does not guarantee true convergence when having a discontinuous design space. This is because of the insufficient knowledge of the BO about the nature of the discontinuity of the unknown true function. In this paper, we have proposed to predict the location of the discontinuity using a BO algorithm on an artificially projected continuous design space from the original discontinuous design space. The proposed approach has been implemented in a thin tube design with the risk of creep-fatigue failure under constant loading of temperature and pressure. The stated risk depends on the location of the designs in terms of safe and unsafe regions, where the discontinuities lie at the transition between those regions; therefore, the discontinuity has also been treated as an unknown creep-fatigue failure constraint. The proposed BO algorithm has been trained to maximize sampling toward the unknown transition region, to act as a high accuracy classifier between safe and unsafe designs with minimal training cost. The converged solution has been validated for different design parameters with classification error rate and function evaluations at an average of <1% and ∼150, respectively. Finally, the performance of our proposed approach in terms of training cost and classification accuracy of thin tube design is shown to be better than the existing machine learning (ML) algorithms such as Support Vector Machine (SVM), Random Forest (RF), and Boosting.

A two-stage online remaining useful life prediction framework for supercapacitors based on the fusion of deep learning network and state estimation algorithm

Developing an excellent quantitative trading strategy to obtain a high Sharpe ratio requires optimizing several parameters at the same time. Example parameters include the window length of a moving average sequence, the choice of trading instruments, and the thresholds used to generate trading signals. Simultaneously optimizing all these parameters to seek a high Sharpe ratio is a daunting and time-consuming task, partly because of the unknown mechanism determining the Sharpe ratio. This article proposes using Bayesian optimization to systematically search for the optimal parameter configuration that leads to a high Sharpe ratio. The author shows that the proposed intelligent search strategy performs better than manual search, a common practice that proves to be inefficient. The author’s framework also can easily be extended to other parameter selection tasks in portfolio optimization and risk management.