An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization

The traditional multi-objective efficient global optimization (EGO) algorithms have been hybridized and adapted to solving the expensive aerodynamic shape optimization problems based on high-fidelity numerical simulations. Although the traditional EGO algorithms are highly efficient in solving some of the optimization problems with very complex landscape, it is not preferred to solve most of the aerodynamic shape optimization problems with relatively low-degree multi-modal design spaces. A new infill criterion encouraging more local exploitation has been proposed by hybridizing two traditional multi-objective expected improvements (EIs), namely, statistical multi-objective EI and expected hypervolume improvement, in order to improve their robustness and efficiency in aerodynamic shape optimization. Different analytical test problems and aerodynamic shape optimization problems have been investigated. In comparison with traditional multi-objective EI algorithms and a standard evolutionary multi-objective optimization algorithm, the proposed method is shown to be more robust and efficient in the tests due to its hybrid characteristics, easier handling of sub-optimization problems, and enhanced exploitation capability.

Material parameters identification by inverse analysis using finite element computations leads to the resolution of complex and time-consuming optimization problems. One way to deal with these complex problems is to use meta-models to limit the number of objective function computations. In this paper, the Efficient Global Optimization (EGO) algorithm is used. The EGO algorithm is applied to specific objective functions, which are representative of material parameters identification issues. Isotropic and anisotropic correlation functions are tested. For anisotropic correlation functions, it leads to a significant reduction of the computation time. Besides, they appear to be a good way to deal with the weak sensitivity of the parameters. In order to decrease the computation time, a parallel strategy is defined. It relies on a virtual enrichment of the meta-model, in order to compute q new objective functions in a parallel environment. Different methods of choosing the qnew objective functions are presented and compared. Speed-up tests show that Kriging Believer (KB) and minimum Constant Liar (CLmin) enrichments are suitable methods for this parallel EGO (EGO-p) algorithm. However, it must be noted that the most interesting speed-ups are observed for a small number of objective functions computed in parallel. Finally, the algorithm is successfully tested on a real parameters identification problem.

Both evolutionary Bayesian optimization (EBO) and Bayesian evolutionary optimization (BEO) are used to solve time-consuming optimization problems, where in the EBO method, the evolutionary algorithm is adopted to optimize the acquisition function for finding its optimal solution to be evaluated using the time consuming objective functions, while in the BEO approach, surrogates are utilized to replace the expensive objective functions and the acquisition function is used in the model management to choose individuals for evaluations using the expensive objective functions. In this paper, we perform comparisons on these two approaches to see which method is better for solving computationally expensive multi-objective problems. The expected improvement (EI) is used as the acquisition function in both EBO and BEO approaches. Two strategies are proposed to select individuals to be evaluated using the real computationally expensive objectives. The experimental results conducted on ZDT and DTLZ test problems showed that the BEO approach is more efficient than the EBO approach for solving multi-objective problems.

A surrogate-assisted multi-objective evolutionary algorithm with dimension-reduction for production optimization

Surrogate-based method can dramatically reduce the number of expensive function evaluations in real-world multi- objective optimization problems (MOP). When the number of objectives is small, using surrogate models combined with expected hypervolume improvement (EHVI) infill sampling criterion (ISC) has been proved to be efficient to provide a set of solutions with good diversity and good proximity to the Pareto front (PF) in aerodynamic shape optimization. However, traditional hypervolume-based infilling strategies use only one kind of ISC to generate one or multiple sample points, the advantages of various kinds of ISC cannot be comprehensively utilized and the parallelization is not easy to implement. This paper proposes a combined multi-point infilling strategy based on Kriging models and develops an efficient global multi-objective constrained optimization method (EGMOCO) to solve multi- objective aerodynamic shape optimization with complex constraints. Multiple sample points are generated by using four ISC considering hypervolume at each iteration and then evaluated in parallel. Firstly, the performance of EGMOCO is compared with that of single criterion EHVI strategy on six benchmarks within the same computational budget to prove its effectiveness, and then EGMOCO is implemented in an aerodynamic shape optimization problem with complex constraints. The result shows that EGMOCO has good performance in balancing local exploitation and global exploration with faster convergence rate and high robustness, the whole PF can be fully explored in limited evaluations and the constraint handling is effective especially for real-world problems with complex and nonlinear constraints, the comprehensive aerodynamic performance of the airfoil is greatly improved. It can be confirmed that Kriging-based multi-objective optimization method combined with multi-point infilling strategy performs better than single infilling criterion EHVI, since different sample infilling criteria can complement with each other, both local exploitation and global exploration can be considered and well balanced.

The profile and axial designs of a volute tongue have great influences on the internal flow conditions and aerodynamic performance of a forward-curved blade centrifugal fan. In order to achieve a high quality of internal flow fields and fan aerodynamic performance, we propose an inclined bionic volute tongue (IBVT) described with rotation angle and scale coefficient at the front and back walls of the volute, which varies the profile of the volute tongue and the distance between the impeller and the volute tongue in the axial direction. Meanwhile, an adaptive pseudo expected improvement matrix (APEIM) parallel multi-objective efficient global optimization (EGO) algorithm is developed by introducing a flexible number of added points for each iteration of optimization on account of the optimization problem itself. On this basis, IBVT is optimized to maximize the fan aerodynamic performance in terms of total pressure efficiency and static pressure rise by virtue of numerical simulation. The results show that the developed APEIM parallel multi-objective EGO algorithm is capable of accurately and efficiently finding the Pareto front solutions. IBVT is considered to have the effectiveness of aerodynamic performance improvement for a forward-curved blade centrifugal fan. Within the fan with IBVT, the flow fields in the axial direction become more uniform with the dimension reduction of the inactive zone around the front span. The impeller-volute tongue interaction is greatly improved with the decreases of reversed and recirculated flow coefficients. IBVT also benefits the reduction of flow losses in the volute. By adopting the analysis of variance (ANOVA), the aerodynamic performance of the fan is highly affected by the rotation angle and scale coefficient of IBVT at the front wall of the volute. This work is helpful to guide the design of the volute tongue for a forward-curved blade centrifugal fan.

Machine learning algorithms often have many hyper-parameters that can be tuned to improve empirical performance. However, manually exploring the complex search spaces is tedious and cannot guarantee to find satisfactory outcomes. Recently, the Efficient Global Optimization (EGO) algorithm for solving the hyper-parameter optimization problem have shown substantial improvements. However, these algorithms cannot easily be expanded to include the selection of an algorithm. Automatically determining and optimizing an algorithm is known as the combined algorithm selection and hyper-parameter optimization (CASH) problem. In this paper, a novel mixed integer efficient global optimization algorithm and its variants are proposed to solve the CASH problem efficiently. The proposed algorithm is compared with a wide set of state-of-the-art algorithms in both a Black Box Optimization and a CASH problem setting. For the CASH problem setup, seven machine learning algorithms are optimized for a large set of classification t asks. Results show that the proposed algorithms outperform on the black box optimization task and can also outperform alternative state-of-the-art approaches on the CASH problem for specific instances.

Antenna structure design constitutes a computationally expensive optimization problem due to the requirement for full-wave electromagnetic (EM) simulations. Surrogate-assisted evolutionary algorithms offer a promising approach for addressing such challenges. However, several challenges remain in solving expensive, highly constrained antenna design problems. This paper introduces a surrogate-assisted dynamic constrained multi-objective evolutionary algorithm framework to tackle expensive and highly constrained antenna design optimization tasks. A multi-layer perceptron (MLP) is employed as the surrogate model to approximate EM evaluations and alleviate the computational burden, while a dynamic scale-constrained boundary strategy is implemented to handle highly constraints. The effectiveness of the proposed method is validated on a set of constrained benchmark problems and two antenna design cases.

In multiobjective optimization, the R2 indicator is widely used for designing the indicator-based algorithms, and the Tchebycheff approach is commonly employed in the decomposition-based algorithms. Despite their wide use, the connection between these two different paradigms is still not well understood, particularly in the field of multiobjective efficient global optimization (MOEGO). Considering that expected improvement (EI) is a cornerstone in efficient global optimization (EGO), this article first studies the relationship between R2-based EI and Tchebycheff-based EI. Then, we introduce a many-to-few (M2F) decomposition framework, offering a new perspective for linking the R2-based method and the Tchebycheff decomposition approach. By incorporating M2F decomposition into MOEGO, a new algorithm called R2/D-EGO is proposed. At each iteration, R2/D-EGO utilizes the Tchebycheff decomposition paradigm to generate a set of candidate solutions, each one corresponding to a different weight vector. Subsequently, a subset of query points is selected from the candidates based on the lower bound of R2-based EI. Empirical results indicate that the proposed R2/D-EGO is highly competitive in comparison with both the R2-based and decomposition-based MOEGO algorithms in the parallel (or batch) setting.

In the aircraft design methodology, there are many methods to measuring the UAVs performance, such as, the minimize drag, the lift required or the take-off/landing performance etc. The multi-objective optimization is one of the popular method for UAVs design problem. In addition, in the UAVs design problem is required for high computational cost such as computation of fluid dynamics. The solution to the said problem could be reached through the method of the Efficient Global Optimization algorithm (or abbreviated, the EGO). However, the EGO was, in the first instant, intended for very limited use; namely, it was utilized as a solution for a single-objective optimization problem with just one additional sampling. Then, the EGO method must be required for long computational time for single-additional sampling procedure. The objective of this particular research was to study the EGO with multi-objective multi-additional sampling as a solution to the UAVs airfoil design problem. The Expected Hypervolume Improvement (EHVI) is applied with the EGO process with an intention to find a solution to the multi-objective optimization problem. Furthermore, there was a proposal to use multiple additional sampling methods in the efficiency improvement of the additional sampling process in EGO, and at the same time, keeping the performance of exploration based on EHVI maximization at the same constant There are two main goals in the application of this algorithm to UAVs airfoil design optimization, which include minimizing aerodynamic drag and maximizing UAVs airfoil thickness at the trailing edge. The Reynolds-averaged Navier-Stokes simulation is applied for aerodynamic evaluation. By adopting the airfoil design, the results were the reduction in the aerodynamic drag, as well as 5% improvement of the thickness of the airfoil at the trailing edge when compared with the airfoil initial design.

Evolutionary algorithms have been widely used to tackle multi-objective optimization problems. Despite many successful applications there is a lack of theoretical knowledge on how and why this type of algorithm works particularly well for multi-objective problems. This special issue presents current advances to the theoretical understanding of evolutionary algorithms for multi-objective optimization. It comprises four very high quality papers which have been selected via a rigorous reviewing process. The first paper “On the Effect of Populations in Evolutionary Multi-Objective Optimisation” by Oliver Giel and Per Kristian Lehre studies the basic question whether a population is provably useful to compute the whole Pareto front of a multi-objective optimization problem. They compare restart strategies of several single-objective approaches based on one individual to a natural multi-objective approach using a population. Their analyses point out the benefits of the multi-objective evolutionary algorithm in a rigorous way. Using illustrative example functions they provide not only proven results but also a good intuition about why population-based approaches can outperform individual-based approaches in multi-objective optimization. The paper “Exploring the Runtime of an Evolutionary Algorithm for the Multi-Objective Shortest Path Problem” by Christian Horoba examines the behavior of evolutionary algorithms for the classical NP-hard multi-objective shortest path problem. The author considers an evolutionary approach based on -dominance and proves that this algorithm constitutes a fully polynomial-time randomized approximation scheme (FPRAS) for the problem. The algorithm achieves this by mimicking a problemspecific algorithm for that problem. This provides general insight into why evolutionary algorithms can perform well for problems that allow for efficient problem-specific algorithms. Hypervolume-based algorithms have become very popular in evolutionary multiobjective optimization. In the paper, “An Efficient Algorithm for Computing Hypervolume Contributions,” Karl Bringmann and Tobias Friedrich present a faster algorithm for determining the individual with the smallest hypervolume contribution in a given population. Furthermore, they study the problem of removing λ > 1 individuals with the smallest hypervolume contribution from a given population and present an elegant and fast method to carry out this task.

Designing and understanding the mechanism of non-stoichiometric materials with enhanced properties is challenging, both experimentally and even computationally, due to the large number of chemical spaces and their distributions through the material. In the current work, it is proposed a Machine Learning approach coupled with the Efficient Global Optimization (EGO) method-an Adaptive Design (AD)-to model local defects in materials from first-principle calculations. Our method takes into account the smallest sample set as possible, envisioning the material defect structure relationship with target properties for new insights. As an example, the AD framework allows us to study the stability and the structure of the modified goethite (Fe0.875Al0.125OOH) by considering a proper defect distribution, from first-principle calculations. The chemical space search for the modified goethite was evaluated by starting from different sizes and configurations of the samples as well as different surrogate models (ANN and Gaussian Process; GP), acquisition functions, and descriptors. Our results show that the same local solution of several defect arrangements in Fe0.875Al0.125OOH is found regardless of the initial sample and regression model. This indicates the efficiency of our search method. We also discuss the role of the descriptors in the accelerated global search for defects in material modeling. We conclude that the AD method applied in material defects is a successful approach in automating the search within huge chemical spaces from first-principle calculations by considering small samples. This method can be applied to mechanistic elucidation of non-stoichiometric materials, solid solutions, alloys, and Schottky and Frenkel defects, essential for material design and discovery. Graphical abstract.

Multi-objective optimization algorithms aim to identify a set of non-dominated solutions that capture the complex and often conflicting trade-offs between objectives. In black-box settings, where the input-output relationship is usually analytically intractable, these algorithms face two major challenges: limited evaluation budgets, especially when objective evaluations are costly, and computational difficulties in proposing new candidates. While all non-dominated solutions hold equal mathematical value, the Decision-Maker is typically interested in only the most preferred trade-offs. This adds an additional challenge for preference-driven optimization: the cost of human interaction. To address these challenges, this paper proposes leveraging surrogate-assisted evolutionary multi-objective optimization methods, combined with advanced discretization techniques and strategic filtering of preferred solutions to mitigate the overall computational burden. After evaluating against similar methods in the literature, the experimental results suggest the method to be competitive in enhancing both the efficiency and precision of the decision-making process.

Despite the success of Evolutionary Algorithms in solving complex problems, they may require many function evaluations. This becomes an issue when dealing with costly problems. Surrogate models may overcome this difficulty, though their use in problems with medium to large dimensionality is underexplored in the literature. Problems with multiple conflicting objectives can be formulated as Multi-objective Optimization Problems (MOPs). MOPs have received a great attention lately, mainly with Multi-objective Evolutionary Algorithms (MOEAs). This paper proposes ELMOEA/D-DE, a surrogate-assisted MOEA, for solving expensive MOPs in small evaluation budgets. ELMOEA/D-DE encompasses a state-of-the-art MOEA based on decomposition, Differential Evolution (DE) operators and Extreme Learning Machines. This paper tests three variants of ELMOEA/D-DE, using different DE operators, for solving five known benchmark MOPs with 10 to 60 decision variables. All variants achieve good results in terms of hyper volume metric and the best variant with operator DE/rand/1/bin is compared with two state-of-the-art approaches (MOEA/D-RBF and a non-surrogate-based MOEA), achieving the best results in all but one problems instances.

In a real life situation, due to the complexity of the industrial systems and non-linearity of their behavior, it is very difficult to achieve optimum performance of system for desired industrial goals using uncertain, vague and imprecise data. Herein, an approach has been proposed through which the behavior of the system is analyzed in the form of well- known six reliability indices by using triangular fuzzy numbers which allow the consideration of expert opinions, linguistic variables, operating conditions in reliability information. Using their behavior analysis a fuzzy multi-objective optimization problem (FMOOP) has been formulated. Due to the conflicting nature of the multiple objectives, the decision making is difficult and it leads to the Pareto optimal solutions instead of single optimal solutions. Many evolutionary algorithms (EAs) already exist in the literature for solving a multi-objective optimization problem (MOOP), and are termed as multi-objective evolutionary algorithms (MOEAs). Particle swarm optimization (PSO) is one of such MOEA which demonstrates the ability to identity a Pareto-optimal front efficiently. Here, a crisp optimization problem is reformulated from FMOOP by taking into account the preference of decision maker (DM) and then PSO is applied to solve the resulting fuzzified MOOP. The presented approach is applied in order to solve the multi-objective series-parallel system reliability optimization problem for a crystallization unit of a urea plant.