An Improved Blind Kriging Surrogate Model for Design Optimization Problems

To improve the efficiency and quality of solving expensive engineering design optimization problems, recently metamodel-based design and optimization technologies have been widely employed. In this paper, a novel adaptive metamodel-based optimization method called sequential radial basis function using support vector machine is proposed to solve the practical engineering optimization problems involving computationally expensive objective and constraints. The proposed metamodel-based optimization method uses the radial basis function metamodel to approximate the expensive simulations in the interesting sampling region, which is a subregion of the design space. To identify the interesting sampling region, a novel feature space fuzzy -means clustering method is proposed to obtain the cluster center of superior cheap points classified by support vector machine. The procedure of sequential radial basis function using support vector machine is presented first, and the interesting sampling region and feature space fuzzy -means clustering methods are discussed in detail. Several numerical benchmark problems are used to compare the proposed method with other well-known metamodel-based design and optimization methods including efficient global optimization, multiple surrogate efficient global optimization methodology, mode-pursuing sampling, constraint importance mode pursuing sampling, and constrained optimization by radial basis function interpolation. The comparison results show that the proposed method generally outperforms the competitive methods in terms of efficiency, convergence, and robustness. Finally, sequential radial basis function using support vector machine is applied in an Earth observation satellite multidisciplinary design optimization problem. The results illustrate that compared with genetic algorithm and sequential quadratic programming, the proposed method can find a better solution with fewer function evaluations, which demonstrates the practicality and effectiveness of the proposed method in solving real-world engineering design optimization problems.

In this paper, a surrogate-assisted differential evolution (DE) algorithm is proposed to solve the computationally expensive optimization problems. In it, the Kriging model is used to approximate the objective function, while DE employs a mechanism to dynamically select the best performing combinations of parameters (amplification factor, crossover rate and population size). The performance of the algorithm is tested on the WCCI2014 competition on expensive single objective optimization problems. The experimental results demonstrate that the proposed algorithm has the ability to obtain good solutions.

Optimizing additive manufacturing path pattern for Ti-6Al-4V thin rods using a combinatorial radial basis function surrogate-assisted genetic algorithm

Efficient hierarchical surrogate-assisted differential evolution for high-dimensional expensive optimization

On the one hand, surrogate-assisted evolutionary algorithms are established as a method of choice for expensive black-box optimization problems. On the other hand, the growth in computing facilities has seen a massive increase in potential computational power, granted the users accommodate their approaches with the offered parallelism. While a number of studies acknowledge the impact of parallelism for single-objective expensive optimization assisted by surrogates, extending such techniques to the multi-objective setting has not yet been properly investigated, especially within the state-of-the-art decomposition framework. We first highlight the different degrees of parallelism in existing surrogate-assisted multi-objective evolutionary algorithms based on decomposition (S-MOEA/D). We then provide a comprehensive analysis of the key steps towards a successful parallel S-MOEA/D approach. Through an extensive benchmarking effort relying on the well-established bbob-biobj test functions, we analyze the performance of the different algorithm designs with respect to the problem dimensionality and difficulty, the amount of parallel cores available, and the supervised learning models considered. In particular, we show the difference in algorithm scalability based on the selected surrogate-assisted approaches, the performance impact of distributing the model training task and the efficacy of the designed parallel-surrogate methods.

Practitioners often encounter computationally expensive multiobjective optimization problems to be solved in a variety of real-world applications. On the purpose of challenging these problems, we propose a new surrogate-based multiobjective optimization algorithm that does not require a large evaluation budget. It is called Multiobjective Tree-structured Parzen Estimator (MOTPE) and is an extension of the tree-structured Parzen estimator widely used to solve expensive single-objective optimization problems. Our empirical evidences reveal that MOTPE can approximate Pareto fronts of many benchmark problems better than existing methods with a limited budget. In this paper, we discuss furthermore the influence of MOTPE configurations to understand its behavior.

Surrogate-assisted evolutionary framework with an ensemble of teaching-learning and differential evolution for expensive optimization

When solving expensive multi-objective optimization problems, surrogate models are often used to reduce the number of true evaluations. Based on predictions from the surrogate models, promising candidate solutions, also referred to as infill solutions, can be identified for evaluation to expedite the search towards the optimum. This infill process in turn involves optimization of certain criteria derived from the surrogate models. In this study, predicted hypervolume maximization is considered as the infill criterion for expensive multi/many-objective optimization. In particular, we examine the effect of normalization bounds on the performance of the algorithm building on our previous study on bi-objective optimization. We propose a more scalable approach based on “surrogate corner” search that shows improved performance where some of the conventional techniques face challenges. Numerical experiments on a range of benchmark problems with up to 5 objectives demonstrate the efficacy and reliability of the proposed approach.

Many real-world applications can be formulated as expensive multimodal optimization problems (EMMOPs). When surrogate-assisted evolutionary algorithms (SAEAs) are employed to tackle these problems, they not only face the problem of selecting surrogate models but also need to tackle the problem of discovering and updating multiple modalities. Different optimization problems and different stages of evolutionary algorithms (EAs) generally require different types of surrogate models. To address this issue, in this article, we present a multisurrogate-assisted multitasking particle swarm optimization algorithm to seek multiple optimal solutions of EMMOPs at a low computational cost. The proposed algorithm first transforms an EMMOP into a multitasking optimization problem by integrating various surrogate models, and designs a multitasking niche particle swarm algorithm to solve it. Following that, a surrogate model management strategy based on the skill factor and clustering is developed to effectively balance the number of real function evaluations and the prediction accuracy of candidate optimal solutions. In addition, an adaptive local search strategy based on the trust region is proposed to enhance the capability of swarm in exploiting potential optimal modalities. We compare the proposed algorithm with five state-of-the-art SAEAs and seven multimodal EAs on 19 benchmark functions and the building energy conservation problem and experimental results show that the proposed algorithm can obtain multiple highly competitive optimal solutions.

Surrogate-assisted evolutionary algorithms provide an effective means for complex and computationally expensive optimization problems. However, due to the scarcity of training samples, the prediction accuracy of frequently-used regression surrogate models can hardly be guaranteed as the difficulty of the problem increases, resulting in performance degradation of the whole algorithm. Since real-world problems rarely exist in isolation, it is expected to alleviate the above issue by properly exploiting the knowledge shared across different problems. In this context, this study proposes a novel evolutionary multitasking optimization algorithm assisted by a classifier rather than a regression model for expensive multitasking problems, where the accuracy of the classifier is boosted by knowledge transfer. To be specific, a support vector classifier (SVC) is first developed and integrated into a classic evolutionary algorithm, i.e., covariance matrix adaptation evolution strategy (CMA-ES). With a low computational cost, it helps CMA-ES to prescreen parent solutions from the current population. Following that, a knowledge transfer strategy is designed to enrich the training samples for each task-oriented classifier by sharing high-quality solutions among different tasks, where a PCA-based subspace alignment technique is employed. Extensive experiments indicate that the SVC-assisted CMA-ES gains significant superiority over general CMA-ES in terms of both robustness and scalability, and the knowledge transfer strategy further helps it earn a competitive edge over some state-of-the-art algorithms on expensive multitasking optimization problems.

The surrogate-assisted optimization algorithms (SAOAs) are very promising for solving computationally expensive optimization problems (EOPs). Generally, the performance of a SAOA is determined by the quality of its surrogate model and the infill sampling criterion. In this paper, we propose a radial basis function (RBF) assisted optimization algorithm with batch infill sampling criterion for solving EOPs (short for RBFBS). In RBFBS, the quality of RBF model is adjusted by choosing a good shape parameter via solving a sub-expensive hyperparameter optimization problem. Moreover, a batch infill sampling criterion that includes a bi-objective-based sampling approach and a single-objective-based sampling approach is proposed to get a batch of samples for expensive evaluation. The experimental results on various benchmark problems show that RBFBS is very promising for expensive optimization.

Accelerating surrogate assisted evolutionary algorithms for expensive multi-objective optimization via explainable machine learning

A surrogate-assisted multi-objective evolutionary algorithm with dimension-reduction for production optimization

Surrogate-assisted evolutionary algorithms (SAEAs) have shown great potential to solve computationally expensive optimization problems (EOPs). Their two key components, i.e., the optimizer and the surrogate model, both need to select solutions to promote further evolution and to update the model, respectively. However, the corresponding selection strategies are designed independently and mainly consider the predicted fitness of candidate solutions. Consequently, they can hardly complement each other well and are much likely to mislead the population evolution since the surrogate model necessarily has some prediction errors. Directing against this issue, this study proposes a unified dual-space-driven selection (DSDS) strategy and develops a new SAEA by taking differential evolution and radial basis function as the optimizer and the surrogate model, respectively. Besides the predicted fitness in objective space of candidate solutions, DSDS also considers their distribution in decision space by measuring how close they are to the geometric centers of their respective neighborhoods with a new indicator called neighborhood centrality (NC). It tries to select the solutions with good fitness and small NC values under a multi-objective sample selection framework and thus helps the resulting SAEA steadily search in multiple neighborhoods of excellent solutions. The performance of the new SAEA was extensively tested on six commonly used benchmark functions with five different dimensions from 20 to 200 as well as two typical real-word traffic signal optimization cases. Experimental results demonstrate that it possesses more competitive performance and stronger robustness than state-of-the-art SAEAs.

The development of multi-objective evolutionary algorithms (MOEAs) assisted by surrogate models has significantly increased in the last few years. However, in realworld applications, the high modality and dimensionality that functions normally have, often causes problems to such models. Therefore, if the Pareto optimal set of a multi-objective optimization problem is located in a search space in which the surrogate model is not able to shape the corresponding region, the search could be misinformed and thus converge to wrong regions. This has motivated the idea of incorporating refinement mechanisms to such approaches, in order to improve the search. In this paper, we present a local search mechanism which improves the search of a MOEA assisted by surrogate models. Our preliminary results indicate that our proposed approach can produce good quality results when it is restricted to performing only between 1,000 and 5,000 fitness function evaluations. Our proposed approach is validated using a set of standard test problems and an airfoil design problem.