Differential Evolution Using Surrogate Model Based on Pairwise Ranking Estimation for Constrained Optimization Problems

Surrogate-assisted evolutionary framework with an ensemble of teaching-learning and differential evolution for expensive optimization

Surrogate-assisted evolutionary algorithms (SAEAs) have shown great potential to solve computationally expensive optimization problems (EOPs). Their two key components, i.e., the optimizer and the surrogate model, both need to select solutions to promote further evolution and to update the model, respectively. However, the corresponding selection strategies are designed independently and mainly consider the predicted fitness of candidate solutions. Consequently, they can hardly complement each other well and are much likely to mislead the population evolution since the surrogate model necessarily has some prediction errors. Directing against this issue, this study proposes a unified dual-space-driven selection (DSDS) strategy and develops a new SAEA by taking differential evolution and radial basis function as the optimizer and the surrogate model, respectively. Besides the predicted fitness in objective space of candidate solutions, DSDS also considers their distribution in decision space by measuring how close they are to the geometric centers of their respective neighborhoods with a new indicator called neighborhood centrality (NC). It tries to select the solutions with good fitness and small NC values under a multi-objective sample selection framework and thus helps the resulting SAEA steadily search in multiple neighborhoods of excellent solutions. The performance of the new SAEA was extensively tested on six commonly used benchmark functions with five different dimensions from 20 to 200 as well as two typical real-word traffic signal optimization cases. Experimental results demonstrate that it possesses more competitive performance and stronger robustness than state-of-the-art SAEAs.

It is well known that in evolutionary algorithms (EAs), different reproduction operators may be suitable for different problems or in different running stages. To improve the algorithm performance, the ensemble of multiple operators has become popular. Most ensemble techniques achieve this goal by choosing an operator according to a probability learned from the previous experience. In contrast to these ensemble techniques, in this paper we propose a cheap surrogate model-based multioperator search strategy for evolutionary optimization. In our approach, a set of candidate offspring solutions are generated by using the multiple offspring reproduction operators, and the best one according to the surrogate model is chosen as the offspring solution. Two major advantages of this approach are: 1) each operator can generate a solution for competition compared to the probability-based approaches and 2) the surrogate model building is relatively cheap compared to that in the surrogate-assisted EAs. The model is used to implement multioperator ensemble in two popular EAs, that is, differential evolution and particle swarm optimization. Thirty benchmark functions and the functions presented in the CEC 2013 are chosen as the test suite to evaluate our approach. Experimental results indicate that the new approach can improve the performance of single operator-based methods in the majority of the functions.

Many real-world applications can be formulated as expensive multimodal optimization problems (EMMOPs). When surrogate-assisted evolutionary algorithms (SAEAs) are employed to tackle these problems, they not only face the problem of selecting surrogate models but also need to tackle the problem of discovering and updating multiple modalities. Different optimization problems and different stages of evolutionary algorithms (EAs) generally require different types of surrogate models. To address this issue, in this article, we present a multisurrogate-assisted multitasking particle swarm optimization algorithm to seek multiple optimal solutions of EMMOPs at a low computational cost. The proposed algorithm first transforms an EMMOP into a multitasking optimization problem by integrating various surrogate models, and designs a multitasking niche particle swarm algorithm to solve it. Following that, a surrogate model management strategy based on the skill factor and clustering is developed to effectively balance the number of real function evaluations and the prediction accuracy of candidate optimal solutions. In addition, an adaptive local search strategy based on the trust region is proposed to enhance the capability of swarm in exploiting potential optimal modalities. We compare the proposed algorithm with five state-of-the-art SAEAs and seven multimodal EAs on 19 benchmark functions and the building energy conservation problem and experimental results show that the proposed algorithm can obtain multiple highly competitive optimal solutions.

Accelerating surrogate assisted evolutionary algorithms for expensive multi-objective optimization via explainable machine learning

A surrogate-assisted multi-objective evolutionary algorithm with dimension-reduction for production optimization

Efficient hierarchical surrogate-assisted differential evolution for high-dimensional expensive optimization

Although surrogate-assisted evolutionary algorithms have been widely utilized in the science and engineering fields, efficient management of surrogates is still a challenging task because of the complicated action mechanisms of surrogates. Trying to investigate how surrogates influence the convergence of evolutionary algorithms, this paper performs an analysis of a (1+1) surrogate-assisted evolutionary algorithm with Gaussian mutations, where a linear surrogate model is employed to approximately evaluate the candidate solutions. Numerical results demonstrate that the linear surrogate model greatly improves performance of evolutionary algorithms on a unimodal problem when the standard deviation of Gaussian mutation σ is small. However, when σ is set big, the improving functions of linear surrogate models are not significant. However, for a multi-modal problem, the positive function of the linear surrogate model is not significant, because the employed Gaussian mutation cannot keep a balance between exploration and exploitation.KeywordsCandidate SolutionSphere FunctionGaussian MutationMultimodal ProblemGaussian Random NumberThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

To solve noisy and expensive multi-objective optimization problems, there are only a few function evaluations can be used due to the limitation of time and/or money. Because of the influence of noises, the evaluations are inaccurate. It is challenging for the existing surrogate-assisted evolutionary algorithms. Due to the influence of noises, the performance of the surrogate model constructed by these algorithms is degraded. At the same time, noises would mislead the evolution direction. More importantly, because of the limitations of function evaluations, noise treatment methods consuming many function evaluations cannot be applied. An adaptive model switch-based surrogate-assisted evolutionary algorithm is proposed to solve such problems in this paper. The algorithm establishes radial basis function networks for denoising. An adaptive model switch strategy is adopted to select suited surrogate model from Gaussian regression and radial basis function network. It adaptively selects the sampling strategies based on the maximum improvement in the convergence, diversity, and approximation uncertainty to make full use of the limited number of function evaluations. The experimental results on a set of test problems show that the proposed algorithm is more competitive than the five most advanced surrogate-assisted evolutionary algorithms.

We consider a job shop scheduling problem with uncertain processing times modelled as triangular fuzzy numbers and propose a multiobjective surrogate-assisted evolutionary algorithm to optimise not only the schedule’s fuzzy makespan but also the robustness of schedules with respect to different perturbations in the durations. The surrogate model is defined to avoid evaluating the robustness measure for some individuals and estimate it instead based on the robustness values of neighbouring individuals, where neighbour proximity is evaluated based on the similarity of fuzzy makespan values. The experimental results show that by using fitness estimation, it is possible to reach good fitness levels much faster than if all individuals are evaluated.

Process optimization of a Non-circular drawing (NCD) sequence of a pearlitic steel wire was performed to improve the mechanical properties of a drawn wire based on surrogate assisted meta-heuristic algorithms. The objective function was introduced to minimize inhomogeneity of effective strain distribution at the cross-section of the drawn wire, which could deteriorate delamination characteristics of the drawn wires. The design variables introduced were die geometry and reduction of area of the NCD sequence. Several surrogate models and their combinations with the weighted sum technique were utilized. In the process optimization of the NCD sequence, the surrogate models were used to predict effective strain distributions at the cross-section of the drawn wire. Optimization using Differential evolution (DE) algorithm was performed, while the objective function was calculated from the predicted effective strains. The accuracy of all surrogate models was investigated, while optimum results were compared with the previous study available in the literature. It was found that hybrid surrogate models can improve prediction accuracy compared to a single surrogate model. The best result was obtained from the combination of Kriging (KG) and Support vector regression (SVR) models, while the second best was obtained from the combination of four surrogate models: Polynomial response surface (PRS), Radial basic function (RBF), KG, and SVR. The optimum results found in this study showed better effective strain homogeneity at the cross-section of the drawn wire with the same total reduction of area of the previous work available in the literature for fewer number of passes. The multi-surrogate models with the weighted sum technique were found to be powerful in improving the delamination characteristics of the drawn wire and reducing the production cost.

In recent years, various methods from the evolutionary computation (EC) field have been applied to electromagnetic (EM) design problems and have shown promising results. However, due to the high computational cost of the EM simulations, the efficiency of directly using evolutionary algorithms is often very low (e.g., several weeks' optimization time), which limits the application of these methods for many industrial applications. To address this problem, a new method, called surrogate model assisted differential evolution for antenna synthesis (SADEA), is presented in this paper. The key ideas are: (1) A Gaussian Process (GP) surrogate model is constructed on-line to predict the performances of the candidate designs, saving a lot of computationally expensive EM simulations. (2) A novel surrogate model-aware evolutionary search mechanism is proposed, directing effective global search even when a traditional high-quality surrogate model is not available. Three complex antennas and two mathematical benchmark problems are selected as examples. Compared with the widely used differential evolution and particle swarm optimization, SADEA can obtain comparable results, but achieves a 3 to 7 times speed enhancement for antenna design optimization.

Adaptive surrogate-assisted multi-objective evolutionary algorithm using an efficient infill technique

Surrogate-assisted evolutionary algorithms have been paid more and more attention to solve computationally expensive problems. However, model management still plays a significant importance in searching for the optimal solution. In this paper, a new method is proposed to measure the approximation uncertainty, in which the differences between the solution and its neighbour samples in the decision space, and the ruggedness of the objective space in its neighborhood are both considered. The proposed approximation uncertainty will be utilized in the surrogate-assisted global search to find a solution for exact objective evaluation to improve the exploration capability of the global search. On the other hand, the approximated fitness value is adopted as the infill criterion for the surrogate-assisted local search, which is utilized to improve the exploitation capability to find a solution close to the real optimal solution as much as possible. The surrogate-assisted global and local searches are conducted in sequence at each generation to balance the exploration and exploitation capabilities of the method. The performance of the proposed method is evaluated on seven benchmark problems with 10, 20, 30 and 50 dimensions, and one real-world application with 30 and 50 dimensions. The experimental results show that the proposed method is efficient for solving the low- and medium-dimensional expensive optimization problems by compared to the other six state-of-the-art surrogate-assisted evolutionary algorithms.

Despite the success of Evolutionary Algorithms in solving complex problems, they may require many function evaluations. This becomes an issue when dealing with costly problems. Surrogate models may overcome this difficulty, though their use in problems with medium to large dimensionality is underexplored in the literature. Problems with multiple conflicting objectives can be formulated as Multi-objective Optimization Problems (MOPs). MOPs have received a great attention lately, mainly with Multi-objective Evolutionary Algorithms (MOEAs). This paper proposes ELMOEA/D-DE, a surrogate-assisted MOEA, for solving expensive MOPs in small evaluation budgets. ELMOEA/D-DE encompasses a state-of-the-art MOEA based on decomposition, Differential Evolution (DE) operators and Extreme Learning Machines. This paper tests three variants of ELMOEA/D-DE, using different DE operators, for solving five known benchmark MOPs with 10 to 60 decision variables. All variants achieve good results in terms of hyper volume metric and the best variant with operator DE/rand/1/bin is compared with two state-of-the-art approaches (MOEA/D-RBF and a non-surrogate-based MOEA), achieving the best results in all but one problems instances.