A new filled function method for global optimization

Parallel radial basis function methods for the global optimization of expensive functions

High dimensional global optimization problems arise frequently in process systems engineering. This is a result of the complex mechanistic relationships that describe process systems, and/or their large-scale nature. High dimensional optimization problems can often be more easily solved by instead solving a sequence of reduced-dimension subproblems. Surrogate models can allow the formulation of reduced-dimension subproblems by approximating the key features of the original model. Surrogate-based optimization (SBO) is to use surrogate modeling to solve a sequence of approximate reduced-dimension subproblems, in order to converge to a high quality solution to the original high dimensional problem. Here we review the key characteristics of SBO frameworks and their application to process systems optimization problems.

We introduce a new framework for the global optimization of computationally expensive multimodal functions when derivatives are unavailable. The proposed Stochastic Response Surface (SRS) Method iteratively utilizes a response surface model to approximate the expensive function and identifies a promising point for function evaluation from a set of randomly generated points, called candidate points. Assuming some mild technical conditions, SRS converges to the global minimum in a probabilistic sense. We also propose Metric SRS (MSRS), which is a special case of SRS where the function evaluation point in each iteration is chosen to be the best candidate point according to two criteria: the estimated function value obtained from the response surface model, and the minimum distance from previously evaluated points. We develop a global optimization version and a multistart local optimization version of MSRS. In the numerical experiments, we used a radial basis function (RBF) model for MSRS and the resulting algorithms, Global MSRBF and Multistart Local MSRBF, were compared to 6 alternative global optimization methods, including a multistart derivative-based local optimization method. Multiple trials of all algorithms were compared on 17 multimodal test problems and on a 12-dimensional groundwater bioremediation application involving partial differential equations. The results indicate that Multistart Local MSRBF is the best on most of the higher dimensional problems, including the groundwater problem. It is also at least as good as the other algorithms on most of the lower dimensional problems. Global MSRBF is competitive with the other alternatives on most of the lower dimensional test problems and also on the groundwater problem. These results suggest that MSRBF is a promising approach for the global optimization of expensive functions.

<div class="section abstract"><div class="htmlview paragraph">Plenum system of automotive vehicles draws atmospheric air using HVAC blower motor through air inlet grille at the bottom of the windshield and uses lower plenum for rainwater drainage after which air flow enters into awning and sugar scoop for evaporator air flow requirement. Awning and sugar scoop component is used in plenum air flow system to ensure the air flow with the required velocity that would meet the performance metric of total pressure drop of plenum air flow system. The optimal design of plenum component is essential since blower motor energy needs to be drawn from the expensive battery pack and any wastage of energy would lead to reduction in range of the Electric Vehicle. The conventional design process utilizes incremental design iteration, engineering intuition and requires significant effort from engineers to meet performance targets within the specified design space. This also may or may not ensure an optimal design.</div><div class="htmlview paragraph">In this work, a Machine learning tool with extreme gradient boost algorithm is used for development of a robust meta model that is used for quick prediction of awning and sugar scoop performance parameters such as total pressure drop and velocity of air prediction. On this response surface, application of a meta heuristic [<span class="xref">1</span>] based global optimization method is demonstrated which uses simplex homological global optimization (SHGO) and differential evolutionary (DE) method for optimum awning and sugar scoop design. Simplex homological global optimization (SHGO) is the concept of homology group growth which is used to solve a global and derivative free optimization problem of arbitrarily high dimensions. Similarly Differential evolution (DE) is a population-based metaheuristic search algorithm [<span class="xref">2</span>] that optimizes a problem which can quickly explore very large design spaces. Using these global optimization algorithms, it is observed that the optimized design of the awning and sugar scoop does result in significant reduction of total pressure drop that helps improve the required performance substantially compared to baseline design parameters within the given design space.</div></div>

A velocity-based butterfly optimization algorithm for high-dimensional optimization and feature selection

Computer analysis and simulation based design optimization requires more computationally efficient global optimization tools. In this work, a new global optimization algorithm based on design experiments, region elimination and response surface model, namely Approximated Unimodal Region Elimination Method (AUREM), is introduced. The approach divides the field of interest into several unimodal regions using design experiment data; identify and rank the regions that most likely contain the global minimum; form a response surface model with additional design experiment data over the most promising region; identify its minimum, remove this processed region, and move to the next most promising region. By avoiding redundant searches, the approach identifies the global optimum with reduced number of objective function evaluations and computation effort. The new algorithm was tested using a variety of benchmark global optimization problems and compared with several widely used global optimization algorithms. The experiments results present comparable search accuracy and superior computation efficiency, making the new algorithm an ideal tool for computer analysis and simulation black-box based global design optimization.

Particle Swarm Optimization (PSO) is an evolution-nary computation technique. Separate adjustment to inertia weight and learning factors in PSO undermines the integrity and intelligent characteristic in the evolutionary process of particle swarm to some extent, thus it is not suitable for solving most complicated optimization problems. On the basis of previous researches, the aim of this study was to improve the computational efficiency of PSO and avoid premature convergence for multimodal, higher dimensional complicated optimization problems by considering the mutual influences of inertia weight and learning factors on the updates of particle's veloci-ties. A great number of experiment data provided evidence that the nonlinear adjustment to inertia weight, cognitive learning factor and social learning factor within a certain interval is a good selection. Moreover, the simulation results on four typical functions show that the improved strategy of PSO proposed in the paper is available for solving multimodal and higher dimensional complicated optimization problems, and can accelerate convergence speed, improve optimization quality effectively in comparison to the algorithms of standard PSO and existing relevant improved PSO.

Improved sequential optimization method (SOM) and dimension reduction optimization method (DROM) are presented for high dimensional optimization design problems of electromagnetic devices. Improved SOM can simultaneously optimize the statistical approximate models and optimization algorithms more efficiently compared with SOM. Using DROM, a high dimensional problem can be converted into a low dimensional problem with expert experience or some design of experiment techniques. Then two TEAM benchmark problems (Problem 22 and Problem 25) are investigated to illustrate the efficiency of the proposed methods. From the experimental results, we can see that the presented methods can obviously reduce the computational cost of finite element analysis, while the optimal results also satisfy design specification.

Global optimisation problems in high-dimensional and infinite-dimensional spaces arise in various real-world applications such as engineering, economics and finance, geophysics, biology, machine learning, optimal control, etc. Among stochastic approaches to global optimisation, biology-inspired methods are currently very popular in the literature. Bio-inspired approaches imitate natural ecological and evolutionary processes and are reported to be efficient in a large number of practical study cases. However, many bio-inspired methods can possess some vital drawbacks. For example, due to their semi-empirical nature, convergence to the globally optimal solution cannot always be guaranteed. Another major obstacle is that the existing methods often struggle with higher dimensionality of the space of parameters, which results in a slow convergence. Moreover, it is often difficult to adjust the dimensionality of the space of parameters in the corresponding computer code for a practical realisation of the optimisation method. Here, we present a bio-inspired global stochastic optimisation method applicable in Hilbert function spaces. The proposed method is an evolutionary algorithm inspired by the Darwin’s famous idea of survival of the fittest and is, therefore, referred to as the ‘Survival of the Fittest Algorithm’ (SoFA). Mathematically, the convergence of SoFA is a consequence of the fundamental property of localisation of probabilistic measure in a Hilbert space and we rigorously prove the convergence of the introduced algorithm for a generic class of functionals. The approach is simple in terms of practical coding. As an insightful, real-world problem, we apply our method to find the globally optimal trajectory for the daily vertical migration of zooplankton in the ocean and lakes, this phenomenon is considered to be the largest synchronised movement of biomass on Earth. We maximise fitness in a function space derived from a von-Foerster stage-structured population model with biologically realistic parameters. We show that for problems of fitness maximisation in high-dimensional spaces, SoFA provides better performance as compared to some other stochastic global optimisation algorithms. We highlight the links between the new optimisation algorithm and natural selection processes in ecosystems occurring within a population via gradual exclusion of competitive con-specific strains.

This paper describes a new parallel global surrogate-based algorithm Global Optimization in Parallel with Surrogate (GOPS) for the minimization of continuous black-box objective functions that might have multiple local minima, are expensive to compute, and have no derivative information available. The task of picking P new evaluation points for P processors in each iteration is addressed by sampling around multiple center points at which the objective function has been previously evaluated. The GOPS algorithm improves on earlier algorithms by (a) new center points are selected based on bivariate non-dominated sorting of previously evaluated points with additional constraints to ensure the objective value is below a target percentile and (b) as iterations increase, the number of centers decreases, and the number of evaluation points per center increases. These strategies and the hyperparameters controlling them significantly improve GOPS’s parallel performance on high dimensional problems in comparison to other global optimization algorithms, especially with a larger number of processors. GOPS is tested with up to 128 processors in parallel on 14 synthetic black-box optimization benchmarking test problems (in 10, 21, and 40 dimensions) and one 21-dimensional parameter estimation problem for an expensive real-world nonlinear lake water quality model with partial differential equations that takes 22 min for each objective function evaluation. GOPS numerically significantly outperforms (especially on high dimensional problems and with larger numbers of processors) the earlier algorithms SOP and PSD-MADS-VNS (and these two algorithms have outperformed other algorithms in prior publications).

Abstract: Inversion of self-potential anomaly for 2-D inclined sheets of infinite horizontal extent has been studied. Least-square inversion and very fast simulated annealing global optimization has been used to model the five parameters of self potential anomaly. The method of least square and very fast simulated annealing global optimization method is compared and analyzed. Very fast simulated annealing can model the noisy and field data of self potential anomaly very precisely than linear inversion technique. However, time taken by very fast simulated annealing inversion is larger than linearized inversion. The comparative analysis has been done on synthetic data (noise free and noisy) and two field data from Bavarian woods anomaly, Germany and Surda anomaly, India to show the efficacy of both the methods. The estimated parameters were compared with those from previous studies using various global optimization algorithms, mainly neural network, genetic algorithm and particle swarm optimization on the same field data sets. It can be concluded that the global optimization algorithms considered in this study were able to yield compatible solutions with those from least-square methods. The present global optimization method is in good agreement with the other global optimization methods in terms of results and computation time.

A reconsideration on the resolution of phase stability analysis using stochastic global optimization methods: Proposal of a reliable set of benchmark problems

Global optimization has been an area of active research for the past few decades. There are many different global optimization algorithms that have been proposed and studied. Among these, differential evolution (DE) is a promising population‐based stochastic global optimization algorithm. It has been gaining popularity in the recent past due to its capability to handle non‐convex, non‐differentiable and multimodal objective functions. In this work, a hybrid global optimization (HGO) method with a novel stopping criterion is proposed. The proposed algorithm integrates DE with a taboo list of taboo search and self‐adaptive strategies. The taboo list/check prevents revisiting the same area, thus increasing the population diversity and computational efficiency. The self‐adaptive strategy reduces the algorithm parameters to be provided and makes the algorithm more robust. Furthermore, a novel stopping criterion based on the number of rejected points is developed. The effectiveness of the proposed stopping criterion and HGO is assessed on more than 30 benchmark problems with 2–30 variables. The performance of HGO is compared with some state‐of‐the‐art global optimization algorithms recently published in the literature. The results show that the reliability of finding the global optimum by HGO is slightly better than that by the latest DE techniques in the literature, and the number of function evaluations (NFE) of HGO is significantly reduced. Finally, HGO is tested for parameter estimation in dynamic models. In conclusion, the proposed HGO algorithm with the novel stopping criterion is found to be reliable, efficient and suitable for application problems. Copyright © 2011 Curtin University of Technology and John Wiley & Sons, Ltd.

Memetic algorithms are effective algorithms to obtain reliable and accurate solutions for complex continuous optimization problems. Nowadays, high dimensional optimization problems are an interesting field of research. The high dimensionality introduces new problems for the optimization process, requiring more scalable algorithms that, at the same time, could explore better the higher domain space around each solution. In this work, we proposed a memetic algorithm, MA-SW-Chains, for large scale global optimization. This algorithm assigns to each individual a local search intensity that depends on its features, by chaining different local search applications. MA-SW-Chains is an adaptation to large scale optimization of a previous algorithm, MA-CMA-Chains, to improve its performance on high-dimensional problems. Finally, we present the results obtained by our proposal using the benchmark problems defined in the Special Session of Large Scale Global Optimization on the IEEE Congress on Evolutionary Computation in 2010.

The main objective of this paper is to solve the inverse convection heat transfer problems with particle swarm optimization method. An enhanced particle swarm optimization (EPSO) algorithm is proposed to overcome the shortcoming of earlier convergence of standard PSO algorithms. The performance of EPSO is tested by some benchmark functions; it is shown that EPSO has a strong antilocal trap capability especially for high dimensional multimodal optimization problems. At last, EPSO is used to identify the unknown boundary heat flux in a channel flow. According to the computational results of four test problems, it is clear that the proposed EPSO algorithm is able to estimate the unknown heat flux accurately even when the input data contain measurement error.