Evolutionary algorithms and elliptical copulas applied to continuous optimization problems

Both Estimation of Distribution Algorithms (EDAs) and Copula Theory are hot topics in different research domains. The key of EDAs is modeling and sampling the probability distribution function which need much time in the available algorithms. Moreover, the modeled probability distribution function can not reflect the correct relationship between variables of the optimization target. Copula Theory provides a correlation between univariable marginal distribution functions and the joint probability distribution function. Therefore, Copula Theory could be used in EDAs. Because Archimedean copulas possess many nice properties, an EDA based on Archimedean copulas is presented in this paper. The experimental results show the effectiveness of the proposed algorithm.

Estimation of distribution algorithms (EDAs) are intelligent systems that take advantage of statistical learning techniques. The distribution of promising regions in the search space is estimated and probabilistically guide the new particles’ searching towards them in EDAs. But EDAs cannot solve complex optimisation problems reliably and efficiently because of premature convergence and difficulty of complex probabilistic model learning. This paper presents a PCA-EDA algorithm which introduces principal component analysis (PCA) into simple Gaussian model-based EDA. PCA-EDA is aimed to keep the balance of accuracy and efficiency of probabilistic model learning, as well as to avoid premature convergence by PCA’s ability of analysis of Gaussian model’s variance. From the results of numerical experiments, it is showed that the proposed method is feasible and effective for complex optimisation problems.

Finding a good model and efficiently estimating the distribution is still an open challenge in estimation of distribution algorithms (EDAs). Factorization encoded by models in most of the EDAs are constrained. However for optimization of many real-world problems, finding the model capable of representing complex interactions without much computational complexity overhead is the key challenge. On the other hand factor graph which is the most natural graphical model for representing additively decomposable functions is rarely employed in EDAs. In this paper we introduce Factor Graph based EDA (FGEDA) which learns factor graph as the model and estimate the probability distribution represented by the learned factor graph using Markov blanket canonical factorization. The class of factorization that is employed for approximation of distribution in FGEDA is expanded relative to famous EDAs. We have used matrix factorization for learning the factor graph of the problem based on the pairwise mutual information between pair of variables. Gibbs sampling and BB-wise crossover are used to generate new samples. Empirical evaluation as well as theoretical analysis of the approach show the efficiency and power of FGEDA in the optimization of functions with complex interactions. It is showed experimentally that FGEDA outperform other well-known EDAs.

A comparative study of hybrid estimation distribution algorithms in solving the facility layout problem

<strong class="journal-contentHeaderColor">Abstract.</strong> Biogeochemical model behaviour for micronutrients is typically hard to constrain because of the sparsity of observational data, the difficulty of determining parameters in situ, and uncertainties in observations and models. Here, we assess the influence of data distribution, model uncertainty, and the misfit function on objective parameter optimisation in a model of the oceanic cycle of zinc (Zn), an essential micronutrient for marine phytoplankton with a long whole-ocean residence time. We aim to investigate whether observational constraints are sufficient for reconstruction of biogeochemical model behaviour, given that the Zn data coverage provided by the GEOTRACES Intermediate Data Product 2017 is sparse. Furthermore, we aim to assess how optimisation results are affected by the choice of the misfit function and by confounding factors such as analytical uncertainty in the data or biases in the model related to either seasonal variability or the larger-scale circulation. The model framework applied herein combines a marine Zn cycling model with a state-of-the-art estimation of distribution algorithm (Covariance Matrix Adaption Evolution Strategy, CMA-ES) to optimise the model towards synthetic data in an ensemble of 26 optimisations. Provided with a target field that can be perfectly reproduced by the model, optimisation retrieves parameter values perfectly regardless of data coverage. As differences between the model and the system underlying the target field increase, the choice of the misfit function can greatly impact optimisation results, while limitation of data coverage is in most cases of subordinate significance. In cases where optimisation to full or limited data coverage produces relatively distinct model behaviours, we find that applying a misfit metric that compensates for differences in data coverage between ocean basins considerably improves agreement between optimisation results obtained with the two data situations.

Estimation of Distribution Algorithms (EDAs) is new kinds of colony evolution algorithms. It produces its new generation by constructing probability distribution model through counting excellent information of individuals of present colony EDAs first, and then sampling the model. To solve the NP-Hard question as EDAs searching optimum network structure, a new Maximum Entropy Estimation of Distribution Algorithm (MEEDA) is proposed; and the paper puts forward the MEEDA to optimize two degree of freedom PID controller parameters. The simulation results show that the optimal 2-DOF PID controller has simultaneously both the characteristics of command tracking and disturbance rejection, so we can see the simulation verifies the effectiveness of the EDA algorithm.

Estimation of distribution algorithms (EDAs) are a class of evolutionary optimisation algorithms based on probability distribution model. This article extends the basic EDAs for tackling multi-objective optimisation problems by incorporating Archimedean copulas for constructing probability distribution model, and using the concept of Pareto dominance. In the algorithm, the marginal distributions from the current population are firstly estimated by kernel estimation method and are used to estimate the parameter of the Archimedean copula function generator by using the maximum likelihood method. Afterwards, the multivariate Archimedean copula sample algorithm is used to generate current offsprings population by sampling the n-dimensional Laplace transform Archimedean copula. The population with the current population and current offsprings population is sorted based on non-domination, and the best individuals are selected to form the next population based on rank and the crowding distance. The proposed algorithm is tested to compare with NSGA-II, PAES and SPEA2 using a set of benchmark functions. Both convergence and diversity metrics are used to evaluate the performance of the algorithm. The experimental results show that the algorithm outperforms NSGA-II, PAES and SPEA2 in two metrics.

Markov random field (MRF) modelling techniques have been recently proposed as a novel approach to probabilistic modelling for estimation of distribution algorithms (EDAs). An EDA using this technique was called distribution estimation using Markov random fields (DEUM). DEUM was later extended to DEUMd. DEUM and DEUMd use a univariate model of probability distribution, and have been shown to perform better than other univariate EDAs for a range of optimization problems. This paper extends DEUM to use a bivariate model and applies it to the Ising spin glass problems. We propose two variants of DEUM that use different sampling techniques. Our experimental result show a noticeable gain in performance.

This paper presents our work on Estimation of Distribution Algorithms (EDAs) that address sequencing problems, i.e., the task of finding the best ordering of a set of items or an optimal schedule to perform a given set of operations. Specifically, we focus on using probabilistic models based on $n$-gram statistics. These models have been used extensively in modeling the statistical properties of sequences. We start with an EDA that uses a bigram model, then extend this scheme to higher-order models. However, directly replacing the bigram model with a higher-order model results in premature convergence. We give an explanation on this situation, along with some empirical support. We then introduce a technique for combining multiple models of different orders, which allows for smooth transition from lower-order models to higher-order ones. Furthermore, this technique can also be used to incorporate other heuristics as well as prior knowledge about the problem into the search process. Promising preliminary results on solving Traveling Salesman Problems (TSPs) are presented.

Estimation of Distribution Algorithms (EDAs) are a set of algorithms that belong to the field of Evolutionary Computation. In EDAs there are neither crossover nor mutation operators. Instead, the new population of individuals is sampled from a probability distribution, which is estimated from a database that contains the selected individuals from the previous generation. Thus, the interrelations between the different variables that represent the individuals may be explicitly expressed through the joint probability distribution associated with the individuals selected at each generation.

Many real-world optimization problems are subjected to uncertainties that may be characterized by the presence of noise in the objective functions. The estimation of distribution algorithm (EDA), which models the global distribution of the population for searching tasks, is one of the evolutionary computation techniques that deals with noisy information. This paper studies the potential of EDAs; particularly an EDA based on restricted Boltzmann machines that handles multi-objective optimization problems in a noisy environment. Noise is introduced to the objective functions in the form of a Gaussian distribution. In order to reduce the detrimental effect of noise, a likelihood correction feature is proposed to tune the marginal probability distribution of each decision variable. The EDA is subsequently hybridized with a particle swarm optimization algorithm in a discrete domain to improve its search ability. The effectiveness of the proposed algorithm is examined via eight benchmark instances with different characteristics and shapes of the Pareto optimal front. The scalability, hybridization, and computational time are rigorously studied. Comparative studies show that the proposed approach outperforms other state of the art algorithms.

Most of EMO algorithms use crossover operator for generating new solutions. There have been proposed various kinds of crossover in this field and most crossover approaches are good at global optimization but not effective for the problem with strong nonlinearity and dependency.On the other hand, Estimation of Distribution Algorithm (E-DA) is known as an effective approach without using crossover for generating new solutions. EDA use an estimation of distribution operator for generating new solutions and this operator is known as to be effective for the problem with strong nonlinearity and dependency.In this paper, a new approach to generate new solutions combing crossover and estimation of distribution is proposed. The main purpose of this approach is to generate high-quality solutions more effectively by combing each other’s strength. This approach is named as Combined with Estimation of Distribution (MOEA/D-CED) because this approach is incorporated with MOEA/D. Through applying to some benchmark problems in this field, the characteristics and effectiveness of MOEA/D-CED were confirmed by the comparison with original MOEA/D and MO-CMA-ES.

The blocking job shop problem (BJSSP) is a variant of the classical job shop problem, where a job completed on one machine cannot be transferred to the next machine until the latter becomes available, causing the current machine to remain blocked. Many real-world problems have been modeled as BJSSP. This problem is classified as strongly NP-hard. The existing literature indicates that some proposed methods fail to guarantee the generation of feasible solutions during the search process, necessitating solution reconstruction. In this study, we developed an Estimation Distribution Algorithm (EDA) designed to operate strictly within the BJSSP feasible solution space. The experiments showed that no specific levels of the factors were found to be significant in determining the quality of the solutions found by the EDA across all problem sets. On the other hand, incorporating assignment probabilities into the construction of EDA solutions facilitated diversity among the generated solutions. The runtime of the EDA for the proposed problem sets in the experiments, using the factor levels defined, ranged from 20 s for problems with 10 jobs and 5 machines to 600 s for those with 30 jobs and 10 machines. The proposed EDA begins by generating a random initial population of solutions. The makespan (Cmax) is then computed for each individual. The next generation is composed by three sets of individuals. The first group corresponds to a percentage of the best-performing individuals of the current generation. The second group is generated by using a probability distribution function derived from the structure of the selected solutions, while the remaining individuals are created randomly to enhance the algorithm’s exploratory capability. Our results demonstrate that the proposed method improves upon some of the previously best-known results in the current state of the art.

The Estimation of Distribution Algorithms(EDAs) is a new paradigm for Evolutionary Computation. This new class of algorithms generalizes Genetic Algorithms(GAs) by replacing the crossover and mutation operators by learning and sampling the probability distribution of the best individuals of the population at each iteration of the algorithm. In this paper, we review the EDAs for the solution of combinatorial optimization problems and optimization in continuous domains. The paper gives a brief overview of the multiobjective problems(MOP) and estimation of distribution algorithms(EDAs). We introduce a representative algorithm called RMMEDA (Regularity Model Based Multi-objective Estimation of Distribution Algorithm). In order to improve the convergence performance of the algorithm, we improve the traditional RM-MEDA. The improvement we make is using part of the parent population with better performance instead of the entire parent population to establish a more accurate manifold model, and the RM-MEDA based on elitist strategy theory is proposed. Experimental results show that the improved RM-MEDA performs better on the convergence metric and the algorithm runtime than the original one.KeywordsEDARMMEDAElitist Strategy

Markov Networks (also known as Markov Random Fields) have been proposed as a new approach to probabilistic modelling in Estimation of Distribution Algorithms (EDAs). An EDA employing this approach called Distribution Estimation Using Markov Networks (DEUM) has been proposed and shown to work well on a variety of problems, using a unique fitness modelling approach. Previously DEUM has only been demonstrated on univariate and bivariate complexity problems. Here we show that it can be extended to a difficult multivariate problem and is capable of accurately modelling a fitness function and locating an optimum with a very small number of function evaluations.