The Compact Genetic Algorithm is Efficient under Extreme Gaussian Noise

Estimation of distribution algorithms (EDAs) are a successful branch of evolutionary algorithms (EAs) that evolve a probabilistic model instead of a population. Analogous to genetic drift in EAs, EDAs also encounter the phenomenon that the random sampling in the model update can move the sampling frequencies to boundary values not justified by the fitness. This can result in a considerable performance loss. This work gives the first tight quantification of this effect for three EDAs and one ant colony optimizer, namely for the univariate marginal distribution algorithm, the compact genetic algorithm, population-based incremental learning, and the max-min ant system with iteration-best update. Our results allow to choose the parameters of these algorithms in such a way that within a desired runtime, no sampling frequency approaches the boundary values without a clear indication from the objective function. This paper for the Hot-off-the-Press track at GECCO 2020 summarizes the work Sharp Bounds for Genetic Drift in Estimation of Distribution Algorithms by B. Doerr and W. Zheng, which has been accepted for publication in the IEEE Transactions on Evolutionary Computation [5].

As faults are unavoidable in large scale multiprocessor systems, it is important to be able to determine which units of the system are working and which are faulty. System-level diagnosis is a long-standing realistic approach to detect faults in multiprocessor systems. Diagnosis is based on the results of tests executed on the system units. In this work we evaluate the performance of evolutionary algorithms applied to the diagnosis problem. Experimental results are presented for both the traditional genetic algorithm (GA) and specialized versions of the GA. We then propose and evaluate specialized versions of Estimation of Distribution Algorithms (EDA) for system-level diagnosis: the compact GA and Population-Based Incremental Learning both with and without negative examples. The evaluation was performed using four metrics: the average number of generations needed to find the solution, the average fitness after up to 500 generations, the percentage of tests that got to the optimal solution and the average time until the solution was found. An analysis of experimental results shows that more sophisticated algorithms converge faster to the optimal solution.

A comparative study of hybrid estimation distribution algorithms in solving the facility layout problem

Training artificial neural networks is a complex task of great practical importance. Besides classical ad-hoc algorithms such as backpropagation, this task can be approached by using Evolutionary Computation, a highly configurable and effective optimization paradigm. This chapter provides a brief overview of these techniques, and shows how they can be readily applied to the resolution of this problem. Three popular variants of Evolutionary Algorithms —Genetic Algorithms, Evolution Strategies and Estimation of Distribution Algorithms— are described and compared. This comparison is done on the basis of a benchmark comprising several standard classification problems of interest for neural networks. The experimental results confirm the general appropriateness of Evolutionary Computation for this problem. Evolution Strategies seem particularly proficient techniques in this optimization domain, and Estimation of Distribution Algorithms are also a competitive approach.KeywordsEvolutionary AlgorithmsArtificial Neural NetworksSupervised TrainingHybridization

Estimation of distribution algorithms (EDAs) solve an optimization problem heuristically by finding a probability distribution focused around its optima. Starting with the uniform distribution, points are sampled with respect to this distribution and the distribution is changed according to the function values of the sampled points. Although there are many successful experiments suggesting the usefulness of EDAs, there are only few rigorous theoretical results apart from convergence results without time bounds. Here we present first rigorous runtime analyses of a simple EDA, the compact genetic algorithm (cGA), for linear pseudo-Boolean functions on n variables. We prove a general lower bound for all functions and a general upper bound for all linear functions. Simple test functions show that not all linear functions are optimized in the same runtime by the cGA.

Evolutionary algorithms and elliptical copulas applied to continuous optimization problems

Practical optimization problems frequently include uncertainty about the quality measure, for example due to noisy evaluations. Thus, they do not allow for a straightforward application of traditional optimization techniques. In these settings meta-heuristics are a popular choice for deriving good optimization algorithms, most notably evolutionary algorithms which mimic evolution in nature. Empirical evidence suggests that genetic recombination is useful in uncertain environments because it can stabilize a noisy fitness signal. With this paper we want to support this claim with mathematical rigor.The setting we consider is that of noisy optimization. We study a simple noisy fitness function that is derived by adding Gaussian noise to a monotone function. First, we show that a classical evolutionary algorithm that does not employ sexual recombination (the \(( \mu +1) \)-EA) cannot handle the noise efficiently, regardless of the population size. Then we show that an evolutionary algorithm which does employ sexual recombination (the Compact Genetic Algorithm, short: cGA) can handle the noise using a graceful scaling of the population.KeywordsEvolutionary AlgorithmSexual RecombinationMinimum Span Tree ProblemNoisy OptimizationCompact Genetic AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An important problem in the study of evolutionary algorithms is how to continuously predict promising solutions while simultaneously escaping from local optima. In this paper, we propose an elitist probability schema (EPS) for the first time, to the best of our knowledge. Our schema is an index of binary strings that expresses the similarity of an elitist population at every string position. EPS expresses the accumulative effect of fitness selection with respect to the coding similarity of the population. For each generation, EPS can quantify the coding similarity of the population objectively and quickly. One of our key innovations is that EPS can continuously predict promising solutions while simultaneously escaping from local optima in most cases. To demonstrate the abilities of the EPS, we designed an elitist probability schema genetic algorithm and an elitist probability schema compact genetic algorithm. These algorithms are estimations of distribution algorithms (EDAs). We provided a fair comparison with the persistent elitist compact genetic algorithm (PeCGA), quantum-inspired evolutionary algorithm (QEA), and particle swarm optimization (PSO) for the 0---1 knapsack problem. The proposed algorithms converged quicker than PeCGA, QEA, and PSO, especially for the large knapsack problem. Furthermore, the computation time of the proposed algorithms was less than some EDAs that are based on building explicit probability models, and was approximately the same as QEA and PSO. This is acceptable for evolutionary algorithms, and satisfactory for EDAs. The proposed algorithms are successful with respect to convergence performance and computation time, which implies that EPS is satisfactory.

Estimation of distribution algorithm (EDA) is a new class of evolutionary algorithms with a wide range of real-world applications. However, it has been well known that the performance of EDA is not satisfactory enough if its population size is small. But to simply increase its population size may result in slow convergence. To the best knowledge of the authors', very few work has been done on improving the performance of EDA under small population size. This paper illustrates why EDA does not work well under small population size and proposes a novel approach termed as Over-Selection to boost EDA under small population size. Experimental results on several benchmark problems demonstrate that Over-Selection based EDA is often able to achieve a better solution without significantly increasing its time consumption when compared with the original version of EDA.

The compact Genetic Algorithm (cGA) is an Estimation of Distribution Algorithm that generates offspring population according to the estimated probabilistic model of the parent population instead of using traditional recombination and mutation operators. The cGA only needs a small amount of memory; therefore, it may be quite useful in memory-constrained applications. This paper introduces a theoretical framework for studying the cGA from the convergence point of view in which, we model the cGA by a Markov process and approximate its behavior using an Ordinary Differential Equation (ODE). Then, we prove that the corresponding ODE converges to local optima and stays there. Consequently, we conclude that the cGA will converge to the local optima of the function to be optimized.

Robust estimation of distribution algorithms via fitness landscape analysis for optimal low-thrust orbital maneuvers

Feature subset selection for unsupervised learning, is a very important topic in artificial intelligence because it is the base for saving computational resources. In this implementation we use a typical testor's methodology in order to incorporate an importance index for each variable. This paper presents the general framework and the way two hybridized meta-heuristics work in this NP-complete problem. The evolutionary mechanisms are based on the Univariate Marginal Distribution Algorithm (UMDA) and the Genetic Algorithm (GA). GA and UMDA --- Estimation of Distribution Algorithm (EDA) use a very useful rapid operator implemented for finding typical testors on a very large dataset and also, both algorithms, have a local search mechanism for improving time and fitness. Experiments show that EDA is faster than GA because it has a better exploitation performance; nevertheless, GA' solutions are more consistent.

The layout positioning problem of facilities on a straight line is known as Single Row Facility Layout Problem (SRFLP). The objective of SRFLP, categorized as NP-Complete problem, is to arrange the layout such that the sum of distances between all facilities’ pairs can be minimized. Extended Artificial Chromosome Genetic Algorithm (eACGA) is a promising algorithm that has been proposed recently. eACGA extends the probabilistic model in Estimation of Distribution Algorithms (EDAs) and then hybridize it with Genetic Algorithms (GAs). eACGA is proven to produce an excellent solution for scheduling problem. In this paper, we modify the eACGA to solve SRFLP. Computational results on benchmark problems show the effectiveness of eACGA for solving SRFLP. Key Words: single row facility layout, estimation distribution algorithm, genetic algorithm.

Aging is a general mechanism that some randomized search heuristics employ to increase the diversity of their collection of search points. A more diverse collection of search points is believed to improve the search heuristic’s performance for difficult problems. The most prominent randomized search heuristics with aging are evolutionary algorithms and artificial immune systems. While it is known that randomized search heuristics with aging can be very much more efficient than randomized search heuristics without aging the details of the origin of such benefits are difficult to understand. We contribute to this understanding by presenting a detailed and structured analysis of aging. We prove that in addition to diversity with respect to search points diversity with respect to age plays a key role. We analyze different ways of dealing with age diversity by means of theoretical as well as empirical analyses. Major results include a more structured understanding of aging and showcases where age diversity can make the difference between efficient and completely inefficient optimization.

Decomposition to smaller sub-problems is a general approach in problem solving. Many of the real-world problems can be decomposed into a number of sub-problems which may be solved easier. Appropriate decomposition is a significant issue specially for optimization problems where the optimal solution is usually obtained by combining the solutions of sub-problems. Estimation of distribution algorithms (EDAs) are a type of evolutionary algorithms that learn a model of problem from the population of candidate solutions. This model is intended to capture the interactions between problem variables, thus facilitating problem decomposition and is used to generate new solutions. In this paper, a novel type of problems is presented that is designed to challenge the model building process in discrete EDAs. The main idea is to propose a set of problems that their candidate solutions can be simultaneously decomposed into different sub-problems. This means that the candidate solution of a problem may be interpreted by two or more different structure where only one is true, resulting in the optimal solution to that problem. Some of these decompositions or structures may be more likely according to the low-order statistics collected from the population of candidate solutions, but may not necessarily lead to the optimal solution. Learning the correct structure/decomposition is a challenge for the model building process in EDA. The experimental results show that the proposed problems are indeed difficult for EDAs even when expressive models such as Bayesian networks are used to capture the interactions in the problem.