Hot off the Press: First Steps Towards a Runtime Analysis When Starting With a Good Solution

The mathematical runtime analysis of evolutionary algorithms traditionally regards the time an algorithm needs to find a solution of a certain quality when initialized with a random population. In practical applications it may be possible to guess solutions that are better than random ones. We start a mathematical runtime analysis for such situations. We observe that different algorithms profit to a very different degree from a better initialization. We also show that the optimal parameterization of an algorithm can depend strongly on the quality of the initial solutions. To overcome this difficulty, self-adjusting and randomized heavy-tailed parameter choices can be profitable. Finally, we observe a larger gap between the performance of the best evolutionary algorithm we found and the corresponding black-box complexity. This could suggest that evolutionary algorithms better exploiting good initial solutions are still to be found. These first findings stem from analyzing the performance of the \((1+1)\) evolutionary algorithm and the static, self-adjusting, and heavy-tailed \((1+(\lambda,\lambda))\) genetic algorithms on the OneMax benchmark. We are optimistic that the question of how to profit from good initial solutions is interesting beyond these first examples.

The maximum cut (MAX-CUT) problem is to find a bipartition of the vertices in a given graph such that the number of edges with ends in different sets reaches the largest. Though, several experimental investigations have shown that evolutionary algorithms (EAs) are efficient for this NP-complete problem, there is little theoretical work about EAs on the problem. In this paper, we theoretically investigate the performance of EAs on the MAX-CUT problem. We find that both the (1+1) EA and the (1+1) EA*, two simple EAs, efficiently achieve approximation solutions of (m/2)+(1/4)s(G) and (m/2)+(1/2)(√{8m+1}-1), where m and s(G) are respectively the number of edges and the number of odd degree vertices in the input graph. We also reveal that for a given integer k the (1+1) EA* finds a cut of size at least k in expected runtime O(nm+1/δ(4k)) and a cut of size at least (m/2)+k in expected runtime O(n(2)m+1/δ((64/3)k(2))), where δ is a constant mutation probability and n is the number of vertices in the input graph. Finally, we show that the (1+1) EA and the (1+1) EA* are better than some local search algorithms in one instance, and we also show that these two simple EAs may not be efficient in another instance.

Geometric crossover is a formal class of crossovers that includes many well-known recombination operators across representations. In previous work, it was shown that all evolutionary algorithms with geometric crossover (but no mutation) do the same form of convex search regardless of the underlying representation, the specific selection mechanism, offspring distribution, search space, and problem at hand. Furthermore, it was suggested that the generalised convex search could perform well on generalised forms of concave and approximately concave fitness landscapes regardless of the underlying space and representation. In this article, we deepen this line of enquiry and study the runtime of generalised convex search on concave fitness landscapes. This is a first step toward linking a geometric theory of representations and runtime analysis in the attempt to (1) set the basis for a more general, unified approach for the runtime analysis of evolutionary algorithms across representations, and (2) identify the essential matching features of evolutionary search behaviour and landscape topography that cause polynomial performance. We present a general runtime result that can be systematically instantiated to specific search spaces and representations and present its specifications to three search spaces. As a corollary, we obtain that the convex search algorithm optimises LeadingOnes in [Formula: see text] fitness evaluations, which is faster than all unbiased unary black box algorithms.

Genetic algorithm with advanced mechanisms applied to the protein structure prediction in a hydrophobic-polar model and cubic lattice

We evaluate the small sample size (SSS) performance of evolutionary algorithms (EAs) for relevance feedback (RF) in image retrieval. We focus on the requirement to learn the user’s information need based on a small — between 2 and 25 — number of positive and negative training images. Despite this being a fundamental requirement, none of the existing works dealing with EAs for RF systematically evaluates their SSS performance. To address this issue, we compare four variants of EAs for RF. Common for all variants is the hierarchical, region-based image similarity model, with region and feature weights as parameters. The difference between the variants is in the objective function of the EA used to adjust the model parameters. The objective functions include: (O-1) precision; (O-2) average rank; (O-3) ratio of within-class (i.e., positive images) and between-class (i.e., positive and negative images) scatter; and (O-4) combination of O-2 and O-3. We note that — unlike O-1 and O-2 — O-3 and O-4 are not used in any of the existing works dealing with EAs for RF. The four variants are evaluated on five test databases, containing 61,895 general-purpose images, in 619 semantic categories. Results of the evaluation reveal that variants with objective functions O-3 and O-4 consistently outperform those with O-1 and O-2. Furthermore, comparison with the representative of the existing RF methods shows that EAs are both effective and efficient approaches for SSS learning in region-based image retrieval.

The definition of an investment plan to implement in a distribution power system, is a task that constantly faced by utilities. This work presents a methodology for determining the investment plan for a distribution power system under a short-term, using as a criterion for evaluating investment projects, associated costs and customers benefit from its implementation. Given the number of projects carried out annually on the system, the definition of an investment plan requires the use of computational tools to evaluate, a set of possibilities, the one that best suits the needs of the present system and better results. That is why in the job, implementing a multi objective evolutionary algorithm SPEA (Strength Pareto Evolutionary Algortithm), which, based on the principles of Pareto optimality, it deliver to the planning expert, the best solutions found in the optimization process. The performance of the algorithm is tested using a set of projects to determine the best among the possible plans. We analyze also the effect of operators on the performance of evolutionary algorithm and results..

Modeling the behavior of algorithms is the realm of evolutionary algorithm theory. From a practitioner's point of view, theory must provide some guidelines regarding which algorithm/parameters to use in order to solve a particular problem. Unfortunately, most theoretical models of evolutionary algorithms are difficult to apply to realistic situations. However, in recent work (Graff and Poli, 2008, 2010), where we developed a method to practically estimate the performance of evolutionary program-induction algorithms (EPAs), we started addressing this issue. The method was quite general; however, it suffered from some limitations: it required the identification of a set of reference problems, it required hand picking a distance measure in each particular domain, and the resulting models were opaque, typically being linear combinations of 100 features or more. In this paper, we propose a significant improvement of this technique that overcomes the three limitations of our previous method. We achieve this through the use of a novel set of features for assessing problem difficulty for EPAs which are very general, essentially based on the notion of finite difference. To show the capabilities or our technique and to compare it with our previous performance models, we create models for the same two important classes of problems-symbolic regression on rational functions and Boolean function induction-used in our previous work. We model a variety of EPAs. The comparison showed that for the majority of the algorithms and problem classes, the new method produced much simpler and more accurate models than before. To further illustrate the practicality of the technique and its generality (beyond EPAs), we have also used it to predict the performance of both autoregressive models and EPAs on the problem of wind speed forecasting, obtaining simpler and more accurate models that outperform in all cases our previous performance models.

Metaheuristic optimization procedures such as evolutionary algorithms are usually driven by an objective function that rates the quality of a candidate solution. However, it is not clear in practice whether an objective function adequately rewards intermediate solutions on the path to the global optimum and it may exhibit deceptiveness, epistasis, neutrality, ruggedness, and a lack of causality. In this paper, we introduce the frequency fitness H, subject to minimization, which rates how often solutions with the same objective value have been discovered so far. The ideas behind this method are that good solutions are difficult to find and that if an algorithm gets stuck at a local optimum, the frequency of the objective values of the surrounding solutions will increase over time, which will eventually allow it to leave that region again. We substitute a frequency fitness assignment process (FFA) for the objective function into several different optimization algorithms. We conduct a comprehensive set of experiments: the synthesis of algorithms with genetic programming (GP), the solution of MAX-3SAT problems with genetic algorithms, classification with Memetic Genetic Programming, and numerical optimization with a <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$(1+1)$</tex></formula> Evolution Strategy, to verify the utility of FFA. Given that they have no access to the original objective function at all, it is surprising that for some problems (e.g., the algorithm synthesis task) the FFA-based algorithm variants perform significantly better. However, this cannot be guaranteed for all tested problems. Thus, we also analyze scenarios where algorithms using FFA do not perform better or perform even worse than with the original objective functions.

The runtime analysis of evolutionary algorithms using crossover as search operator has recently produced remarkable results indicating benefits and drawbacks of crossover and illustrating its working principles. Virtually all these results are restricted to upper bounds on the running time of the crossover-based algorithms. This work addresses this lack of lower bounds and rigorously bounds the optimization time of simple algorithms using uniform crossover on the search space $$\{0,1\}^n$$ from below via two novel techniques called decoupling and family graphs. First, a simple steady-state crossover-based evolutionary algorithm without selection pressure is analyzed and shown that after $$O(\mu \log \mu )$$ generations, bit positions are sampled almost independently with marginal probabilities corresponding to the fraction of one-bits at the corresponding position in the initial population. In the presence of weak selective pressure induced by the probabilistic application of tournament selection, it is demonstrated that the inheritance probability at an arbitrary locus quickly approaches a uniform distribution over the initial population up to additive factors that depend on the effect of selection. Afterwards, the algorithm is analyzed by a novel generalization of the family tree technique originally introduced for mutation-only EAs. Using these so-called family graphs, almost tight lower bounds on the optimization time on the OneMax benchmark function are shown.

Many software engineering tasks can potentially be automated using search heuristics. However, much work is needed in designing and evaluating search heuristics before this approach can be routinely applied to a software engineering problem. Experimental methodology should be complemented with theoretical analysis to achieve this goal. Recently, there have been significant theoretical advances in the runtime analysis of evolutionary algorithms (EAs) and other search heuristics in other problem domains. We suggest that these methods could be transferred and adapted to gain insight into the behaviour of search heuristics on software engineering problems while automating software engineering.

One promising method of improving Evolutionary Algorithm (EA) performance is to improve its fine tuning capabilities by using an additional local optimization operator in the evolutionary cycle. This customization on the traditional EA is typically called a Memetic Algorithm (MA). Adding the appropriate local optimization algorithm can increase performance, while a poor choice can decrease performance. Thus, some knowledge is required to select the correct algorithm. In many cases the optimal algorithm selection is not known and it may not be static, but could change during evolution. This investigation combines a method of local optimizer evolution using Push Genetic Programming with a method of automatic, self-configuration called Supportive Coevolution. This combination creates a novel MA that coevolves local optimization operators with target fitness function solution candidates. Implementation methodology is shown and experimentation details with corresponding results are presented. Some additional parameters that were discovered for performance tuning are discussed along with a study of their impact on the algorithm's performance. Discussion of some interesting insights followed by some suggestions for further investigation are also provided. Results show the proposed technique can improve the performance of an EA by providing automatically configured, coevolved local optimization operators to a MA.

Evolutionary algorithms is a class of randomized heuristics inspired by natural evolution. They are applied in many different contexts, in particular in optimization, and analysis of such algorithms has seen tremendous advances in recent years. In this book the author provides an introduction to the methods used to analyze evolutionary algorithms and other randomized search heuristics. He starts with an algorithmic and modular perspective and gives guidelines for the design of evolutionary algorithms. He then places the approach in the broader research context with a chapter on theoretical perspectives. By adopting a complexity-theoretical perspective, he derives general limitations for black-box optimization, yielding lower bounds on the performance of evolutionary algorithms, and then develops general methods for deriving upper and lower bounds step by step. This main part is followed by a chapter covering practical applications of these methods. The notational and mathematical basics are covered in an appendix, the results presented are derived in detail, and each chapter ends with detailed comments and pointers to further reading. So the book is a useful reference for both graduate students and researchers engaged with the theoretical analysis of such algorithms.

In this paper we study the differences in performance among different implementations, in Java, of the data structures used in an evolutionary algorithm (EA). Typical studies on EAs performance deal only with different abstract representations of the data and pay no attention to the fact that each data representation has to be implemented in a concrete programming language which, in general, offers several possibilities, with differences in time consumed, that may be worthy of consideration.

The performance of evolutionary algorithms (EAs) may be enhanced whether the choice of some parameters, as mutation rate and crossover method, is made appropriately. Several methods to adjust those parameters have been developed in order to enhance EAs performance. For this reason, it is important to understand EA dynamics. This paper presents a new population dynamics model to describe and predict the diversity at one generation. The formulation is based on the selection probability density function of each individual. The proposed population dynamics is modeled for an infinite population with generational evolution method. The model was tested in several case studies of different population sizes. The results suggest that the prediction error decreases with the population size increasement.

Artificial neural networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. This paper examines the use of different evolutionary algorithms, imperialist competitive algorithm (ICA), genetic algorithm (GA), ICA-GA and recursive ICA-GA (R-ICA-GA) to train a classification problem on a multi layer perceptron (MLP) neural network. All of named evolutionary training algorithms are compared together in this paper. The first goal of the paper is to apply new evolutionary optimization algorithms ICA-GA and R-ICA-GA for training the ANN and the second goal of the paper is to compare different evolutionary algorithms. It is shown that the ICA-GA has the best performance, in number of epochs, compared to the other algorithms. For this purpose, learning algorithms are applied on six known datasets (WINE, PIMA, WDBC, IRIS, SONAR and GLASS) which are used for classification problems.