Low-Budget Exploratory Landscape Analysis on Multiple Peaks Models

In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks in the domain of continuous optimization problems, ranging, for example, from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is-to the best of our knowledge-very limited. Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1) a strong correlation between multiple features, as well as (2) its very limited applicability to multiobjective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, among others, point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data. Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. We pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multiobjective optimization problems. Our proposed framework can either be used out of the box for analyzing single- and multiobjective continuous optimization problems, or subsequently fine-tuned to various tasks focusing on algorithm behavior and problem understanding.

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

This paper analyzes the performance of the Fractal Decomposition Algorithm (FDA) metaheuristic applied to low-dimensional continuous optimization problems. This algorithm was originally developed specifically to deal efficiently with high-dimensional continuous optimization problems by building a fractal-based search tree with a branching factor linearly proportional to the number of dimensions. Here, we aim to answer the question of whether FDA could be equally effective for low-dimensional problems. For this purpose, we evaluate the performance of FDA on the Black Box Optimization Benchmark (BBOB) for dimensions 2, 3, 5, 10, 20, and 40. The experimental results show that overall the FDA in its current form does not perform well enough. Among different function groups, FDA shows its best performance on Misc. moderate and Weak structure functions.KeywordsContinuous optimizationMetaheuristicsFractal decompositionBlack Box Optimization Benchmark

Many optimization algorithm benchmarking platforms allow users to share their experimental data to promote reproducible and reusable research. However, different platforms use different data models and formats, which drastically complicates the identification of relevant datasets, their interpretation, and their interoperability. Therefore, a semantically rich, ontology-based, machine-readable data model that can be used by different platforms is highly desirable. In this paper, we report on the development of such an ontology, which we call OPTION (OPTImization algorithm benchmarking ONtology). Our ontology provides the vocabulary needed for semantic annotation of the core entities involved in the benchmarking process, such as algorithms, problems, and evaluation measures. It also provides means for automatic data integration, improved interoperability, and powerful querying capabilities, thereby increasing the value of the benchmarking data. We demonstrate the utility of OPTION, by annotating and querying a corpus of benchmark performance data from the BBOB collection of the COCO framework and from the Yet Another Black-Box Optimization Benchmark (YABBOB) family of the Nevergrad environment. In addition, we integrate features of the BBOB functional performance landscape into the OPTION knowledge base using publicly available datasets with exploratory landscape analysis. Finally, we integrate the OPTION knowledge base into the IOHprofiler environment and provide users with the ability to perform meta-analysis of performance data.

Population-based approaches have given us new search strategies and ideas in order to solve optimization problems. Usually, these methods are based on the performance carried out by a finite number of agents, which by the interaction between them they evolve and work all over the search space. Also, it is well-known that the correct employment of parameter values in this kind of method can positively impact their performance and behavior. In this context, the present work focuses on the design of a hybrid architecture which smartly balances the population size on run-time. In order to smartly balance and control the population size, a modular approach, named Linear Modular Population Balancer (LMPB), is proposed. The main ideas behind the designed architecture include the solving strategy behind a population-based metaheuristic, the influence of learning components based on multiple statistical modeling methods which transform the dynamic data generated into knowledge, and the possibilities to tackle both discrete and continuous optimization problems. In this regard, three modules are proposed for LMPB, which concern tasks such as the management of the population-based algorithm, parameter setting, probabilities, learning methods, and selection mechanism for the population size to employ. In order to test the viability and effectiveness of our proposed approach, we solve a set of well-known benchmark functions and the multidimensional knapsack problem (MKP). Additionally, we illustrate promising solving results, compare them against state-of-the-art methods which have proved to be good options for solving optimization problems, and give solid arguments for future work in the necessity to keep evolving this type of proposed architecture.

Virulence Optimization Algorithm

Exploratory landscape analysis (ELA) supports supervised learning approaches for automated algorithm selection and configuration by providing sets of features that quantify the most relevant characteristics of the optimization problem at hand. In black-box optimization, where an explicit problem representation is not available, the feature values need to be approximated from a small number of sample points. In practice, uniformly sampled random point sets and Latin hypercube constructions are commonly used sampling strategies. In this work, we analyze how the sampling method and the sample size influence the quality of the feature value approximations and how this quality impacts the accuracy of a standard classification task. While, not unexpectedly, increasing the number of sample points gives more robust estimates for the feature values, to our surprise we find that the feature value approximations for different sampling strategies do not converge to the same value. This implies that approximated feature values cannot be interpreted independently of the underlying sampling strategy. As our classification experiments show, this also implies that the feature approximations used for training a classifier must stem from the same sampling strategy as those used for the actual classification tasks. As a side result we show that classifiers trained with feature values approximated by Sobol' sequences achieve higher accuracy than any of the standard sampling techniques. This may indicate improvement potential for ELA-trained machine learning models.

Optimization problems with continuous variables are very often in real-life scenarios. This is why several studies have focused on the use of metaheuristics to solve these problems, known as continuous optimization problems. This chapter focuses on the robust population-based metaheuristic Variable Mesh Optimization (VMO). We review the fundamental aspects of VMO for continuous optimization. Moreover, we analyze the variants of VMO that have been proposed to solve multimodal problems, i.e. those with multiple optima at several search subspaces. The chapter uses a homogeneous algorithmic framework to describe VMO and its variants with niching strategies for multimodal problems.KeywordsMetaheuristicsVariable mesh optimizationContinuous optimization problemsMultimodal optimization problemsNiching

In single-objective optimization different optimization strategies exist depending on the structure and characteristics of the underlying problem. In particular, the presence of so-called funnels in multimodal problems offers the possibility of applying techniques exploiting the global structure of the function. The recently proposed Exploratory Landscape Analysis approach automatically identifies problem characteristics based on a moderately small initial sample of the objective function and proved to be effective for algorithm selection problems in continuous black-box optimization. In this paper, specific features for detecting funnel structures are introduced and combined with the existing ones in order to classify optimization problems regarding the funnel property. The effectiveness of the approach is shown by experiments on specifically generated test instances and validation experiments on standard benchmark problems.

Connectionist networks for pivot selection in linear programming

Memetic algorithms arise as very effective algorithms to obtain reliable and high accurate solutions for complex continuous optimization problems. Nowadays, high dimensional optimization problems are an interesting field of research. The high dimensionality introduces new problems for the optimization process, making recommendable to test the behavior of the optimization algorithms to large-scale problems. The local search method must be applied with a higher intensity, specially to most promising solutions, to explore the higher domain space around each solution. In this work, we present a preliminar study of a memetic algorithm that assigns to each individual a local search intensity that depends on its features, by chaining different local search applications. This algorithm have obtained good results in continuous optimization and we study whether is a good algorithm for large scale optimizations problems. We make experiments of our proposal using the benchmark problems defined in the Special Session or Competition on Large Scale Global Optimisation, on the IEEE Congress on Evolutionary Computation in 2008. First, we test different local search methods to identify the best one. Then, we compare the proposed algorithm with the algorithms used into the competition, obtaining that our proposal is a very promising algorithm for this type of high-dimensional problems: with dimension 500 our proposal is the second best of the compared algorithms, and the best memetic algorithm.

One of the most promising innovations in the area of heuristics is the development of evolutionary algorithms. A valuable and novel proposition in this area is ant algorithms. Researchers examining the behavior of real ants developed algorithms and applied them to many optimization problems. Based on classical ant algorithm, a method for solving optimization problem with continuous parameters using ant colony algorithm is proposed in this paper. In the method, the size of artificial ant colony is determined according to the constrained field of the problem, the amount of the change in objective function is introduced as heuristic factor of the algorithm. The searching region is reduced, moved and modified according to the transition probability dynamically. Our experimental results in continuous optimization problem show that this method has much higher convergence speed and the disadvantage of classical ant colony algorithm of not being suitable for solving continuous optimization problems is overcome.

In this paper, we consider a general class of nonlinear mixed discrete programming problems. By introducing continuous variables to replace the discrete variables, the problem is first transformed into an equivalent nonlinear continuous optimization problem subject to original constraints and additional linear and quadratic constraints. Then, an exact penalty function is employed to construct a sequence of unconstrained optimization problems, each of which can be solved effectively by unconstrained optimization techniques, such as conjugate gradient or quasi-Newton methods. It is shown that any local optimal solution of the unconstrained optimization problem is a local optimal solution of the transformed nonlinear constrained continuous optimization problem when the penalty parameter is sufficiently large. Numerical experiments are carried out to test the efficiency of the proposed method.

Tree-seed algorithm (TSA) is a nature-inspired and population-based algorithm for solving continuous optimization problems. The tree-seed relationship is the main motivation of this algorithm. TSA has only two peculiar parameters which are the total number of trees in the stand (pop) and the controller of the seed production (search tendency). Although many problems have been solved in the literature by TSA which is the successful optimizer for low dimensional unconstrained continuous problems, real-life problems have not been addressed yet. In this study, six continuous unconstrained real-life optimization problems (gas transmission compressor design, optimal capacity of gas production facility, gear train design, frequency modulation sounds parameter identification, the spread spectrum radar polyphase code design with 10 decision variables, the spread spectrum radar polyphase code design with 20 decision variables) have been solved. It is seen that choosing the number of the population as 50 and the value of search tendency as 0.1 is appropriate according to experimental results for these problems.

Quantum computing is a promising and emerging field of computer science with the potential to improve artificial intelligence techniques, particularly those related to optimization. However, current quantum devices operate in the Noisy Intermediate-Scale Quantum era, in which quantum noise can significantly compromise computational reliability. Evolutionary algorithms are inherently stochastic and may therefore be less sensitive to such noise, or even benefit from it. This study evaluates the performance of a hybrid quantum genetic algorithm on a set of combinatorial and continuous optimization problems under various realistic noise models. The findings demonstrate that quantum noise does not hinder the algorithm's performance and can even enhance its exploratory capabilities in some cases. This suggests that noise-aware quantum evolutionary strategies have the potential to be effective even with today's imperfect quantum hardware.