Detecting Funnel Structures by Means of Exploratory Landscape Analysis

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

Exploratory Landscape Analysis (ELA) measures have been shown to predict algorithm performance; hence, they are being applied on critical tasks such as automatic algorithm selection and problem generation. This paper provides a cautionary examination on their use in black-box continuous optimization. We explore the effect that translations have on the measures, when the cost function is defined within a bound-constrained region. Furthermore, we examine the robustness of the neighborhood structure after dimensionality reduction. The results demonstrate that a measure may transition abruptly due a translation. Therefore, we should not generalize the measures of an instance nor report average values of a measure as belonging to the generating function. Moreover, dimensionality reduction could alter the neighborhood structure, such that the regions corresponding to significantly different functions overlap.

In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks in the domain of continuous optimization problems, ranging, for example, from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is-to the best of our knowledge-very limited. Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1) a strong correlation between multiple features, as well as (2) its very limited applicability to multiobjective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, among others, point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data. Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. We pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multiobjective optimization problems. Our proposed framework can either be used out of the box for analyzing single- and multiobjective continuous optimization problems, or subsequently fine-tuned to various tasks focusing on algorithm behavior and problem understanding.

Exploratory landscape analysis (ELA) supports supervised learning approaches for automated algorithm selection and configuration by providing sets of features that quantify the most relevant characteristics of the optimization problem at hand. In black-box optimization, where an explicit problem representation is not available, the feature values need to be approximated from a small number of sample points. In practice, uniformly sampled random point sets and Latin hypercube constructions are commonly used sampling strategies. In this work, we analyze how the sampling method and the sample size influence the quality of the feature value approximations and how this quality impacts the accuracy of a standard classification task. While, not unexpectedly, increasing the number of sample points gives more robust estimates for the feature values, to our surprise we find that the feature value approximations for different sampling strategies do not converge to the same value. This implies that approximated feature values cannot be interpreted independently of the underlying sampling strategy. As our classification experiments show, this also implies that the feature approximations used for training a classifier must stem from the same sampling strategy as those used for the actual classification tasks. As a side result we show that classifiers trained with feature values approximated by Sobol' sequences achieve higher accuracy than any of the standard sampling techniques. This may indicate improvement potential for ELA-trained machine learning models.

When selecting the best suited algorithm for an unknown optimization problem, it is useful to possess some a priori knowledge of the problem at hand. In the context of single-objective, continuous optimization problems such knowledge can be retrieved by means of Exploratory Landscape Analysis (ELA), which automatically identifies properties of a landscape, e.g., the so-called funnel structures, based on an initial sample. In this paper, we extract the relevant features (for detecting funnels) out of a large set of landscape features when only given a small initial sample consisting of 50 x D observations, where D is the number of decision space dimensions. This is already in the range of the start population sizes of many evolutionary algorithms. The new Multiple Peaks Model Generator (MPM2) is used for training the classifier, and the approach is then very successfully validated on the Black-Box Optimization Benchmark (BBOB) and a subset of the CEC 2013 niching competition problems.

An important challenge in black-box optimization is to be able to understand the relative performance of different algorithms on problem instances. This challenge has motivated research in exploratory landscape analysis and algorithm selection, leading to a number of frameworks for analysis. However, these procedures often involve significant assumptions, or rely on information not typically available. In this paper we propose a new, model-based framework for the characterization of black-box optimization problems using Gaussian Process regression. The framework allows problem instances to be compared to each other in a relatively simple way. The model-based approach also allows us to assess the goodness of fit and Gaussian Processes lead to an efficient means of model comparison. The implementation of the framework is described and validated on several test sets as one benchmark problem is slowly transformed into another.

Many optimization algorithm benchmarking platforms allow users to share their experimental data to promote reproducible and reusable research. However, different platforms use different data models and formats, which drastically complicates the identification of relevant datasets, their interpretation, and their interoperability. Therefore, a semantically rich, ontology-based, machine-readable data model that can be used by different platforms is highly desirable. In this paper, we report on the development of such an ontology, which we call OPTION (OPTImization algorithm benchmarking ONtology). Our ontology provides the vocabulary needed for semantic annotation of the core entities involved in the benchmarking process, such as algorithms, problems, and evaluation measures. It also provides means for automatic data integration, improved interoperability, and powerful querying capabilities, thereby increasing the value of the benchmarking data. We demonstrate the utility of OPTION, by annotating and querying a corpus of benchmark performance data from the BBOB collection of the COCO framework and from the Yet Another Black-Box Optimization Benchmark (YABBOB) family of the Nevergrad environment. In addition, we integrate features of the BBOB functional performance landscape into the OPTION knowledge base using publicly available datasets with exploratory landscape analysis. Finally, we integrate the OPTION knowledge base into the IOHprofiler environment and provide users with the ability to perform meta-analysis of performance data.

Problem understanding and the ability to assign problems to distinct classes can improve the usability of metaheuristics. A popular problem-independent method for the characterization of optimization problems is exploratory landscape analysis (ELA). It consists of a sequence of operations that describe the hypersurfaces formed by fitness and other characteristic properties of the problem solutions on the basis of a limited number of samples. Sampling is the initial step of ELA that selects a limited number of candidate solutions for which are the characteristic properties evaluated. The solutions and the computed properties serve as the main inputs for the rest of ELA. In this work, we study the impact of different sampling strategies on machine learning-based classification of bi-objective problems on the basis of FLACCO features. A series of computational experiments demonstrates that different sampling strategies affect the value of the resulting landscape features, their suitability for problem classification, and the overhead of the sampling process. An in-depth analysis of the results also shows the relationship between classification accuracy and the structure of the training data set.

When designing a benchmark problem set, it is important to create a set of benchmark problems that are a good generalization of the set of all possible problems. One possible way of easing this difficult task is by using artificially generated problems. In this paper, one such single-objective continuous problem generation approach is analyzed and compared with the COCO benchmark problem set, a well know problem set for benchmarking numerical optimization algorithms. Using Exploratory Landscape Analysis and Singular Value Decomposition, we show that such representations allow us to further explore the relations between the problems by applying visualization and correlation analysis techniques, with the goal of decreasing the bias in benchmark problem assessment.

In this tutorial we highlight the optimal working methodology for discovering novel heterogeneous catalysts using modern tools. First, we give a structure to the discovery and optimisation process, explaining its iterative nature. Then, we focus in turn on each step of catalyst synthesis, catalysts testing, integrating low-level and high-level descriptor models into the workflow, and explorative data analysis. Finally, we explain the importance of experimental and model validation, and show how by combining experimental design, descriptor modelling, and experimental validation you can increase the chances of discovering and optimising good catalysts. The basic principles are illustrated with four concrete examples: oxidative methane coupling; catalytic hydrogenation of 5-ethoxymethylfurfural; optimising bimetallic catalysts in a continuous reactor system, and linking material properties to chemisorption energies for a variety of catalysts. Based on the above examples and principles, we then return to the general case, and discuss the application of data-driven workflows in catalyst discovery and optimisation.

Single-objective continuous optimization can be challenging, especially when dealing with multimodal problems. This work sheds light on the effects that multi-objective optimization may have in the single-objective space. For this purpose, we examine the inner mechanisms of the recently developed sophisticated local search procedure SOMOGSA. This method solves multimodal single-objective continuous optimization problems based on first expanding the problem with an additional objective (e.g., a sphere function) to the bi-objective domain and subsequently exploiting local structures of the resulting landscapes. Our study particularly focuses on the sensitivity of this multiobjectivization approach w.r.t. (1) the parametrization of the artificial second objective, as well as (2) the position of the initial starting points in the search space. As SOMOGSA is a modular framework for encapsulating local search, we integrate Nelder–Mead local search as optimizer in the respective module and compare the performance of the resulting hybrid local search to its original single-objective counterpart. We show that the SOMOGSA framework can significantly boost local search by multiobjectivization. Hence, combined with more sophisticated local search and metaheuristics, this may help solve highly multimodal optimization problems in the future.

In this paper, we propose a set of six test problems for single-objective bilevel optimization. The test-collection represents various difficulties which are commonly encountered in practical bilevel optimization problems. To support experiments with problems of different size, all of the test problems are scalable in terms of the number of variables. The problem set is also accompanied by a construction procedure, which helps to generate new test problems with controlled difficulties in convergence and interaction patterns between the two optimization levels. To provide a baseline result for easy comparisons, we have solved a 10 variable instance for each of the test problems using a simple bilevel evolutionary algorithm. The results presented may be used as a benchmark while evaluating the performance of any bilevel optimization algorithm.

Machine learning research has been able to solve problems in multiple domains. Machine learning represents an open area of research for solving optimisation problems. The optimisation problems can be solved using a metaheuristic algorithm, which can find a solution in a reasonable amount of time. However, the time required to find an appropriate metaheuristic algorithm, that would have the convenient configurations to solve a set of optimisation problems properly presents a problem. The proposal described in this article contemplates an approach that automatically creates metaheuristic algorithms given a set of optimisation problems. These metaheuristic algorithms are created by modifying their logical structure via the execution of an evolutionary process. This process employs an extension of the reinforcement learning approach that considers multi-agents in their environment, and a learning agent composed of an analysis process and a process of modification of the algorithms. The approach succeeded in creating a metaheuristic algorithm that managed to solve different continuous domain optimisation problems from the experiments performed. The implications of this work are immediate because they describe a basis for the generation of metaheuristic algorithms in an online-evolution.

Abstract: This paper presents a new multi-objective evolu-tionary algorithm based on differential evolution. The pro-posed approach adopts a secondary population in order to re-tain the nondominated solutions found during the evolution-ary process. Additionally, the approach also incorporates theconcept of † -dominance to get a good distribution of the solu-tions retained. The main goal of this work was to keep thefast convergence exhibited by Differential Evolution in globaloptimization when extending this heuristic to multi-objectiveoptimization. We adopted standard test functions and perfor-mance measures reported in the specialized literature to vali-date our proposal. Our results are compared with respect toanother multi-objective evolutionary algorithm based on differ-ential evolution (Pareto Differential Evolution) and with respectto two approaches that are representative of the state-of-the-artin the area: the NSGA-II and † -MOEA. I. Introduction Most real world problems involve the simultaneous opti-mization of two or more (often conflicting) objectives. Thesolution of such problems (called “multi-objective”) is differ-ent from that of a single-objective optimization problem. Themain difference is that multi-objective optimization problemsnormally have not one but a set of solutions which are allequally good.In the past, a wide variety of evolutionary algorithms (EA’s)have been used to solve multi-objective optimization prob-lems [5]. However, from the several types of EAs available,few researchers have attempted to extend Differential Evo-lution (DE) [25] to solve multi-objective optimization prob-lems. DE has been very successful in the solution of a va-riety of continuous (single-objective) optimization problemsin which it has shown a great robustness and a very fast con-vergence. These are precisely the characteristics of DE thatmake it attractive to extend it to solve multi-objective opti-mization problems.In this paper, we propose a new multi-objective evolution-ary algorithm based on differential evolution. Our approachuses an external archive in which the relaxed form of Paretodominance known as

An SAT problem is attacked, which is one of the most fundamental problems in Computer Science. A number of algebraic reformulations of SAT are presented as problems of Boolean and continuous optimization. They are based on applying jointly theories of convex extensions of functions, functional continuous representations of sets, and Euclidean combinatorial configurations. This results in the possibility of applying powerful tools of discrete and continuous optimization, including convex, to the exact solution of SAT and approximate one with error estimate evaluation.