Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

When designing a benchmark problem set, it is important to create a set of benchmark problems that are a good generalization of the set of all possible problems. One possible way of easing this difficult task is by using artificially generated problems. In this paper, one such single-objective continuous problem generation approach is analyzed and compared with the COCO benchmark problem set, a well know problem set for benchmarking numerical optimization algorithms. Using Exploratory Landscape Analysis and Singular Value Decomposition, we show that such representations allow us to further explore the relations between the problems by applying visualization and correlation analysis techniques, with the goal of decreasing the bias in benchmark problem assessment.

In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks in the domain of continuous optimization problems, ranging, for example, from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is-to the best of our knowledge-very limited. Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1) a strong correlation between multiple features, as well as (2) its very limited applicability to multiobjective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, among others, point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data. Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. We pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multiobjective optimization problems. Our proposed framework can either be used out of the box for analyzing single- and multiobjective continuous optimization problems, or subsequently fine-tuned to various tasks focusing on algorithm behavior and problem understanding.

The Guided Genetic Algorithm (GCA) is a hybrid of Genetic Algorithm and Guided Local Search, a meta-heuristic search algorithm. As the search progresses, GGA modifies both the fitness function and fitness template of candidate solutions based on feedback from constraints. The fitness template is then used to bias crossover and mutation. The Radio Link Frequency Assignment Problem (RLFAP) is a class of problem that has practical relevance to both military and civil applications. In this paper, we show how GGA can be applied to the RLFAP. We focus on an abstraction of a real life military application that involves the assigning of frequencies to radio links. GGA was tested on a set of eleven benchmark problems provided by the French military. This set of problems has been studied intensively by a number of prominent groups in Europe. It covers a variety of needs in military applications, including the satisfaction of constraints, finding optimal solutions that satisfy all the constraints and optimization of some objective functions whenever no solution exist (“partial constraint satisfaction”). Not only do these benchmark problems vary in problem nature, they are reasonably large for military applications (up to 916 variables, and up to 5548 constraints). This makes them a serious challenge to the generality, reliability as well as efficiency of algorithms. We show in this paper that GGA is capable of producing excellent results reliably in the whole set of benchmark problems.

In single-objective optimization different optimization strategies exist depending on the structure and characteristics of the underlying problem. In particular, the presence of so-called funnels in multimodal problems offers the possibility of applying techniques exploiting the global structure of the function. The recently proposed Exploratory Landscape Analysis approach automatically identifies problem characteristics based on a moderately small initial sample of the objective function and proved to be effective for algorithm selection problems in continuous black-box optimization. In this paper, specific features for detecting funnel structures are introduced and combined with the existing ones in order to classify optimization problems regarding the funnel property. The effectiveness of the approach is shown by experiments on specifically generated test instances and validation experiments on standard benchmark problems.

PSO-sono: A novel PSO variant for single-objective numerical optimization

Exploratory landscape analysis (ELA) supports supervised learning approaches for automated algorithm selection and configuration by providing sets of features that quantify the most relevant characteristics of the optimization problem at hand. In black-box optimization, where an explicit problem representation is not available, the feature values need to be approximated from a small number of sample points. In practice, uniformly sampled random point sets and Latin hypercube constructions are commonly used sampling strategies. In this work, we analyze how the sampling method and the sample size influence the quality of the feature value approximations and how this quality impacts the accuracy of a standard classification task. While, not unexpectedly, increasing the number of sample points gives more robust estimates for the feature values, to our surprise we find that the feature value approximations for different sampling strategies do not converge to the same value. This implies that approximated feature values cannot be interpreted independently of the underlying sampling strategy. As our classification experiments show, this also implies that the feature approximations used for training a classifier must stem from the same sampling strategy as those used for the actual classification tasks. As a side result we show that classifiers trained with feature values approximated by Sobol' sequences achieve higher accuracy than any of the standard sampling techniques. This may indicate improvement potential for ELA-trained machine learning models.

Benchmark problems for numerical implementations of phase field models

An important challenge in black-box optimization is to be able to understand the relative performance of different algorithms on problem instances. This challenge has motivated research in exploratory landscape analysis and algorithm selection, leading to a number of frameworks for analysis. However, these procedures often involve significant assumptions, or rely on information not typically available. In this paper we propose a new, model-based framework for the characterization of black-box optimization problems using Gaussian Process regression. The framework allows problem instances to be compared to each other in a relatively simple way. The model-based approach also allows us to assess the goodness of fit and Gaussian Processes lead to an efficient means of model comparison. The implementation of the framework is described and validated on several test sets as one benchmark problem is slowly transformed into another.

Many optimization algorithm benchmarking platforms allow users to share their experimental data to promote reproducible and reusable research. However, different platforms use different data models and formats, which drastically complicates the identification of relevant datasets, their interpretation, and their interoperability. Therefore, a semantically rich, ontology-based, machine-readable data model that can be used by different platforms is highly desirable. In this paper, we report on the development of such an ontology, which we call OPTION (OPTImization algorithm benchmarking ONtology). Our ontology provides the vocabulary needed for semantic annotation of the core entities involved in the benchmarking process, such as algorithms, problems, and evaluation measures. It also provides means for automatic data integration, improved interoperability, and powerful querying capabilities, thereby increasing the value of the benchmarking data. We demonstrate the utility of OPTION, by annotating and querying a corpus of benchmark performance data from the BBOB collection of the COCO framework and from the Yet Another Black-Box Optimization Benchmark (YABBOB) family of the Nevergrad environment. In addition, we integrate features of the BBOB functional performance landscape into the OPTION knowledge base using publicly available datasets with exploratory landscape analysis. Finally, we integrate the OPTION knowledge base into the IOHprofiler environment and provide users with the ability to perform meta-analysis of performance data.

When selecting the best suited algorithm for an unknown optimization problem, it is useful to possess some a priori knowledge of the problem at hand. In the context of single-objective, continuous optimization problems such knowledge can be retrieved by means of Exploratory Landscape Analysis (ELA), which automatically identifies properties of a landscape, e.g., the so-called funnel structures, based on an initial sample. In this paper, we extract the relevant features (for detecting funnels) out of a large set of landscape features when only given a small initial sample consisting of 50 x D observations, where D is the number of decision space dimensions. This is already in the range of the start population sizes of many evolutionary algorithms. The new Multiple Peaks Model Generator (MPM2) is used for training the classifier, and the approach is then very successfully validated on the Black-Box Optimization Benchmark (BBOB) and a subset of the CEC 2013 niching competition problems.

A new differential evolution algorithm for single objective optimization is presented in this paper. The proposed algorithm uses a self-adaptation mechanism for parameter control, divides its population into more subpopulations, applies more DE strategies, promotes population diversity, and eliminates the individuals that are not changed during some generations. The experimental results obtained by our algorithm on the benchmark consisting of 25 test functions with dimensions D = 10, D = 30, and D = 50 as provided for the CEC 2013 competition and special session on Real Parameter Single Objective Optimization are presented.

Recently, multi-tasking optimization (MTO) has become a rising research topic in the field of evolutionary computation that has attracted increasing attention of academia. Comparing with single-objective optimization (SOO) and multi-objective optimization (MOO), MTO can solve different optimization tasks simultaneously by utilizing inter-task similarities and complementarities. Based on crossover operator, the classical multifactorial evolutionary algorithm (MFEA) transfers inter-task knowledge. To broaden the search region and accelerate the convergence, this paper integrates differential evolution (DE) and opposition-based learning (OBL) into MFEA and hence proposes MFEA/DE-OBL. The motivation of integrating DE and OBL is that they have different search neighborhoods and strong complementarity with simulated binary crossover (SBX) used in MFEA. Furthermore, integrating DE and OBL can help MFEA jump out of local optima. The effectiveness and efficiency of integrating DE and OBL into MFEA are experimentally studied on a set of benchmark problems with different degrees of similarities. Experimental results demonstrate that the proposed MFEA/DE-OBL dramatically improves the performance compared with the MFEA.

The need for Out-Of-Position (OOP) simulation capabilities in crash safety software has risen in importance in the automotive industry after the final ruling of FMVSS 208 by the National Highway Transportation Safety Administration (NHTSA) in 2000. However, because of current technical challenges, the thermodynamics airbag models are not capable of either accurately simulating the flow-bag interactions under OOP conditions, or differentiating the effect of some important design changes, such as vent locations, inflator configuration, and flow diverging devices. The development of these capabilities entails overcoming tremendous technical challenges and numerical difficulties in computational fluid dynamics (CFD) and crash simulations due to the complexity and extreme conditions of OOP. This paper summarizes the developments of algorithms used in the context of an Arbitrary Lagrangian-Eulerian (ALE) formulation. The main developments of this study include the gas dynamic model for mixing gases, the special treatment of inflator gas flows, a penalty based fluid-structure coupling algorithm, and a permeability algorithm for porous fabrics. To expedite the developments and resolve the technical difficulties, a set of benchmark problems was used in this study. Each of the benchmark problems addresses specific technical difficulties of airbag OOP simulations by comparing simulation results to analytical solutions, well-known numerical solutions, or test results. The benchmark set was designed to start from simple CFD problems and progress to the most complicated OOP applications such that the weakness and algorithmic errors of the simulation code can be easily identified. Simulation results of the benchmark problems and the issues addressed by them are discussed in this paper. The simulation results have demonstrated that LS-DYNA now has the capabilities to simulate certain OOP problems.

A reconsideration on the resolution of phase stability analysis using stochastic global optimization methods: Proposal of a reliable set of benchmark problems

Problem understanding and the ability to assign problems to distinct classes can improve the usability of metaheuristics. A popular problem-independent method for the characterization of optimization problems is exploratory landscape analysis (ELA). It consists of a sequence of operations that describe the hypersurfaces formed by fitness and other characteristic properties of the problem solutions on the basis of a limited number of samples. Sampling is the initial step of ELA that selects a limited number of candidate solutions for which are the characteristic properties evaluated. The solutions and the computed properties serve as the main inputs for the rest of ELA. In this work, we study the impact of different sampling strategies on machine learning-based classification of bi-objective problems on the basis of FLACCO features. A series of computational experiments demonstrates that different sampling strategies affect the value of the resulting landscape features, their suitability for problem classification, and the overhead of the sampling process. An in-depth analysis of the results also shows the relationship between classification accuracy and the structure of the training data set.