Selecting a small number of representative non-dominated solutions by a hypervolume-based solution selection approach

In recent years, evolutionary multiobjective optimization (EMO) algorithms have frequently been used for engineering problems with some conflicting objective functions to be simultaneously optimized. EMO algorithms can provide a number of Pareto optimal solutions to users. Two scenarios are considered in the practical use of EMO algorithms. One is that a decision maker selects a single solution from the obtained ones after the EMO process. The other is that a decision maker utilizes the solutions to analyze the relationship between design variables and objective functions of the corresponding problem. In this paper, we apply fuzzy genetics-based machine learning to the second scenario in order to generate if-then rule-based classifiers which represent the relationship between design variables and objective functions. We also utilize this method during the EMO process to pre-screen candidate offspring solutions. The classifier detects non-promising offspring solutions. Then, they are discarded before their fitness evaluation, so that the computation resource is used only for promising solutions. We apply this method to one engineering problem and examine its effect on the search performance of an EMO algorithm.

Various evolutionary multiobjective optimization (EMO) algorithms have been used in the field of evolutionary fuzzy systems (EFS), because EMO algorithms can easily handle multiple objective functions such as the accuracy maximization and complexity minimization for fuzzy system design. Most EMO algorithms used in EFS are Pareto dominance-based algorithms such as NSGA-II, SPEA2, and PAES. There are a few studies where other types of EMO algorithms are used in EFS. In this paper, we apply a multiobjective evolutionary algorithm based on decomposition called MOEA/D to EFS for fuzzy classifier design. MOEA/D is one of the most well-known decomposition-based EMO algorithms. The key idea is to divide a multiobjective optimization problem into a number of single-objective problems using a set of uniformly distributed weight vectors in a scalarizing function. We propose a new scalarizing function called an accuracy-oriented function (AOF) which is specialized for classifier design. We examine the effects of using AOF in MOEA/D on the search ability of our multiobjective fuzzy genetics-based machine learning (GBML). We also examine the synergy effect of MOEA/D with AOF and parallel distributed implementation of fuzzy GBML on the generalization ability.

It has been reported for multi-objective knapsack problems that the recombination of similar parents often improves the performance of evolutionary multi-objective optimization (EMO) algorithms. Recently performance improvement was also reported by exchanging only a small number of genes between two parents (i.e., crossover with a very small gene exchange probability) without choosing similar parents. In this paper, we examine these performance improvement schemes through computational experiments where NSGA-II is applied to 500-item knapsack problems with 2-10 objectives. We measure the parent-parent distance and the parent-offspring distance in computational experiments. Clear performance improvement is observed when the parent-offspring distance is small. To further examine this observation, we implement a distance-based crossover operator where the parent-offspring distance is specified as a user-defined parameter. Performance of NSGA-II is examined for various parameter values. Experimental results show that an appropriate parameter value (parent-offspring distance) is surprisingly small. It is also shown that a very small parameter value is beneficial for diversity maintenance.

One advantage of evolutionary multiobjective optimization (EMO) algorithms over classical approaches is that many non-dominated solutions can be simultaneously obtained by their single run. This paper shows how this advantage can be utilized in genetic rule selection for the design of fuzzy rule-based classification systems. Our genetic rule selection is a two-stage approach. In the first stage, a pre-specified number of candidate rules are extracted from numerical data using a data mining technique. In the second stage, an EMO algorithm is used for finding non-dominated rule sets with respect to three objectives: to maximize the number of correctly classified training patterns, to minimize the number of rules, and to minimize the total rule length. Since the first objective is measured on training patterns, the evolution of rule sets tends to overfit to training patterns. The question is whether the other two objectives work as a safeguard against the overfitting. In this paper, we examine the effect of the three-objective formulation on the generalization ability (i.e., classification rates on test patterns) of obtained rule sets through computer simulations where many non-dominated rule sets are generated using an EMO algorithm for a number of high-dimensional pattern classification problems.

In the past few years, multi-objective optimization algorithms have been extensively applied in several fields including engineering design problems. A major reason is the advancement of evolutionary multi-objective optimization (EMO) algorithms that are able to find a set of non-dominated points spread on the respective Pareto-optimal front in a single simulation. Besides just finding a set of Pareto-optimal solutions, one is often interested in capturing knowledge about the variation of variable values over the Pareto-optimal front. Recent innovization approaches for knowledge discovery from Pareto-optimal solutions remain as a major activity in this direction. In this article, a different data-fitting approach for continuous parameterization of the Pareto-optimal front is presented. Cubic B-spline basis functions are used for fitting the data returned by an EMO procedure in a continuous variable space. No prior knowledge about the order in the data is assumed. An automatic procedure for detecting gaps in the Pareto-optimal front is also implemented. The algorithm takes points returned by the EMO as input and returns the control points of the B-spline manifold representing the Pareto-optimal set. Results for several standard and engineering, bi-objective and tri-objective optimization problems demonstrate the usefulness of the proposed procedure.

Preference-based Evolutionary Multiobjective Optimization (EMO) algorithms approximate the region of interest (ROI) of the Pareto optimal front defined by the preferences of a decision maker (DM). Here, we propose a preference-based EMO algorithm, in which the preferences are given by means of aspiration and reservation points. The aspiration point is formed by objective values which the DM wants to achieve, while the reservation point is constituted by values for the objectives not to be worsened. Internally, the first generations are performed in order to generate an initial approximation set according to the reservation point. Next, in the remaining generations, the algorithm adapts the search for new non-dominated solutions depending on the dominance relation between the solutions obtained so far and both the reservation and aspiration points. This allows knowing if the given points are achievable or not; this type of information cannot be known before the solution process starts. On this basis, the algorithm proceeds according to three different scenarios with the aim of re-orienting the search directions towards the ROI formed by the Pareto optimal solutions with objective values within the given aspiration and reservation values. Computational results show the potential of our proposal in 2, 3 and 5-objective test problems, in comparison to other state-of-the-art algorithms.

The hypervolume indicator has frequently been used for comparing evolutionary multi-objective optimization (EMO) algorithms. A reference point is needed for hypervolume calculation. However, its specification has not been discussed in detail from a viewpoint of fair performance comparison. A slightly worse point than the nadir point is usually used for hypervolume calculation in the EMO community. In this paper, we propose a reference point specification method for fair performance comparison of EMO algorithms. First, we discuss the relation between the reference point specification and the optimal distribution of solutions for hypervolume maximization. It is demonstrated that the optimal distribution of solutions strongly depends on the location of the reference point when a multi-objective problem has an inverted triangular Pareto front. Next, we propose a reference point specification method based on theoretical discussions on the optimal distribution of solutions. The basic idea is to specify the reference point so that a set of well-distributed solutions over the entire linear Pareto front has a large hypervolume and all solutions in such a solution set have similar hypervolume contributions. Then, we examine whether the proposed method can appropriately specify the reference point through computational experiments on various test problems. Finally, we examine the usefulness of the proposed method in a hypervolume-based EMO algorithm. Our discussions and experimental results clearly show that a slightly worse point than the nadir point is not always appropriate for performance comparison of EMO algorithms.

In a short span of about 14 years, evolutionary multi-objective optimization (EMO) has established itself as a mature field of research and application with an extensive literature, many commercial softwares, numerous freely downloadable codes, a dedicated biannual conference running successfully four times so far since 2001, special sessions and workshops held at all major evolutionary computing conferences, and full-time researchers from universities and industries from all around the globe. In this paper, we make a brief outline of EMO principles, some EMO algorithms, and focus on current research and application potential of EMO. Besides, simply finding a set of Pareto-optimal solutions, EMO research has now diversified in hybridizing its search with multi-criterion decision-making tools to arrive at a single preferred solution, in utilizing EMO principle in solving different kinds of single-objective optimization problems efficiently, and in various interesting application domains which were not possible to be solved adequately due to the lack of a suitable solution technique.

Evolutionary Multi-objective Optimization (EMO) is a hot research direction nowadays and one of the state-of-the-art evolutionary multi-objective optimization algorithms ——NSGA-II has gain wide attention and application in many fields. Gene Expression Programming (GEP) has a powerful search capability, but falls into local optimum easily. Based on the transformed GEP, NSGA-II and the virus evolution mechanism, a new multi-objective evolutionary algorithm GEP Virus NSGA-II is proposed. With the infection operation of virus population, the diversity of the host population is increased, and it’s much easier to jump out of the local optimum. And this algorithm has got good experimental results on 9 standard test problems.

Evolutionary algorithms have been widely used to tackle multi-objective optimization problems. Despite many successful applications there is a lack of theoretical knowledge on how and why this type of algorithm works particularly well for multi-objective problems. This special issue presents current advances to the theoretical understanding of evolutionary algorithms for multi-objective optimization. It comprises four very high quality papers which have been selected via a rigorous reviewing process. The first paper “On the Effect of Populations in Evolutionary Multi-Objective Optimisation” by Oliver Giel and Per Kristian Lehre studies the basic question whether a population is provably useful to compute the whole Pareto front of a multi-objective optimization problem. They compare restart strategies of several single-objective approaches based on one individual to a natural multi-objective approach using a population. Their analyses point out the benefits of the multi-objective evolutionary algorithm in a rigorous way. Using illustrative example functions they provide not only proven results but also a good intuition about why population-based approaches can outperform individual-based approaches in multi-objective optimization. The paper “Exploring the Runtime of an Evolutionary Algorithm for the Multi-Objective Shortest Path Problem” by Christian Horoba examines the behavior of evolutionary algorithms for the classical NP-hard multi-objective shortest path problem. The author considers an evolutionary approach based on -dominance and proves that this algorithm constitutes a fully polynomial-time randomized approximation scheme (FPRAS) for the problem. The algorithm achieves this by mimicking a problemspecific algorithm for that problem. This provides general insight into why evolutionary algorithms can perform well for problems that allow for efficient problem-specific algorithms. Hypervolume-based algorithms have become very popular in evolutionary multiobjective optimization. In the paper, “An Efficient Algorithm for Computing Hypervolume Contributions,” Karl Bringmann and Tobias Friedrich present a faster algorithm for determining the individual with the smallest hypervolume contribution in a given population. Furthermore, they study the problem of removing λ > 1 individuals with the smallest hypervolume contribution from a given population and present an elegant and fast method to carry out this task.

Evolutionary algorithm has gained a worldwide popularity among multi-objective optimization. The paper proposes a self-adaptive evolutionary algorithm (called SEA) for multi-objective optimization. In the SEA, the probability of crossover and mutation, P cand P m, are varied depending on the fitness values of the solutions. Fitness assignment of SEA realizes the twin goals of maintaining diversity in the population and guiding the population to the true Pareto Front; fitness value of individual not only depends on improved density estimation but also depends on non-dominated rank. The density estimation can keep diversity in all instances including when scalars of all objectives are much different from each other. SEA is compared against the Non-dominated Sorting Genetic Algorithm (NSGA-II) on a set of test problems introduced by the MOEA community. Simulated results show that SEA is as effective as NSGA-II in most of test functions, but when scalar of objectives are much different from each other, SEA has better distribution of non-dominated solutions.

In evolutionary multi-objective optimization (EMO) algorithms, diversity maintenance has been mainly discussed in the objective space in order to search for uniformly distributed non-dominated solutions along the entire Pareto front. In this paper, we propose three types of many-objective test problems with multiple Pareto optimal regions in the decision space. One type has multiple equivalent Pareto optimal regions. Another type has different but somewhat similar Pareto optimal regions. The other type has Pareto and local Pareto optimal regions. Our many-objective test problems are generated by placing multiple polygons of the same or similar shapes in a decision space. The ith objective is the minimization of the distance from a solution to the nearest ith vertex over all polygons. Thus the number of objectives is the same as the number of vertices of the polygons. The number of equivalent or similar Pareto regions in the decision space is the same as the number of the polygons.

Multi-clustering via evolutionary multi-objective optimization

Handling of many-objective problems is a hot issue in the evolutionary multiobjective optimization (EMO) community. It is well-known that frequently-used EMO algorithms such as NSGA-II and SPEA do not work well on many-objective problems whereas they have been successfully applied to a large number of test problems and real-world application tasks with two or three objectives. The main difficulty in the handling of many-objective problems is that almost all solutions in the current population of an EMO algorithm are non-dominated with each other. This means that Pareto dominance relation cannot generate enough selection pressure toward the Pareto front. As a result, Pareto dominance-based EMO algorithms such as NSGA-II and SPEA cannot drive the current population toward the Pareto front efficiently in a high-dimensional objective space of a many-objective problem. A simple idea for introducing additional selection pressure toward the Pareto front is the use of scalarizing fitness functions. In this paper, we examine the effect of using weighted sum fitness functions for parent selection and generation update on the performance of NSGA-II for many-objective 0/1 knapsack problems.

An important issue in hypervolume-based evolutionary multi-objective optimization (EMO) algorithms is the specification of a reference point for hypervolume calculation. However, its appropriate specification has not been carefully studied in the literature. Some recent studies have pointed out the importance and difficulty of the reference point specification. Its appropriate specification depends on problem characteristics such as the Pareto front shape and the number of objectives. In this paper, the difficulty of the reference point specification in hypervolume-based EMO algorithms is circumvented by using two reference points. Instead of using only a single reference point, we propose a new hypervolume-based EMO algorithm that can effectively utilize two reference points cooperatively. Experimental results show that the proposed algorithm has good and robust performance on a wide range of test problems. In comparison to hypervolume-based EMO algorithms with only a single reference point, the proposed algorithm can find a wider and more uniformly distributed solution set. On a recently proposed real-world problem suite, the proposed algorithm shows competitive performance in comparison to state-of-the-art algorithms.