Distance-Based Analysis of Crossover Operators for Many-Objective Knapsack Problems

Various evolutionary multiobjective optimization (EMO) algorithms have been used in the field of evolutionary fuzzy systems (EFS), because EMO algorithms can easily handle multiple objective functions such as the accuracy maximization and complexity minimization for fuzzy system design. Most EMO algorithms used in EFS are Pareto dominance-based algorithms such as NSGA-II, SPEA2, and PAES. There are a few studies where other types of EMO algorithms are used in EFS. In this paper, we apply a multiobjective evolutionary algorithm based on decomposition called MOEA/D to EFS for fuzzy classifier design. MOEA/D is one of the most well-known decomposition-based EMO algorithms. The key idea is to divide a multiobjective optimization problem into a number of single-objective problems using a set of uniformly distributed weight vectors in a scalarizing function. We propose a new scalarizing function called an accuracy-oriented function (AOF) which is specialized for classifier design. We examine the effects of using AOF in MOEA/D on the search ability of our multiobjective fuzzy genetics-based machine learning (GBML). We also examine the synergy effect of MOEA/D with AOF and parallel distributed implementation of fuzzy GBML on the generalization ability.

A large number of non-dominated solutions are often obtained by a single run of an evolutionary multiobjective optimization (EMO) algorithm. In the EMO research area, it is usually assumed that a single solution is to be chosen from the obtained non-dominated solutions by the decision maker. It is, however, time-consuming and not easy for the decision maker to examine a large number of obtained non-dominated solutions. Motivated by these discussions, we proposed single-objective and multiobjective formulations of solution selection problems to present only a small number of representative non-dominated solutions to the decision maker in our former study. The basic idea is to minimize the number of solutions to be presented while maximizing their hypervolume. A number of single-objective formulations can be derived from such a two-objective solution selection problem. In this paper, single-objective rule selection is performed as a post-processing procedure of EMO algorithms to select a prespecified number of non-dominated solutions (e.g., 10 or 20 solutions). Through computational experiments on multiobjective 0/1 knapsack problems, we examine the characteristic features of selected non-dominated solutions. We also examine the effect of the choice of a reference point for hypervolume calculation on the distribution of selected non-dominated solutions.

In recent years, evolutionary multiobjective optimization (EMO) algorithms have frequently been used for engineering problems with some conflicting objective functions to be simultaneously optimized. EMO algorithms can provide a number of Pareto optimal solutions to users. Two scenarios are considered in the practical use of EMO algorithms. One is that a decision maker selects a single solution from the obtained ones after the EMO process. The other is that a decision maker utilizes the solutions to analyze the relationship between design variables and objective functions of the corresponding problem. In this paper, we apply fuzzy genetics-based machine learning to the second scenario in order to generate if-then rule-based classifiers which represent the relationship between design variables and objective functions. We also utilize this method during the EMO process to pre-screen candidate offspring solutions. The classifier detects non-promising offspring solutions. Then, they are discarded before their fitness evaluation, so that the computation resource is used only for promising solutions. We apply this method to one engineering problem and examine its effect on the search performance of an EMO algorithm.

The hypervolume indicator has frequently been used for comparing evolutionary multi-objective optimization (EMO) algorithms. A reference point is needed for hypervolume calculation. However, its specification has not been discussed in detail from a viewpoint of fair performance comparison. A slightly worse point than the nadir point is usually used for hypervolume calculation in the EMO community. In this paper, we propose a reference point specification method for fair performance comparison of EMO algorithms. First, we discuss the relation between the reference point specification and the optimal distribution of solutions for hypervolume maximization. It is demonstrated that the optimal distribution of solutions strongly depends on the location of the reference point when a multi-objective problem has an inverted triangular Pareto front. Next, we propose a reference point specification method based on theoretical discussions on the optimal distribution of solutions. The basic idea is to specify the reference point so that a set of well-distributed solutions over the entire linear Pareto front has a large hypervolume and all solutions in such a solution set have similar hypervolume contributions. Then, we examine whether the proposed method can appropriately specify the reference point through computational experiments on various test problems. Finally, we examine the usefulness of the proposed method in a hypervolume-based EMO algorithm. Our discussions and experimental results clearly show that a slightly worse point than the nadir point is not always appropriate for performance comparison of EMO algorithms.

Handling of many-objective problems is a hot issue in the evolutionary multiobjective optimization (EMO) community. It is well-known that frequently-used EMO algorithms such as NSGA-II and SPEA do not work well on many-objective problems whereas they have been successfully applied to a large number of test problems and real-world application tasks with two or three objectives. The main difficulty in the handling of many-objective problems is that almost all solutions in the current population of an EMO algorithm are non-dominated with each other. This means that Pareto dominance relation cannot generate enough selection pressure toward the Pareto front. As a result, Pareto dominance-based EMO algorithms such as NSGA-II and SPEA cannot drive the current population toward the Pareto front efficiently in a high-dimensional objective space of a many-objective problem. A simple idea for introducing additional selection pressure toward the Pareto front is the use of scalarizing fitness functions. In this paper, we examine the effect of using weighted sum fitness functions for parent selection and generation update on the performance of NSGA-II for many-objective 0/1 knapsack problems.

Evolutionary algorithms have been widely used to tackle multi-objective optimization problems. Despite many successful applications there is a lack of theoretical knowledge on how and why this type of algorithm works particularly well for multi-objective problems. This special issue presents current advances to the theoretical understanding of evolutionary algorithms for multi-objective optimization. It comprises four very high quality papers which have been selected via a rigorous reviewing process. The first paper “On the Effect of Populations in Evolutionary Multi-Objective Optimisation” by Oliver Giel and Per Kristian Lehre studies the basic question whether a population is provably useful to compute the whole Pareto front of a multi-objective optimization problem. They compare restart strategies of several single-objective approaches based on one individual to a natural multi-objective approach using a population. Their analyses point out the benefits of the multi-objective evolutionary algorithm in a rigorous way. Using illustrative example functions they provide not only proven results but also a good intuition about why population-based approaches can outperform individual-based approaches in multi-objective optimization. The paper “Exploring the Runtime of an Evolutionary Algorithm for the Multi-Objective Shortest Path Problem” by Christian Horoba examines the behavior of evolutionary algorithms for the classical NP-hard multi-objective shortest path problem. The author considers an evolutionary approach based on -dominance and proves that this algorithm constitutes a fully polynomial-time randomized approximation scheme (FPRAS) for the problem. The algorithm achieves this by mimicking a problemspecific algorithm for that problem. This provides general insight into why evolutionary algorithms can perform well for problems that allow for efficient problem-specific algorithms. Hypervolume-based algorithms have become very popular in evolutionary multiobjective optimization. In the paper, “An Efficient Algorithm for Computing Hypervolume Contributions,” Karl Bringmann and Tobias Friedrich present a faster algorithm for determining the individual with the smallest hypervolume contribution in a given population. Furthermore, they study the problem of removing λ > 1 individuals with the smallest hypervolume contribution from a given population and present an elegant and fast method to carry out this task.

An important implementation issue in the design of hybrid evolutionary multiobjective optimization algorithms with local search (i.e., multiobjective memetic algorithms) is how to strike a balance between local search and global search. If local search is applied to all individuals at every generation, almost all computation time is spent by local search. As a result, global search ability of memetic algorithms is not well utilized. We can use three ideas for decreasing the computation load of local search. One idea is to apply local search to only a small number of individuals. This idea can be implemented by introducing a local search probability, which is used to choose only a small number of initial solutions for local search from the current population. Another idea is a periodical (i.e., intermittent) use of local search. This idea can be implemented by introducing a local search interval (e.g., every 10 generations), which is used to specify when local search is applied. The other idea is an early termination of local search. Local search for each initial solution is terminated after a small number of neighbors are examined. This idea can be implemented by introducing a local search length, which is the number of examined neighbors in a series of iterated local search from a single initial solution. In this paper, we discuss the use of these three ideas to strike a local-global search balance. Through computational experiments on a two-objective 500-item knapsack problem, we compare various settings of local search such as short local search from all individuals at every generation, long local search from only a few individuals at every generation, and periodical long local search from all individuals. Global search in this paper means genetic search by crossover and mutation in multiobjective memetic algorithms.

Evolutionary Multi-objective Optimization (EMO) is a hot research direction nowadays and one of the state-of-the-art evolutionary multi-objective optimization algorithms ——NSGA-II has gain wide attention and application in many fields. Gene Expression Programming (GEP) has a powerful search capability, but falls into local optimum easily. Based on the transformed GEP, NSGA-II and the virus evolution mechanism, a new multi-objective evolutionary algorithm GEP Virus NSGA-II is proposed. With the infection operation of virus population, the diversity of the host population is increased, and it’s much easier to jump out of the local optimum. And this algorithm has got good experimental results on 9 standard test problems.

Within the field of evolutionary multi-objective optimization, there exists a class of problems where most evolutionary multi-objective optimization algorithms (EMOAs) suffer from a loss of diversity that remains unrecoverable for extended periods, namely “imbalanced problems”. Currently, the field of multi-objective optimization lacks rigorous quantitative characterization of imbalanced multi-objective optimization problems (MOPs) and systematic understanding of their impact on algorithmic performance. To address these limitations, we introduce two novel quantities: the global and local shrinkage-spread rates, which characterize population convergence and diversity dynamics during global exploration and local exploitation phases, respectively. Based on these quantities, we provide the first mathematical characterization of imbalanced MOPs and derive key theoretical properties through rigorous probabilistic analysis. Our theoretical result shows that the probability of achieving substantial diversity recovery is below 0.1 under given conditions, demonstrating that diversity recovery becomes extremely challenging once population diversity is lost. We further construct ten imbalanced MOPs in which the difficulty of maintaining population diversity can be precisely modulated through adjustable parameters. Using these benchmark problems, we conduct comparative experiments evaluating four representative EMOAs, i.e., NSGA-II, MOEA/D, RVEA, MOEA/D-M2M and DrEA, in terms of their diversity maintenance capabilities on imbalanced MOPs. The results provide both theoretical insights and practical guidance for designing robust EMOAs capable of handling imbalanced MOPs.

Multi-clustering via evolutionary multi-objective optimization

Objective functions in combinatorial optimization are discrete. The number of possible values of a discrete objective function is totally different from problem to problem. Optimization of a discrete objective function is often very difficult. In the case of multiobjective optimization, a different objective function has a different number of possible values. This means that each axis of the objective space has a different granularity. Some axes may have fine granularities while others are coarse. In this paper, we examine the effect of discrete objective functions with different granularities on the search behavior of EMO (evolutionary multiobjective optimization) algorithms through computational experiments. Experimental results show that a discrete objective function with a coarse granularity slows down the search of EMO algorithms along that objective. An interesting observation is that such a slow-down along one objective often leads to the speed-up of the search along other objectives. We also examine the effect of adding a small random noise to each discrete objective function in order to increase the number of possible objective values.

In the field of single-objective optimization, hybrid variants of gradient based methods and evolutionary algorithms have been shown to performance better than the pure evolutionary method. This same idea has been used with Evolutionary Multiobjective Optimization (EMO), obtaining also very promising results. In most of the cases, gradient information is used as part of the mutation operator, in order to move every generated point to the exact Pareto front. This means that gradient information is used along the whole process, and then consumes computational resources also along the whole process. On the other hand, in our approach we will use gradient information only at the beginning of the process, and will show that quality of the results is not decreased while computational cost is. We will use a steepest descent method to generate some efficient points to be used to seed an EMO method. The main goal will be generating some efficient points in the exact front using the less evaluations as possible, and let the EMO method use these points to spread along the whole Pareto front. In our approach, we will solve box-constrained continuous problems, gradients will be approximated using quadratic regressions and the EMO method will be based on Rough Sets theory Hernandez-Diaz et al. (Parallel Problem Solving from Nature (PPSN IX) 9th International Conference, 2006).

In the past few years, multi-objective optimization algorithms have been extensively applied in several fields including engineering design problems. A major reason is the advancement of evolutionary multi-objective optimization (EMO) algorithms that are able to find a set of non-dominated points spread on the respective Pareto-optimal front in a single simulation. Besides just finding a set of Pareto-optimal solutions, one is often interested in capturing knowledge about the variation of variable values over the Pareto-optimal front. Recent innovization approaches for knowledge discovery from Pareto-optimal solutions remain as a major activity in this direction. In this article, a different data-fitting approach for continuous parameterization of the Pareto-optimal front is presented. Cubic B-spline basis functions are used for fitting the data returned by an EMO procedure in a continuous variable space. No prior knowledge about the order in the data is assumed. An automatic procedure for detecting gaps in the Pareto-optimal front is also implemented. The algorithm takes points returned by the EMO as input and returns the control points of the B-spline manifold representing the Pareto-optimal set. Results for several standard and engineering, bi-objective and tri-objective optimization problems demonstrate the usefulness of the proposed procedure.

Evolutionary algorithm has gained a worldwide popularity among multi-objective optimization. The paper proposes a self-adaptive evolutionary algorithm (called SEA) for multi-objective optimization. In the SEA, the probability of crossover and mutation, P cand P m, are varied depending on the fitness values of the solutions. Fitness assignment of SEA realizes the twin goals of maintaining diversity in the population and guiding the population to the true Pareto Front; fitness value of individual not only depends on improved density estimation but also depends on non-dominated rank. The density estimation can keep diversity in all instances including when scalars of all objectives are much different from each other. SEA is compared against the Non-dominated Sorting Genetic Algorithm (NSGA-II) on a set of test problems introduced by the MOEA community. Simulated results show that SEA is as effective as NSGA-II in most of test functions, but when scalar of objectives are much different from each other, SEA has better distribution of non-dominated solutions.

In a short span of about 14 years, evolutionary multi-objective optimization (EMO) has established itself as a mature field of research and application with an extensive literature, many commercial softwares, numerous freely downloadable codes, a dedicated biannual conference running successfully four times so far since 2001, special sessions and workshops held at all major evolutionary computing conferences, and full-time researchers from universities and industries from all around the globe. In this paper, we make a brief outline of EMO principles, some EMO algorithms, and focus on current research and application potential of EMO. Besides, simply finding a set of Pareto-optimal solutions, EMO research has now diversified in hybridizing its search with multi-criterion decision-making tools to arrive at a single preferred solution, in utilizing EMO principle in solving different kinds of single-objective optimization problems efficiently, and in various interesting application domains which were not possible to be solved adequately due to the lack of a suitable solution technique.