On Benchmark Problems and Metrics for Decision Space Performance Analysis in Multi-Objective Optimization

Many multi-objective evolutionary algorithms (MOEAs) have been successful in approximating the Pareto Front. However, well-distributed solutions in the objective and decision spaces are still required in many real-life applications. In this paper, a novel MOEA is proposed to this problem. Distinct from other MOEAs, the proposed algorithm suggests a framework, which includes two crowding estimation methods, multiple selection methods for mating and search strategies for variation, to improve the MOEA' s searching ability, and the diversity of its solutions. The algorithm emphasizes the importance of using the decision space and the objective space diversities. The objective space crowding and decision space crowding distances are designed using different ideas. To produce new individuals, three different types of mating selections and their respective search strategies are constructed for the main population and the two sparse populations, with the help of the two crowding measurements. Finally, based on the experimental tests on 17 unconstrained multi-objective optimization problems, the proposed algorithm is demonstrated to have better results compared to several state-of-the-art MOEAs. A detailed analysis on the effectiveness and robustness of the framework is also presented.

In multiobjective optimization, it is nontrivial for decision makers to articulate preferences without a priori knowledge, which is particularly true when the number of objectives becomes large. Depending on the shape of the Pareto front, optimal solutions such as knee points may be of interest. Although several multi- and many-objective optimization test suites have been proposed, little work has been reported focusing on designing multiobjective problems whose Pareto front contains complex knee regions. Likewise, few performance indicators dedicated to evaluate an algorithm's ability of accurately locating all knee points in high-dimensional objective space have been suggested. This paper proposes a set of multiobjective optimization test problems whose Pareto front consists of complex knee regions, aiming to assess the capability of evolutionary algorithms to accurately identify all knee points. Various features related to knee points have been taken into account in designing the test problems, including symmetry, differentiability, and degeneration. These features are also combined with other challenges in solving the optimization problems, such as multimodality, linkage between decision variables, nonuniformity, and scalability of the Pareto front. The proposed test problems are scalable to both decision and objective spaces. Accordingly, new performance indicators are suggested for evaluating the capability of optimization algorithms in locating the knee points. The proposed test problems, together with the performance indicators, offer a new means to develop and assess preference-based evolutionary algorithms for solving multi- and many-objective optimization problems.

Finding a good Pareto front (PF) approximation and locating sufficient equivalent Pareto optimal solutions are two important goals of the multimodal multiobjective optimization (MMO). Preserving the diversity in decision and objective spaces is a core task in the MMO accordingly. Although various “soft isolation” approaches, such as niching methods, have been proposed to promote the diversity and find multiple Pareto optimal solutions in the decision space, they may perform poorly on complex MMO problems (MMOPs) due to high environmental selection pressure and complex geometry of Pareto optimal sets (PSs). To alleviate the above-mentioned challenging task, a “hard/physical isolation” method called zoning search (ZS) is proposed to maintain the diversity in the decision space and reduce the problem complexity in this article. In the ZS, some decision variables of MMOPs are selected randomly and then divided into several segments, i.e., the entire search space is partitioned into many subspaces. Clearly, the population diversity can be naturally maintained in the decision space and the problem complexity is reduced by the ZS in each subspace. The effectiveness of the ZS is systematically evaluated by 11 recently proposed MMOPs. The experimental results demonstrate that the ZS can effectively assist a selected multimodal multiobjective evolutionary algorithm (MMOEA) in finding more and better distributed equivalent Pareto optimal solutions in the decision space, and keep its performance in the objective space unchanged. Additionally, if additional computational resources are given, the ZS can further help the selected MMOEA to improve its performance in the decision space when compared with a soft isolation method used in the corresponding MMOEA. Overall, the ZS is a simple and promising approach to balance the broad search and the deep search in solving MMOPs.

Multioperator search strategy for evolutionary multiobjective optimization

A fast sampling based evolutionary algorithm for million-dimensional multiobjective optimization

Under certain smoothness assumptions, the Pareto set of a continuous multiobjective optimization problem is a piecewise continuous manifold in the decision space, which can be derived from the Karush-Kuhn-Tucker condition. Despite that a number of multiobjective evolutionary algorithms (MOEAs) have been proposed, their performance on multiobjective optimization problems with complicated Pareto sets (MOP-cPS) is still unsatisfying. In this article, we adopt the concept of manifold and propose a manifold learning-inspired mating strategy to enhance the diversity maintenance in MOEAs for solving MOP-cPS efficiently. In the proposed strategy, all of the individuals are first clustered into different manifolds according to their distribution in the objective space, and then the mating reproduction is restricted among individuals in the same manifold. Moreover, we embed the proposed mating strategy in three representative MOEAs and compare the embedded MOEAs with their original versions using the assortative genetic operators on a variety of MOP-cPS. The experimental results demonstrate the significant performance improvements benefitting from the proposed mating restriction strategy.

Dynamic multi-objective optimization problem (DMOP) denotes the multi-objective optimization problem which varies over time. As changes in DMOP may exist some patterns which are predictable, to solve DMOP, a number of research efforts have been made to develop evolutionary search with prediction approaches to estimate the changes of the problem. A common practice of existing prediction approaches is to predict the change of Pareto-optimal solutions (POS) based on the historical solutions obtained in the decision space. However, the change of a DMOP may occur in both decision and objective spaces. Prediction only in the decision space thus may not be able to give the proper estimation of the problem change. Taking this cue, in this paper, we propose an evolutionary search with multi-view prediction for solving DMOP. In contrast to existing prediction methods, the proposed approach conducts prediction from the views of both decision and objective spaces. To estimate dynamic changes in DMOP, a kernelized autoencoding model is derived to perform the multi-view prediction in a Reproducing Kernel Hilbert Space (RKHS), which holds a closed-form solution. To examine the performance of the proposed method, comprehensive empirical studies on the commonly used DMOP benchmarks, as well as a real-world case study on the movie recommendation problem, are presented. The obtained experimental results verified the efficacy of the proposed method for solving both benchmark and real-world DMOPs.

Difficult Pareto set topology refers to multi-objective problems with geometries of the Pareto set such that neighboring optimal solutions in objective space differ in several or all variables in decision space. These problems can present a tough challenge for evolutionary multi-objective algorithms to find a good approximation of the optimal Pareto set well-distributed in decision and objective space. One important challenge optimizing these problems is to keep or restore diversity in decision space. In this work, we propose a method that learns a model of the topology of the solutions in the population by performing parametric spline interpolations for all variables in decision space. We use Catmull-Rom parametric curves as they allow us to deal with any dimension in decision space. The proposed method is appropriated for bi-objective problems since their optimal set is a one-dimensional curve according to the Karush-Kuhn-Tucker condition. Here, the proposed method is used to promote restarts from solutions generated by the model. We study the effectiveness of the proposed method coupled to NSGA-II and two variations of MOEA/D on problems with difficult Pareto set topology. These algorithms approach very differently the Pareto set. We argue and discuss their behavior and its implications for model building.

There can be a complicated mapping relation between decision variables and objective functions in multi-objective optimization problems (MOPs). It is uncommon that decision variables influence objective functions equally. Decision variables act differently in different objective functions. Hence, often, the mapping relation is unbalanced, which causes some redundancy during the search in a decision space. In response to this scenario, we propose a novel memetic (multi-objective) optimization strategy based on dimension reduction in decision space (DRMOS). DRMOS firstly analyzes the mapping relation between decision variables and objective functions. Then, it reduces the dimension of the search space by dividing the decision space into several subspaces according to the obtained relation. Finally, it improves the population by the memetic local search strategies in these decision subspaces separately. Further, DRMOS has good portability to other multi-objective evolutionary algorithms (MOEAs); that is, it is easily compatible with existing MOEAs. In order to evaluate its performance, we embed DRMOS in several state of the art MOEAs to facilitate our experiments. The results show that DRMOS has the advantage in terms of convergence speed, diversity maintenance, and portability when solving MOPs with an unbalanced mapping relation between decision variables and objective functions.

Decomposition is an effective and popular strategy used by evolutionary algorithms to solve multiobjective optimization problems (MOPs). It can reduce the difficulty of directly solving MOPs, increase the diversity of the obtained solutions, and facilitate parallel computing. However, with the increase of the number of decision variables, the performance of multiobjective evolutionary algorithms (MOEAs) often deteriorates sharply. The advantages of the decomposition strategy are not fully exploited when solving such large-scale MOPs (LSMOPs). To this end, this paper proposes a parallel MOEA based on two-space decomposition (TSD) to solve LSMOPs. The main idea of the algorithm is to decompose the objective space and decision space into multiple subspaces, each of which is expected to contain some complete Pareto-optimal solutions, and then use multiple populations to conduct parallel searches in these subspaces. Specifically, the objective space decomposition approach adopts the traditional reference vector-based method, whereas the decision space decomposition approach adopts the proposed method based on a diversity design subspace (DDS). The algorithm uses a message passing interface (MPI) to implement its parallel environment. The experimental results demonstrate the effectiveness of the proposed DDS-based method. Compared with the state-of-the-art MOEAs in solving various benchmark and real-world problems, the proposed algorithm exhibits advantages in terms of general performance and computational efficiency.

Different from multiobjective optimization problems (MOPs), multimodal MOPs (MMOPs) focus on both decision and objective spaces rather than only objective one. Thus, finding a good Pareto front approximation and finding the maximal number of equivalent Pareto optimal solutions for each objective vector in the Pareto front are two core tasks for them. Although some multimodal multiobjective evolutionary algorithms have been proposed to handle them, they can quickly converge to the easy-to-find equivalent Pareto optimal solutions, thereby losing their ability to improve solution diversity in decision space and performance in objective space. To address the above issues, this work proposes a new information utilization method. Its core idea is to randomly extract a certain amount of decision variable information from the current optimal solutions to construct an information vector, which is, in turn, used to assist the generation of elite solutions. The proposed method can assist any available intelligent optimizers to improve their performance in solving MMOPs. This is confirmed by experimental results obtained from solving 22 such problems from CEC2019 and 12 scalable imbalanced distance minimization problems through a number of optimizers. Finally, we apply the proposed method to credit card fraud detection problems to show its practical significance.

The Pareto front (Pareto set) of a continuous optimization problem with m objectives is a (m-1) dimensional piecewise continuous manifold in the objective space (the decision space) under some mild conditions. Based on this regularity property in the objective space, we have recently developed several multiobjective estimation of distribution algorithms (EDAs). However, this property has not been utilized in the decision space. Using the regularity property in both the objective and decision space, this paper proposes a simple EDA for multiobjective optimization. Since the location information has not efficiently used in EDAs, a combination of EDA and differential evolution (DE) is suggested for improving the algorithmic performance. The hybrid method and the pure EDA method proposed in this paper, and a DE based method are compared on several test instances. Experimental results have shown that the algorithm with the proposed strategy is very promising.

In this paper, we propose a framework to accelerate the computational efficiency of evolutionary algorithms on large-scale multiobjective optimization. The main idea is to track the Pareto optimal set (PS) directly via problem reformulation. To begin with, the algorithm obtains a set of reference directions in the decision space and associates them with a set of weight variables for locating the PS. Afterwards, the original large-scale multiobjective optimization problem is reformulated into a low-dimensional single-objective optimization problem. In the reformulated problem, the decision space is reconstructed by the weight variables and the objective space is reduced by an indicator function. Thanks to the low dimensionality of the weight variables and reduced objective space, a set of quasi-optimal solutions can be obtained efficiently. Finally, a multiobjective evolutionary algorithm is used to spread the quasi-optimal solutions over the approximate Pareto optimal front evenly. Experiments have been conducted on a variety of large-scale multiobjective problems with up to 5000 decision variables. Four different types of representative algorithms are embedded into the proposed framework and compared with their original versions, respectively. Furthermore, the proposed framework has been compared with two state-of-the-art algorithms for large-scale multiobjective optimization. The experimental results have demonstrated the significant improvement benefited from the framework in terms of its performance and computational efficiency in large-scale multiobjective optimization.

Many real-world optimization problems have more than one objective (and these objectives are often conflicting). In most cases, there is no single solution being optimized with regards to all objectives. Deal with such problems, Multi-Objective Evolutionary Algorithms (MOEAs) have shown a great potential. There has been a popular trend in getting suitable solutions and increasing the convergence of MOEAs, that is consideration of Decision Maker (DM) during the optimization process (interacting with DM) for checking, analyzing the results and giving the preference.In [2] and its next version in [9], a system of rays is used to maintain the diversity of the population during evolutionary process. The niching values are from the distance of solutions and rays are used to guide the evolutionary process ensures the balance between exploration and exploitation of the population. When DM interactive with the evolutionary process, a set of reference points are given as his/her expected area in objective space. Based on the concept of the system of rays, in [11], and its updated version in [12] the authors proposed a ray based interactive method with a set of reference points in different ways. In this paper, we indicated some issues on the method and we suggest to use a geometric buffer based technique to improve the method. We carried out a case study on several test problems and obtained quite good results.

Multi-objective sand cat swarm optimization based on adaptive clustering for solving multimodal multi-objective optimization problems