A Memetic Optimization Strategy Based on Dimension Reduction in Decision Space

A fast sampling based evolutionary algorithm for million-dimensional multiobjective optimization

Multi-objective evolutionary algorithms (MOEAs) of the state of the art are created with the only purpose of dealing with the number of objective functions in a multi-objective optimization problem (MOP) and treat the decision variables of a MOP as a whole. However, when dealing with MOPs with a large number of decision variables (more than 100) their efficacy decreases as the number of decision variables of the MOP increases. On the other hand, problem decomposition, in terms of decision variables, has been found to be extremely efficient and effective for solving large scale optimization problems. Nevertheless, most of the currently available approaches for large scale optimization rely on models based on cooperative coevolution or linkage learning methods that use multiple subpopulations or preliminary analysis, respectively, which is computationally expensive (in terms of function evaluations) when used within MOEAs. In this work, we study the effect of what we call operational decomposition, which is a novel framework based on coevolutionary concepts to apply MOEAs’s crossover operator without adding any extra cost. We investigate the improvements that NSGA-III can achieve when combined with our proposed coevolutionary operators. This new scheme is capable of improving efficiency of a MOEA when dealing with large scale MOPs having from 200 up to 1200 decision variables.

Multioperator search strategy for evolutionary multiobjective optimization

Under certain smoothness assumptions, the Pareto set of a continuous multiobjective optimization problem is a piecewise continuous manifold in the decision space, which can be derived from the Karush-Kuhn-Tucker condition. Despite that a number of multiobjective evolutionary algorithms (MOEAs) have been proposed, their performance on multiobjective optimization problems with complicated Pareto sets (MOP-cPS) is still unsatisfying. In this article, we adopt the concept of manifold and propose a manifold learning-inspired mating strategy to enhance the diversity maintenance in MOEAs for solving MOP-cPS efficiently. In the proposed strategy, all of the individuals are first clustered into different manifolds according to their distribution in the objective space, and then the mating reproduction is restricted among individuals in the same manifold. Moreover, we embed the proposed mating strategy in three representative MOEAs and compare the embedded MOEAs with their original versions using the assortative genetic operators on a variety of MOP-cPS. The experimental results demonstrate the significant performance improvements benefitting from the proposed mating restriction strategy.

Cooperative co-evolutionary algorithm for multi-objective optimization problems with changing decision variables

Decomposition is an effective and popular strategy used by evolutionary algorithms to solve multiobjective optimization problems (MOPs). It can reduce the difficulty of directly solving MOPs, increase the diversity of the obtained solutions, and facilitate parallel computing. However, with the increase of the number of decision variables, the performance of multiobjective evolutionary algorithms (MOEAs) often deteriorates sharply. The advantages of the decomposition strategy are not fully exploited when solving such large-scale MOPs (LSMOPs). To this end, this paper proposes a parallel MOEA based on two-space decomposition (TSD) to solve LSMOPs. The main idea of the algorithm is to decompose the objective space and decision space into multiple subspaces, each of which is expected to contain some complete Pareto-optimal solutions, and then use multiple populations to conduct parallel searches in these subspaces. Specifically, the objective space decomposition approach adopts the traditional reference vector-based method, whereas the decision space decomposition approach adopts the proposed method based on a diversity design subspace (DDS). The algorithm uses a message passing interface (MPI) to implement its parallel environment. The experimental results demonstrate the effectiveness of the proposed DDS-based method. Compared with the state-of-the-art MOEAs in solving various benchmark and real-world problems, the proposed algorithm exhibits advantages in terms of general performance and computational efficiency.

Reference sets generated with uniformly distributed weight vectors on a unit simplex are widely used by several multi-objective evolutionary algorithms (MOEAs). They have been employed to tackle multi-objective optimization problems (MOPs) with four or more objective functions, i.e., the so-called many-objective optimization problems. These MOEAs have shown a good performance on MOPs with regular Pareto front shapes, i.e., simplex-like shapes. However, it has been observed that in many cases, their performance degrades on MOPs with irregular Pareto front shapes. In this paper, we designed a new selection mechanism that aims to promote a Pareto front shape invariant performance of MOEAs that use weight vector-based reference sets. The newly proposed selection mechanism takes advantage of weight vector-based reference sets and seven pair-potential functions. It was embedded into the non-dominated sorting genetic algorithm III (NSGA-III) to increase its performance on MOPs with different Pareto front geometries. We use the DTLZ and DTLZ\(^{-1}\) test problems to perform an empirical study about the usage of these pair-potential functions for this selection mechanism. Our experimental results show that the pair-potential functions can enhance the distribution of solutions obtained by weight vector-based MOEAs on MOPs with irregular Pareto front shapes. Also, the proposed selection mechanism permits maintaining the good performance of these MOEAs on MOPs with regular Pareto front shapes.

For large-scale multi-objective optimization, it is particularly challenging for evolutionary algorithms to converge to the Pareto Front. Most existing multi-objective evolutionary algorithms (MOEAs) handle convergence and diversity in a mutually dependent manner during the evolution process. In this case, the performance degradation of one solution may lead to the deterioration of the performance of the other solution. This paper proposes a two-stage multi-objective optimization algorithm based on decision variable clustering (LSMOEA-VT) to solve large-scale optimization problems. In LSMOEA-VT, decision variables are divided into two categories and use dimensionality reduction methods to optimize the variables that affect evolutionary convergence. Following this, we performed an interdependence analysis to break down the convergence variables into multiple subcomponents that are more tractable. Furthermore, a non-dominated dynamic weight aggregation method is used to enhance the diversity of the population. To evaluate the performance of our proposed algorithm, we performed extensive comparative experiments against four optimization algorithms across a diverse set of benchmark problems, including eight multi-objective optimization problems and nine large-scale optimization problems. The experimental results show that our proposed algorithm performs well on some test functions and has certain competitiveness.

A number of benchmark problems exist for evaluating multi-objective evolutionary algorithms (MOEAs) in the objective space. However, the decision space performance analysis is a recent and relatively less explored topic in evolutionary multi-objective optimization research. Among other implications, such analysis can lead to designing more realistic test problems, gaining better understanding about optimal and robust design areas, and design and evaluation of knowledge-based optimization algorithms. This paper complements the existing research in this area and proposes a new method to generate multi-objective optimization test problems with clustered Pareto sets in hyper-rectangular defined areas of decision space. The test problem is parametrized to control number of decision variables, number and position of optimal areas in the decision space and modality of fitness landscape. Three leading MOEAs, including NSGA-II, NSGA-III, and MOEA/D, are evaluated on a number of problem instances with varying characteristics. A new metric is proposed that measures the performance of algorithms in terms of their coverage of the optimal areas in the decision space. The empirical analysis presented in this research shows that the decision space performance may not necessarily be reflective of the objective space performance and that all algorithms are sensitive to population size parameter for the new test problems.

In a real life situation, due to the complexity of the industrial systems and non-linearity of their behavior, it is very difficult to achieve optimum performance of system for desired industrial goals using uncertain, vague and imprecise data. Herein, an approach has been proposed through which the behavior of the system is analyzed in the form of well- known six reliability indices by using triangular fuzzy numbers which allow the consideration of expert opinions, linguistic variables, operating conditions in reliability information. Using their behavior analysis a fuzzy multi-objective optimization problem (FMOOP) has been formulated. Due to the conflicting nature of the multiple objectives, the decision making is difficult and it leads to the Pareto optimal solutions instead of single optimal solutions. Many evolutionary algorithms (EAs) already exist in the literature for solving a multi-objective optimization problem (MOOP), and are termed as multi-objective evolutionary algorithms (MOEAs). Particle swarm optimization (PSO) is one of such MOEA which demonstrates the ability to identity a Pareto-optimal front efficiently. Here, a crisp optimization problem is reformulated from FMOOP by taking into account the preference of decision maker (DM) and then PSO is applied to solve the resulting fuzzified MOOP. The presented approach is applied in order to solve the multi-objective series-parallel system reliability optimization problem for a crystallization unit of a urea plant.

Metaheuristics based on the Artificial Immune System (AIS) framework, especially those inspired by the Immune Network theory, are known to be capable of stimulating the generation of diverse sets of solutions for a given problem, even though they generally implement very simple mechanisms to control the dynamics of the network. However, there are several studies in the AIS literature that propose more elaborate computational models, generally based on the concept of concentration of antibodies, which better explain the behavior of such networks. Therefore, in this work we propose a novel immune-inspired algorithm for real-parameter optimization, named cob-aiNet (Concentrationbased Artificial Immune Network), that adopts the concept of concentration of antibodies to better control the dynamics of the network, so that a broader coverage of promising regions of the search space can be achieved. This property of cob-aiNet was verified in a series of experimental analyses, in which the algorithm was compared to several techniques based on distinct paradigms, including two popular AISs from the literature. The experiments have shown that cob-aiNet, besides being able to maintain diversity during all the iterations, is also competitive with respect to the approximation of the global optima of the problems. Therefore, it was also proposed in this work an extension of cob-aiNet to deal with multiobjective optimization problems, which was named cob-aiNet[MO] (Concentration-based Artificial Immune Network for Multiobjective Optimization). Like a small set of techniques from the literature, cob-aiNet[MO] is capable of dealing with multiobjective optimization problems that also require a proper maintenance of diversity in the decision space, not only to overcome difficulties introduced by multimodality but also to facilitate the post-optimization decision making process. A series of experimental analyses were also made with cob-aiNet[MO], and it was observed that this algorithm presented better results in most of the considered problems, with respect to both the approximation of the Pareto front and diversity maintenance.

In multiobjective optimization problems (MOPs), the Pareto set consists of efficient solutions that represent the best trade-offs between the conflicting objectives. Many forms of uncertainty affect the MOP, including uncertainty in the decision variables, parameters or objectives. A source of uncertainty that is not studied in the evolutionary multiobjective optimization (EMO) literature is the decision-maker's attitude to risk (DMAR) even though it has great significance in real-world applications. Often the decision-makers change over the course of the decision-making process and thus, some relevant information about preferences of future decision-makers is unknown at the time a decision is made. This poses a major risk to organizations because a new decision-maker may simply reject a decision that has been made previously. When an EMO technique attempts to generate the set of nondominated solutions for a problem, then DMAR-related uncertainty needs to be reduced. Solutions generated by an EMO technique should be robust against perturbations caused by the DMAR. In this paper, we focus on the DMAR as a source of uncertainty and present two new types of robustness in MOP. In the first type, dominance robustness (DR), the robust Pareto solutions are those which, if perturbed, would have a high chance to move to another Pareto solution. In the second type, preference robustness (PR), the robust Pareto solutions are those that are close to each other in configuration space. Dominance robustness captures the ability of a solution to move along the Pareto optimal front under some perturbative variation in the decision space, while PR captures the ability of a solution to produce a smooth transition (in the decision variable space) to its neighbors (defined in the objective space). We propose methods to quantify these robustness concepts, modify existing EMO techniques to capture robustness against the DMAR, and present test problems to examine both DR and PR.

Finding a good Pareto front (PF) approximation and locating sufficient equivalent Pareto optimal solutions are two important goals of the multimodal multiobjective optimization (MMO). Preserving the diversity in decision and objective spaces is a core task in the MMO accordingly. Although various “soft isolation” approaches, such as niching methods, have been proposed to promote the diversity and find multiple Pareto optimal solutions in the decision space, they may perform poorly on complex MMO problems (MMOPs) due to high environmental selection pressure and complex geometry of Pareto optimal sets (PSs). To alleviate the above-mentioned challenging task, a “hard/physical isolation” method called zoning search (ZS) is proposed to maintain the diversity in the decision space and reduce the problem complexity in this article. In the ZS, some decision variables of MMOPs are selected randomly and then divided into several segments, i.e., the entire search space is partitioned into many subspaces. Clearly, the population diversity can be naturally maintained in the decision space and the problem complexity is reduced by the ZS in each subspace. The effectiveness of the ZS is systematically evaluated by 11 recently proposed MMOPs. The experimental results demonstrate that the ZS can effectively assist a selected multimodal multiobjective evolutionary algorithm (MMOEA) in finding more and better distributed equivalent Pareto optimal solutions in the decision space, and keep its performance in the objective space unchanged. Additionally, if additional computational resources are given, the ZS can further help the selected MMOEA to improve its performance in the decision space when compared with a soft isolation method used in the corresponding MMOEA. Overall, the ZS is a simple and promising approach to balance the broad search and the deep search in solving MMOPs.

Many real-world optimization problems are characterized by multiple conflicting objectives, which are known as multi-objective optimization problems (MOPs). In the last two decades, evolutionary algorithms have shown promising performance in solving various MOPs, and a large number of multi-objective evolutionary algorithms (MOEAs) have been proposed. In order to determine the most suitable MOEA for a specific MOP, it is usually necessary to perform experiments to compare the performance of multiple candidate MOEAs. In 2017, an evolutionary multi-objective optimization platform was proposed by us, called PlatEMO, which provides the source codes of many state-of-the-art MOEAs and helps researchers perform batch experiments on these MOEAs. In this work, we illustrate the method of using the newest version of PlatEMO to solve MOPs in applications, by means of a case study on the feature selection problem, which is an important and difficult task in machine learning and data mining. This paper details the method of adding the feature selection problem to PlatEMO, and presents the experimental results of eight MOEAs on nine datasets in feature selection.

Despite the success of Evolutionary Algorithms in solving complex problems, they may require many function evaluations. This becomes an issue when dealing with costly problems. Surrogate models may overcome this difficulty, though their use in problems with medium to large dimensionality is underexplored in the literature. Problems with multiple conflicting objectives can be formulated as Multi-objective Optimization Problems (MOPs). MOPs have received a great attention lately, mainly with Multi-objective Evolutionary Algorithms (MOEAs). This paper proposes ELMOEA/D-DE, a surrogate-assisted MOEA, for solving expensive MOPs in small evaluation budgets. ELMOEA/D-DE encompasses a state-of-the-art MOEA based on decomposition, Differential Evolution (DE) operators and Extreme Learning Machines. This paper tests three variants of ELMOEA/D-DE, using different DE operators, for solving five known benchmark MOPs with 10 to 60 decision variables. All variants achieve good results in terms of hyper volume metric and the best variant with operator DE/rand/1/bin is compared with two state-of-the-art approaches (MOEA/D-RBF and a non-surrogate-based MOEA), achieving the best results in all but one problems instances.