A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization

Balancing convergence and diversity in the decision space is essential in solving multimodal multi-objective optimization problems (MMOPs), which have multiple equivalent Pareto optimal sets (PSs) with the same Pareto optimal front (PF). For MMOPs with an imbalance between convergence and diversity in the decision space (MMOP-ICD), numerous efficient multimodal multiobjective evolutionary algorithms (MMEAs) avoid premature convergence and search for the imbalanced PS by relaxing the traditional convergence-first selection mechanism. Unfortunately, existing MMEAs suffer from convergence degradation due to excessive relaxation of the convergence-first selection mechanism. Therefore, this paper proposes an improved convergence-relaxed mechanism that includes an enhanced local convergence indicator and a two-stage mating selection. The enhanced local convergence indicator introduces the global convergence indicator into the local convergence indicator. The local convergence indicator can locate more equivalent PSs and prevent premature convergence caused by the global convergence indicator. The global convergence indicator can improve the convergence quality of the solution selected by the local convergence indicator. Then, the two-stage mating selection is used to enhance the diversity in the decision space and balance the improved convergence. Experimental results and statistical analysis show that the proposed algorithm is significantly superior to other state-of-the-art MMEAs.

A multi-modal multi-objective optimization problem is a special kind of multi-objective optimization problem with multiple Pareto subsets. In this paper, we propose an efficient multi-modal multi-objective optimization algorithm based on the widely used MOEA/D algorithm. In our proposed algorithm, each weight vector has its own sub-population. With a clearing mechanism and a greedy removal strategy, our proposed algorithm can effectively preserve equivalent Pareto optimal solutions (i.e., different Pareto optimal solutions with same objective values). Experimental results show that our proposed algorithm can effectively preserve the diversity of solutions in the decision space when handling large-scale multi-modal multi-objective optimization problems.

Dual-space distribution metric-based evolutionary algorithm for multimodal multi-objective optimization

Multimodal multiobjective optimization problems (MMOPs) have attracted wide attention in recent years. This kind of problem is very challenging since they need to locate different Pareto-optimal solution sets (PSs) that correspond to the same Pareto front. To resolve it, this article proposes a novel multimodal multiobjective differential evolution (DE) algorithm with species conservation, which develops a new way of locating different PSs. Specifically, the proposed algorithm adopts species conservation to determine different PSs in known areas, while it uses a variant of DE as the basic optimizer to explore new areas. There are three operators in species conservation: 1) species division; 2) seed determination; and 3) seed conservation. Species division mainly partitions the joint population of parents and children into various species in the decision space for retaining different PSs. Seed determination selects superior solutions from each species as seeds that need to be kept in the next generation. Seed conservation is to ensure that all species seeds are retained in the new generation by substituting no promising solutions with them, thereby guarantee not missing some known areas that may contain different PSs. Besides, the DE variant is utilized to produce diverse solutions to find new areas in the decision space where PSs may exist. The comparative experiments with ten state-of-the-art algorithms have been performed on the CEC 2019 MMOPs test set and two real-world problems. The experimental results have verified that the proposed algorithm has a competitive performance for MMOPs.

In recent years, numerous efficient and effective multimodal multi-objective evolutionary algorithms (MMOEAs) have been developed to address multimodal multi-objective optimization problems (MMOPs) involving multiple equivalent sets of Pareto optimal solutions to be found simultaneously. However, the Pareto optimal solutions may have various contracting or expending shapes, and have random locations in the decision space. In addition, uniform decision distribution does not imply good objective distribution. Therefore, many existing MMOEAs are very difficult to guide the individuals converged to every Pareto subregion with good distribution in both the decision space and the objective space. In this paper, we present a multi-population evolutionary algorithm to search for the equivalent global Pareto optimal solutions. The original population should be divided into two groups of subpopulations with equal size. The first subpopulation is designed to search for the optimal solutions in objective space. At the same time. the second subpopulation focus to obtain high-quality optimal solutions in the decision space. The multi-population strategy is helpful to improve the decision and objective distributions simultaneously, and address the MMOPs effectively. The proposed algorithm is compared against five state-of-the-art MMOEAs. The experimental results indicate the proposed algorithm provides better performance than competing MMOEAs on IEEE CEC 2019 MMOPs test suite.

A ring-hierarchy-based evolutionary algorithm for multimodal multi-objective optimization

Finding a good Pareto front (PF) approximation and locating sufficient equivalent Pareto optimal solutions are two important goals of the multimodal multiobjective optimization (MMO). Preserving the diversity in decision and objective spaces is a core task in the MMO accordingly. Although various “soft isolation” approaches, such as niching methods, have been proposed to promote the diversity and find multiple Pareto optimal solutions in the decision space, they may perform poorly on complex MMO problems (MMOPs) due to high environmental selection pressure and complex geometry of Pareto optimal sets (PSs). To alleviate the above-mentioned challenging task, a “hard/physical isolation” method called zoning search (ZS) is proposed to maintain the diversity in the decision space and reduce the problem complexity in this article. In the ZS, some decision variables of MMOPs are selected randomly and then divided into several segments, i.e., the entire search space is partitioned into many subspaces. Clearly, the population diversity can be naturally maintained in the decision space and the problem complexity is reduced by the ZS in each subspace. The effectiveness of the ZS is systematically evaluated by 11 recently proposed MMOPs. The experimental results demonstrate that the ZS can effectively assist a selected multimodal multiobjective evolutionary algorithm (MMOEA) in finding more and better distributed equivalent Pareto optimal solutions in the decision space, and keep its performance in the objective space unchanged. Additionally, if additional computational resources are given, the ZS can further help the selected MMOEA to improve its performance in the decision space when compared with a soft isolation method used in the corresponding MMOEA. Overall, the ZS is a simple and promising approach to balance the broad search and the deep search in solving MMOPs.

Multi-objective sand cat swarm optimization based on adaptive clustering for solving multimodal multi-objective optimization problems

Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems

Multimodal multiobjective optimization problems, characterized by multiple solutions mapping to identical objective vectors, are ubiquitous in real-world applications. Despite their prevalence, most existing multimodal multiobjective evolutionary algorithms (MMOEAs) predominantly focus on identifying global Pareto sets, often overlooking the equally significant local Pareto sets. While some algorithms attempt to address local Pareto sets, their performance in the objective space remains suboptimal. The inherent challenge lies in the fact that a single strategy cannot effectively tackle problems with and without local Pareto fronts. This study proposes a novel approach that first detects the presence of local Pareto fronts using a neural network, thereby enabling adaptive adjustments to the algorithm’s selection strategy and search scope. Based on this detection mechanism, we design a microscale searching multimodal multiobjective evolutionary algorithm (MMOEA_MS). Through extensive experiments on twenty-two benchmark problems, MMOEA_MS demonstrates superior performance in identifying local Pareto fronts and outperforms existing algorithms in the objective space. This study highlights the effectiveness of MMOEA_MS in solving multimodal multiobjective optimization problems with diverse Pareto front characteristics, thereby addressing key limitations of current methodologies.

Effectively solving multimodal multiobjective optimization problems (MMOPs) requires maintaining an optimal balance between the diversity and the convergence. Traditional algorithms often struggle with environmental selection adaptability, leading to suboptimal performance across diverse MMOPs. This article innovatively integrates the actor-critic reinforcement learning (RL) with the evolutionary algorithm, significantly improving environmental selection adaptability through synergistic online learning between its actor and critic components. An RL process that dynamically optimizes the niche size is established, which critically determines the trade-off between the diversity and convergence preferences. The process is formulated by defining: 1) convergence and diversity measures as the state; 2) niche size adjustment as the continuous action; and 3) state improvement as the reward. Two specialized fully connected neural networks are employed as the actor network for action generation and the critic network for value estimation, which collaboratively adapt to population states through real-time online learning. This adaptive niching technology, integrated with local convergence quality assessment, enables comprehensive evaluation of both diversity and potential convergence. Extensive experimental validation demonstrates the superior performance of the proposed algorithm against ten state-of-the-art algorithms across 48 benchmark problems and a real-world application. The results consistently show significant improvements in balance maintenance and overall optimization effectiveness compared to existing algorithms.

Recently, multimodal multiobjective optimization problems (MMOPs) have received increasing attention. Their goal is to find a Pareto front and as many equivalent Pareto optimal solutions as possible. Although some evolutionary algorithms for them have been proposed, they mainly focus on the convergence rate in the decision space while ignoring solutions diversity. In this paper, we propose a new multiobjective fireworks algorithm for them, which is able to balance exploitation and exploration in the decision space. We first extend a latest single-objective fireworks algorithm to handle MMOPs. Then we make improvements by incorporating an adaptive strategy and special archive guidance into it, where special archives are established for each firework, and two strategies (i.e., explosion and random strategies) are adaptively selected to update the positions of sparks generated by fireworks with the guidance of special archives. Finally, we compare the proposed algorithm with eight state-of-the-art multimodal multiobjective algorithms on all 22 MMOPs from CEC2019 and several imbalanced distance minimization problems. Experimental results show that the proposed algorithm is superior to compared algorithms in solving them. Also, its runtime is less than its peers'.

A two-archive model based evolutionary algorithm for multimodal multi-objective optimization problems

There have been few researches on solving multimodal multiobjective optimization problems, whereas they are commonly seen in real-world applications but difficult for the existing evolutionary optimizers. In this paper, we propose a novel multimodal multiobjective evolutionary algorithm using two-archive and recombination strategies. In the proposed algorithm, the properties of decision variables and the relationships among them are analyzed at first to guide the evolutionary search. Then, a general framework using two archives, i.e., the convergence and the diversity archives, is adopted to cooperatively solve these problems. Moreover, the diversity archive simultaneously employs a clustering strategy to guarantee diversity in the objective space and a niche-based clearing strategy to promote the same in the decision space. At the end of evolution process, solutions in the convergence and the diversity archives are recombined to obtain a large number of multiple Pareto optimal solutions. In addition, a set of benchmark test functions and a performance metric are designed for multimodal multiobjective optimization. The proposed algorithm is empirically compared with two state-of-the-art evolutionary algorithms on these test functions. The comparative results demonstrate that the overall performance of the proposed algorithm is significantly superior to the competing algorithms.

A novel scalable test problem suite for multimodal multiobjective optimization