Combining CMA-ES and MOEA/DD for many-objective optimization

A Multiobjective Optimization Problem (MOP) requires the optimization of several objective functions simultaneously, usually in conflict with each other. One of the most efficient algorithms for solving MOPs is MOEA/D (Multiobjective Evolutionary Algorithm Based on Decomposition), which decomposes a MOP into single-objective optimization subproblems and solves them using information from neighboring subproblems. MOEA/D variants with other evolutionary operators have emerged over the years, improving their efficiency in various MOPs. Recently, the IWOA (Improved Whale Optimization Algorithm) was proposed, an optimization algorithm bioinspired by the whale hunting method hybridized with Differential Evolution, which presented excellent results in single-objective optimization problems. This work proposes the MOEA/D-IWOA algorithm, which associates characteristics of the evolutionary operators of the IWOA to MOEA/D. Computational experiments were accomplished to analyze the performance of the MOEA/D-IWOA in benchmark MOPs suites. The results were compared with those obtained by the MOEA/D, Non-dominated Sorting Genetic Algorithm II (NSGA-II), Third Evolution Step of Generalized Differential Evolution (GDE3), Improving the Strength Pareto Evolutionary Algorithm (SPEA2), and Indicator-Based Evolutionary Algorithm (IBEA) algorithms in the Hypervolume and Inverted Generational Distance Plus (IGD+) indicators. The MOEA/D-IWOA proved to be competitive, with a good performance profile, in addition to presenting the best results in some POMs.

The Multi-Objective Evolutionary Algorithm based on Decomposition (MOEA/D) is a popular algorithm for solving Multi-Objective Problems (MOPs). The main component of MOEA/D is to decompose a MOP into easier sub-problems using a set of weight vectors. The choice of the number of weight vectors significantly impacts the performance of MOEA/D. However, the right choice for this number varies, given different MOPs and search stages. We adaptively change the number of vectors by removing unnecessary vectors and adding new ones in empty areas of the objective space. Our MOEA/D variant uses the Consolidation Ratio to decide when to change the number of vectors and to decide where to add or remove these weighted vectors. We investigate the effects of this adaptive MOEA/D against MOEA/D with a poorly chosen set of vectors, a MOEA/D with fine-tuned vectors and MOEA/D with Adaptive Weight Adjustment on two commonly used benchmark functions. We analyse the algorithms in terms of hypervolume, IGD and entropy performance. Our results show that the proposed method is equivalent to MOEA/D with fine-tuned vectors and superior to MOEA/D with poorly defined vectors. Thus, our adaptive mechanism mitigates problems related to the choice of the number of weight vectors in MOEA/D, increasing the final performance of MOEA/D by filling empty areas of the objective space and avoiding premature stagnation of the search progress.

Multiobjective evolutionary algorithms based on decomposition (MOEA/D) decompose a multiobjective optimization problem (MOP) into a group of subproblems and optimizes them at the same time. The reproduction method in MOEA/D, which generates offspring solutions, has crucial effect on the performance of algorithm. As the difficulties of MOPs increases, it requires much higher efficiency for the reproduction methods in MOEA/D. However, for the complex optimization problems whose PS shape is complicated, the original reproduction method used in MOEA/D might not be suitable to generate excellent offspring solutions. In order to improve the property of the reproduction method for MOEA/D, this paper proposes an external archive matching strategy which selects solutions’ most matching archive solutions as parent solutions. The offspring solutions generated by this strategy can maintain a good convergence ability. To balance convergence and diversity, a perturbed learning scheme is used to extend the search space of the solutions. The experimental results on three groups of test problems reveal that the solutions obtained by MOEA/D-EAM have better convergence and diversity than the other four state-of-the-art algorithms.

Given that the definition of the multi-objective optimization problem is raised when number of objectives is increased in number at the optimization problem, where not only the number of objectives but also the computational resources which are needed to solve the problem, is also more desired. Therefore, novel approaches had required to solve multi-objective optimization problem in a reasonable time. One of this novel approach is utilization of the decomposition method with the evolutionary algorithm/operator. This algorithm was called multi-objective evolutionary algorithm based on decomposition (MOEA/D). Later on, variants have been proposed to improve the performance of the MOEA/D algorithm. However, a general comparison between these variants has needed for demonstrate the performance of these algorithm. For this reason, in this research the variants of MOEA/D algorithms have implemented on benchmark problems (DTLZ and MaF) and the performances has compared with each other. Two metrics had selected to evaluate/compare the performances of the variants. The metrics are IGD and Spread metrics. The results at the end of the implementations suggest that adaptive weighting idea is the most promising idea to increase the performance of the MOEA/D algorithm.

There are a large number of multi-objective optimization problems in real-world applications, like in games, that need to be solved in real time. In order to meet this pressing need, we suggests a method of parallelizing the multi-objective evolutionary algorithm based on decomposition (MOEA/D). Furthermore, a novel task decomposition strategy and scalarizing method without the ideal point are proposed for meeting the requirements of real-time and precision of the game. By combining the novel scalarizing function and GPU-based CUDA technology with the MOEA/D, a parallel MOEA/D for real-time multi-objective optimization problems is developed, namely P-MOEA/D. Experimental studies on ZDT and DTLZ benchmark problems suggest that the P-MOEA/D algorithm is efficient and fast.

Wireless sensor networks design requires high quality location assignment and energy efficient power assignment for maximizing the network coverage and lifetime. Classical deployment and power assignment approaches optimize these two objectives individually or by combining them together in a single objective or by constraining one and optimizing the other. In this article a multi-objective deployment and power assignment problem (DPAP) is formulated and a multi-objective evolutionary algorithm based on decomposition (MOEA/D) is specialized. Following the MOEA/D's framework the above multiobjective optimization problem (MOP) is decomposed into many scalar single objective problems. The sub-problems are solved simultaneously by using neighborhood information. Additionally, unique problem-specific, parameter-rising, genetic operators and local search heuristics were designed specifically for the DPAP. In addition, a new encoding scheme is designed to represent a WSN based on the DPAP's design variables. Simulation results show that MOEA/D provides a high quality set of alternative solutions without any prior knowledge on the objectives preference.

A new resource allocation strategy based on the relationship between subproblems for MOEA/D

The Markowitz mean-variance (MV) model is the basis of modern portfolio theory, the goal of which is to choose an optimal set of weights with the maximum expected return for a given level of risk. However, the MV model assumes that returns are normally distributed, which ignores the asymmetry of returns in real life. The mean-variance-skewness (MVS) portfolio framework introduces the skewness of the return as an extension of the classical MV model. Because of the increasing conflicting objective functions, the multi-objective portfolio optimization problem is more difficult to solve. This paper investigates the use of a multi-objective evolutionary algorithm based on decomposition (MOEA/D) for solving the portfolio optimization problem in the stock market. The results show that MOEA/D can produce Pareto fronts efficiently with three objectives to be optimized.

A decomposition based memetic algorithm for multi-objective vehicle routing problem with time windows

Randomized direct search algorithms for continuous domains, such as evolution strategies, are basic tools in machine learning. They are especially needed when the gradient of an objective function (e.g., loss, energy, or reward function) cannot be computed or estimated efficiently. Application areas include supervised and reinforcement learning as well as model selection. These randomized search strategies often rely on normally distributed additive variations of candidate solutions. In order to efficiently search in non-separable and ill-conditioned landscapes the covariance matrix of the normal distribution must be adapted, amounting to a variable metric method. Consequently, covariance matrix adaptation (CMA) is considered state-of-the-art in evolution strategies. In order to sample the normal distribution, the adapted covariance matrix needs to be decomposed, requiring in general Θ(n 3) operations, where n is the search space dimension. We propose a new update mechanism which can replace a rank-one covariance matrix update and the computationally expensive decomposition of the covariance matrix. The newly developed update rule reduces the computational complexity of the rank-one covariance matrix adaptation to Θ(n 2) without resorting to outdated distributions. We derive new versions of the elitist covariance matrix adaptation evolution strategy (CMA-ES) and the multi-objective CMA-ES. These algorithms are equivalent to the original procedures except that the update step for the variable metric distribution scales better in the problem dimension. We also introduce a simplified variant of the non-elitist CMA-ES with the incremental covariance matrix update and investigate its performance. Apart from the reduced time-complexity of the distribution update, the algebraic computations involved in all new algorithms are simpler compared to the original versions. The new update rule improves the performance of the CMA-ES for large scale machine learning problems in which the objective function can be evaluated fast.

To obtain propellant formulations with superior comprehensive and robustness performance, the study establishes a multi-objective optimization model that accounts for uncertainties. The model adopts a bi-layer structure. The inner layer computes performance bounds to construct uncertainty intervals, which are subsequently transformed into deterministic performance via interval order relations. The outer layer optimizes component mass fractions using MOEA/D (Multi-objective Evolutionary Algorithm Based on Decomposition) to maximize the deterministic performance. The study leverages Large Language Models (LLMs) as pre-trained optimizers to automate the operator design of MOEA/D. Designers can identify formulations that satisfy the performance requirements and robustness criteria by adjusting uncertainty levels and MOEA/D weight coefficients. The results on ZDTs and UFs demonstrate that MOEA/D-LLM achieves approximately a 4.0% improvement in hypervolume values compared to MOEA/D. Additionally, the NEPE propellant optimization case shows that MOEA/D-LLM improves the computational speed by about 13.05% and enhances hypervolume values by around 2.7% compared to MOEA/D. The specific impulse increases by 1.11%, the generation of aluminum oxide and hydrogen chloride decreases by approximately 18.43% and 16.40%, respectively, and the impact sensitivity is reduced by about 1.67%.

Design of multi-band antenna involves multiple characteristics such as return loss in multiple operation bands. To apply MOEA/D (multi-objective evolutionary algorithm based on decomposition) to antenna structure optimization, it was introduced into this field for the first time. MOEA/D framework was demonstrated at first. Then it was in conjunction with electromagnetic solver, HFSS (high frequency structure simulator) to optimize and design tri-band bow-tie antenna to serve as an example. The evolutionary results showed that MOEA/D worked efficiently and generated multiple candidate structures at one single iteration, and that MOEA/D had lower computational overhead than NSGA-II (non-dominated sorting genetic algorithm II) for this problem. Therefore, MOEA/D shows great potential for antenna structure optimization and design.

The adaptive operator selection (AOS) and the adaptive parameter control are widely used to enhance the search power in many multiobjective evolutionary algorithms. This paper proposes a novel adaptive selection strategy with bandits for the multiobjective evolutionary algorithm based on decomposition (MOEA/D), named latest stored information based adaptive selection (LSIAS). An improved upper confidence bound (UCB) method is adopted in the strategy, in which the operator usage rate and abandonment of extreme fitness improvement are introduced to improve the performance of UCB. The strategy uses a sliding window to store recent valuable information about operators, such as factors, probabilities, and efficiency. Four common used DE operators are chosen with the AOS, and two kinds of assist information on operator are selected to improve the operators search power. The operator information is updated with the help of LSIAS and the resulting algorithmic combination is called MOEA/D-LSIAS. Compared to some well-known MOEA/D variants, the LSIAS demonstrates the superior robustness and fast convergence for various multiobjective optimization problems. The comparative experiments also demonstrate improved search power of operators with different assist information on different problems.

Test task scheduling problem (TTSP) is a complex optimization problem and has many local optima. In this paper, a hybrid chaotic multiobjective evolutionary algorithm based on decomposition (CMOEA/D) is presented to avoid becoming trapped in local optima and to obtain high quality solutions. First, we propose an improving integrated encoding scheme (IES) to increase the efficiency. Then ten chaotic maps are applied into the multiobjective evolutionary algorithm based on decomposition (MOEA/D) in three phases, that is, initial population and crossover and mutation operators. To identify a good approach for hybrid MOEA/D and chaos and indicate the effectiveness of the improving IES several experiments are performed. The Pareto front and the statistical results demonstrate that different chaotic maps in different phases have different effects for solving the TTSP especially the circle map and ICMIC map. The similarity degree of distribution between chaotic maps and the problem is a very essential factor for the application of chaotic maps. In addition, the experiments of comparisons of CMOEA/D and variable neighborhood MOEA/D (VNM) indicate that our algorithm has the best performance in solving the TTSP.

In optimization problems it is often necessary to perform an optimization based on more than one objective. The goal of the multiobjective optimization is usually to find an approximation of the Pareto front which contains solutions that represent the best possible trade-offs between the objectives. In a multiobjective scenario it is important to both improve the solutions in terms of the objectives and to find a wide variety of available options. Evolutionary algorithms are often used for multiobjective optimization because they maintain a population of individuals which directly represent a set of solutions of the optimization problem. multiobjective evolutionary algorithm based on decomposition (MOEA/D) is one of the most effective multiobjective algorithms currently used in the literature. This paper investigates several methods which increase the selective pressure to the outside of the Pareto front in the case of the MOEA/D algorithm. Experiments show that by applying greater selective pressure in the outwards direction the quality of results obtained by the algorithm can be increased. The proposed methods were tested on nine test instances with complicated Pareto sets. In the tests the new methods outperformed the original MOEA/D algorithm.