Data mining using parallel Multi-Objective Evolutionary algorithms on graphics hardware

Many real-world optimization problems are characterized by multiple conflicting objectives, which are known as multi-objective optimization problems (MOPs). In the last two decades, evolutionary algorithms have shown promising performance in solving various MOPs, and a large number of multi-objective evolutionary algorithms (MOEAs) have been proposed. In order to determine the most suitable MOEA for a specific MOP, it is usually necessary to perform experiments to compare the performance of multiple candidate MOEAs. In 2017, an evolutionary multi-objective optimization platform was proposed by us, called PlatEMO, which provides the source codes of many state-of-the-art MOEAs and helps researchers perform batch experiments on these MOEAs. In this work, we illustrate the method of using the newest version of PlatEMO to solve MOPs in applications, by means of a case study on the feature selection problem, which is an important and difficult task in machine learning and data mining. This paper details the method of adding the feature selection problem to PlatEMO, and presents the experimental results of eight MOEAs on nine datasets in feature selection.

Reference sets generated with uniformly distributed weight vectors on a unit simplex are widely used by several multi-objective evolutionary algorithms (MOEAs). They have been employed to tackle multi-objective optimization problems (MOPs) with four or more objective functions, i.e., the so-called many-objective optimization problems. These MOEAs have shown a good performance on MOPs with regular Pareto front shapes, i.e., simplex-like shapes. However, it has been observed that in many cases, their performance degrades on MOPs with irregular Pareto front shapes. In this paper, we designed a new selection mechanism that aims to promote a Pareto front shape invariant performance of MOEAs that use weight vector-based reference sets. The newly proposed selection mechanism takes advantage of weight vector-based reference sets and seven pair-potential functions. It was embedded into the non-dominated sorting genetic algorithm III (NSGA-III) to increase its performance on MOPs with different Pareto front geometries. We use the DTLZ and DTLZ\(^{-1}\) test problems to perform an empirical study about the usage of these pair-potential functions for this selection mechanism. Our experimental results show that the pair-potential functions can enhance the distribution of solutions obtained by weight vector-based MOEAs on MOPs with irregular Pareto front shapes. Also, the proposed selection mechanism permits maintaining the good performance of these MOEAs on MOPs with regular Pareto front shapes.

This paper presents a parallel hybrid genetic algorithm (GA) for solving the sum-of-pairs multiple protein sequence alignment. A new chromosome representation and its corresponding genetic operators are proposed. A multi-population GENITOR-type GA is combined with local search heuristics. It is then extended to run in parallel on a multiprocessor system for speeding up. Experimental results of benchmarks from the BAliBASE show that the proposed method is superior to MSA, OMA, and SAGA methods with regard to quality of solution and running time. It can be used for finding multiple sequence alignment as well as testing cost functions.

In a real life situation, due to the complexity of the industrial systems and non-linearity of their behavior, it is very difficult to achieve optimum performance of system for desired industrial goals using uncertain, vague and imprecise data. Herein, an approach has been proposed through which the behavior of the system is analyzed in the form of well- known six reliability indices by using triangular fuzzy numbers which allow the consideration of expert opinions, linguistic variables, operating conditions in reliability information. Using their behavior analysis a fuzzy multi-objective optimization problem (FMOOP) has been formulated. Due to the conflicting nature of the multiple objectives, the decision making is difficult and it leads to the Pareto optimal solutions instead of single optimal solutions. Many evolutionary algorithms (EAs) already exist in the literature for solving a multi-objective optimization problem (MOOP), and are termed as multi-objective evolutionary algorithms (MOEAs). Particle swarm optimization (PSO) is one of such MOEA which demonstrates the ability to identity a Pareto-optimal front efficiently. Here, a crisp optimization problem is reformulated from FMOOP by taking into account the preference of decision maker (DM) and then PSO is applied to solve the resulting fuzzified MOOP. The presented approach is applied in order to solve the multi-objective series-parallel system reliability optimization problem for a crystallization unit of a urea plant.

When solving multi-objective optimization problems (MOPs) with big data, traditional multi-objective evolutionary algorithms (MOEAs) meet challenges because they demand high computational costs that cannot satisfy the demands of online data processing involving optimization. The gradient heuristic optimization methods show great potential in solving large scale numerical optimization problems with acceptable computational costs. However, some intrinsic limitations make them unsuitable for searching for the Pareto fronts. It is believed that the combination of these two types of methods can deal with big MOPs with less computational cost. The main contribution of this paper is that a multi-objective memetic algorithm based on decomposition for big optimization problems (MOMA/D-BigOpt) is proposed and a gradient-based local search operator is embedded in MOMA/D-BigOpt. In the experiments, MOMA/D-BigOpt is tested on the multi-objective big optimization problems with thousands of variables. We also combine the local search operator with other widely used MOEAs to verify its effectiveness. The experimental results show that the proposed algorithm outperforms MOEAs without the gradient heuristic local search operator.

This paper proposes parallel asynchronous versions of promising multi-objective evolutionary algorithms, implemented over a network of personal computers, with the aim of designing an optimal telecommunication network in the presence of multiple conflicting objectives as cost and performance. The resulting tool provides a set of Pareto optimal solutions, facilitating the decision making process of designing a telecommunication network with a mix of different technologies.

Multi-objective evolutionary algorithms (MOEAs) of the state of the art are created with the only purpose of dealing with the number of objective functions in a multi-objective optimization problem (MOP) and treat the decision variables of a MOP as a whole. However, when dealing with MOPs with a large number of decision variables (more than 100) their efficacy decreases as the number of decision variables of the MOP increases. On the other hand, problem decomposition, in terms of decision variables, has been found to be extremely efficient and effective for solving large scale optimization problems. Nevertheless, most of the currently available approaches for large scale optimization rely on models based on cooperative coevolution or linkage learning methods that use multiple subpopulations or preliminary analysis, respectively, which is computationally expensive (in terms of function evaluations) when used within MOEAs. In this work, we study the effect of what we call operational decomposition, which is a novel framework based on coevolutionary concepts to apply MOEAs’s crossover operator without adding any extra cost. We investigate the improvements that NSGA-III can achieve when combined with our proposed coevolutionary operators. This new scheme is capable of improving efficiency of a MOEA when dealing with large scale MOPs having from 200 up to 1200 decision variables.

The Pareto set (PS) of real multi-objective optimization problems (MOPs) are often unknown and complex, so, it is significant for multi-objective evolutionary algorithms (MOEAs) to solve complex PS MOPs (CPS_MOPs namely). In this paper, we combined Latin hypercube sampling (LHS) with MOEA, proposed a LHS based MOEA (LHS-MOEA). We suggested two kinds of LHS-MOEA, in which LHS local search and evolutionary operator are combined to handle CPS_MOPs. Through some experiments, the results demonstrate that LHS-MOEA performs much better than the traditional prevalent MOEA — NSGA-II in solving CPS_MOPs.KeywordsMulti-objective Evolutionary AlgorithmLatin Hypercube SamplingComplex Pareto SetLocal SearchEvolutionary Operator

In this paper, we report a parallel hybrid genetic algorithm (HGA) on consumer-level graphics cards. HGA extends the classical genetic algorithm by incorporating the Cauchy mutation operator from evolutionary programming. In our parallel HGA, all steps except the random number generation procedure are performed in graphics processing unit (GPU) and thus our parallel HGA can be executed effectively and efficiently. We propose the pseudo-deterministic selection method which is comparable to the traditional global selection approach with significant execution time performance advantages. We perform experiments to compare our parallel HGA with our previous parallel FEP (fast evolutionary programming) and demonstrate that the former is much more effective and efficient than the latter. The parallel and sequential implementations of HGA are compared in a number of experiments, it is observed that the former outperforms the latter significantly. The effectiveness and efficiency of the pseudo-deterministic selection method is also studied.

Optimization problems in many industrial applications are very hard to solve. Many examples of them can be found in the design of aeronautical systems. In this field, the designer is frequently faced with the problem of considering not only a single design objective, but several of them, i.e., the designer needs to solve a Multi-Objective Optimization Problem (MOP). In aeronautical systems design, aerodynamics plays a key role in aircraft design, as well as in the design of propulsion system components, such as turbine engines. Thus, aerodynamic shape optimization is a crucial task, and has been extensively studied and developed. Multi-Objective Evolutionary Algorithms (MOEAs) have gained popularity in recent years as optimization methods in this area, mainly because of their simplicity, their ease of use and their suitability to be coupled to specialized numerical simulation tools. In this chapter, we will review some of the most relevant research on the use of MOEAs to solve multi-objective and/or multi-disciplinary aerodynamic shape optimization problems. In this review, we will highlight some of the benefits and drawbacks of the use of MOEAs, as compared to traditional design optimization methods. In the second part of the chapter, we will present a case study on the application of MOEAs for the solution of a multi-objective aerodynamic shape optimization problem.

Decomposition is an effective and popular strategy used by evolutionary algorithms to solve multiobjective optimization problems (MOPs). It can reduce the difficulty of directly solving MOPs, increase the diversity of the obtained solutions, and facilitate parallel computing. However, with the increase of the number of decision variables, the performance of multiobjective evolutionary algorithms (MOEAs) often deteriorates sharply. The advantages of the decomposition strategy are not fully exploited when solving such large-scale MOPs (LSMOPs). To this end, this paper proposes a parallel MOEA based on two-space decomposition (TSD) to solve LSMOPs. The main idea of the algorithm is to decompose the objective space and decision space into multiple subspaces, each of which is expected to contain some complete Pareto-optimal solutions, and then use multiple populations to conduct parallel searches in these subspaces. Specifically, the objective space decomposition approach adopts the traditional reference vector-based method, whereas the decision space decomposition approach adopts the proposed method based on a diversity design subspace (DDS). The algorithm uses a message passing interface (MPI) to implement its parallel environment. The experimental results demonstrate the effectiveness of the proposed DDS-based method. Compared with the state-of-the-art MOEAs in solving various benchmark and real-world problems, the proposed algorithm exhibits advantages in terms of general performance and computational efficiency.

Despite the success of Evolutionary Algorithms in solving complex problems, they may require many function evaluations. This becomes an issue when dealing with costly problems. Surrogate models may overcome this difficulty, though their use in problems with medium to large dimensionality is underexplored in the literature. Problems with multiple conflicting objectives can be formulated as Multi-objective Optimization Problems (MOPs). MOPs have received a great attention lately, mainly with Multi-objective Evolutionary Algorithms (MOEAs). This paper proposes ELMOEA/D-DE, a surrogate-assisted MOEA, for solving expensive MOPs in small evaluation budgets. ELMOEA/D-DE encompasses a state-of-the-art MOEA based on decomposition, Differential Evolution (DE) operators and Extreme Learning Machines. This paper tests three variants of ELMOEA/D-DE, using different DE operators, for solving five known benchmark MOPs with 10 to 60 decision variables. All variants achieve good results in terms of hyper volume metric and the best variant with operator DE/rand/1/bin is compared with two state-of-the-art approaches (MOEA/D-RBF and a non-surrogate-based MOEA), achieving the best results in all but one problems instances.

Recent decades have witnessed great advancements in multiobjective evolutionary algorithms (MOEAs) for multiobjective optimization problems (MOPs). However, these progressively improved MOEAs have not necessarily been equipped with scalable and learnable problem-solving strategies for new and grand challenges brought by the scaling-up MOPs with continuously increasing complexity from diverse aspects, mainly, including expensive cost of function evaluations, many objectives, large-scale search space, time-varying environments, and multitask. Under different scenarios, divergent thinking is required in designing new powerful MOEAs for solving them effectively. In this context, research studies on learnable MOEAs with machine learning techniques have received extensive attention in the field of evolutionary computation. This article begins with a general taxonomy of scaling-up MOPs and learnable MOEAs, followed by an analysis of the challenges that these MOPs pose to traditional MOEAs. Then, we synthetically overview recent advances of learnable MOEAs in solving various scaling-up MOPs, focusing primarily on four attractive directions (i.e., learnable evolutionary discriminators for environmental selection, learnable evolutionary generators for reproduction, learnable evolutionary evaluators for function evaluations, and learnable evolutionary transfer modules for sharing or reusing optimization experience). The insight of learnable MOEAs is offered to readers as a reference to the general track of the efforts in this field.

Multi-Objective Optimization Problems (MOPs) have attracted growing attention during the last decades. Multi-Objective Evolutionary Algorithms (MOEAs) have been extensively used to address MOPs because are able to approximate a set of non-dominated high-quality solutions. The Multi-Objective Quadratic Assignment Problem (mQAP) is a MOP. The mQAP is a generalization of the classical QAP which has been extensively studied, and used in several real-life applications. The mQAP is defined as having as input several flows between the facilities which generate multiple cost functions that must be optimized simultaneously. In this study, we propose PasMoQAP, a parallel asynchronous memetic algorithm to solve the Multi-Objective Quadratic Assignment Problem. PasMoQAP is based on an island model that structures the population by creating sub-populations. The memetic algorithm on each island individually evolve a reduced population of solutions, and they asynchronously cooperate by sending selected solutions to the neighboring islands. The experimental results show that our approach significatively outperforms all the island-based variants of the multi-objective evolutionary algorithm NSGA-II. We show that PasMoQAP is a suitable alternative to solve the Multi-Objective Quadratic Assignment Problem.

The potential and effectiveness of the newly developed Pareto-based multiobjective evolutionary algorithms (MOEA) for solving a real-world power system multiobjective nonlinear optimization problem are comprehensively discussed and evaluated in this paper. Specifically, nondominated sorting genetic algorithm, niched Pareto genetic algorithm, and strength Pareto evolutionary algorithm (SPEA) have been developed and successfully applied to an environmental/economic electric power dispatch problem. A new procedure for quality measure is proposed in this paper in order to evaluate different techniques. A feasibility check procedure has been developed and superimposed on MOEA to restrict the search to the feasible region of the problem space. A hierarchical clustering algorithm is also imposed to provide the power system operator with a representative and manageable Pareto-optimal set. Moreover, an approach based on fuzzy set theory is developed to extract one of the Pareto-optimal solutions as the best compromise one. These multiobjective evolutionary algorithms have been individually examined and applied to the standard IEEE 30-bus six-generator test system. Several optimization runs have been carried out on different cases of problem complexity. The results of MOEA have been compared to those reported in the literature. The results confirm the potential and effectiveness of MOEA compared to the traditional multiobjective optimization techniques. In addition, the results demonstrate the superiority of the SPEA as a promising multiobjective evolutionary algorithm to solve different power system multiobjective optimization problems.