Realization of an Adaptive Memetic Algorithm Using Differential Evolution and Q-Learning: A Case Study in Multirobot Path Planning

This paper provides a novel approach to design an adaptive memetic algorithm by utilizing the composite benefits of Artificial Bee Colony for global search and Q-learning for local refinement. Four variants of Differential Evolution including the currently best Self-adaptive Differential Evolution algorithm have been used here to study the relative performance of the proposed adaptive memetic algorithm with respect to runtime, cost function evaluation and accuracy (offset in cost function from the theoretical optimum after termination of the algorithm). Computer simulations undertaken on a well-known set of 25 benchmark functions reveals that incorporation of Q-learning in Artificial Bee Colony makes the corresponding algorithm more efficient in both runtime and accuracy. The performance of the proposed adaptive memetic algorithm has also been studied on a multi-robot path-planning problem.

Differential evolution (DE) is a relatively new technique that has recently been used to optimize the design for water distribution systems (WDSs). Several parameters need to be determined in the use of DE, including population size, N; mutation weighting factor, F; crossover rate, CR, and a particular mutation strategy. It has been demonstrated that the search behavior of DE is especially sensitive to the F and CR values. These parameters need to be fine-tuned for different optimization problems because they are generally problem-dependent. A self-adaptive differential evolution (SADE) algorithm is proposed to optimize the design of WDSs. Three new contributions are included in the proposed SADE algorithm: (1) instead of pre-specification, the control parameters of F and CR are encoded into the chromosome of the SADE algorithm, and hence are adapted by means of evolution; (2) F and CR values of the SADE algorithm apply at the individual level rather than the generational level normally used by the traditional DE algorithm; and (3) a new convergence criterion is proposed for the SADE algorithm as the termination condition, thereby avoiding pre-specifying a fixed number of generations or computational budget to terminate the evolution. Four WDS case studies have been used to demonstrate the effectiveness of the proposed SADE algorithm. The results show that the proposed algorithm exhibits good performance in terms of solution quality and efficiency. The advantage of the proposed SADE algorithm is that it reduces the effort required to fine-tune algorithm parameter values.

In swarm intelligence, Particle Swarm Optimization (PSO) and Differential Evolution (DE) have been successfully applied in many optimization tasks, and a large number of variants, where novel algorithm operators or components are implemented, has been introduced to boost the empirical performance. In this paper, we first propose to combine the variants of PSO or DE by modularizing each algorithm and incorporating the variants thereof as different options of the corresponding modules. Then, considering the similarity between the inner workings of PSO and DE, we hybridize the algorithms by creating two populations with variation operators of PSO and DE respectively, and selecting individuals from those two populations. The resulting novel hybridization, called PSODE, encompasses most up-to-date variants from both sides, and more importantly gives rise to an enormous number of unseen swarm algorithms via different instantiations of the modules therein. In detail, we consider 16 different variation operators originating from existing PSO- and DE algorithms, which, combined with 4 different selection operators, allow the hybridization framework to generate 800 novel algorithms. The resulting set of hybrid algorithms, along with the combined 30 PSO- and DE algorithms that can be generated with the considered operators, is tested on the 24 problems from the well-known COCO/BBOB benchmark suite, across multiple function groups and dimensionalities.

This paper investigates the suitability of the Particle Swarm Optimization (PSO) and the Differential Evolution (DE) algorithms in solving expensive optimization problems. Eight PSO variants, and eight DE variants are experimentally compared among each other. The Comparing Continuous Optimizers (COCO) methodology was adopted in comparing these variants on the noiseless BBOB test bed. Based on the results, we provide useful insights regarding the algorithms' relative efficiency and effectiveness under an expensive budget of function evaluations, and draw suggestions about which algorithm should be used depending on what we know about our optimization problem in terms of evaluation budget, dimensionality, and function structure. Furthermore, we propose possible future research directions addressing the algorithms limitations. Overall, DE variants perform well in low dimensions, whereas in higher dimensions, several PSO variants surpass DE algorithms. Among the top performers, JADE and χPSO are the robust algorithms for solving expensive budget problems.

Particle Swarm Optimization or Differential Evolution—A comparison

A memetic algorithm is an evolutionary algorithm augmented with a local search. For many applications, researchers have applied variations of memetic algorithms and have gained very positive experimental results. But the theory of these variations of memetic algorithms is still underdeveloped. This paper defines the (1+1) adaptive memetic algorithm with a dynamic mutation probability, and analyzes two types of local searches. We then propose different classes of functions for studying the performance of evolutionary algorithms. We give time complexity analysis that proves our two local searches can outperform each other on different functions. Also we show that memetic algorithms with dynamic mutation probabilities can out-perform memetic algorithms with static mutation probabilities, and vice versa.

Particle swarm optimization (PSO) algorithm has shown promising performances on various benchmark functions and engineering optimization problems. However, it is still difficult to achieve a satisfying trade-off between exploration and exploitation for all the optimization problems and different evolving stages. Furthermore, control parameters of some related mechanisms need pre-experience by the requirement of trial-and-error scheme. This paper presents a novel PSO algorithm, which adaptively adopts various search strategies, called Self-adapting Hybrid Strategy PSO (SaHSPS). Unlike some other peer PSO variants, this method dynamically changes the probabilities of different strategies according to their previous successful searching memories, without any additional control parameters. The probabilities of different strategies would be re-initialized according to a proposed dynamic probabilistic model to diverse the population. Besides, particles are updated by probabilistically selected strategies after niching PSO with Ring topology. Moreover, a dynamic updating mechanism by niching PSO is proposed to guarantee the parallel searching capability during the whole evolution process. Thus, this proposed algorithm might be problem-independent and search-stage-independent, yielding more satisfying solutions on various optimization problems. A comprehensive experimental study is conducted on 28 benchmark functions of CEC 2013 special session on real-parameter optimization, including shifted, rotated, multi-modal, high conditioned, expanded and composition problems, compared with several state-of-the-art variants of PSO and differential evolution (DE) algorithms. Comparison results show that SaHSPS obtains outstanding performances on the majority of the test problems. Moreover, a practical engineering problem, real power loss minimization of IEEE 30-bus power system, is used to further evaluate SaHSPS. The numerical results, compared with other stochastic search algorithms, show that SaHSPS could find high-quality solutions with higher probability.

Differential Evolution (DE) is a very simple but powerful Evolutionary Algorithm (EA) for real-world applications. There are two aspects affecting the overall optimization performance significantly, one is parameter control scheme and the other is trial vector generation strategy. As all the control parameters are employed in generating trial vectors, from this perspective of view, the trial vector generation strategy dominates the overall optimization performance. Therefore, trial vector generation strategies are often designed first, and then the corresponding parameter control schemes are well-tuned thereafter. Both of them constitute the main body of a new DE variant. Here in the paper, a novel DE variant with a new designed trial vector generation strategy as well as novel parameter control scheme is proposed to tackle freight trains scheduling problem. By incorporating both a time-stamp mechanism of the external archive and a novel parameter control scheme, the new algorithm can secure better optimization performance not only on man-made benchmarks but also on real-world applications. Both the man-made benchmarks from Congress on Evolutionary Computation (CEC) test suite and the real-world freight train scheduling problem are employed in the validation of the new DE variant. The experiment results show that it is competitive with other state-of-the-art DE techniques for these optimization problems.

This brief paper reports a hybrid algorithm we developed recently to solve the global optimization problems of multimodal functions, by combining the advantages of two powerful population-based metaheuristics—differential evolution (DE) and particle swarm optimization (PSO). In the hybrid denoted by DEPSO, each individual in one generation chooses its evolution method, DE or PSO, in a statistical learning way. The choice depends on the relative success ratio of the two methods in a previous learning period. The proposed DEPSO is compared with its PSO and DE parents, two advanced DE variants one of which is suggested by the originators of DE, two advanced PSO variants one of which is acknowledged as a recent standard by PSO community, and also a previous DEPSO. Benchmark tests demonstrate that the DEPSO is more competent for the global optimization of multimodal functions due to its high optimization quality.

Differential evolution (DE) is a powerful evolutionary algorithm for global optimization problems. Generally, appropriate mutation strategies and proper equilibrium between global exploration and local exploitation are significant to the performance of DE. From this consideration, in this paper, we present a novel DE variant, abbreviated to DMIE-DE, to further enhance the optimization capacity of DE by developing a dual mutations collaboration mechanism with elites guiding and inferiors eliminating techniques. More specifically, an explorative mutation strategy DE/current-to-embest with an elite individual serving as part of the difference vector and an exploitative mutation strategy DE/ebest-to-rand with selecting an elite individual as the base vector are employed simultaneously to achieve the balance between local and global performance of the whole population instead of only one mutation strategy in classical DE algorithm. The control parameters F and CR for above mutation strategies are updated adaptively to supplement the optimization ability of DMIE-DE based on a rational probability distribution model and the successful experience from the previous iterations. Moreover, an inferior solutions eliminating technique is embedded to enhance the convergence speed and compensate cost of the fitness evaluation times during the evaluation process. To evaluate the performance of DMIE-DE, experiments are conducted by comparing with five state-of-the-art DE variants on solving 29 test functions in CEC2017 benchmark set. The experimental results indicate that the performance of DMIE-DE is significantly better than, or at least comparable to the considered DE variants.

Differential Evolution with exponential crossover can be also competitive on numerical optimization

MEMPSODE: A global optimization software based on hybridization of population-based algorithms and local searches

Differential evolution (DE) is an efficient population-based search algorithm for solving numerical optimization problems. However, the performance of DE is very sensitive to the choice of mutation strategies and their associated control parameters. In this paper, we propose a self-adaptive multi-population differential evolution algorithm, called SAMDE. The population is randomly divided into three equally sized sub-populations, each with different mutation strategies. At the end of each generation, all sub-populations are updated independently and recombined. Each sub-population uses an adaptive mechanism for selecting how current generation control parameters are generated. An improved mutation strategy, “rand assemble/1”, is proposed, its base vector is composed proportionally of three randomly selected individuals. The performance of SAMDE is evaluated on the suite of CEC 2005 benchmark functions. A comparative study is carried out with other state-of-the-art optimization techniques. The results show that SAMDE has a competitive performance compared to several other efficient DE variants.

Differential evolution with orthogonal array‐based initialization and a novel selection strategy

Enhancing the search ability of differential evolution through orthogonal crossover