Improved Electromagnetics Optimization: The covariance matrix adaptation evolutionary strategy

Graph neural networks are becoming increasingly popular in deep learning due to their ability to process data in irregular structures and graphs thus preserving additional spatial dependencies due to the arrangement of nodes. This network allows for better accuracy results in classification problems, where information from neighbors is also crucial. However, training such a model is complex and time-consuming and challenging. To date, some metaheuristic algorithms have been used primarily to optimize convolutional networks and find suitable hyperparameters, these are: genetic algorithm, particle swarm optimization, differential evolution or covariance matrix adaptation evolution strategy. In this paper, we propose a metaheuristic algorithm of Simulated Annealing with Uniform distribution for optimization of weights in GCNs, as a hybrid in combination with gradient optimizers. The performance of our technique was tested on the QM7 Dataset, where it was split into two datasets: imbalanced and balanced. Experimental results confirm that our proposed optimization method outperformed other standalone SOTA optimization models, including gradient and heuristics methods, demonstrating in each case to lower loss function values, higher accuracy values for balanced dataset and higher AUC (macro) values for imbalanced dataset.

This chapter is based on a recently published paper [9] of authors of this chapter in which a method for solving bound-constrained non-linear global optimization problems has been proposed. The algorithm obtains a sphere and then generates new trial solutions on its surface. Hence, this algorithm has been named as Spherical Search (SS) algorithm. This chapter starts with an introduction to the SS algorithm and then discusses different components and steps of the algorithm, viz., initialization of population, the concept of a spherical surface, the procedure of generation of trial solutions, selection of new population using greedy selection, stopping criteria, steps of the algorithm, and space and time complexity of the algorithm. Then, the algorithm has been applied to solve 30 bound-constrained global optimization benchmark problems of IEEE CEC 2014 suite and the results of the spherical search algorithm on these benchmark problems have been compared with the results of variants of well-known algorithms such as particle swarm optimization, genetic algorithm, covariance matrix adapted evolution strategy, and Differential Evolution on these problems to demonstrate its performance. Further, the SS algorithm has been applied to solve a model order reduction problem, an example of a real-life complex optimization problem.

This chapter considers a class of optimization techniques that were developed to imitate processes found in nature. Nature is a wonderful source of inspiration for global optimization because so many aspects of natural phenomenon can be mimicked and employed for solving challenging design problems, from the very process of evolution to the coordinated search behavior of various swarming organisms. Nature inspired search algorithms have played an important role in electromagnetic design, as they have proven to be very robust at solving complex problems with many design parameters. Also, as the field of metamaterials has developed, optimization has become an important tool in the quest to overcome performance limitations such as high loss and narrow bandwidth, which have limited the widespread use of metamaterials in practical device applications. In the first part of this chapter, three prominent nature inspired optimization algorithms are described in detail, including the genetic algorithm (GA), particle swarm optimization (PSO), and the covariance matrix adaptation evolutionary strategy (CMA-ES). Following this, several examples of metamaterial surfaces are presented that have each been optimized by one of the three nature inspired techniques. Finally, two homogenization techniques that can be employed to invert scattering parameters for a slab of metamaterial to obtain isotropic or anisotropic effective medium parameters are examined and used in conjunction with a GA to overcome previous limitations in terms of loss and angular stability in metamaterials.

This paper proposes an application of Evolutionary Algorithms (EAs) such as Differential Evolution (DE), Evolutionary Programming (EP), Real coded Genetic Algorithm (RGA), Covariance Matrix Adaptation Evolution Strategy (CMAES) and Particle Swarm Optimization (PSO) to Reactive Power Planning (RPP) problem. RPP is a non-smooth and non-differentiable optimization problem for a multi-objective function. Three loading conditions (Normal load, 1.25% and 1.5% of normal load) have been considered. The IEEE 30 bus system is used to validate the effectiveness of the Evolutionary Algorithms. Simulation results shows that, the DE algorithm gives better results compared to other algorithms for all the loading conditions and it can be better suited for the multi objective RPP problem.

A model-based fault diagnosis in a nonlinear bioreactor using an inverse problem approach and evolutionary algorithms

Broadband antennas find many applications in modern communication systems, such as Wi-Fi, 5G, and SatCom. Multiple competing optimization methods are available for the application to antenna design, while it would be preferable to know in advance if any method is superior. Here, an application of the cross-entropy method, along with particle swarm optimization and covariance matrix adaptation evolutionary strategy, to the design of broadband antennas is presented. The first example is an aperture-coupled microstrip patch antenna that has 9.5 dBi peak directivity and 53% bandwidth after optimization. It is then used as a feed in a high-gain broadband resonant cavity antenna. Using an all-dielectric superstrate with a transverse permittivity gradient, a compact thin resonant cavity antenna with a peak directivity of 19 dBi and 40% 3-dB bandwidth was designed. A comparative analysis of the cross-entropy method, particle swarm optimization, and covariance matrix adaptation evolutionary strategy applied to these two problems was carried out to provide the basis for further optimization of antennas in radio frequency and microwave frequency bands. We found that although all three methods reached a similar solution, the cross-entropy method has a speed advantage. It improves our ability to optimize existing designs and has wider applicability beyond antenna engineering.

Evolutionary algorithm solution and KKT based optimality verification to multi-area economic dispatch

Optimization of fed-batch fermentation processes using the Backtracking Search Algorithm

Carbonate rocks have a very complex pore system due to the presence of interparticle and intra-particle porosities. This makes the acquisition and analysis of the petrophysical data, and the characterization of carbonate rocks a big challenge. In this study, functional network tool is used to develop a model to predict water saturation using petrophysical well logs as input data and the dean-stark measured water saturation as an output parameter. The data comprised of more than 200 well log points corresponding to available core data. The developed FN model was optimized by using several optimization algorithms such as differential evolution (DE), particle swarm optimization (PSO), and covariance matrix adaptation evolution strategy (CMAES). FN model optimized with PSO found to be the most robust artificial intelligence (AI) model to predict water saturation in carbonate rocks. The results showed that the proposed model predicted the water saturation with an accuracy of 97% when related to the experimental core values. In this study in addition to the development of optimized FN model, an explicit empirical correlation is also extracted from the optimized FN model. To validate the proposed correlation, three most commonly applied water saturation models (Simandoux, Bardon and Pied model, Fertl and Hammack Model, Waxman-Smits, and Indonesian) from literature were selected and subjected to same well log data as the AI model to estimate water saturation. The estimated water saturation values for AI and other saturation models were then compared with experimental values of testing data and the results showed that AI model was able to predict water saturation with an error of less than 5% while the saturation models did the same with lesser accuracy of error up to 50%. This work clearly shows that computer-based machine learning techniques can determine water saturation with a high precision and the developed correlation works extremely well in prediction mode.

Features of several well-known global and local optimization methods were discussed and their robustness and efficiency were tested on several artificial test functions in Matlab environment. The tested local methods were the interior-point, the quasi-Newton method, Nelder–Mead simplex, the pattern search, the NEWUOA and the BOBYQA methods. The global methods were the genetic algorithm (GA), the simulated annealing (SA), the particle swarm optimization (PSO), and the covariance matrix adaptation evolutionary strategy (CMA-ES) methods (see subsections 2.2 and 2.3 for their details). Furthermore, a novel global optimization method, called FOCusing robusT Optimization with Uncertainty-based Sampling (FOCTOPUS), which proved to be very efficient in the optimization of constrained and highly correlated parameters of combustion kinetic models, was also tested. The test functions were selected in such a way that they had a variety of features: uni-modal and multi-modal, differentiable and non-differentiable, separable and non-separable, low dimensional and high dimensional. The following test functions were used: the 20D Alpine function, the 4D Ackley function, the Cross-in-tray 2D function, the Hartmann-6D function, the Holder table 2D function, the 5D Rastrigin function, the 5D Rosenbrock function, the 4D modified Rosenbrock function, the 20D Zakharov function and a typical 2D multi-modal function. The general conclusion here is that, the global methods performed well on the multi-modal and high-dimensional test functions while the local methods were superior in the case of low-dimensional and uni-modal test functions. For the highly multi-modal test functions, the GA was better than all the other methods. The FOCTOPUS method proved to be inferior to GA for most of the test functions, thus its application cannot be generally recommended.

When designing a well field for efficiently extracting groundwater or petroleum, it is common for designers to rely on computational optimization methods to determine the optimal placement of wells. The goal of these methods is to find a well-field solution that maximizes the value of a defined objective function, and to do so while utilizing the least amount of computational effort. To achieve this, researchers have developed algorithms based on a wide range of heuristics. Within groundwater management, popular methods include particle swarm optimization (PSO) and genetic algorithms such as differential evolution (DE). This study seeks to investigate a recently developed method called Extremal Optimization for Well Placement Problems (EO-WPP), and to compare its performance with established methods like PSO and DE. EO-WPP is an optimization method based on the extremal optimization (EO) algorithm. EO optimizes by iteratively identifying and modifying the least effective components of a solution. By following this heuristic, the EO algorithm has the potential to quickly find optimal solutions while requiring minimal computational effort. To test this, the performance of DE, PSO and EO-WPP was compared on four benchmark problems. Two of these are the Rastrigin and the Rosenbrock benchmark functions. These functions were used because of their quick evaluation and their popularity in optimization literature. The third benchmark is a synthetic groundwater model, built to test the methods under the context of groundwater management. The final benchmark is a field problem using the Aberdeen groundwater model in South Dakota. The results reveal that EO-WPP was able to outperform DE and PSO on all tested benchmarks. EO-WPP is an effective and efficient optimization tool for well placement design in groundwater management.

ReaxFF is a computationally efficient force field to simulate complex reactive dynamics in extended molecular models with diverse chemistries, if reliable force-field parameters are available for the chemistry of interest. If not, they must be optimized by minimizing the error ReaxFF makes on a relevant training set. Because this optimization is far from trivial, many methods, in particular genetic algorithms (GAs), have been developed to search for the global optimum in parameter space. Recently, two alternative parameter calibration techniques were proposed, i.e. Monte-Carlo Force Field optimizer (MCFF) and Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). In this work, CMA-ES, MCFF and a GA method (OGOLEM) are systematically compared using three training sets from the literature. GA shows the smallest risk of getting trapped into a local minimum, whereas CMA-ES is capable of reaching the lowest errors for two third of the cases. For each method, we provide reasonable default settings and our analysis offers useful guidelines for their usage in future work. An important side effect impairing the parameter optimization is numerical noise. A detailed analysis reveals that it can be reduced, e.g. by using exclusively unambiguous geometry optimizations in the training set. Even without this noise, many distinct near-optimal parameter vectors can be found, which opens new avenues for improving the training set and detecting overfitting artifacts.

Short-term solar radiation forecasting using hybrid deep residual learning and gated LSTM recurrent network with differential covariance matrix adaptation evolution strategy

Comparison of emerging metaheuristic algorithms for optimal hydrothermal system operation

The big bang--big crunch (BB--BC) algorithm has been proposed as a novel optimization method that relies on one of the theories of the evolution of the universe, namely the big bang and big crunch theory. It has been shown that this algorithm is capable of quick convergence, even in long, narrow parabolic-shaped flat valleys or in the existence of several local extremes. In this work, the BB--BC optimization algorithm is hybridized with local search moves in between the ``banging'' and ``crunching'' phases of the algorithm. These 2 original phases of the BB--BC algorithm are preserved, but the representative point (``best'' or ``fittest'' point) attained after the crunching phase of the iteration is modified with some local directional moves, using the previous representative points so far attained, with the hope that a better representative point would be obtained. The effect of imposing local directional moves is inspected by comparing the results of the original and enhanced BB--BC algorithm on various benchmark test functions. Moreover, the crunching phase of the algorithm is improved with the addition of a simplex-based local search routine. The results of the new hybridized algorithm are compared with the state-of-the-art variants of widely used evolutionary computation methods, namely genetic algorithms, covariance matrix adaptation evolutionary strategies, and particle swarm optimization. The results over the benchmark test functions have proven that the BB--BC algorithm, enhanced with local directional moves, has provided more accuracy with the same computation time or for the same number of function evaluations.