An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques

A stepwise optimization approach based on Gaussian process (GP) surrogate model is proposed to determine the process parameters and improve the quality control for injection molding. In order to improve the global performance in this optimization, an enhanced probability of improvement criterion is also introduced. Firstly, GP surrogate model is constructed with the initial samples which are obtained from an optimal design of experiment method. GP is capable of giving both a prediction and an estimate of the confidence for the prediction simultaneously. Secondly, an enhanced probability of improvement criterion is used to find the direction of adding training samples and optimize the surrogate model. Since the global optimal region of the model become accurate efficiently after steps of optimizing the surrogate model, the proposed enhanced probability of improvement criterion can switch more swiftly to global optima compared with other improvement criterion. Finally, an auto front grille molding process is taken as an example to illustrate the method. The results show that the proposed optimization method can effectively decrease the warpage of injection-molded parts.

In this paper, an adaptive optimization method based on Gaussian process (GP) surrogate model is proposed to minimize the warpage of injection molding parts. GP surrogate model combining design of experiment (DOE) methods is used to build an approximate function relationship between warpage and process parameters, replacing the expensive simulation analysis in the optimization iterations. First, establish an approximation function of the relationship between warpage and process parameters by a small size of design of experiment with GP surrogate model. And then, an enhanced probability improvement criterion is used to determine how additional training samples could be added to optimize the surrogate model. Comparing with expected improvement criterion, proposed enhanced probability improvement criterion can switch to global optima more swiftly. Finally, a front grille molding processing is taken as an example to illustrate the criterion. The results show that the proposed optimization method can effectively decrease the warpage of injection molding parts.

The composite kernel learning (CKL) method is introduced to efficiently construct composite kernels for Gaussian process (GP) surrogate models with applications in engineering design. The mixture of kernel functions is cast as a weighted-sum model in which the weights are treated as extra hyperparameters to yield a higher optimum likelihood. The CKL framework aims to improve the accuracy of the GP and relieves the difficulty of kernel selection. In this paper, the combination of five kernel functions are studied, namely, Gaussian, Matérn-3/2, Matérn-5/2, exponential, and cubic, with each kernel sharing the same length scale. Numerical studies were performed on a set of engineering problems to assess the approximation capability of GP-CKL. The results show that, in general, GP-CKL yields a lower approximation error and higher robustness as compared to GP models with single kernel and model selection methods. Numerical experiments show that it is worth using the GP-CKL method for achieving higher accuracy with extra burdens on the number of hyperparameters and a longer training time. Besides, experiments were also performed with variable length scales for each kernel and each variable. However, this variant of CKL is not suggested because it is prone to overfitting and is significantly more expensive than the other GP variants.

Soft computing techniques including fuzzy logic have been successfully applied to Wireless Body Area Networks (WBANs). However, most of the existing research works rely on manual design of the Fuzzy Logic Controller (FLC). To address this issue, in this paper, we propose to use Evolutionary Computation (EC) techniques to automate the design of FLCs for cross layer medium access control in WBANs. With the goal of improving network reliability while keeping the communication delay at a low level, we have particularly experimented on three different coding schemes. The influence of fitness functions has also been examined carefully in order to achieve a good balance between reliability and performance. We have also evaluated the effectiveness of two widely used evolutionary algorithms. Particularly, Particle Swarm Optimisation (PSO) is shown to be more effective than Differential Evolution (DE) for our design problem. Moreover, the FLC designed by our approach is also shown to outperform some related algorithms as well as the IEEE 802.15.4 standard.

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

It is well known that in evolutionary algorithms (EAs), different reproduction operators may be suitable for different problems or in different running stages. To improve the algorithm performance, the ensemble of multiple operators has become popular. Most ensemble techniques achieve this goal by choosing an operator according to a probability learned from the previous experience. In contrast to these ensemble techniques, in this paper we propose a cheap surrogate model-based multioperator search strategy for evolutionary optimization. In our approach, a set of candidate offspring solutions are generated by using the multiple offspring reproduction operators, and the best one according to the surrogate model is chosen as the offspring solution. Two major advantages of this approach are: 1) each operator can generate a solution for competition compared to the probability-based approaches and 2) the surrogate model building is relatively cheap compared to that in the surrogate-assisted EAs. The model is used to implement multioperator ensemble in two popular EAs, that is, differential evolution and particle swarm optimization. Thirty benchmark functions and the functions presented in the CEC 2013 are chosen as the test suite to evaluate our approach. Experimental results indicate that the new approach can improve the performance of single operator-based methods in the majority of the functions.

The lac operen in E. coli is one of the earliest examples of an inducible system of genes being under both positive and negative control that is capable of showing bistability. In this paper, we present a methodology to infer synthetic threshold Boolean regulatory networks of a reduced model of the lac operon using evolutionary computation. The formulation consists in a vector representation of the solutions (networks) and a fitness function specially designed to correctly simulate the bistability through the models' fixed points. We compared the effectiveness and efficiency (runtime) of the proposed approach using three evolutionary computation techniques: differential evolution, genetic algorithms, and particle swarm optimization. The results showed that the three algorithms are capable of finding solutions, being differential evolution the most effective, whereas genetic algorithms was the least effective and efficient in terms of runtime. Particle swarm optimization obtained a good trade-off between effectiveness versus efficiency. One of the inferred solutions was analyzed showing some interesting biological insights, as well as correctly being able to model bistability without any spurious attractors. Overall, the proposed formulation was effective to infer bistable lac operon models under the threshold Boolean network paradigm.

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

The optimization of input variables (typically feeding trajectories over time) in fed-batch fermentations has gained special attention, given the economic impact and the complexity of the problem. Evolutionary Computation (EC) has been a source of algorithms that have shown good performance in this task. In this chapter, Differential Evolution (DE) is proposed to tackle this problem and quite promising results are shown. DE is tested in several real world case studies and compared with other EC algorihtms, such as Evolutionary Algorithms and Particle Swarms. Furthermore, DE is also proposed as an alternative to perform online optimization, where the input variables are adjusted while the real fermentation process is ongoing. In this case, a changing landscape is optimized, therefore making the task of the algorithms more difficult. However, that fact does not impair the performance of the DE and confirms its good behaviour.

Modified Teaching–Learning-Based Optimization algorithm for global numerical optimization—A comparative study

Particle Swarm Optimization or Differential Evolution—A comparison

Source localization with a geoacoustic model requires optimizing the model over a parameter space of range and depth with the objective of matching a predicted sound field to a field measured on an array. We propose a sample-efficient sequential Bayesian optimization strategy that models the objective function as a Gaussian process (GP) surrogate model conditioned on observed data. Using the mean and covariance functions of the GP, a heuristic acquisition function proposes a candidate in parameter space to sample, balancing exploitation (sampling around the best observed objective function value) and exploration (sampling in regions of high variance in the GP). The candidate sample is evaluated, and the GP conditioned on the updated data. Optimization proceeds sequentially until a fixed budget of evaluations is expended. We demonstrate source localization for a shallow-water waveguide using Monte Carlo simulations and experimental data from an acoustic source tow. Compared to grid search and quasi-random sampling strategies, simulations and experimental results indicate the Bayesian optimization strategy converges on optimal solutions rapidly.

Feature selection is an important research issue in classification. As an effective global optimization technique, Particle Swarm Optimization (PSO) algorithm has been widely employed to solve feature selection problems. Population initialization is an important stage in Evolutionary Computation (EC) techniques. Therefore, in EC research field, many works focus on population initialization and it has been verified that suitable population initialization methods can significantly improve the performance of EC techniques. The efficient of PSO significantly deteriorates when being used to solve the large-scale feature selection problems. At present, only a few works have been done to enhance the performance of PSO for feature selection problems through population initialization. Meanwhile, there are many excellent traditional filter feature selection methods. Their characteristic is quick in speed but low in effect. However, they can provide a lot of useful heuristic information. Therefore, this paper focuses on improving the performance of PSO for feature selection problems by designing filter-based population initialization methods. In the proposed method, the filters are firstly used to evaluate features. Based on the obtained heuristic information, the initialization method combining the mixed initialization and the threshold selection is designed. The experiments are carried out on several datasets and the proposed initialization method is compared to some related initialization methods. Besides, the K-Nearest Neighbour (KNN) and other compatible fitness learners are used for performance assessment for feature selection. The results show that the proposed method using KNN classifier is promising to improve the performance of PSO for feature selection problems.

There have been many different views and definitions about evolu- tionary computation. Some regard evolutionary computation as genetic algorithms (GAs), although GAs are only one of many possible types of evolutionary algorithms (EAs). In this chapter, we will take a much broader view of what evolutionary computation is by emphasizing its computational nature. In short, evolutionary computation refers to the study of computational systems that use ideas and draw inspirations from natural evolution. Evolutionary computation techniques can be used to solve a wide range of practical problems in optimization, machine learning and design. This chapter gives a brief introduction to evolutionary computation. A few important issues that may have been overlooked in evolutionary computation will be emphasized. Pointers to further details in the literature will be given whenever appropriate.

Neural networks and fuzzy systems are two soft‐computing paradigms for system modelling. Adapting a neural or fuzzy system requires to solve two optimization problems: structural optimization and parametric optimization. Structural optimization is a discrete optimization problem which is very hard to solve using conventional optimization techniques. Parametric optimization can be solved using conventional optimization techniques, but the solution may be easily trapped at a bad local optimum. Evolutionary computation is a general‐purpose stochastic global optimization approach under the universally accepted neo‐Darwinian paradigm, which is a combination of the classical Darwinian evolutionary theory, the selectionism of Weismann, and the genetics of Mendel. Evolutionary algorithms are a major approach to adaptation and optimization. In this paper, we first introduce evolutionary algorithms with emphasis on genetic algorithms and evolutionary strategies. Other evolutionary algorithms such as genetic programming, evolutionary programming, particle swarm optimization, immune algorithm, and ant colony optimization are also described. Some topics pertaining to evolutionary algorithms are also discussed, and a comparison between evolutionary algorithms and simulated annealing is made. Finally, the application of EAs to the learning of neural networks as well as to the structural and parametric adaptations of fuzzy systems is also detailed.