Dynamic GP fitness cases in static and dynamic optimisation problems

<p>Symbolic regression (SR) is a function identification process, the task of which is to identify and express the relationship between the input and output variables in mathematical models. SR is named to emphasise its ability to find the structure and coefficients of the model simultaneously. Genetic Programming (GP) is an attractive and powerful technique for SR, since it does not require any predefined model and has a flexible representation. However, GP based SR generally has a poor generalisation ability which degrades its reliability and hampers its applications to science and real-world modeling. Therefore, this thesis aims to develop new GP approaches to SR that evolve/learn models exhibiting good generalisation ability. This thesis develops a novel feature selection method in GP for high-dimensional SR. Feature selection can potentially contribute not only to improving the efficiency of learning algorithms but also to enhancing the generalisation ability. However, feature selection is seldom considered in GP for high-dimensional SR. The proposed new feature selection method utilises GP’s built-in feature selection ability and relies on permutation to detect the truly relevant features and discard irrelevant/noisy features. The results confirm the superiority of the proposed method over the other examined feature selection methods including random forests and decision trees on identifying the truly relevant features. Further analysis indicates that the models evolved by GP with the proposed feature selection method are more likely to contain only the truly relevant features and have better interpretability. To address the overfitting issue of GP when learning from a relatively small number of instances, this thesis proposes a new GP approach by incorporating structural risk minimisation (SRM), which is a framework to estimate the generalisation performance of models, into GP. The effectiveness of SRM highly depends on the accuracy of the Vapnik-Chervonenkis (VC) dimension measuring model complexity. This thesis significantly extends an experimental method (instead of theoretical estimation) to measure the VC-dimension of a mixture of linear and nonlinear regression models in GP for the first time. The experimental method has been conducted using uniform and non-uniform settings and provides reliable VC-dimension values. The results show that our methods have an impressively better generalisation gain and evolve more compact model, which have a much smaller behavioural difference from the target models than standard GP and GP with bootstrap, The proposed method using the optimised non-uniform setting further improves the one using the uniform setting. This thesis employs geometric semantic GP (GSGP) to tackle the unsatisfied generalisation performance of GP for SR when no overfitting occurs. It proposes three new angle-awareness driven geometric semantic operators (GSO) including selection, crossover and mutation to further explore the geometry of the semantic space to gain a greater generalisation improvement in GP for SR. The angle-awareness brings new geometric properties to these geometric operators, which are expected to provide a greater leverage for approximating the target semantics in each operation, and more importantly, to be resistant to overfitting. The results show that compared with two kinds of state-of-the-art GSOs, the proposed new GSOs not only drive the evolutionary process fitting the target semantics more efficiently but also significantly improve the generalisation performance. A further comparison on the evolved models shows that the new method generally produces simpler models with a much smaller size and containing important building blocks of the target models.</p>

Solving online dynamic time-linkage problems under unreliable prediction

Dynamic multiobjective optimization problem (DMOP) is a class of complex dynamic optimization problem (DOP), its widely exists in many real-world problems. Firstly, an core estimation of distribution model which used to approximately estimate the Pareto solution in the next environment is given, when a change in the environment is detected, the algorithm uses the collected information from the previous searching environments to predict the location of the Pareto solution in the next environment. Furthermore, a dynamic multiobjective optimization evolutionary algorithm with core estimation of distribution is proposed. The computer simulations are made on two dynamic multiobjective optimization problems, and the results indicate the proposed algorithm is effective.

Nonlinear model identification of an experimental ball-and-tube system using a genetic programming approach

Genetic Programming (GP) is a form of nature-inspired computing, introduced over 30 years ago, with notable success in problems such as symbolic regression. However, there remains a lot of relatively unexploited potential for solving hard, real-world problems. There is consensus in the GP community that the lack of effective real-world benchmark problems negatively impacts the quality of research [4]. When a GP system is initialised, a number of parameters must be provided. The optimal setup configuration is often not known, due to the fact that many of the values are problem and domain specific, meaning the GP system is unable to produce satisfactory results. We believe that the implementation of continuous self-adaptation, along with the introduction of tunable and suitably difficult benchmark problems, will allow for the creation of more robust GP systems that are resilient to failure.

<p>One of the greater issues in Genetic Programming (GP) is the computational effort required to run the evolution and discover a good solution. Phenomena such as program bloating (where genetic programs rapidly grow in size) can quickly exhaust available memory resources and slow down the evolutionary process, while the heavy cost of performing fitness evaluation can make problems which have a lot of available data very slow to solve. These issues may limit GP in some tasks it can appropriately be applied to, as well as inhibit its applications in time/space sensitive environments. In this thesis, we look at developing solutions to some of these issues in GP computational cost. First, we develop an algebraic program simplification method based on simple rules and hashing techniques, and use this method in conjunction with the standard GP on a variety of tasks. Our results suggest that program simplification can lead to a significant reduction in program size, while not significantly changing the effectiveness of the systems in finding solution programs. Secondly, we analyse the effects of program simplification on the internal GP "building blocks" to investigate whether simplification is a destructive or constructive force. Using two models for building blocks (numerical-nodes and the more complex fixed-depth subtree), we track building blocks through GP runs on a symbolic regression problem, both with and without using simplification. We find that the program simplification process can both disrupt and construct building blocks in the GP populations. However, GP systems using simplification appear to retain important building blocks, and the simplification process appears to lead to an increase in genetic diversity. These may help explain why using simplification does not reduce the effectiveness of GP systems in solving tasks. Lastly, we develop two methods of reducing the cost of fitness evaluation by reducing the number of node evaluations performed. The first method is elitism avoidance, which avoids re-evaluating programs which have been placed in the population using elitismreproduction. Thismethod reduces the CPU time for evolving solutions for six different GP tasks. The second method is a subtree caching mechanism which store fitness evaluations for subtrees in a cache so that they may be fetched when these subtrees are encountered in future fitness evaluations. Results suggest that using this mechanism can significantly reduce both the number of node evaluations and the overall CPU time used in evolving solutions, without reducing the fitness of the solutions produced.</p>

We propose a novel method to solve the integrated scheduling and dynamic optimization problem for sequential batch processes. The scheduling problem and the dynamic optimization problems are collaborated by a Stackelberg game (leader–followers game). Mathematically, the integrated problem is formulated into a bilevel program. The scheduling problem in the upper level acts as the leader, while the dynamic optimization problems in the lower level are the followers. The follower problems have their own objectives, but the leader problem can coordinate the follower problems to pursue its objective. To efficiently solve the bilevel program, we develop a decomposition algorithm. It first solves the lower-level problems to determine the response functions. The response functions are then represented by piecewise linear functions to solve the upper-level problem. The integrated method is consistent with the ISA 95 standard and can be easily implemented in an IT infrastructure following the standard. The performance of the proposed method is demonstrated in a complicated batch plant in comparison with the traditional sequential method which solves the scheduling problem and the dynamic optimization problems separately, and the simultaneous method which formulates the integrated problem into a monolithic model.

Genetic programming (GP) is applied to the design of fuzzy logic controllers (FLCs) for mobile robot path tracking. GP is applied to automatic discovery of full knowledge bases for use in fuzzy logic control applications. An extension to a rule learning GP system is presented that achieves this objective. In addition, GP is employed to handle selection of fuzzy set intersection operators (t-norms). The new GP system is applied to design a mobile robot path tracking controller and performance is shown to be comparable to that of a manually designed controller. GP was successfully applied to discover FLCs capable of steering a mobile robot to track straight-line paths in the plane. Instances of simultaneous evolution of membership functions and rules showed that GP was capable of evolving a FLC that demonstrated satisfactory responsiveness to various initial conditions while utilizing minimal human interface.

Quantum particles-enhanced multiple Harris Hawks swarms for dynamic optimization problems

This paper investigates the efficiency of using semantic and syntactic distance metrics in fitness sharing with Genetic Programming (GP). We modify the implementation of fitness sharing to speed up its execution, and used two distance metrics in calculating the distance between individuals in fitness sharing: semantic distance and syntactic distance. We applied fitness sharing with these two distance metrics to a class of real-valued symbolic regression. Experimental results show that using semantic distance in fitness sharing helps to significantly improve the performance of GP more frequently, and results in faster execution times than with the syntactic distance. Moreover, we also analyse the impact of the fitness sharing parameters on GP performance helping to indicate appropriate values for fitness sharing using a semantic distance metric.KeywordsGenetic programmingFitness sharingSemanticSyntactic

Genetic programming (GP) is an evolutionary computation paradigm for the automatic induction of syntactic expressions. In general, GP performs an evolutionary search within the space of possible program syntaxes, for the expression that best solves a given problem. The most common application domain for GP is symbolic regression, where the goal is to find the syntactic expression that best fits a given set of training data. However, canonical GP only employs a syntactic search, thus it is intrinsically unable to efficiently adjust the (implicit) parameters of a particular expression. This work studies a Lamarckian memetic GP, that incorporates a local search (LS) strategy to refine GP individuals expressed as syntax trees. In particular, a simple parametrization for GP trees is proposed, and different heuristic methods are tested to determine which individuals should be subject to a LS, tested over several benchmark and real-world problems. The experimental results provide necessary insights in this insufficiently studied aspect of GP, suggesting promising directions for future work aimed at developing new memetic GP systems.KeywordsGenetic ProgrammingLocal SearchMemetic Algorithms

This paper presents a novel nonparametric efficiency analysis technique based on the Genetic Programming (GP) in order to measure efficiency of Iran electric power plants. GP model was used to predict the output of power plants with respect to input data. The method, we presented here, is capable of finding a best performance among power plant based on the set of input data, GP predicted results and real outputs. The advantage of using GP over traditional statistical methods is that in prediction with GP, the researcher doesn’t need to assume the data characteristic of the dependent variable or output and the independent variable or input. In this proposed methodology to calculate the efficiency scores, a novel algorithm was introduced which worked on the basis of predicted and real output values. To validate our model, the results of proposed algorithm for calculating efficiency rank of power plants were compared to traditional method. Real data was presented for illustrative our proposed methodology. Results showed that by utilizing the capability of input-output pattern recognition of GP, this method provides more realistic results and outperform in identification of efficient units than the conventional methods.

Symbolic regression is a process to find a mathematical expression that represents the relationship between a set of explanatory variables and a measured variable. It has become a best-known problem for GP (genetic programming), as GP can use the tree representation to represent solutions as expression trees. Since the success of memetic algorithms (MAs (Memetic algorithms (MAs) can be regarded as a class of methods that combine population-based global search and local search [6, 30])) has proved the importance of local search in augmenting the global search ability of GP, GP with local search is investigated to solve symbolic regression tasks in this work. An important design issue of MAs is the balance between the global exploration of GP and the local exploitation, which has a great influence on the performance and efficiency of MAs. This work proposes a GP-based memetic algorithm for symbolic regression, termed as aMeGP (a daptive Me metic GP), which can balance global exploration and local exploitation adaptively. Compared with GP, two improvements are made in aMeGP to invoke and stop local search adaptively during evolution. The proposed aMeGP is compared with GP-based and nonGP-based symbolic regression methods on both benchmark test functions and real-world applications. The results show that aMeGP is generally better than both GP-based and nonGP-based reference methods with its evolved solutions achieving lower root mean square error (RMSE) for most test cases. Moreover, aMeGP outperforms the reference GP-based methods in the convergence ability, which can converge to lower RMSE values with faster or similar speeds.

Comparing and ranking information is an important topic in social and information sciences, and in particular on the web. Its objective is to measure the difference of the preferences of voters on a set of candidates and to compute a consensus ranking. Commonly, each voter provides a total order or a bucket order of all candidates, where bucket orders allow ties. In this work we consider the generalization of total and bucket orders to partial orders and compare them by the nearest neighbor and the Hausdorff Kendall tau distances. For total and bucket orders these distances can be computed in $\mathcal{O}(n \log n)$ time. We show that the computation of the nearest neighbor Kendall tau distance is NP-hard, 2-approximable and fixed-parameter tractable for a total and a partial order. The computation of the Hausdorff Kendall tau distance for a total and a partial order is shown to be coNP-hard. The rank aggregation problem is known to be NP-complete for total and bucket orders, even for four voters and solvable in $\mathcal{O}(n\log n)$ for two voters. It is NP-complete for two partial orders and the nearest neighbor Kendall tau distance. For the Hausdorff Kendall tau distance it is in $\mathbf{\Sigma_2^p}$ , but not in NP or coNP unless $\ensuremath{\mathbf{NP}} = \ensuremath{\mathbf{coNP}} $ , even for four voters.

The evolutionary approach of Genetic Programming (GP) has been applied extensively to model various non-linear systems. The distinct advantage of using GP is that prior assumptions for the selection of a model structure are not required. The GP automatically evolves the optimal model structure and its parameters that best describe the system characteristics. However, the evolution of an optimal model structure is highly dependent on the experimental designs used to sample the problem (system) domain and capture its characteristics. The literature reveals that very few researchers have studied the effect of various experimental designs on the performance of GP models and therefore the optimum choice of an experimental design is still unknown. This paper studies the effect of various experimental designs on the performance of GP models on two non-linear test functions. The objective of the paper is to identify the most robust (best) experimental design for effective modeling of non-linear test functions using GP. The analysis reveals that for the test function 1, the experimental design that gives best performance of GP models is response surface faced design and for test function 2, the best experimental design is 5-level full factorial design. Thus, the result concludes that the selection of the robust experimental design is a crucial preprocessing step for the effective modeling of non-linear systems using GP.