Controlling Individuals Growth in Semantic Genetic Programming through Elitist Replacement.

<p>Symbolic regression (SR) is a function identification process, the task of which is to identify and express the relationship between the input and output variables in mathematical models. SR is named to emphasise its ability to find the structure and coefficients of the model simultaneously. Genetic Programming (GP) is an attractive and powerful technique for SR, since it does not require any predefined model and has a flexible representation. However, GP based SR generally has a poor generalisation ability which degrades its reliability and hampers its applications to science and real-world modeling. Therefore, this thesis aims to develop new GP approaches to SR that evolve/learn models exhibiting good generalisation ability. This thesis develops a novel feature selection method in GP for high-dimensional SR. Feature selection can potentially contribute not only to improving the efficiency of learning algorithms but also to enhancing the generalisation ability. However, feature selection is seldom considered in GP for high-dimensional SR. The proposed new feature selection method utilises GP’s built-in feature selection ability and relies on permutation to detect the truly relevant features and discard irrelevant/noisy features. The results confirm the superiority of the proposed method over the other examined feature selection methods including random forests and decision trees on identifying the truly relevant features. Further analysis indicates that the models evolved by GP with the proposed feature selection method are more likely to contain only the truly relevant features and have better interpretability. To address the overfitting issue of GP when learning from a relatively small number of instances, this thesis proposes a new GP approach by incorporating structural risk minimisation (SRM), which is a framework to estimate the generalisation performance of models, into GP. The effectiveness of SRM highly depends on the accuracy of the Vapnik-Chervonenkis (VC) dimension measuring model complexity. This thesis significantly extends an experimental method (instead of theoretical estimation) to measure the VC-dimension of a mixture of linear and nonlinear regression models in GP for the first time. The experimental method has been conducted using uniform and non-uniform settings and provides reliable VC-dimension values. The results show that our methods have an impressively better generalisation gain and evolve more compact model, which have a much smaller behavioural difference from the target models than standard GP and GP with bootstrap, The proposed method using the optimised non-uniform setting further improves the one using the uniform setting. This thesis employs geometric semantic GP (GSGP) to tackle the unsatisfied generalisation performance of GP for SR when no overfitting occurs. It proposes three new angle-awareness driven geometric semantic operators (GSO) including selection, crossover and mutation to further explore the geometry of the semantic space to gain a greater generalisation improvement in GP for SR. The angle-awareness brings new geometric properties to these geometric operators, which are expected to provide a greater leverage for approximating the target semantics in each operation, and more importantly, to be resistant to overfitting. The results show that compared with two kinds of state-of-the-art GSOs, the proposed new GSOs not only drive the evolutionary process fitting the target semantics more efficiently but also significantly improve the generalisation performance. A further comparison on the evolved models shows that the new method generally produces simpler models with a much smaller size and containing important building blocks of the target models.</p>

Genetic Programming (GP) has been around for over two decades and has been used in a wide range of practical applications producing human competitive results in several domains. In this paper we present a discussion and a proposal of a GP algorithm that could be conveniently implemented on an embedded system, as part of a broader research project that pursues the implementation of a complete GP system in a Field Programmable Gate Array (FPGA). Motivated by the significant time savings associated with such a platform, as well as low power consumption, low maintenance requirements, small size of the system and the possibility of performing several parallel processes. The proposal is focused on the Geometric Semantic Genetic Programming (GSGP) approach that has been recently introduced with promising results. GSGP induces a unimodal fitness landscape, simplifying the search process. The experimental work considers five variants of GSGP, that incorporate local search strategies, optimal mutations and alignment in error space. Best results were obtained by a simple variant that uses both the optimal mutation step and the standard geometric semantic mutation, using three difficult real-world problems to evaluate the methods, outperforming the original GSGP formulation in terms of fitness and empirical convergence.

Semantic genetic programming is a recent, rapidly growing trend in Genetic Programming (GP) that aims at opening the 'black box' of the evaluation function and make explicit use of more information on program behavior in the search. In the most common scenario of evaluating a GP program on a set of input-output examples (fitness cases), the semantic approach characterizes program with a vector of outputs rather than a single scalar value (fitness). The past research on semantic GP has demonstrated that the additional information obtained in this way facilitates designing more effective search operators. In particular, exploiting the geometric properties of the resulting semantic space leads to search operators with attractive properties, which have provably better theoretical characteristics than conventional GP operators. This in turn leads to dramatic improvements in experimental comparisons. The aim of the tutorial is to give a comprehensive overview of semantic methods in genetic programming, illustrate in an accessible way a formal geometric framework for program semantics to design provably good mutation and crossover operators for traditional GP problem domains, and to analyze rigorously their performance (runtime analysis). A number of real-world applications of this framework will be also presented. Other promising emerging approaches to semantics in GP will be reviewed. In particular, the recent developments in the behavioral programming, which aims at characterizing the entire program behavior (and not only program outputs) will be covered as well. Current challenges and future trends in semantic GP will be identified and discussed. Selected methods and concepts will be accompanied with live software demonstrations. Also, efficient implementation of semantic search operators may be challenging. We will illustrate very efficient, concise and elegant implementations of these operators, which are available for download from the web.

High-performance concrete, besides aggregate, cement, and water, incorporates supplementary cementitious materials, such as fly ash and blast furnace slag, and chemical admixture, such as superplasticizer. Hence, it is a highly complex material and modeling its behavior represents a difficult task. This paper presents an evolutionary system for the prediction of high performance concrete strength. The proposed framework blends a recently developed version of genetic programming with a local search method. The resulting system enables us to build a model that produces an accurate estimation of the considered parameter. Experimental results show the suitability of the proposed system for the prediction of concrete strength. The proposed method produces a lower error with respect to the state-of-the art technique. The paper provides two contributions: from the point of view of the high performance concrete strength prediction, a system able to outperform existing state-of-the-art techniques is defined; from the machine learning perspective, this case study shows that including a local searcher in the geometric semantic genetic programming system can speed up the convergence of the search process.

Geometric Semantic Genetic Programming (GSGP) is a recently proposed form of Genetic Programming (GP) that searches directly the space of the underlying semantics of the programs. The fitness landscape seen by the GSGP variation operators is unimodal with a linear slope by construction and, consequently, easy to search. Despite this advantage, the offspring produced by these operators grow very quickly. A new implementation of the same operators was proposed that computes the semantics of the offspring without having to explicitly build their syntax. This allowed GSGP to be used for the first time in real-life multidimensional datasets. GSGP presented a surprisingly good generalization ability, which was justified by some properties of the geometric semantic operators. In this paper, we show that the good generalization ability of GSGP was the result of a small implementation deviation from the original formulation of the mutation operator, and that without it the generalization results would be significantly worse. We explain the reason for this difference, and then we propose two variants of the geometric semantic mutation that deterministically and optimally adapt the mutation step. They reveal to be more efficient in learning the training data, and they also achieve a competitive generalization in only a single operator application. This provides a competitive alternative when performing semantic search, particularly since they produce small individuals and compute fast.

There are two important limitations of standard tree-based genetic programming (GP). First, GP tends to evolve unnecessarily large programs, what is referred to as bloat. Second, it uses inefficient search operators that operate at the syntax level. The first problem has been the subject of a fair amount of research over the years. Regarding the second problem, one approach is to use alternative search operators, for instance geometric semantic operators. However, another approach is to introduce greedy local search strategies, combining the syntactic search performed by standard GP with local search strategies for solution tuning, which is a simple strategy that has comparatively received much less attention. This work combines a recently proposed bloat-free GP called neat-GP with a local search strategy. One benefit of using a bloat-free GP is that it reduces the size of the parameter space confronted by the local searcher, offsetting some of the added computational cost. The algorithm is validated on a real-world problem with promising results.

Cartesian Genetic Programming (CGP) is a graph-based genetic programming technique developed by Miller & Thompson in 2000 which has several advantages over the traditional form of tree-based Genetic programming (GP). In the standard form, CGP uses only mutation as a genetic operator whereas the use of traditional crossover operators of GP when applied to CGP led to disrupted results. Various researchers have come out with new genetic operators which when applied to standard CGP or its variants have given better results either in the form of better convergence or efficiency. In this paper, we performed a comparative evaluation of four genetic operators, namely standard crossover, Clegg’s crossover, Graph-based crossover, and forking operator, on standard CGP and tested them on standard benchmark datasets such as Koza-2, Koza-3, and Nguyen-2. Our evaluation shows that both Clegg’s crossover and forking operator performs better than the others.KeywordsSoft computingEvolutionary algorithmGenetic programmingGenetic operators

Graph representations promise several desirable properties for genetic programming (GP); multiple-output programs, natural representations of code reuse and, in many cases, an innate mechanism for neutral drift. Each graph GP technique provides a program representation, genetic operators and overarching evolutionary algorithm. This makes it difficult to identify the individual causes of empirical differences, both between these methods and in comparison to traditional GP. In this work, we empirically study the behaviour of Cartesian genetic programming (CGP), linear genetic programming (LGP), evolving graphs by graph programming and traditional GP. By fixing some aspects of the configurations, we study the performance of each graph GP method and GP in combination with three different EAs: generational, steady-state and (1+lambda ). In general, we find that the best choice of representation, genetic operator and evolutionary algorithm depends on the problem domain. Further, we find that graph GP methods can increase search performance on complex real-world regression problems and, particularly in combination with the (1 + lambda) EA, are significantly better on digital circuit synthesis tasks. We further show that the reuse of intermediate results by tuning LGP’s number of registers and CGP’s levels back parameter is of utmost importance and contributes significantly to better convergence of an optimization algorithm when solving complex problems that benefit from code reuse.

This document is devoted to the task of object detection and recognition in digital images by using genetic programming. The goal was to improve and simplify existing approaches. The detection and recognition are achieved by means of extracting the features. A genetic program is used to extract and classify features of objects. Simple features and primitive operators are processed in genetic programming operations. We are trying to detect and to recognize objects in SAR images. Due to the new approach described in this article, five and seven types of objects were recognized with good recognition results.

Efficient truck dispatching is crucial for container port operations. Dynamic container port truck dispatching, a complex online optimization problem, poses significant challenges due to its uncertain and non-linear nature. This paper presents a novel neural network assisted genetic programming (NN-GP) approach, which combines the global search of genetic programming (GP) and the local search of recurrent neural network (RNN). In this framework, the RNN further refines GP individuals after genetic operations (crossovers and mutations), enhancing solution adaptability and precision in response to dynamic and uncertain scenarios. The proposed method leverages RNN's understanding of temporal dynamics and GP's robust exploration of the solution space, effectively addressing the dynamic container truck dispatching problem. Experiments using real-world container port data demonstrate that the RNN-GP model outperforms traditional heuristic methods and standalone GP algorithms, reducing dispatching time and increasing port efficiency. This research highlights the potential of hybridizing machine learning techniques with GP in solving complex real-world optimization problems.

This paper describes a domain independent approach to the use of genetic programming for object detection problems. Rather than using raw pixels or high level domain specific features, this approach uses domain independent statistical features as terminals in genetic programming. Besides position invariant statistics such as mean and standard deviation, this approach also uses position dependent pixel statistics such as moments and local region statistics as terminals. Based on an existing fitness function which uses linear combination of detection rate and alarm rate, we introduce a new measure called false alarm to the fitness function. In addition to the standard arithmetic operators, this approach also uses a conditional operator if in the function set. This approach is tested on two object detection problems. The experiments suggest that position dependent pixel statistics computed from local (central) regions and nonlinear condition functions are effective to object detection problems. Fitness functions with alarm area can reflect the smoothness of evolved genetic programs. This approach works well for the detecting small regular multiple class objects on a relatively uncluttered background.

The data at DKRZ are separated into data produced by climate models and data from observations. The local climate data archive contains 90% model data and 10% observational data. The basic data access problems are correlated with existing storage limitations. Data are preferably stored in the directories of the operating system. The raw data files are stored in the mass storage archive on sequential storage devices. The basic idea to realize the access requirements, searching the metadata catalogue, data extraction, searching climate data, and standard operations on the data, is to implement a climate database system including a graphical user interface. CERA, the Climate and Environmental Data Retrieval and Archiving system, consists of three major parts, the data model, the climate data hierarchy and the climate data processing.

This paper presents an approach for designing classifiers for a multiclass problem using Genetic Programming (GP). The proposed approach takes an integrated view of all classes when GP evolves. An individual of the population will be represented using multiple trees. The GP is trained with a set of N training samples in steps. A concept of unfitness of a tree is used in order to improve genetic evolution. Weak trees having poor performance are given more chance to participate in the genetic operations, and thus improve themselves. In this context, a new mutation operation called nondestructive directed point mutation is used, which reduces the destructive nature of mutation operation. The approach is being demonstrated by experimenting on some datasets.

Melanoma is the deadliest form of skin cancer that causes around 75% of deaths worldwide. However, most of the skin cancers can be cured, especially if detected and treated early. Existing approaches have employed various feature extraction methods, where different types of features are used individually for skin image classification which may not provide sufficient information to the classification algorithm necessary to discriminate between classes, leading to sub-optimal performance. This study develops a novel skin image classification method using multi-tree genetic programming (GP). To capture local information from gray and color skin images, Local Binary Pattern is used in this work. In addition, for capturing global information, variation in color within the lesion and the skin regions, and domain-specific lesion border shape features are extracted. GP with a multi-tree representation is employed to use multiple types of features. Genetic operators such as crossover and mutation are designed accordingly in order to select a single type of features at terminals in one tree of the GP individual. The performance of the proposed method is assessed using two skin image datasets having images captured from multiple modalities, and compared with six most commonly used classification algorithms as well as the standard (single-tree) wrapper and embedded GP methods. The results show that the proposed method has significantly outperformed all these classification methods. Being interpretable and fast in terms of the computation time, this method can help dermatologist identify prominent skin image features, specific to a type of skin cancer in real-time situations.

We have investigated structural distance metrics for linear genetic programs. Causal connections between changes of the genotype and changes of the phenotype form a necessary condition for analyzing structural differences between genetic programs and for the two objectives of this paper: (i) Distance information between individuals is used to control structural diversity of population individuals actively by a two-level tournament selection. (ii) Variation distance is controlled on the effective code for different genetic operators - including a mutation operator that works closely with the applied distance metric. Numerous experiments have been performed for three benchmark problems.