Grammatical Evolution Mapping for Semantically-Constrained Genetic Programming

Big data driven genetic improvement for maintenance of legacy software systems

Evolutionary computation (EC) is the study of computational systems that borrow ideas from and are inspired by natural evolution and adaptation (Yao & Xu, 2006, pp. 1-18). EC covers a number of techniques based on evolutionary processes and natural selection: evolutionary strategies, genetic algorithms and genetic programming (Keedwell & Narayanan, 2005). Evolutionary strategies are an approach for efficiently solving certain continuous problems, yielding good results for some parametric problems in real domains. Compared with genetic algorithms, evolutionary strategies run more exploratory searches and are a good option when applied to relatively unknown parametric problems. Genetic algorithms emulate the evolutionary process that takes place in nature. Individuals compete for survival by adapting as best they can to the environmental conditions. Crossovers between individuals, mutations and deaths are all part of this process of adaptation. By substituting the natural environment for the problem to be solved, we get a computationally cheap method that is capable of dealing with any problem, provided we know how to determine individuals’ fitness (Manrique, 2001). Genetic programming is an extension of genetic algorithms (Couchet, Manrique, Ríos & Rodríguez- Patón, 2006). Its aim is to build computer programs that are not expressly designed and programmed by a human being. It can be said to be an optimization technique whose search space is composed of all possible computer programs for solving a particular problem. Genetic programming’s key advantage over genetic algorithms is that it can handle individuals (computer programs) of different lengths. Grammar-guided genetic programming (GGGP) is an extension of traditional GP systems (Whigham, 1995, pp. 33-41). The difference lies in the fact that they employ context-free grammars (CFG) that generate all the possible solutions to a given problem as sentences, establishing this way the formal definition of the syntactic problem constraints, and use the derivation trees for each sentence to encode these solutions (Dounias, Tsakonas, Jantzen, Axer, Bjerregard & von Keyserlingk, D. 2002, pp. 494-500). The use of this type of syntactic formalisms helps to solve the so-called closure problem (Whigham, 1996). To achieve closure valid individuals (points that belong to the search space) should always be generated. As the generation of invalid individuals slows down convergence speed a great deal, solving this problem will very much improve the GP search capability. The basic operator directly affecting the closure problem is crossover: crossing two (or any) valid individuals should generate a valid offspring. Similarly, this is the operator that has the biggest impact on the process of convergence towards the optimum solution. Therefore, this article reviews the most important crossover operators employed in GP and GGGP, highlighting the weaknesses existing nowadays in this area of research. We also propose a GGGP system. This system incorporates the original idea of employing ambiguous CFG to overcome these weaknesses, thereby increasing convergence speed and reducing the likelihood of trapping in local optima. Comparative results are shown to empirically corroborate our claims.

This chapter evaluates the performance of various methods to constant creation in Grammatical Evolution (GE), and validates the results by comparing against those from a reasonably standard Genetic Programming (GP) setup. Specifically, the chapter compares a standard GE method to constant creation termed digit concatenation with what this chapter calls compact methods to constant creation. Constant creation in GE is an important issue due to the disruptive nature of ripple crossover, which can radically remap multiple terminals in an individual, and we investigate if more compact methods, which are more similar to the GP style of constant creation (Ephemeral Random Constants (ERCs), perform better. The results are surprising. Against common wisdom, a standard GE approach of digit concatenation does not produce individuals that are any larger than those from methods which are designed to use less genetic material. In fact, while GP characteristically evolves increasingly larger individuals, GE—after an initial growth or drop in sizes—tends to keep individual sizes stable despite no explicit mechanisms to control size growth. Furthermore, various GE setups perform acceptably well on unseen test data and typically outperform GP. Overall, these results encourage a belief that standard GE methods to symbolic regression are relatively resistant to pathogenic evolutionary tendencies of code bloat and overfitting.

In this thesis, a number of search techniques are proposed as a solution for program and discrete controller synthesis (DCS). Classic synthesis techniques facilitate exhaus- tive search, while genetic programming has recently proven the potential of generic search techniques. But is genetic programming the right search technique for the synthesis prob- lem? In this thesis we challenge this belief and argue in favor of simulated annealing, a different class of general search techniques. We show that, in hindsight, the success of genetic programming has drawn from what is arguably a hybrid between simulated annealing and genetic programming, and compare the fitness of classic genetic program- ming, the hybrid form, and pure simulated annealing. Our experimental evaluation suggests that pure simulated annealing offers better results for automated programming than techniques based on genetic programming. Discrete Controller Synthesis (DCS) and Program Synthesis have similar goals: they are automated techniques to infer a control strategy and an implementation, respectively, that is correct by construction. We also investigate the application of the search tech- niques that we have been used for program synthesis for the computation of deterministic strategies solving symbolic Discrete Controller Synthesis (DCS) problems, where a model of the system under control is given along with desired objective behaviours. We experi- mentally confirm that relative performance results are similar to program synthesis, and give a complexity analysis of our simulated annealing algorithm for symbolic DCS. From the performance results we obtain, we draw the conclusion that simulated annealing, when combined with efficient model-checking techniques, is worth further investigating to solve symbolic DCS problems. A tool is designed to explore the parameter space of different synthesis techniques. Besides using it to synthesise a discrete control strategies for reactive systems (controller synthesis) and for protocol adapters for the coordination of different threads (software synthesis), we can also use it to study the influence of turning various screws in the syn- thesis process. For simulated annealing, PranCS allows the user to define the behaviour of the cooling schedule. For genetic programming, the user can select the population size.

Program synthesis is a problem domain that due to its importance is tackled by many different fields, one being Genetic Programming. Two variants, Grammar-Guided Genetic Programming (G3P) and PushGP, have been applied to a vast general program synthesis benchmark suite and solved a variety of problems although with varying success rates. While G3P achieved higher success rates on some problems, PushGP was able to find solutions to more problem instances. Reason why G3P fails at some problems might be missing functionality in the grammars or knowledge that has to discovered during the runs. In this paper the current shortcomings of G3P are analysed and the papers contributions include an example of extending grammars for program synthesis, a fairer comparison between PushGP and G3P with a more similar function set as well as new results on problems that have not been solved with G3P and one that has not been solved with PushGP.

In the previous Chapter we highlighted the fact that very little research has been conducted into the area of Genetic Programming (GP) in dynamic environments. In this book we outline the foundations of research to date with Grammatical Evolution (GE) for these kinds of non-stationary environments. As described earlier, GE possesses a number of features that differentiate it significantly from GP and it is these features that present the most interesting avenues for exploration in relation to dynamic environments, more so than in their application to static problems.In this chapter we start out by detailing in Section 4.1 the very first steps which we have taken with GE into the domain of non-stationary environments. Following this, in Section 4.2, we discuss the potential strengths of GE for the challenges presented by a dynamic world. Finally outline in Section 4.3 how we build the foundations upon which GE can be developed for application in these formidable environments.KeywordsGenetic ProgrammingGenetic CodeDynamic EnvironmentNeutral NetworkEvolutionary SearchThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper introduces an approach to evolving computer programs using an Attribute Grammar (AG) extension of Grammatical Evolution (GE) to eliminate ineffective pieces of code with the help of context-sensitive information.The standard Context-Free Grammars (CFGs) used in GE, Genetic Programming (GP) (which uses a special type of CFG with just a single non-terminal) and most other grammar-based system are not well-suited for codifying information about context. AGs, on the other hand, are grammars that contain functional units that can help determine context which, as this paper demonstrates, is key to removing ineffective code.The results presented in this paper indicate that, on a selection of grammars, the prevention of the appearance of ineffective code through the use of context analysis significantly improves the performance of and resistance to code bloat over both standard GE and GP for both Santa Fe Trail (SFT) and Los Altos Hills (LAH) trail version of the ant problem with same amount of energy used.

Genetic Programming (GP) is a powerful software induction technique that can be applied to solve a wide variety of problems. However, most researchers develop tailor-made GP tools for solving specific problems. These tools generally require significant modifications in their kernel to be adapted to other domains. In this paper, we explore the Grammar-Guided Genetic Programming (GGGP) approach as an alternative to overcome such limitation. We describe a GGGP based framework, named Chameleon, that can be easily configured to solve different problems. We explore the use of Chameleon in two domains, not usually addressed by works in the literature: in the task of mining relational databases and in the software testing activity. The presented results point out that the use of the grammar-guided approach helps us to obtain more generic GP frameworks and that they can contribute in the explored domains.

Dispatching rules are most commonly used to solve scheduling problems under dynamic conditions. Since designing new dispatching rules is a time-consuming process, it can be automated by using various machine learning and evolutionary computation methods. In previous research, genetic programming has been the most commonly used method for automatically designing new dispatching rules. However, there are many other evolutionary methods that use representations other than genetic programming that can be used to create dispatching rules. Some, such as gene expression programming, have already been used successfully, while others, such as Cartesian genetic programming or grammatical evolution, have not yet been used to generate dispatching rules. In this paper, six different methods (genetic programming, gene expression programming, Cartesian genetic programming, grammatical evolution, stack representation, and analytic programming) for generating dispatching rules for the unrelated machines environment are tested and the results for various scheduling criteria are analysed. It is also analysed how different individual sizes in the tested methods affect the performance and average size of the generated dispatching rules. The results show that, with the exception of grammatical evolution and analytic programming, all tested methods perform quite similarly, with results depending on the selected scheduling criterion. The results also show that Cartesian genetic programming is the most resistant to the occurrence of bloat and evolves dispatching rules with the smallest average size.

In our previous studies, Genetic Programming (GP), Probabilistic Incremental Program Evolution (PIPE) and Ant Programming (AP) have been used to optimal design of Flexible Neural Tree (FNT). In this paper Grammar Guided Genetic Programming (GGGP) was employed to optimize the architecture of FNT model. Based on the predefined instruction sets, a flexible neural tree model can be created and evolved. This framework allows input variables selection, over-layer connections and different activation functions for the various nodes involved. The free parameters embedded in the neural tree are optimized by particle swarm optimization algorithm. Empirical results on stock index prediction problems indicate that the proposed method is better than the neural network and genetic programming forecasting models.

BackgroundIn this work, we developed many machine learning classifiers to assist in diagnosing respiratory changes associated with sarcoidosis, based on results from the Forced Oscillation Technique (FOT), a non-invasive method used to assess pulmonary mechanics. In addition to accurate results, there is a particular interest in their interpretability and explainability, so we used Genetic Programming since the classification is made with intelligible expressions and we also evaluate the feature importance in different experiments to find the more discriminative features.Methodology/principal findingsWe used genetic programming in its traditional tree form and a grammar-based form. To check if interpretable results are competitive, we compared their performance to K-Nearest Neighbors, Support Vector Machine, AdaBoost, Random Forest, LightGBM, XGBoost, Decision Trees and Logistic Regressor. We also performed experiments with fuzzy features and tested a feature selection technique to bring even more interpretability. The data used to feed the classifiers come from the FOT exams in 72 individuals, of which 25 were healthy, and 47 were diagnosed with sarcoidosis. Among the latter, 24 showed normal conditions by spirometry, and 23 showed respiratory changes. The results achieved high accuracy (AUC > 0.90) in two analyses performed (controls vs. individuals with sarcoidosis and normal spirometry and controls vs. individuals with sarcoidosis and altered spirometry). Genetic Programming and Grammatical Evolution were particularly beneficial because they provide intelligible expressions to make the classification. The observation of which features were selected most frequently also brought explainability to the study of sarcoidosis.ConclusionsThe proposed system may provide decision support for clinicians when they are struggling to give a confirmed clinical diagnosis. Clinicians may reference the prediction results and make better decisions, improving the productivity of pulmonary function services by AI-assisted workflow.

In recent years, Automated Program Repair (APR) has been extensively studied in academia and even drawn wide attention from the industry. However, APR techniques can be extremely time consuming since (1) a large number of patches can be generated for a given bug, and (2) each patch needs to be executed on the original tests to ensure its correctness. In the literature, various techniques (e.g., based on learning, mining, and constraint solving) have been proposed/studied to reduce the number of patches. Intuitively, every patch can be treated as a software revision during regression testing; thus, traditional Regression Test Selection (RTS) techniques can be leveraged to only execute the tests affected by each patch (as the other tests would keep the same outcomes) to further reduce patch execution time. However, few APR systems actually adopt RTS and there is still a lack of systematic studies demonstrating the benefits of RTS and the impact of different RTS strategies on APR. To this end, this article presents the first extensive study of widely used RTS techniques at different levels (i.e., class/method/statement levels) for 12 state-of-the-art APR systems on over 2M patches. Our study reveals various practical guidelines for bridging the gap between APR and regression testing, including: (1) the number of patches widely used for measuring APR efficiency can incur skewed conclusions, and the use of inconsistent RTS configurations can further skew the conclusions; (2) all studied RTS techniques can substantially improve APR efficiency and should be considered in future APR work; (3) method- and statement-level RTS outperform class-level RTS substantially and should be preferred; (4) RTS techniques can substantially outperform state-of-the-art test prioritization techniques for APR, and combining them can further improve APR efficiency; and (5) traditional Regression Test Prioritization (RTP) widely studied in regression testing performs even better than APR-specific test prioritization when combined with most RTS techniques. Furthermore, we also present the detailed impact of different patch categories and patch validation strategies on our findings.

This paper describes a new heuristic search optimisation algorithm capable of automatically generating programs in any language as solutions to a problem using an arbitrary BNF-based grammar. The approach maintains two populations of agents: the first, a set of partially generated programs that are built as a result of the agents traversing in parallel the entire search space of possible programs as determined by the grammar; and the second, a set of completely generated programs that are tested to see how they perform in the problem. Both populations are updated during each iteration by using a fitness function to prune out poorly performing agents. The effectiveness of the algorithm is evaluated on variations of the Santa Fe Trail problem. Experimental results show that the algorithm is capable of finding the optimal solution of 165 steps (i.e. the path itself as described by the three move, left turn and right turn operators without any conditional operator) whereas the best solutions found by Genetic Programming and Grammatical Evolution typically involve several hundred more steps. When using a grammar that omits the conditional operator, the algorithm again finds the optimal solution, unlike Grammatical Evolution which finds no solution at all.

Fault localization (FL) and automated program repair (APR) are two main tasks of automatic software debugging. Compared with traditional methods, deep learning-based approaches have been demonstrated to achieve better performance in FL and APR tasks. However, the existing deep learning-based FL methods ignore the deep semantic features or only consider simple code representations. And for APR tasks, existing template-based APR methods are weak in selecting the correct fix templates for more effective program repair, which are also not able to synthesize patches via the embedded end-to-end code modification knowledge obtained by training models on large-scale bug-fix code pairs. Moreover, in most of FL and APR methods, the model designs and training phases are performed separately, leading to ineffective sharing of updated parameters and extracted knowledge during the training process. This limitation hinders the further improvement in the performance of FL and APR tasks. To solve the above problems, we propose a novel approach called MTL-TRANSFER, which leverages a multi-task learning strategy to extract deep semantic features and transferred knowledge from different perspectives. First, we construct a large-scale open-source bug datasets and implement 11 multi-task learning models for bug detection and patch generation sub-tasks on 11 commonly used bug types, as well as one multi-classifier to learn the relevant semantics for the subsequent fix template selection task. Second, an MLP-based ranking model is leveraged to fuse spectrum-based, mutation-based and semantic-based features to generate a sorted list of suspicious statements. Third, we combine the patches generated by the neural patch generation sub-task from the multi-task learning strategy with the optimized fix template selecting order gained from the multi-classifier mentioned above. Finally, the more accurate FL results, the optimized fix template selecting order, and the expanded patch candidates are combined together to further enhance the overall performance of APR tasks. Our extensive experiments on widely-used benchmark Defects4J show that MTL-TRANSFER outperforms all baselines in FL and APR tasks, proving the effectiveness of our approach. Compared with our previously proposed FL method TRANSFER-FL (which is also the state-of-the-art statement-level FL method), MTL-TRANSFER increases the faults hit by 8/11/12 on Top-1/3/5 metrics (92/159/183 in total). And on APR tasks, the number of successfully repaired bugs of MTL-TRANSFER under the perfect localization setting reaches 75, which is 8 more than our previous APR method TRANSFER-PR. Furthermore, another experiment to simulate the actual repair scenarios shows that MTL-TRANSFER can successfully repair 15 and 9 more bugs (56 in total) compared with TBar and TRANSFER, which demonstrates the effectiveness of the combination of our optimized FL and APR components.

Bankruptcy prediction with neural logic networks by means of grammar-guided genetic programming