Automated multi-objective system identification using grammar-based genetic programming

Bilevel optimization problems require every feasible upper-level solution to satisfy optimality of a lower-level optimization problem. These problems commonly appear in many practical problem solving tasks including optimal control, process optimization, game-playing strategy development, transportation problems, and others. In the context of a bilevel single objective problem, there exists a number of theoretical, numerical, and evolutionary optimization results. However, there does not exist too many studies in the context of having multiple objectives in each level of a bilevel optimization problem. In this paper, we address bilevel multi-objective optimization issues and propose a viable algorithm based on evolutionary multi-objective optimization (EMO) principles. Proof-of-principle simulation results bring out the challenges in solving such problems and demonstrate the viability of the proposed EMO technique for solving such problems. This paper scratches the surface of EMO-based solution methodologies for bilevel multi-objective optimization problems and should motivate other EMO researchers to engage more into this important optimization task of practical importance.

The bilevel optimisation problem (BLP) is a subclass of optimisation problems in which one of the constraints of an optimisation problem is another optimisation problem. BLP is widely used to model hierarchical decision making where the leader and the follower correspond to the upper level and lower level optimisation problem, respectively. In BLP, the optimal solutions to the lower level optimisation problem are the feasible solutions to the upper level problem, which makes it particularly difficult to solve. This paper proposes a novel hardware architecture known as Recursive Pipelined Genetic Propagation (RPGP), to solve BLP efficiently on FPGA. RPGP features a graph of genetic operation nodes which can be scaled to exploit hardware resources. In addition, the topology of the RPGP graph can be changed at run-time to escape from local optima. We evaluate the proposed architecture on an Altera Stratix-V FPGA, using a benchmark bilevel optimisation problem set. Our experimental results show that RPGP can achieve a significant speed-up against previous work.

Grammar-Based Genetic Programming (GBGP) improves the search performance of Genetic Programming (GP) by formalizing constraints and domain specific knowledge in grammar. The building blocks (i.e. the functions and the terminals) in a program can be dependent. Random crossover and mutation destroy the dependence with a high probability, hence breeding a poor program from good programs. Understanding on the syntactic and semantic in the grammar plays an important role to boost the efficiency of GP by reducing the number of poor breeding. Therefore, approaches have been proposed by introducing context sensitive ingredients encoded in probabilistic models. In this paper, we propose Grammar-Based Genetic Programming with Bayesian Network (BGBGP) which learns the dependence by attaching a Bayesian network to each derivation rule and demonstrates its effectiveness in two benchmark problems.

Robust Nonlinear Model Predictive Control with Constraint Satisfaction: A Relaxation-based Approach

Nonlinear model identification of an experimental ball-and-tube system using a genetic programming approach

A distributionally robust optimization model for batch nonlinear switched time-delay system considering uncertain output measurements

Feature selection and construction are important pre-processing techniques in data mining. They allow not only dimensionality reduction but also classification accuracy and efficiency improvement. While feature selection consists in selecting a subset of relevant features from the original feature set, feature construction corresponds to the generation of new high-level features, called constructed features, where each one of them is a combination of a subset of original features. However, different features can have different abilities to distinguish different classes. Therefore, it may be more difficult to construct a better discriminating feature when combining features that are relevant to different classes. Based on these definitions, feature construction could be seen as a BLOP (Bi-Level optimization Problem) where the feature subset should be defined in the upper level and the feature construction is applied in the lower level by performing mutliple followers, each of which generates a set class dependent constructed features. In this paper, we propose a new bi-level evolutionary approach for feature construction called BCDFC that constructs multiple features which focuses on distinguishing one class from other classes using Genetic Programming (GP). A detailed experimental study has been conducted on six high-dimensional datasets. The statistical analysis of the obtained results shows the competitiveness and the outperformance of our bi-level feature construction approach with respect to many state-of-art algorithms.

International audience

One of the main and fundamental tasks of data mining is the automatic induction of classification rules from a set of examples and observations. A variety of methods performing this task have been proposed in the recent literature. Many comparative studies have been carried out in this field. However, the main common feature between thesemethods is that they are designedmanually. In the meanwhile, there have been some successful attempts to automatically design such methods using Grammar-based Genetic Programming (GGP). In this paper, we propose a different system called Automatic Grammar Genetic Programming (AGGP) that can evolve complete java program codes. These codes represent a rule induction algorithm that uses a grammar evolution technique that governs a Backus Naur Form grammar definition mapping to a program. To perform this task, we will use binary strings as inputs to the mapper along with the Backus Naur Form grammar. Such binary strings represent possible potential solutions resulting from the initialized component and Weka building blocks, this would ease the induction process and makes induced programs short. Experimental results prove the efficiency of the proposed method. It is also shown that, compared to some recent and similar manual techniques (Prism, Ripper, Ridor, OneRule) the proposed method outperforms such techniques.Abenchmark of well-known data sets is used for the sake of comparison.

When the lower-level optimal solution set-valued mapping of a bilevel optimization problem is not single-valued, we are faced with an ill-posed problem, which gives rise to the optimistic and pessimistic bilevel optimization problems, as tractable algorithmic frameworks. However, solving the pessimistic bilevel optimization problem is far more challenging than the optimistic one; hence, the literature has mostly been dedicated to the latter class of the problem. The Scholtes relaxation has appeared to be one of the simplest and most efficient ways to solve the optimistic bilevel optimization problem in its Karush-Kuhn-Tucker (KKT) reformulation or the corresponding more general mathematical program with complementarity constraints (MPCC). Inspired by such a success, this paper studies the potential of the Scholtes relaxation in the context of the pessimistic bilevel optimization problem. To proceed, we consider a pessimistic bilevel optimization problem, where all the functions involved are at least continuously differentiable. Then assuming that the lower-level problem is convex, the KKT reformulation of the problem is considered under the Slater constraint qualification. Based on this KKT reformulation, we introduce the corresponding version of the Scholtes relaxation algorithm. We then construct theoretical results ensuring that the limit of a sequence of global/local optimal solutions (resp. stationary points) of the aforementioned Scholtes relaxation is a global/local optimal solution (resp. stationary point) of the KKT reformulation of the pessimistic bilevel program. The results are accompanied by technical constructions ensuring that the Scholtes relaxation algorithm is well-defined or that the corresponding parametric optimization problem is more tractable. Furthermore, we perform some numerical experiments to assess the performance of the Scholtes relaxation algorithm using various examples. In particular, we study the effectiveness of the algorithm in obtaining solutions that can satisfy the corresponding C-stationarity concept.

This paper addresses a bilevel optimization problem with a guaranteed (pessimistic) solution in a rather general form. The problem is reduced to a family of bilevel problems with optimistic solutions and perturbed follower's problems. For the quadratic-linear bilevel optimization problem, a reduction of the bilevel problem to a series of non-convex one-level optimization problems is proposed using the penalty method. Additionally, a solution method based on Global Search Theory (GST) is developed for the obtained problem. The GST consists of two stages: special local search and Global Search Scheme (GSS), which is based on Global Optimality Conditions (GOCs). The computational testing of the developed approach on specially constructed bilevel problems of high dimensions was successfully carried out. Finally, the computational testing on specially constructed bilevel problems of rather high dimensions was successfully carried out, that substantiate the developed approach.

Bi-level optimisation problems (BOPs) are a class of challenging problems with two levels of optimisation tasks. These problems allow to model a large number of real-life situations in which a first decision maker, hereafter the leader, optimises his objective by taking the follower's response to his decisions explicitly into account. In this context, a new proposed algorithm called CODBA-II was suggested to solve combinatorial BOPs. The latter was able to improve the quality of generated bi-level solutions regarding to recently proposed methods. In fact, a wide range of applications fit the bi-level programming framework and real-life implementations still scarce. For this reason, we propose in this paper a co-evolutionary decomposition-based bi-level algorithm for the bi-level knapsack optimisation problem. The computational algorithm turned out to be quite efficient on both computation time and solution quality regarding to other competitive EAs.

System identification in the presence of trends and outliers using sparse optimization

Research in the rule induction algorithm field produced many algorithms in the last 30 years. However, these algorithms are usually obtained from a few basic rule induction algorithms that have been often changed to produce better ones. Having these basic algorithms and their components in mind, this work proposes the use of Grammar-based Genetic Programming (GGP) to automatically evolve rule induction algorithms. The proposed GGP is evaluated in extensive computational experiments involving 11 data sets. Overall, the results show that effective rule induction algorithms can be automatically generated using GGP. The automatically evolved rule induction algorithms were shown to be competitive with well-known manually designed ones. The proposed approach of automatically evolving rule induction algorithms can be considered a pioneering one, opening a new kind of research area.

Although the nonlinear power form model structure is widely accepted by practitioners in the flood regionalization modelling, there is a lack of studies on whether there is a room for further improvement, and if the answer is yes, what should be done to explore alternative model structures. A framework is proposed in this study towards investigating this issue by the following steps: (i) a universal data‐driven model is utilized to see if there is a room for improvement compared with the conventional model, and (ii) if improvement is achieved, this means that there should exist more effective model structures than the current form. However, because the universal data‐driven models are usually opaque, more explicit model structures should be explored, which are convenient for practical usage. In this study, the proposed framework is applied in a case study using the catchment characteristics from the Flood Estimation Handbook in conjunction with the gamma test, support vector machine (SVM) and genetic programming (GP). First, the gamma test is used for the purpose of input variables selection where no model structure needs to be defined as a priori, and therefore, the result can be applied to any model structures for model building. Second, an SVM, which is a powerful data‐driven nonlinear model capable of modelling a variety of nonlinear systems, is applied to the index flood model for the first time. Once the best model is determined using those two data‐driven tools, GP is employed to find an alternative model structure. As the SVM is not formulated for producing explicit model functional form, the GP offers an advantage at this point where it can infer an explicit mathematical model functional form. Copyright © 2012 John Wiley & Sons, Ltd.