Genetic learning for construction and parameter identification of local model networks

Local Model Networks (LMNs) provide a global representation of a nonlinear dynamical system by interpolating between a set of locally valid sub-models distributed across the operating range. Training such networks typically involves heuristic selection of the number of sub-models and their structure followed by the combined estimation of the free sub-model and interpolation function parameters. This paper describes a new genetic learning approach to the construction of LMNs comprising ARX local models and normalised Gaussian interpolation functions. In addition to allowing the simultaneous optimisation of the number of sub-models, model parameters arid interpolation function parameters, the approach provides a flexible framework for targeting transparency and generalisation. Fuzzy logic is used with special features to provide a directional and dynamic search for the genetic algorithm. Several modifications of the classical genetic algorithm are adopted to optimise each local model separately within the overall global model. A linear direct feedback control scheme is derived from the LMN representation of the nonlinear plant and local stability analysis is discussed. Simulation studies on a pH neutralisation process confirm the excellent nonlinear modelling properties of LM networks and illustrate the potential of the proposed control scheme.

Local model (LM) networks are applied to the identification of the global nonlinear dynamics of a turbogenerator excitation loop. A hybrid algorithm is used in conjunction with prior plant information to optimise the learning process. The resulting model was then used to devise a nonlinear generalised minimum variance (GMV) controller. This controller was found to outperform linear GMV controllers tuned at each generator operating point.

In this paper, advanced concepts for the identification of complex nonlinear systems are discussed. Three major problems are addressed: The nonlinearity of the system, noise in the data upon which the model has to be built, and the potential to incorporate qualitative and quantitative prior knowledge about the system. As an integrated solution approach, local model networks (LMNs) with appropriate parameter estimation schemes are proposed. LMNs generally offer a versatile structure for the identification of nonlinear dynamic systems. In order to account for a realistic situation when noise is present both in input and output data, an equality constrained generalised total least squares algorithm for the local model parameter estimation of the LMN is presented; the incorporation of equality constraints allows to mathematically enforce desired system properties. As an application and benchmark problem, the vertical dynamics of a vehicle is considered. After training the LMN on a rough road, excellent predictions of the behaviour of the vehicle at crossing a single obstacle are obtained, thus proving the effectiveness of the proposed algorithm. It is illustrated how both the application of a proper parameter estimation scheme and the integration of system constraints systematically improve the performance of the model.

Both first-principles-based and data-driven methods provide powerful tools for modeling physical systems. As such, there is a strong push for harnessing both approaches by combining them in the form of gray-box models. Automatic differentiation of ordinary differential equation (ODE) models and physical model parts enables flexible gray-box modeling in system identification tasks. Local model networks (LMNs) with their excellent interpretability and sparing use of model parameters are an ideal model class to provide data-driven elements in a gray-box model. Unfortunately, LMNs’ intricate training procedures currently do not allow for end-to-end training within larger model architectures. To remedy this, we propose a new LMN training algorithm and corresponding model called rectified linear unit model tree (ReLUMoT). ReLUMoT provides fully gradient-based training of LMNs, which permits us to train within gray-box ODE models. In essence, ReLUMoT pioneers the use of piecewise linear neural networks (PLNNs) as a training engine for LMNs. This is accomplished by distilling the locally linear structure of a PLNN into an LMN. We thoroughly review all theoretical concepts needed to train gray-box ODE models, ranging from state space modeling to initializing latent states during the training procedure. In particular, we give a critical review of sensitivity analysis methods, discussing their suitability for system identification tasks. We provide extensive experimental validation of ReLUMoT’s usefulness in practical modeling tasks. ReLUMoT is benchmarked against a variety of state-of-the-art modeling algorithms in learning both static and dynamic processes. In black-box settings, ReLUMoT matches or exceeds the performance of existing LMNs. Properly incorporating prior knowledge is shown to lead to vast improvements in model performance and size.

An extended self-organizing map (ESOM) network, which consists of a self-organization phase and an optimization phase, was recently developed to construct a local model network (LMN) automatically using the plant data. However, this previous result suffers two drawbacks: (1) increased computation time in the self-organization phase as the number of local models increases, (2) lack of checking stability conditions for both local models and LMN. To overcome these problems, an improved algorithm for the ESOM network is developed in this paper by employing a competitive learning algorithm for data clustering in the self-organization phase and parametric constraints are formulated in the optimization phase to handle the stability of local models. In addition, the global stability of LMN is addressed. With LMN constructed by the ESOM network, it serves as a basis for building a nonlinear controller that combines several local controllers through the weighting functions obtained by the ESOM algorithm. Literature examples are used to illustrate the proposed ESOM-based modeling and controller design method.

- This paper introduces the Particle Swarm Algorithm (PSA)-based Local Model network (LMN) for modeling and controlling dynamic systems. Structurally, the proposed PSA-LMN merges the fuzzy set theory and wavelets in a unified form. Learning this network comprises two phases, structure learning phase and parameters learning phase. The former is performed using the Adaptive Resonance Theory (ART) algorithm while the latter is performed using the PSA. The PSA is employed to optimize parameters of the fuzzy sets, the wavelets and the free weights of the proposed LMN. Two simulation nonlinear plants are used to test the soundness of the proposed network; one is a single input single output nonlinear plant and the other is multi-variable medical plant. The latter is employed to test the proposed network in control purposes compared with Genetic Algorithm (GA)-based LMN. Better results were obtained using the proposed PSA-based LMN.

This paper presents a model-based optimization approach to determine operation mode of a set of fans in the cooling tower (CT) by considering turbine generator and CT as an integrated system. The optimization problem is formulated as to maximize the net power output, which is the result of subtracting power consumption of fans in CT from power generated by turbine generator. The operations of both units are characterized by a data-driven method, local model network, which presents a set of local models in the formats of linear equations. Stepwise variable selection is employed to identify key variables as a way of reducing the number of variables and avoiding overfitting. Satisfactory fuzzy -mean cluster is used to categorize operation data into several groups to build local models. The optimal problem is then solved by linear programming algorithm. A case study conducted in a commercial plant demonstrates that the proposed approach can increase significant net power output.

04/01864 Intelligent decoupling control of power plant main steam pressure and power output: Liu, H. et al. International Journal of Electrical Power & Energy Systems, 2003, 25, (10), 809–819

A Local Model Networks Based Multivariable Long-Range Predictive Control Strategy for Thermal Power Plants

In this paper, a new approach to evolving fuzzy model identification from streaming data is given. The structure of the model is given as a local model network in Takagi–Sugeno form, and the partitioning of the input–output space is based on metrics in which these local models are defined as prototypes of the clusters. This means that the clusters and the local models share the same parameters; therefore, the number of parameters of the evolving system is much lower in comparison to similar systems of comparable complexity, and the problems of parameter identifiability are not a particular issue. The algorithm adds the local models in an incremental fashion and recursively adapts the local model parameters. The proposed algorithm is tested on three examples to demonstrate the main features. The first example is a simple simulated example with intersecting clusters; the second is a very well-known benchmark that treats the Mackey–Glass time series; the third is an example that shows the classification of the data from a laser rangefinder. These examples show the great potential of the proposed approach in certain applications.

Order Determination and Input Selection with Local Model Networks

Two-motor variable frequency speed-regulating synchronous system is a multivariable nonlinear coupling system. In order to estimate speed and tension of this system more effectively, a method of modeling based on local model networks (LMN) is given. Through using correlative testing signal to drive two-motor synchronous system firstly, series of output response about speed and tension are obtained.Then fuzzy C-means clustering algorithm is used to cluster the data of input-output according to the satisfactory performance index. Finally ,the sum of linear local model identification model for speed and tension is built . The simulation results are presented to demonstrate the validity of the proposed the method.

Just-in-time software defect prediction (JIT-SDP) is an active topic in software defect prediction, which aims to identify defect-inducing changes. Recently, some studies have found that the variability of defect data sets can affect the performance of defect predictors. By using local models, it can help improve the performance of prediction models. However, previous studies have focused on module-level defect prediction. Whether local models are still valid in the context of JIT-SDP is an important issue. To this end, we compare the performance of local and global models through a large-scale empirical study based on six open-source projects with 227417 changes. The experiment considers three evaluation scenarios of cross-validation, cross-project-validation, and timewise-cross-validation. To build local models, the experiment uses the k-medoids to divide the training set into several homogeneous regions. In addition, logistic regression and effort-aware linear regression (EALR) are used to build classification models and effort-aware prediction models, respectively. The empirical results show that local models perform worse than global models in the classification performance. However, local models have significantly better effort-aware prediction performance than global models in the cross-validation and cross-project-validation scenarios. Particularly, when the number of clusters k is set to 2, local models can obtain optimal effort-aware prediction performance. Therefore, local models are promising for effort-aware JIT-SDP.

Adaptive nonlinear system identification of separation unit for raw materials

Radial basis function networks and local model networks have been successfully used for system modelling and prediction. One issue concerning these techniques, not yet satisfactorily solved, is the location of the basis functions in input space. In the paper, a novel modular architecture, which combines a growing neural gas network with local model networks, is proposed. In the construction of the local model networks, the topological neighborhood defined among the growing neural gas network units, is employed. The proposed method was used to make short term predictions of different computer generated time series in the presence of noise. The experimental results have shown that the proposed neural network can be effectively used for chaotic time series prediction achieving performances at least comparable to state-of-the-art methods.