An advanced algorithm for partitioning and parameter estimation in local model networks and its application to vehicle vertical dynamics

Teaching Aids for Modeling and Control of Hybrid Systems (CAMCHS)

Both first-principles-based and data-driven methods provide powerful tools for modeling physical systems. As such, there is a strong push for harnessing both approaches by combining them in the form of gray-box models. Automatic differentiation of ordinary differential equation (ODE) models and physical model parts enables flexible gray-box modeling in system identification tasks. Local model networks (LMNs) with their excellent interpretability and sparing use of model parameters are an ideal model class to provide data-driven elements in a gray-box model. Unfortunately, LMNs’ intricate training procedures currently do not allow for end-to-end training within larger model architectures. To remedy this, we propose a new LMN training algorithm and corresponding model called rectified linear unit model tree (ReLUMoT). ReLUMoT provides fully gradient-based training of LMNs, which permits us to train within gray-box ODE models. In essence, ReLUMoT pioneers the use of piecewise linear neural networks (PLNNs) as a training engine for LMNs. This is accomplished by distilling the locally linear structure of a PLNN into an LMN. We thoroughly review all theoretical concepts needed to train gray-box ODE models, ranging from state space modeling to initializing latent states during the training procedure. In particular, we give a critical review of sensitivity analysis methods, discussing their suitability for system identification tasks. We provide extensive experimental validation of ReLUMoT’s usefulness in practical modeling tasks. ReLUMoT is benchmarked against a variety of state-of-the-art modeling algorithms in learning both static and dynamic processes. In black-box settings, ReLUMoT matches or exceeds the performance of existing LMNs. Properly incorporating prior knowledge is shown to lead to vast improvements in model performance and size.

This article presents a theoretical framework for the identification of nonlinear dynamic MISO systems with orthonormal base function models on fundamental basics of the Volterra theory. In the past the Volterra theory was used for the identification of nonlinear dynamic SISO systems (e.g. Hammerstein and Wiener models). In principle it is possible to extend these approaches to systems with more than one input (i.e. MISO systems). In former times this failed due to a lack of computational performance. In this paper an approach is presented which allows the identification of arbitrary coupled Hammerstein and Wiener models with multiple inputs. Some fundamental considerations for the identification of MISO systems based on arbitrary coupled Hammerstein models are made and extended to MISO systems based on arbitrary coupled Wiener models. This extended Volterra theory results in equations where the unknown parameters can be separated of the input values in a linear manner. In order to approximate the truncated impulse responses of the linear dynamic systems and to reduce the number of unknown parameters orthonormal base functions (OBFs) are introduced. As an example the proposed identification method is applied to a MISO system based on forward and backward coupled Hammerstein and Wiener models.

This paper compares radial basis function networks for identification of nonlinear dynamic systems with classical methods derived from the Volterra series. The performance of these different approaches, such as Hammerstein, Wiener and NDE models, is analysed. Since the centres and variances of the Gaussian radial basis functions will be fixed before learning and only the weights are learned, a linear optimization problem arises. Therefore training the network and parameter estimation becomes comparable in computational effort. It is shown that the classical methods can compete or even perform better than the neural network, if the assumptions for the structure are valid. However, in practical applications when the structure is not known the radial basis function network performs much better than the classical methods.

In this chapter we deal with learning (optimization) methods used in modeling and identification of complex nonlinear systems. We treat methods that enable estimation of model parameters and, possibly, identification of the structure of the models from the measured data (samples).

A method for identifying nonlinear dynamic systems with noise is proposed by using probabilistic universal learning networks (PrULNs). PrULNs are extensions of universal learning networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. But the ULN does not provide any stochastic characteristics of the signals propagating through it. The PrULNs are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. On the other hand it is generally recognized that there exists an overfitting problem when identification of nonlinear dynamic systems with noise is done by neural networks. In this paper, it is shown from simulation results of identification of a nonlinear robot dynamics that PrULNs are useful for avoiding the overfitting.

Discovering nonlinear PDEs with sparse identification of nonlinear dynamical systems (SINDy) is hindered by high dimensionality, noise, and expensive data acquisition. We propose the greedy sampling neural network for sparse identification of nonlinear PDEs (GN-SINDy), a three–stage framework that integrates strategic sampling, differentiable surrogate modelling, and sparse equation discovery. First, a two–way Q-DEIM–based greedy strategy selects maximally informative space–time samples from snapshot data, drastically reducing data requirements. Second, a deep neural network (DNN) is trained as a differentiable surrogate of the solution field, enabling noise–robust analytic derivatives via automatic differentiation. Third, sparse regression with sparsity–promoting estimators [Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016a). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932–3937; Rudy, S. H., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2017). Data-driven discovery of partial differential equations. Science Advances, 3(4), e1602614.] is applied to recover the governing PDE. Building on the DeepMoD paradigm, GN-SINDy embeds greedy sampling into data acquisition and stabilises coefficient estimation through neural–enhanced differentiation. We analyze noise robustness, structural stability, and QR–based sampling strategies to guide sampler and hyperparameter selection. Experiments on Burgers', Allen–Cahn, and Korteweg–de Vries equations show that GN-SINDy reliably recovers governing PDEs using under 1 % of the data, outperforming DeepMoD in efficiency, support recovery, and robustness to noise.

A procedure based on the use of artificial neural networks for the identification of nonlinear dynamic systems is developed and applied to the damped Duffing oscillator under deterministic excitation. The “generalization” ability of neural networks is invoked to predict the response of the same nonlinear oscillator under stochastic excitations of differing magnitude. The analogy between the neural network approach and a qualitatively similar nonparametric identification technique previously developed by the authors is illustrated. Some of the computational aspects of identification by neural networks, as well as their fault-tolerant nature, are discussed. It is shown that neural networks provide high-fidelity mathematical models of structure-unknown nonlinear systems encountered in the applied mechanics field.

Nonlinear dynamic systems are characterized with uncertainties in terms of structure and parameters. These uncertainties cannot be described by deterministic models. The modelling and identification of nonlinear dynamic systems through the measured experimental data is a problem in engineering and technical processes. Therefore, field of system identification have become an important area of research. Fuzzy technology is an effective tool for dealing with complex nonlinear processes that are characterized with uncertain factors. In this paper, a novel approach based on Local Linear method learning in dynamical filter weights neurons for the identification of non-linear dynamic systems is presented. The fuzzy wavelet neural network combines wavelet theory with fuzzy logic and neural networks. Learning fuzzy rules and parameter update in fuzzy wavelet neural network is based on gradient decent method. The proposed approach is said to be Fuzzy Local Linear Wavelet Neural Network based model. It has been explained through examples. The structure is tested for the identification with both wavelet neural network and Fuzzy Local Linear Wavelet Neural Network that shows the comparative performance.KeywordsSystem IdentificationNon-linear SystemWavelet Neural NetworkFuzzy Wavelet Neural NetworkLocal Linear Wavelet Neural NetworkFuzzy Local Linear Wavelet Neural Network

Adaptive nonlinear system identification of separation unit for raw materials

Building the mathematical models to study the dynamics of prey-predator relationship in ecology is a long standing research topic. The only connection that has been established between real data and ecological models is through parameter estimation. Till date, only a few studies have attempted to formulate ecological models directly from real data without prior knowledge of the underlying structure of the system being modeled. Present work is an attempt in this direction. The ecological study of the moose (Alces alces) and wolf (Canis lupus) populations of Isle Royale National Park (USA) is the longest running large mammal prey-predator study in the world. In this study, we formulate mathematical models of moose and wolf dynamics from yearly reported population data from 1959 to 2019 using sparse identification of nonlinear dynamical systems (SINDy) algorithm. We have used SINDy and Ensemble SINDy (E-SINDy) algorithms with model selection techniques on the park data. We have performed global uncertainty and sensitivity analysis of the best obtained model. The findings suggest that the mathematical models obtained using E-SINDy method is a close representation of the moose-wolf relationship in Isle Royale.

This article addresses parameter convergence problem in identification of nonlinear dynamic systems using fuzzy models. We first establish persistent excitation conditions and then propose several detailed algorithms to generate input signals that guarantee the convergence of the parameter estimates in the fuzzy system models to the true values in identifications of second-order nonlinear moving-average and auto-regressive-moving-average systems. Numerical example is given to illustrate the ideas and results.

Detection and classification of non-linearities based on Volterra kernels processing

Structure Identification of Continuous-Time Block-Oriented Nonlinear Systems in the Frequency Domain

A new anomaly detection scheme based on growing structure multiple model system (GSMMS) is proposed in this paper to detect and quantify the effects of anomalies. The GSMMS algorithm combines the advantages of growing self-organizing networks with efficient local model parameter estimation into an integrated framework for modeling and identification of general nonlinear dynamic systems. The identified model then serves as a foundation for building an effective anomaly detection and fault diagnosis system. By utilizing the information about system operation region provided by the GSMMS, the residual errors can be analyzed locally within each operation region. This local decision making scheme can accommodate for unequally distributed residual errors across different operational regions. The performance of the newly proposed method is evaluated through anomaly detection and quantification in an electronically controlled throttle system, which is simulated using a high-fidelity engine simulation software package provided by a major automotive manufacturer for control system development.