Identification and prediction of nonlinear systems using orthonormal functions

This article presents a theoretical framework for the identification of nonlinear dynamic MISO systems with orthonormal base function models on fundamental basics of the Volterra theory. In the past the Volterra theory was used for the identification of nonlinear dynamic SISO systems (e.g. Hammerstein and Wiener models). In principle it is possible to extend these approaches to systems with more than one input (i.e. MISO systems). In former times this failed due to a lack of computational performance. In this paper an approach is presented which allows the identification of arbitrary coupled Hammerstein and Wiener models with multiple inputs. Some fundamental considerations for the identification of MISO systems based on arbitrary coupled Hammerstein models are made and extended to MISO systems based on arbitrary coupled Wiener models. This extended Volterra theory results in equations where the unknown parameters can be separated of the input values in a linear manner. In order to approximate the truncated impulse responses of the linear dynamic systems and to reduce the number of unknown parameters orthonormal base functions (OBFs) are introduced. As an example the proposed identification method is applied to a MISO system based on forward and backward coupled Hammerstein and Wiener models.

Teaching Aids for Modeling and Control of Hybrid Systems (CAMCHS)

The theory and design of linear adaptive filters based on FIR filter structures is well developed and widely applied in practice. However, the same is not true for more general classes of adaptive systems such as linear infinite impulse response adaptive filters (MR) and nonlinear adaptive systems. This situation results because both linear IIR structures and nonlinear structures tend to produce multi-modal error surfaces for which stochastic gradient optimization strategies may fail to reach the global minimum. After briefly discussing the state of the art in linear adaptive filtering, the attention of this paper is turned to MR and nonlinear adaptive systems for potential use in echo cancellation, channel equalization, acoustic channel modeling, nonlinear prediction, and nonlinear system identification. Structured stochastic optimization algorithms that are effective on multimodal error surfaces are then introduced, with particular attention to the particle swarm optimization (PSO) technique. The PSO algorithm is demonstrated on some representative IIR and nonlinear filter structures, and both performance and computational complexity are analyzed for these types of nonlinear systems.

This paper presents how to apply a newly developed general theory for the global transversality and tangency of flows in n-dimensional nonlinear dynamical systems to a 2-D nonlinear dynamical system (i.e. a periodically forced, damped Duffing oscillator). The global tangency and transversality of the periodic and chaotic motions to the separatrix for such a nonlinear system are discussed to help us understand the complexity of chaos in nonlinear dynamical systems. This paper presents the concept that the global transversality and tangency to the separatrix are independent of the Melnikov function (or the energy increment). Chaos in nonlinear dynamical systems makes the exact energy increment quantity to be chaotic no matter if the nonlinear dynamical systems have separatrices or not. The simple zero of the Melnikov function cannot be used to simply determine the existence of chaos in nonlinear dynamical systems. Through this paper, the expectation is that, from now on, one can use the alternative aspect to look into the complexity of chaos in nonlinear dynamical systems. Therefore, in this paper, the analytical conditions for global transversality and tangency of 2-D nonlinear dynamical systems are presented. The first integral quantity increment (i.e. the energy increment) for a certain time interval is achieved for periodic flows and chaos in the 2-D nonlinear dynamical systems. Under the perturbation assumptions and convergent conditions, the Melnikov function is recovered from the first integral quantity increment. A periodically forced, damped Duffing oscillator with a separatrix is investigated as a sampled problem. The corresponding analytical conditions for the global transversality and tangency to the separatrix are obtained and verified by numerical simulations. The switching planes and the corresponding local and global mappings are defined on the separatrix. The mapping structures are developed for local and global periodic flows passing through the separatrix. The mapping structures of global chaos in the damped Duffing oscillator are also discussed. Bifurcation scenarios of the damped Duffing oscillator are presented through the traditional Poincaré mapping section and the switching planes. The first integral quantity increment (i.e. L-function) is presented to observe the periodicity of flows. In addition, the global tangency of periodic flows in such an oscillator is measured by the G-function and G(1)-function, and is verified by numerical simulations. The first integral quantity increment of periodic flows is zero for their complete periodic cycles. Numerical simulations of chaos in such a Duffing oscillator are carried out through the Poincaré mapping sections. The conservative energy distribution, G-function and L-function along the displacement of Poincaré mapping points are presented to observe the complexity of chaos. The first integral quantity increment (i.e. L-function) of chaotic flows at the Poincaré mapping points is nonzero and chaotic. The switching planes of chaos are presented on the separatrix for a better understanding of the global transversality to the separatrix. The switching point distribution on the separatrix is presented and the switching G-function on the separatrix is given to show the global transversality of chaos on the separatrix. The analytical conditions are obtained from the new theory rather than the Melnikov method. The new conditions for the global transversality and tangency are more accurate and independent of the small parameters.

In this thesis, we present a dynamical systems approach to adaptive controllers for locomotion control. The approach is based on a rigorous mathematical framework using nonlinear dynamical systems and is inspired by theories of self-organization. Nonlinear dynamical systems such as coupled oscillators are an interesting approach for the on-line generation of trajectories for robots with many degrees of freedom (e.g. legged locomotion). However, designing a nonlinear dynamical system to satisfy a given specification and goal is not an easy task, and, hitherto no methodology exists to approach this problem in a unified way. Nature presents us with satisfactory solutions for the coordination of many degrees of freedom. One central feature observed in biological subjects is the ability of the neural systems to exploit natural dynamics of the body to achieve efficient locomotion. In order to be able to exploit the body properties, adaptive mechanisms must be at work. Recent work has pointed out the importance of the mechanical system for efficient locomotion. Even more interestingly, such well suited mechanical systems do not need complicated control. Yet, in robotics, in most approaches, adaptive mechanisms are either missing or they are not based on a rigorous framework, i.e. they are based on heuristics and ad-hoc approaches. Over the last three decades there has been enormous progress in describing movement coordination with the help of Synergetic approaches. This has led to the formulation of a theoretical framework: the theory of dynamic patterns. This framework is mathematically rigorous and at the same time fully operational. However, it does not provide any guidelines for synthetic approaches as needed for the engineering of robots with many degrees of freedom, nor does it directly help to explain adaptive systems. We will show how we can extend the theoretical framework to build adaptive systems. For this purpose, we propose the use of multi-scale dynamical systems. The basic idea behind multi-scale dynamical systems is that a given dynamical system gets extended by additional slow dynamics of its parameters, i.e. some of the parameters become state variables. The advantages of the framework of multi-scale dynamical systems for adaptive controllers are 1) fully dynamic description, 2) no separation of learning algorithm and learning substrate, 3) no separation of learning trials or time windows, 4) mathematically rigorous, 5) low dimensional systems. However, in order to fully exploit the framework important questions have to be solved. Most importantly, methodologies for designing the feedback loops have to be found and important theoretical questions about stability and convergence properties of the devised systems have to be answered. In order to tackle this challenge, we first introduce an engineering view on designing nonlinear dynamical systems and especially oscillators. We will highlight the important differences and freedom that this engineering view introduces as opposed to a modeling one. We then apply this approach by first proposing a very simple adaptive toy-system, consisting of a dynamical system coupled to a spring-mass system. Due to its spring-mass dynamics, this system contains clear natural dynamics in the form of resonant frequencies. We propose a prototype adaptive multi-scale system, the adaptive frequency oscillator, which is able to adapt its intrinsic frequency to the resonant frequency of the body dynamics. After a small sidetrack to show that we can use adaptive frequency oscillators also for other applications than for adaptive controllers, namely for frequency analysis, we then come back to further investigation of the adaptive controller. We apply the same controller concept to a simple spring-mass hopper system. The spring-mass system consists of a body with two legs attached by rotational joints. The legs contain spring-damper elements. Finally, we present results of the implementation of the controller on a real robot, the experimental robot PUPPY II. This robot is a under-actuated robot with spring dynamics in the knee joints. It will be shown, that due to the appropriate simplification and concentration on relevant features in the toy-system the controller concepts works without a fundamental change on all systems from the toy system up to the real robot.

In this study, nonlinear dynamic systems are identified by using artificial bee colony (ABC) algorithm and adaptive neuro fuzzy inference system (ANFIS). ABC algorithm is used in training and updating of ANFIS. The most appropriate model is formed by optimizing the antecedent and conclusion parameters that are found in the structure of ANFIS. The dynamic systems that consist of one input and one output (SISO) are used for the identification of nonlinear dynamic systems. The obtained results are compared with fuzzy neural network, neural network and ANFIS-based methods such as RSONFIN, DFNN, RSEFNN-LF, WRFNN and RFNN. The simulation results show that the proposed method is successful in the identification of considered nonlinear dynamic systems.

In many cases, the equations that dynamical systems are based on are unknown and hard to model and predict. On the other hand, machine learning algorithms are based on the data of a solution as it evolves and do not need equations. In the era of abundant data, using machine learning technology to discover accurate mathematical models of dynamical systems directly from time series data becomes increasingly important. Recently, a multi-step deep neural networks (multi-step DNN) model without need of direct access to temporal gradients is proposed, which can accurately learn the evolution from a given set of observed data, identify nonlinear dynamical systems, and forecast future states. However, the architecture lacks the capability to capture long term temporal dependencies from dynamical time-series data. In the paper, based on the multi-step time-stepping schemes, we proposed a new CLDNN model which combine convolutional layer, long short-term memory layer and fully connected layer, to address the aforementioned weakness. The effectiveness of our model is tested for several benchmark problems involving the identification and prediction of complex, nonlinear and chaotic dynamics. The experiment results show that the multi-step CLDNN has better identification and prediction performance than the multi-step DNN. The research provides possible corroboration for developing new deep learning based algorithms for nonlinear system identification.

Nonlinear dynamic systems are characterized with uncertainties in terms of structure and parameters. These uncertainties cannot be described by deterministic models. The modelling and identification of nonlinear dynamic systems through the measured experimental data is a problem in engineering and technical processes. Therefore, field of system identification have become an important area of research. Fuzzy technology is an effective tool for dealing with complex nonlinear processes that are characterized with uncertain factors. In this paper, a novel approach based on Local Linear method learning in dynamical filter weights neurons for the identification of non-linear dynamic systems is presented. The fuzzy wavelet neural network combines wavelet theory with fuzzy logic and neural networks. Learning fuzzy rules and parameter update in fuzzy wavelet neural network is based on gradient decent method. The proposed approach is said to be Fuzzy Local Linear Wavelet Neural Network based model. It has been explained through examples. The structure is tested for the identification with both wavelet neural network and Fuzzy Local Linear Wavelet Neural Network that shows the comparative performance.KeywordsSystem IdentificationNon-linear SystemWavelet Neural NetworkFuzzy Wavelet Neural NetworkLocal Linear Wavelet Neural NetworkFuzzy Local Linear Wavelet Neural Network

In this paper, nonlinear dynamic system identification by using multiobjectively selected RBF network is considered. RBF networks are widely used as a model structure for nonlinear systems. The determination of its structure that is the number of basis functions is prior important step in system identification, and the tradeoff between model complexity and accuracy exists in this problem. By using multiobjective evolutionary algorithms, the candidates of the RBF network structure are obtained in the sense of Pareto optimality. We discuss an application to system identification by using such RBF networks having Pareto optimal structures. Some numerical simulations for nonlinear dynamic systems are carried out to show the applicability of the proposed approach.

International audience

Identification of continuous-time nonlinear systems: The nonlinear difference equation with moving average noise (NDEMA) framework

A structure detection test which distinguishes between linear and nonlinear dynamic effects in the system response and model validity checks which indicate deficiencies in estimated nonlinear models are derived.

A method for identifying nonlinear dynamic systems with noise is proposed by using probabilistic universal learning networks (PrULNs). PrULNs are extensions of universal learning networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. But the ULN does not provide any stochastic characteristics of the signals propagating through it. The PrULNs are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. On the other hand it is generally recognized that there exists an overfitting problem when identification of nonlinear dynamic systems with noise is done by neural networks. In this paper, it is shown from simulation results of identification of a nonlinear robot dynamics that PrULNs are useful for avoiding the overfitting.

Identification of nonlinear systems is one of the important problems in engineering. Chaotic systems are among those nonlinear systems that have been in the center of attention of many researchers because of their complex and unpredictable behaviors. This paper proposes an algorithm for optimally estimating the parameters of system by minimizing the mean of squared errors (MSE) index. In this paper, Teaching Learning Based Optimization (TLBO) algorithm is used for solving both offline and online parameter estimation problems for chaotic systems. The validity of this algorithm in terms of convergence speed and parameter accuracy in comparison to other popular optimization algorithms such as Standard PSO (PSO), Differential Evolution (DE), Adaptive Particle Swarm Optimization (APSO) and Genetic Algorithm (GA) is shown through an illustrative example for the modeling of chaotic systems. Furthermore, in order to demonstrate the feasibility of this algorithm, it is applied to the problem of parameter identification of a well-known nonlinear Lorenz chaotic system. According to simulation results, the proposed algorithm is a very suitable algorithm for online parameter identification for a class of nonlinear chaotic systems.

The efficacy of the multimodel framework (MMF) in modeling and identification of complex, nonlinear, and uncertain systems has been widely recognized in the literature owing to its simplicity, transparency, and mathematical tractability, allowing the use of well-known modeling analysis and control design techniques. The approach proved to be effective in addressing some of the shortcomings of other modeling techniques such as those based on a single nonlinear autoregressive network with exogenous inputs model or neural networks. A great number of researchers have contributed to this active field. Due to the significant amount of contributions and the lack of a recent survey, the review of recent developments in this field is vital. In this two-part paper, we attempt to provide a comprehensive coverage of the multimodel approach for modeling and identification of complex systems. The study contains a classification of different methods, the challenges encountered, as well as recent applications of MMF in various fields. Part 1 of this paper presents an overview of MMF for modeling and identification of nonlinear systems as well as the review of recent developments in the partitioning strategies employed.