Parameter Convergence of Modified Adaptive Laws with Persistent Excitation

Designing the optimal control for the practical systems is challenging due to the unknown system dynamics and unavoidable external disturbances. In this article, the $H_{\infty } $ optimal control problem is investigated for continuous-time linear systems with unknown dynamics. The existing reinforcement learning-based $H_{\infty } $ optimal control methods require persistence of excitation (PE) condition or data storage mechanism to guarantee the convergence of the algorithms. However, PE condition is hard to be monitored online and data storage mechanism requires to store huge amounts of past system data. In order to solve these problems, the initial excitation-based reinforcement learning algorithms are presented to learn the optimal control policy under an online-verifiable initial excitation condition. The properties of the initial excitation-based reinforcement learning algorithms are analyzed, which show that the presented algorithms converge to the optimum under the initial excitation condition. Numerical analysis is provided which demonstrates the correctness of the presented algorithms.

In this paper, a novel synchronous Q-learning method is proposed for solving discrete-time linear quadratic regulator (LQR) problems. To begin with, the Bellman equation corresponding to the optimal Q-function is reformulated into a consistency equation on the parameters of the optimal Q-function and the optimal controller. Then an actor-critic structure is introduced to learn the optimal Q-function and the optimal controller online in real time by using the state samples generated by the behavior policy. Particularly, the proposed synchronous Q-learning scheme simultaneously updates the Q-function approximation and the optimal controller approximation, rather than iterating between policy evaluation and policy improvement. The proposed control scheme is proved to be uniformly ultimately bounded (UUB) under appropriate learning rates, provided that certain persistence of excitation (PE) conditions are satisfied. Besides, the PE conditions can be easily met by injecting appropriate exploration noise into the behavior policy without causing any excitation noise bias. Finally, one simulation example is provided to verify the effectiveness of the proposed synchronous Q-learning method.

This work proposes a distributed composite adaptive synchronization algorithm for multiple uncertain EulerLagrange (EL) systems, where parameter convergence is achieved under a relaxed mathematical condition as compared to the state-of-the-art. Classical adaptive controllers require an analytical condition, called persistence of excitation (PE), to ensure parameter convergence, which results in better transient performance and robustness to disturbance. The PE condition is extended to Cooperative-PE (C-PE) condition for distributed adaptive controllers with cooperative estimation strategies. The PE and C-PE conditions are restrictive in nature since these conditions are not satisfied in most practical applications. Recent literature in adaptive control has relaxed the PE condition to Initial Excitation (IE), which is shown to be sufficient for parameter convergence. The IE condition is argued to be significantly milder than PE and can be satisfied in many practical setting. The proposed result further extends the IE condition to Cooperative-IE (C-IE) condition in distributed adaptive control architecture in the context of synchronizing multiple EL systems. It is established that the C-IE condition is milder than PE, IE, and C-PE conditions. Two-tier filter based estimation algorithm with strategic switching ensures parameter convergence under the C-IE condition and thereby provides exponential convergence of tracking and parameter estimation error to zero. Simulation results validate the efficacy of the proposed algorithm as compared to conventional distributed adaptive controllers in terms of superior tracking and estimation performance.

This paper focuses on an adaptive robust dynamic surface control (ARDSC) with composite adaptation laws (CAL) for a class of uncertain nonlinear systems in semi‐strict feedback form. A simple and effective controller has been obtained by introducing dynamic surface control (DSC) technique and designing novel adaptation laws. First, the ‘explosion of terms’ problem caused by backstepping method in the traditional adaptive robust control (ARC) is avoided. Meanwhile, through a new proof philosophy the asymptotical output tracking that the ARC possesses is theoretically preserved. Second, when persistent excitation (PE) condition satisfies, true parameter estimates could be acquired via designing CALs which integrate the information of estimation errors. Finally, simulation results are presented to illustrate the effectiveness of the proposed method. Copyright © 2010 John Wiley & Sons, Ltd.

Combined Cooperative Adaptive Cruise Control using Collective Initial Excitation based Distributed Parameter Estimator

SummaryIn this paper, a bipartite consensus problem is considered for a high‐order multiagent system with cooperative‐competitive interactions and unknown time‐varying disturbances. A signed graph is used to describe the interaction network associated with the multiagent system. The unknown disturbances are expressed by linearly parameterized models, and distributed adaptive laws are designed to estimate the unknown parameters in the models. For the case that there is no exogenous reference system, a fully distributed adaptive control law is proposed to ensure that all the agents reach a bipartite consensus. For the other case that there exists an exogenous reference system, another fully distributed adaptive control law is also developed to ensure that all the agents achieve bipartite consensus on the state of the exogenous system. The stability of the closed‐loop multiagent systems with the 2 proposed adaptive control laws are analyzed under an assumption that the interaction network is structurally balanced. Moreover, the convergence of the parameter estimation errors is guaranteed with a persistent excitation condition. Finally, simulation examples are provided to demonstrate the effectiveness of the proposed adaptive bipartite consensus control laws for the concerned multiagent system.

This paper addresses the problem of distributed cooperative adaptive identification for time-varying static parametric model systems with a new distributed event-triggered schedule. In the present framework, the cooperative persistent exciting (PE) condition is introduced to remove the existing assumption that the feasible solution of the controllers depended on the solving of line matrix inequality. By considering the communication resources constraints, a new event-triggered strategy is proposed to relax the requirement of continuous monitoring for each system. It is shown that, if the proposed assumpations are statisfied, the closed-loop adaptive systems are uniformly globally exponentially stable (UGES), the parameter estimation error converge to zero uniformly exponentially under a cooperative persistent excitation (PE) condition, and the Zeno behaviour is avoided by intoruding appropriate assumption that removes the state dependence requirement for each system.

Conditions are investigated for exponential convergence of the tracking error in feedforward adaptive systems having insufficient excitation. Particular attention is paid to the adaptive gradient algorithm with periodic excitation in the overparameterized case. A main result is that for a bounded periodic regressor, the tracking error converges exponentially without regard to parameter convergence or to the degree of overparameterization. These results weaken the persistent excitation (PE) conditions and parameter convergence conditions generally considered necessary to ensure exponential tracking convergence in this class of systems.

Many practical adaptive feedforward systems are overparametrized and, for this reason, will not satisfy persistent excitation (PE) conditions. For these systems, a weaker PE condition is proposed under which it is shown that the while parameters may not converge, the cancellation error (the error between the desired and estimated outputs) still converges exponentially. Bounds are given on the exponential rate of convergence useful for understanding the various tradeoffs and for systematic optimization and design purposes. The convergence rate is determined by properties of the confluence matrix (defined herein) that plays a role similar to that played by the autocorrelation matrix for fully PE systems. As a case study, the structure of the confluence matrix is examined in detail for adaptive systems with a tap delay line (TDL) regressor and sinusoid excitation.

This note investigates the fast tracking control problem of underactuated ships via persistent excitation (PE) conditions. By combining a novel transformation with the computed torque method, a decoupling controller related to the surge force is given first to decompose the error model into two cascade subsystems. Then, a stabilizing controller involving the yaw moment is designed. With the help of the proposed cascaded structure, a weaker PE condition than those given in past literature can be used to verify an integral detectability and guarantee global /spl kappa/-exponential convergence by employing several newly developed stability criteria. A new feature of the obtained results is that only one of these reference signals is needed to satisfy the usual PE condition. Simulation results are provided to validate the effectiveness of the proposed scheme.

The problem of parameter estimation for salient permanent magnet synchronous motors (PMSMs) is considered in this paper. It is assumed that stator currents and voltages and rotor position are measurable signals and the only known motor parameter is the number of pole pairs. The proposed estimation algorithm uses dynamic operators applied to the measurable signals to get a linear regression model with unknown motor parameters. As proven, scalar estimators that are designed for each unknown motor parameter ensure finite time convergence of estimation errors to zero. The main benefit of the proposed approach is to relax the stringent assumptions imposed on regression functions such as condition of persistent excitation (PE) and non-square integrability that are often difficult to verify in practice.

Parameter convergence is desirable in adaptive control as it brings several attractive features, including accurate online modelling, exponential tracking, and robust adaptation without parameter drift. However, a strong persistent‐excitation (PE) condition must be satisfied to guarantee parameter convergence in the conventional adaptive control. This study proposes a model reference composite learning control strategy to guarantee parameter convergence without the PE condition. In the composite learning, an integral at a moving‐time window is applied to construct a prediction error, an integral transformation is derived for avoiding the time derivation of plant states in the calculation of the prediction error, and both the tracking error and the prediction error are applied to update parametric estimates. Global exponential stability of the closed‐loop system is established under an interval‐excitation condition which is much weaker than the PE condition. Compared with a concurrent learning technique that has the same aim as this study, the proposed composite learning technique avoids the usage of singular value maximisation and fixed‐point smoothing resulting in a considerable reduction of computational cost. Numerical results have verified effectiveness and superiority of the proposed control strategy.

It is nontrivial to achieve exponential stability even for time-invariant nonlinear systems with matched uncertainties and persistent excitation (PE) condition. In this article, without the need for PE condition, we address the problem of global exponential stabilization of strict-feedback systems with mismatched uncertainties and unknown yet time-varying control gains. The resultant control, embedded with time-varying feedback gains, is capable of ensuring global exponential stability of parametric-strict-feedback systems in the absence of persistence of excitation. By using the enhanced Nussbaum function, the previous results are extended to more general nonlinear systems where the sign and magnitude of the time-varying control gain are unknown. In particular, the argument of the Nussbaum function is guaranteed to be always positive with the aid of nonlinear damping design, which is critical to perform a straightforward technical analysis of the boundedness of the Nussbaum function. Finally, the global exponential stability of parameter-varying strict-feedback systems, the boundedness of the control input and the update rate, and the asymptotic constancy of the parameter estimate are established. Numerical simulations are carried out to verify the effectiveness and benefits of the proposed methods.

A new adaptive law, averaging gradient algorithm (AGA), is presented for continuous-time direct adaptive control. The plant is assumed to have a modeled part in parallel with stable unmodeled dynamics whose gain is μ and to be perturbed by bounded disturbances. The AGA updates the controller parameters using the averaging gradient information taken in a time horizon determined by the richness of the internal signal in the adaptive loop. Under a parameter-dependent persistent excitation condition, it is shown that the controller parameters converge exponentially to a neighborhood of the tuning gain, which reduces to a single point (the tuning gain) in the ideal case (i.e. with unmodeled dynamics and bounded disturbances removed). In the latter case, the exponential convergence may be achieved under a weaker excitation condition. Conditions that relate: i) adaptation speed, ii) stability margin of the closed-loop transfer with the tuning gain, iii) excitation level of the internal signals in the adaptive loop, iv) upper bound on the disturbances and the gain of the unmodeled dynamics are given to insure the global exponential stability of the overall system.

Robust Adaptive Control Algorithms with and without Persistence of Excitation