Optimal adaptive controller scheme for uncertain quantized linear discrete-time system

In this paper, the finite-horizon near optimal adaptive regulation of linear discrete-time systems with unknown system dynamics is presented in a forward-in-time manner by using adaptive dynamic programming and Q-learning. An adaptive estimator (AE) is introduced to relax the requirement of system dynamics, and it is tuned by using Q-learning. The time-varying solution to the Bellman equation in adaptive dynamic programming is handled by utilizing a time-dependent basis function, while the terminal constraint is incorporated as part of the update law of the AE. The Kalman gain is obtained by using the AE parameters, while the control input is calculated by using AE and the system state vector. Next, to relax the need for state availability, an adaptive observer is proposed so that the linear quadratic regulator design uses the reconstructed states and outputs. For the time-invariant linear discrete-time systems, the closed-loop dynamics becomes non-autonomous and involved but verified by using standard Lyapunov and geometric sequence theory. Effectiveness of the proposed approach is verified by using simulation results. The proposed linear quadratic regulator design for the uncertain linear system requires an initial admissible control input and yields a forward-in-time and online solution without needing value and/or policy iterations. Copyright © 2014 John Wiley & Sons, Ltd.

In this chapter, differential games are studied for continuous-time linear and nonlinear systems, including two-player zero-sum games, multi-player zero-sum games, and multi-player nonzero-sum games, via a series of adaptive dynamic programming (ADP) approaches. First, an integral policy iteration algorithm is developed to learn online the Nash equilibrium of two-player zero-sum differential games with completely unknown continuous-time linear dynamics using the state and control data. Second, multi-player zero-sum differential games for a class of continuous-time uncertain nonlinear systems are solved by using a novel iterative ADP algorithm. Via neural network modeling for the system dynamics, the ADP technique is employed to obtain the optimal control pair iteratively so that the iterative value function reaches the optimal solution of the zero-sum differential games. Finally, an online synchronous approximate optimal learning algorithm based on policy iteration is developed to solve multi-player nonzero-sum games of continuous-time nonlinear systems without the requirement of exact knowledge of system dynamics.

In this study, an optimal adaptive control approach is established to solve the robust output tracking problem of a class of continuous time uncertain linear systems based on the policy iteration (PI) in actor-critic algorithm. First, by augmenting the integral variables of the tracking error into state variables, the robust tracking problem is transformed into a robust control problem of an augmented uncertain linear system. It is proven that the robust control law of the augmented system enables the output of the considered system to track a polynomial time signal asymptotically. Second, an optimal control method in the corresponding auxiliary nominal system is established, and based on the Bellman optimality principle, PI algorithms are proposed to solve online tracking controllers for the matched and the mismatched uncertain systems. Finally, for testing the availability of the proposed approach and theoretical results, two numerical experiments are provided.

This paper is concerned with the robustness and H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> control problem of observer-based control for a class of uncertain linear discrete-time systems with both state and input delays. Under no disturbance input, the asymptotic stabilization for uncertain time-delay systems will be guaranteed. Linear Matrix Inequality(LMI) optimization approach is used to design the observer-based H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> control. An example illustrates the application of the proposed technique.

This article considers the infinite horizon robust linear Minkowski regulator through the analysis of the Bellman equation arising from the corresponding infinite horizon minimax optimal control of parametrically uncertain linear discrete time systems with stage cost functions specified in terms of Minkowski functions of convex compact sets that contain the origin in their interior.The existence and uniqueness of the fixed point of the associated Bellman equation, in terms of the minimax value function, over the space of Minkowski functions of compact convex sets that contain the origin in their interior is established under natural conditions.The characterizations of the fixed point minimax value function and its minimax optimizer map are also derived.

A technique to design stabilizing state feedback controllers is presented for linear discrete-time systems subject to possibly time-varying parametric uncertainties. Conditions imposed on the uncertainties are less stringent than those found in the cited literature. The controller design procedure is based on the solution of a Riccati-like algebraic matrix equation. Stability considerations rely on the second method of Lyapunov, To evaluate the system's performance degradation due to uncertainty, an upper bound (guaranteed cost) is derived for the cost of the controlled uncertain system with respect to a designer chosen quadratic cost function. Two examples are given.

In this paper, a class of discrete-time uncertain linear hybrid systems, which is affected by both parameter variations and exterior disturbances, is considered. The goal here is to synthesize a hybrid control law, which includes the continuous variable control law and the discrete event control signal, so as to guarantee the controlled invariance for a specific region in the hybrid state space. It turns out that the continuous variable control law is a piecewise linear state feedback control based on the partition of the given region's continuous variable state space projection. A numerical example is given for illustration.

The authors present a sufficient condition for quadratic stabilizability of uncertain linear discrete time systems using LQ (linear quadratic) regulators. The true system is represented by a nominal model plus additive terms representing the uncertainties in the state and input matrices. With a proper choice of these terms, one may represent the most common types of uncertainties encountered in the literature. The main result of the present work is a robustification procedure for discrete time LQ regulators, which is based on a sufficient condition for quadratic stabilizability. The conservativeness of the proposed condition is discussed and some examples are given. >

This paper considers a subclass of hybrid systems, which is the discrete-time linear switching systems whose changes in the dynamics are determined by exogenous, uncontrollable events, and resets are present. In the presence of uncertain parameter and non-symmetrical constrained control, based on the results of nominal linear switching systems, a necessary and sufficient condition to test the robustness of a state feedback controller is given. The candidate state feedback controller for this test is that stabilizing nominal discrete-time linear switching systems. Also, the method in this paper can be extended to the case which has state constraints. At last, a numerical example is presented to illustrate the technique.

H∞ control of discrete-time uncertain linear systems with quantized feedback

In this paper, robust H/sub /spl infin// control problem is investigated for a class of uncertain linear discrete-time systems with norm-bounded nonlinear uncertainties. The class of systems can be treated as linear nominal parts with nonlinear perturbations on both states and control inputs. By means of linear matrix inequality technique, an approach has been developed to find the tolerable uncertainty bounds for the robust H/sub /spl infin// performance, and the corresponding static (or dynamic) output feedback controllers at the same time.

This paper uses the Popov-Tsypkin multiplier (which has intimate connections to the mixed structured singular value theory) to design robust H/sub /spl infin// estimators for uncertain linear discrete-time systems and considers the application of robust H/sub /spl infin// estimators to robust fault detection. The key to estimator-based, robust fault detection is to generate residuals which are robust against plant uncertainties and external disturbance inputs, which in turn requires the design of robust estimators. The robust H/sub /spl infin// estimation problem is formulated as a Riccati equation feasibility problem in which a cost function is minimized subject to a Riccati equation constraint. The robust H/sub /spl infin// estimator framework is then applied to the robust fault detection of dynamic systems.

This paper investigates a class of uncertain linear discrete-time systems subject to input rate saturation. A predictive sliding mode control approach is proposed which guarantees the control inputs remain bounded in the input rate saturation. Furthermore, the disturbance observer is developed to compensate for the system uncertainty and disturbance. Finally, the simulations demonstrate the effectiveness of the proposed predictive sliding mode control scheme.

In this paper, a novel strategy iterative method based on finite-time extended state observer is proposed to find the online adaptive optimal controller for a class of linear continuous-time systems with completely unknown system dynamics and unmeasurable state variables. In this method, finite-time state observer estimates the system state variables by using the input and output information of the system and the policy iterative method uses the estimated state variables and system input information to iteratively solve the algebraic Riccati equation until the adaptive optimal controller is obtained. It is worth noting that the method proposed in this paper only uses the input and output information of the system, without requiring the apriori knowledge of the system matrices and the state variables. A numerical simulation example will be provided to demonstrate the effectiveness of the method.

This paper studies the online adaptive optimal controller design for a class of nonlinear systems through a novel policy iteration (PI) algorithm. By using the technique of neural network linear differential inclusion (LDI) to linearize the nonlinear terms in each iteration, the optimal law for controller design can be solved through the relevant algebraic Riccati equation (ARE) without using the system internal parameters. Based on PI approach, the adaptive optimal control algorithm is developed with the online linearization and the two-step iteration, i.e., policy evaluation and policy improvement. The convergence of the proposed PI algorithm is also proved. Finally, two numerical examples are given to illustrate the effectiveness and applicability of the proposed method.