Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems With Input Constraints

A novel reinforcement-learning based output-adaptive neural network (NN) controller, also referred as the adaptive-critic NN controller, is developed to track a desired trajectory for a class of complex nonlinear discrete-time systems in the presence of bounded and unknown disturbances. The controller includes an observer for estimating states and the outputs, critic, and two action NNs for generating virtual, and actual control inputs. The critic approximates certain strategic utility function and the action NNs are used to minimize both the strategic utility function and their outputs. All NN weights adapt online towards minimization of a performance index, utilizing gradient-descent based rule. A Lyapunov function proves the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight, and observer estimation. Separation principle and certainty equivalence principles are relaxed; persistency of excitation condition and linear in the unknown parameter assumption is not needed. The performance of this controller is evaluated on a spark ignition (SI) engine operating with high exhaust gas recirculation (EGR) levels and experimental results are demonstrated.

This paper presents a novel neural network (NN) approach to the problem of control and modelling of nonlinear dynamic process through the manipulation of collected input-output data from multi-flash desalination (MSFD) process, using four multilayers feedforward NN's with the backpropagation learning rule to learn and code the nonlinear and complex mapping. This is implemented by allowing the mapping of the NN to generate on intermediate function V(t) from both the collected inputs and outputs of the plant. V(t) is then used to generate the control input signals of the plant by another mapping process. V(t) is a function representing the time state space. Different cases for V(t) are considered and the results are reported. >

A novel neural network (NN) based trajectory tracking controller for a servo system that also incorporates a knowledge-based control scheme is proposed in this paper. A decentralized control scheme, which neither requires a priori knowledge of the plant nor learning of the system dynamics, is introduced to deactivate the coupled dynamics associated with certain systems like robotic manipulators. The NN is employed to classify the system input-output measurements into several patterns depending on the displacement and velocity deviations from the respective desired trajectories. A proportional plus derivative gain control action is determined from a look-up table corresponding to the classification from the NN. Furthermore, an integrator is applied to enhance system performance. Several PD gains are introduced in a staggered format relative to the magnitudes of the displacement and velocity tracking errors, resulting in a controller that is robust to both structured and unstructured uncertainties.

Performance recovery of regional input-to-state stabilization by sampled-data output feedback control for nonlinear systems in the presence of disturbance

Output feedback control for nonlinear systems with uncertainties on output functions and growth rates

This paper deals with the problem of static output feedback controller design for nonlinear polynomial systems using an iterative sums of squares (SOS) approach. The proposed nonlinear feedback control scheme is developed to provide the semi-global exponential stability of the controlled system using Lyapunov theory with polynomial functions. Finally illustrative examples are given to validate the proposed approach.

A nonlinear adaptive output feedback control approach based on LOLIMOT is proposed for a class of nonaffine system in the non-strict feedback form with unknown nonlinearities in this paper. In order to overcome the disadvantages of output feedback control, LOLIMOT which can identify the nonlinear system accurately is applied. During the control process, backstepping approach is utilized to design the adaptive output feedback controller and the internal pseudo-controller is used as one-step predictor. The effectiveness of proposed scheme is shown by simulation on a proper nonlinear system.

PurposeThe purpose of this paper is to propose a decentralized output feedback controller for cooperative attitude regulation of spacecraft formation in absence of angular velocity feedback.Design/methodology/approachThe nonlinear relative attitude dynamic and kinematic equations represented by relative quaternion and relative angular velocity, respectively, are considered in this paper. The lead filter is employed to synthesize virtual angular velocity signal so that the design of output feedback controller is achieved. Lyapunov method is adopted to prove the stability of closed‐loop system. Considering the external disturbance, the theory of L2‐gain disturbance attenuation is employed to improve the designed controller. Numerical simulations are carried out to verify the controllers proposed.FindingsIt is found that the closed‐loop system can be guaranteed asymptotically stable in absence of external disturbance. When disturbance is considered, as long as the sufficient condition proposed is satisfied, the improved controller can render system uniformly ultimately bounded stable.Practical implicationsThe proposed output feedback control scheme can be considered as a fall‐back alternative for the case that the angular velocity sensors fail, or seen as another option for the system without angular velocity sensors at all.Originality/valueUnlike most classical works in the field of output feedback which focus on centralized scheme and neglect the disturbance, the controller proposed in this paper is able to handle the output feedback control problem of multi‐agent formation in a decentralized fashion, so as to avoid the single failure point of a centralized scheme. Meanwhile, the capability of L2‐gain disturbance attenuation is also achieved simultaneously.

This paper is concerned with the problem of output feedback adaptive tracking control for a class of uncertain nonlinear time-delay systems (NTDS) with both uncertain output function and unknown control directions. By introducing a linear state transformation, the original system is transformed into a new system for which output feedback control design becomes feasible. Without the precise information on output function, an input-driven high-gain state observer is presented to estimate the unavailable states of the new system. Moreover, some radial basis function neural networks (RBFNNs) are employed to approximate the lumped unknown nonlinear functions during the procedure of control design. Then, by combining the Lyapunov–Krasovskii functional method, the Nussbaum gain function approach and the backstepping technique, a novel adaptive output feedback control scheme including only one parameter to be updated is developed for such systems. As a result, all signals in the closed-loop system are semi-globally uniformly ultimately bounded (SGUUB) while the tracking error can remain in a small neighbourhood of zero. Finally, two simulation examples are given to validate the effectiveness and applicability of the proposed design method.

In this paper, an application of the Principal Component Analysis (PCA) technique to data-driven approach is addressed. The proposed scheme is applied to OFSP-based adaptive output feedback control systems. It is well recognized that an adaptive control system based on OFSP conditions can achieve asymptotic stability via a static output feedback. However, these conditions are very restrictive on most realistic systems. Therefore, a parallel feedforward compensator (PFC) is introduced to alleviate such restrictions, but remains a steady state error which is removed by utilizing the feedforward input. Besides, the data-driven approach is one of effective control strategies for nonlinear systems in the process control, and the control parameters can be updated rapidly at each equilibrium point. However, there is no research work that had been done in terms of applying the data-driven approach in an adaptive control system based on OFSP. Therefore, once the robust stability of the adaptive control system based on OFSP can be guaranteed, the adaptive gains can be optimized by the data-driven approach such that the output performance is able to be improved. Moreover, the PCA technique is considered to imporove the data-driven approach in terms of computional cost. The proposed scheme is verified through a numerical example, the results of which demonstrate the effectiveness.

The paper is concerned with the application of reinforcement learning techniques to feedback control of nonlinear systems using adaptive fuzzy logic systems (FLS). Even if a good model of the nonlinear system is known, it is often difficult to formulate a control law. The work in this paper addresses this problem by showing how an adaptive FLS can cope with nonlinearities through reinforcement learning with no preliminary off-line learning phase required. The reinforcement learning rules for finding proper fuzzy rules and tuning membership functions (MFs) do not assume that there is a supervisor to decide whether the current control action is correct. Instead, the FLS is indirectly told about the affect of its control action on the system performance. The learning is performed online based on a binary reinforcement signal from a critic without knowing the nonlinearity appearing in the system, and so is called direct reinforcement adaptive learning (DRAL). The learning algorithm is derived from Lyapunov stability analysis, so that both system tracking stability and error convergence can be guaranteed in the closed-loop system.

Discrete-time neural network output feedback control of nonlinear discrete-time systems in non-strict form

This paper is concerned with the output feedback control problem for a class of nonlinear system with time delay. First, a state feedback controller is design and a sufficient condition with exponential stability is obtained. Second, we study the observer design problem. And we proved that the observer designed in this paper ensure the observation error exponential converge to zero. Furthermore, output feedback controller using state feedback control and observer design is studied.

SummaryThis paper presents the design of output feedback controllers for discrete‐time (DT) linear systems. New sufficient LMI conditions are derived for designing static and controllers using decomposition of an auxiliary matrix. The decomposition facilitates linearization of nonlinear term of reduced size to obtain linear matrix inequality criteria. This leads to less conservative results as shown in the numerical examples. In addition, the proposed static output feedback criteria is also used for designing dynamic output feedback controllers for DT systems. Furthermore, a comparative study is also made for the proposed design method with the results existing in the literature. Finally, a DT static output feedback controller is designed for a quarter‐car suspension system. Simulation results are provided to show the efficacy of the proposed design method.

For a class of uncertain nonlinear systems with actuator failure, a dynamic-scaling output feedback control approach is proposed to address the adaptive practical tracking problem. Compared with existing nonlinear fault-tolerant control methods, a pair of new adaptive observer and control law are developed and the main innovations lie in: (i) novel time-varying coupled matrix inequalities are introduced to deal with the actuator failure; (ii) a non-identification adaptive gain is used to compensate the actuator failure, unknown control coefficients and nonlinear parameters in nonlinearities; (iii) our control approach is non-backstepping, which achieves the global output feedback control of nonlinear systems beyond the classic output-feedback form.