Robust Optimal Control Design Using a Differential Game Approach for Open-Loop Linear Quadratic Descriptor Systems

We consider the robust (H???) control problem for linear time-invariant systems over unreliable communication channels that are subject to packet losses from sensors to the controller and from the controller to the actuators. Moreover, the acknowledgments of control packet transmissions are also sent over reverse unreliable communication channels, which is known as the quasi-TCP model. The robust control problem is formulated within a stochastic zero-sum dynamic game framework, from which we obtain an H??? controller that minimizes the worst-case cost function in the presence of an adversary in the dynamical system. We characterize a set of existence conditions of the H??? controller in terms of the disturbance attenuation parameter ?? and packet loss rates. For the least disturbance attenuation scenario (as ?? ??? 8), we show that the H??? control system converges to the corresponding LQG system. We also show that the H??? controller for the TCP- and UDP-cases can be arrived at as appropriate limits of the solution for the quasi-TCP problem. Numerical examples are presented to illustrate the theoretical results.

Existing methods for nonlinear robust control often use scenario‐based approaches to formulate the control problem as large nonlinear optimization problems. The optimization problems are challenging to solve due to their size, especially if the control problems include time‐varying uncertainty. This paper draws from local reduction methods used in semi‐infinite optimization to solve robust optimal control problems with parametric and time‐varying uncertainty. By iteratively adding interim worst‐case scenarios to the problem, methods based on local reduction provide a way to manage the total number of scenarios. We show that the local reduction method for optimal control problems consists of solving a series of simplified optimal control problems to find worst‐case constraint violations. In particular, we present examples where local reduction methods find worst‐case scenarios that are not on the boundary of the uncertainty set. We also provide bounds on the error if local solvers are used. The proposed approach is illustrated with two case studies with parametric and additive time‐varying uncertainty. In the first case study, the number of scenarios obtained from local reduction is 101, smaller than in the case when all extreme scenarios are considered. In the second case study, the number of scenarios obtained from the local reduction is two compared to 512 extreme scenarios. Our approach was able to satisfy the constraints both for parametric uncertainty and time‐varying disturbances, whereas approaches from literature either violated the constraints or became computationally expensive.

Existing methods for nonlinear robust control often use scenario-based approaches to formulate the control problem as nonlinear optimization problems. Increasing the number of scenarios improves robustness, while increasing the size of the optimization problems. Mitigating the size of the problem by reducing the number of scenarios requires knowledge about how the uncertainty affects the system. This paper draws from local reduction methods used in semi-infinite optimization to solve robust optimal control problems with parametric uncertainty. We show that nonlinear robust optimal control problems are equivalent to semi-infinite optimization problems and can be solved by local reduction. By iteratively adding interim globally worst-case scenarios to the problem, methods based on local reduction provide a way to manage the total number of scenarios. In particular, we show that local reduction methods find worst case scenarios that are not on the boundary of the uncertainty set. The proposed approach is illustrated with a case study with both parametric and additive time-varying uncertainty. The number of scenarios obtained from local reduction is 101, smaller than in the case when all 214+3x192 boundary scenarios are considered. A validation with randomly drawn scenarios shows that our proposed approach reduces the number of scenarios and ensures robustness even if local solvers are used.

This paper deals with various duality results for a new class of constrained robust nonlinear control problems. Concretely, we introduce a minimisation problem associated with a vector of functionals of multiple integral type. In addition, the considered multi-objective controlled optimisation problem is governed by several mixed constraints generated by partial derivatives of second order and uncertain data. For this new class of control problems, we establish and characterise Wolfe, Mond-Weir and mixed type robust dual models. To this aim, by using the notion of convex multiple integral functional, generated by a controlled second-order Lagrangian with uncertain data, and the concept of robust weak efficient solution to the problem under study, we build a new mathematical context to state and prove the duality theorems.

Robust and risk sensitive control of discrete time, finite state systems on an infinite horizon is considered, with either state or output feedback. The solution of the state feedback robust control problem is characterized in terms of the value of an average cost dynamic game. The risk sensitive stochastic optimal control problem is solved using the policy iteration algorithm, and the optimal rate is expressed in terms of the value of a stochastic dynamic game with average cost per unit time criterion. By taking a small noise limit a deterministic dynamic game is obtained, which is closely related to the robust control problem. For the problem with output feedback (partial state information) the analysis depends on introducing appropriate information states for the risk sensitive and robust control problems.

This paper considers an approach to perform perturbation analysis of linear quadratic regulator (LQR) control problem for continuous-time descriptor systems. The investigated control problem is based on solving LMIs (Linear Matrix Inequalities) and applying Lyapunov functions. The paper is concerned with obtaining linear perturbation bounds for the continuous-time LQR control problem for descriptor systems. The computed perturbation bounds can be used to study the effect of perturbations in system and controller on feasibility and performance of the considered control problem. A numerical example is also presented in the paper.

Robust optimal control of connected and automated vehicle platoons through improved particle swarm optimization

An interceptor missile with tail fins and pulse thrusters has two different control inputs: one is continuous and the other one is discontinuous. This brings some challenges to the autopilot design and stability analysis. Indirect robust control strategy is applied to handle this problem. Based on the nonlinear dynamics of the interceptor, a nonlinear system for tracking acceleration commands is presented. Treating the error between the actual inputs and the computed inputs as input uncertainty, a robust control problem is formed. It is equivalent to an optimal control problem of the nominal system with a modified cost functional. The state-dependent Riccati equation approach is applied to solve the nonlinear optimal control to obtain a solution to the robust control problem. Numerical results show the effectiveness of the proposed strategy.

The author summarizes the theoretical results in "A discrete optimal control problem for descriptor systems," [IEEE Trans. Automat. Control, vol.34, pp.177-181, 1989] and "Authors' reply to 'Comments on 'A discrete optimal control problem for descriptor systems"," [ibid., to appear], and uses them in the computer-aided optimization problem for descriptor systems. An efficient algorithm for computing the optimal solution sequence of the optimization problem is developed and an example of the macroeconomic systems is shown.

In this paper, we propose a new approach to address robust control problem of nonlinear systems with uncertainties based on an adaptive dynamic programming (ADP) algorithm. After reformulating the robust control problem as an optimal control problem, we propose a modified ADP method to solve the derived Hamilton-Jacobi-Bellman (HJB) equation, where the optimal cost function is approximated by online training a critic neural network (NN). Then the approximated optimal control action can be derived to guarantee the stability of the controlled system with uncertainties. The closed-loop system stability and convergence have been proved. A simulation example is provided to illustrate the effectiveness of the method.

9 - Optimal and robust control system design

In this paper, an infinite horizon optimal robust guaranteed cost control scheme of a class of continuous-time uncertain nonlinear systems is established based on adaptive dynamic programming. The main idea lies in that the optimal robust guaranteed cost control problem can be transformed into an optimal control problem. Actually, the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to help solving the modified Hamilton-Jacobi-Bellman equation corresponding to the nominal system. Then, an additional stabilizing term is introduced to reinforce the updating process of the weight vector and reduce the requirement of an initial stabilizing control. An example is provided to illustrate the effectiveness of the present control approach.

This paper investigates the robust optimal control problem of a class of continuous-time, partially linear, interconnected systems. In addition to the dynamic uncertainties resulted from the interconnected dynamic system, unknown bounded disturbances are taken into account throughout the learning process, wherein the system’s dynamics and the disturbances are assumed unknown. These challenges lead the collected online data to be imperfect. In this scenario, traditional data-driven control techniques, such as adaptive dynamic programming (ADP) and robust ADP, encounter a challenge in approximating the optimal control policy precisely due to imperfect data and computational errors. In this paper, a novel data-driven robust policy iteration method is proposed to simultaneously solve the robust optimal control problems. Without relying on the knowledge of the system’s dynamics, the external disturbances or the complete state, the implementation of the proposed method only needs to access the input and partial state information. Based on the small-gain theorem, the notions of strong unboundedness observability and input-to-output stability, it is guaranteed that the learned robust optimal control gain is stabilizing and that the solution of the closed-loop system is uniformly ultimately bounded despite the existence of dynamic uncertainties and unknown external disturbances. The simulation results reveal the efficiency and practicality of the proposed data-driven control method. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work is motivated by the use of reinforcement learning to improve the quality of designing adaptive optimal controllers for engineering applications. Adaptive dynamic programming methods, in particular policy iteration (PI), are widely used in solving optimal control problems. However, due to the iterative nature of PI, the approximated optimal control policy may be inaccurate and imprecise. Especially, when using imperfect system’s measurements instead of the modelling information. This can result in causing the learned control policy to deviate from the actual optimal policy. This becomes more challenging in the existence of dynamic uncertainties and unknown external disturbances which corrupt the measurements and result in imperfect data. This work investigates the conditions on the uncertainties such that the proposed novel data-driven PI algorithm is robust to system’s uncertainties, unknown external disturbances and imperfect measurements. The approximated robust optimal control policy performs robustly in the existences of imperfect data and uncertainties, and at the same time is close enough to the optimal control policy.

This paper addresses the robust guaranteed cost tracking control problem of nonlinear matched uncertain systems using event-based adaptive dynamic programming (ADP). By constructing an augmented system and using a discounted cost function, the guaranteed cost control problem is transformed into an optimal control problem of the nominal augmented system. The ADP framework is utilized to approximate the solution of the Hamilton-Jacobi-Bellman (HJB) equation associated with the optimal control problem. The event-based guaranteed cost is derived, and its relation with the time-based one is discussed. Furthermore, it is proven that the derived event-based controller can make the tracking error uniformly ultimately bounded (UUB). The applicability of the suggested technique is exhibited using a simulation study.

Optimal control of aluminum electrolysis production process (AEPP) has long been a challenging industrial issue due to its inherent difficulty in establishing an accurate dynamic model. In this paper, a novel robust optimal control algorithm based on adaptive dynamic programming (ADP) is proposed for the AEPP, where the system subjects to input constraints. First, to establish an accurate dynamic model for the AEPP system, recursive neural network (RNN) is employed to reconstruct the system dynamic using the input-output production data. To ensure input constraints are not to exceed the bound of the actuator, the optimal control problem of the AEPP is formulated under a new nonquadratic form performance index function. Then, considering the perturbation of the AEPP, the robust control problem is effectively converted to the constrained optimal control problem via system transformation. Furthermore, a single critic network framework is developed to obtain the approximate solution of the Hamilton-Jacobi-Bellman (HJB) equation. Finally, the proposed ADP controller is applied to the AEPP system to validate the effectiveness and performance.