State feedback control for stochastic regular linear quadratic tracking problem with input time delay

The ship steering system is a typical electro-hydraulic servo system, which has the characteristics of non self-balance, large inertia, heavy load and uncertain disturbance. The load of the ship determines the depth of the draught. The depth of the draught determines the load of the steering system. The load of the rudder directly affects the dynamic and static tracking accuracy of the steering angle of the rudder. Therefore, the ship load is an important part of the ship steering system, and the establishment of a complete and accurate mathematical model is the necessary condition for the accurate control of the system. Based on the characteristics of the ship steering system, a general mathematical description with an uncertain disturbance system is presented. In order to obtain good performance of tracking control and resistance to load disturbance, a feedforward linear quadratic optimal tracking control method is proposed. The ship steering system designed based on this method can not only make the steering angle fast and accurately track the given, but also effectively suppress the heavy load uncertain disturbance. The simulation results show that feedforward linear quadratic optimal tracking control system has good tracking control performance and strong resistance to heavy load disturbance.

Speed regulation with simultaneous electric and fuel energy management in electrified aircraft engines is considered. An optimal control approach is formulated and solved that uses a weighted measure of stored energy and fuel consumption as cost function. The semiactive virtual control technique, originally developed for robots with regenerative drives, is suitably extended and used to treat the torque applied by the electromechanical subsystem upon the engine as an independent control input to be optimized. The standard proportional-integral loop used for the primary fuel flow control is maintained for compatibility with industry practice and gain scheduling and limit management logic schemes. A single tunable weight reflects design preferences for fuel or stored electric energy preservation. The magnitude of the weight yields feedback gains which can effectively reverse the direction of power flow between the engine and the electromechanical system, providing a direct way to induce power extraction or injection in transient and steady regimes. In the steady-state, infinite-horizon case, the optimal torque controller is shown to reduce to simple proportional action on the shaft speed plus a bias, paralleling standard linear quadratic tracking with a custom cost function. The paper includes simulation results using the gas turbine model included in NASA’s open-source T-MATS package (Toolbox for the Modeling and Analysis of Thermodynamic Systems) and a representative electromechanical system.

This paper presents an optimal observer design framework using a recently emerging method, approximate dynamic programming (ADP), to minimize a predefined cost function. We first exploit the duality between the linear optimal observer and the linear quadratic tracking (LQT) control. We show that the optimal observer design can be formulated as an optimal control problem subject to a specific cost function, and thus the solution can be obtained by solving an algebraic Riccati equation (ARE). For nonlinear systems, we further introduce an optimal observer design formulation and suggest a modified policy iteration method. Finally, to solve the problem online we propose a framework based on ADP and specifically on an approximator structure. Namely, a critic approximator is used to estimate the optimal value function, and a newly developed tuning law is proposed to find the parameters online. The stability and the performance are guaranteed with rigorous proofs. Numerical simulations are given to validate the theoretical studies.

Stochastic adaptive d‐step‐ahead optimal control is analyzed in this paper. An adaptive controller using the least squares (LS) algorithm and an input matching technique is proposed to combine the globally convergent estimation character of the adaptive tracking and optimally‐based input specification behavior by using a d‐step‐ahead quadratic cost function. The identification convergence rate of the parameter estimation is based on the LS algorithm. With adaptive control, the d‐step‐ahead closed‐loop stochastic system is globally stable and the adaptive controller converges to the d‐step‐ahead optimal controller. Lastly, the consistency of parameter estimators is analyzed and the convergence rate of parameter estimation is obtained. Copyright © 2004 John Wiley & Sons, Ltd.

Hybrid electrical vehicles (HEVs) are able to improve the fuel economy with reduced emissions due to their effective power management and regenerative power utilization. However, developing an optimal supervisory control strategy to distribute power is quite challenging due to the high degrees of freedom introduced by the multiple power sources in HEVs. Conventional approach uses so-called power follower scheme, and the power management can also be achieved by minimizing the equivalent fuel consumption at each given instant without guaranteeing that the battery state of charge (SOC) maintains at its target level. This paper proposes a model predictive control (MPC) strategy based on the linear quadratic tracking (LQT) control to follow the predicted driver power demand over a given horizon while keeping the battery SOC operated at its target level. The LQT controller is developed based upon the linearized control-oriented model and used to optimally track the predicted desired torque trajectory by minimizing the total equivalent fuel consumption of the internal combustion (IC) engine and electric motor (EM) over a given finite prediction horizon at each operational point. The performance of the MPC based LQT controller is validated in simulations under four typical driving cycles. The simulation results show that the fuel economy is greatly improved over the baseline power follower control strategy especially under FTP and IM240 driving cycles, and at the same time the battery state of charge is maintained at the desired level.

Control of time-delay systems represents one important class of control problems. A lot of work has been done in recent years. Much of the research work has been focused on the stability analysis and stabilization of time-delay systems based on the Lyapunov functional and linear matrix inequality (LMI) approaches. While the LMI approach does provide an e±cient tool, the results are mostly only su±cient and only numerical solutions are available. Our research aims to present analytical solutions by some algebraic approaches, which can provide an insightful understanding of control problem of time-delay systems. On the other hand, there has also been significant recent interest in networked multi-agent system (MAS) where unavoidably there exist delays in information acquisition as well as information exchange between agents in addition to possible input/state delays in each agent. This motivates the study of MASs with input and communication delays. The thesis is divided into two parts: Part I is focused on optimal control of single agent systems with multiple input delays; Part II is on distributed consensus control of MAS with communication and input delays. More specifically, in Part I, we shall study the linear quadratic regulation (LQR) and tracking for a single agent with linear dynamics. We first focus on LQR for linear discrete-time systems with multiple delays in a single input channel. By introducing a backward stochastic system, we establish a relationship between the LQR problem for the original system and optimal estimation for a backward stochastic system which can be solved by using projection approach. The explicit optimal controllers are developed for both finite time and infinite time horizon cases. A parallel result for linear continuous-time systems is also established, where the optimal feedback gain is given in terms of the solution of Riccati-type partial differential equations (PDEs).

This thesis reviews the ideas of reinforcement learning and looks into the detail of three popular methods: dynamic programming, Monte Carlo method and temporal-difference method. Dynamic programming is the core idea of reinforcement learning. Grid world and cliff walking problems show how reinforcement learning works to find the optimal path, or policy. The thesis also reviews the knowledge of linear quadratic control problem. Two common methods are reviewed. One is the standard method for LQR problem and the other one is a causal solution with an augmented system. Some concepts of them are related to dynamic programming. Related to this, some algorithms of solving linear quadratic tracking problem are presented based on reinforcement learning. To this end, augmented systems are created and the problem appears in the form of two system state vectors with different value matrices. Both offline algorithms with known system dynamics and online algorithms without system dynamics are investigated. According to the state vector of systems and the property of strategies, these algorithms are clearly classified. The simulation is done to testify the algorithms. The linear quadratic tracking control of motor speed is carried out with one second sampling time in an experimental setting. It turns out that the controller gain converges and the motor speed can track the reference perfectly.

Prosthesis is an artificial device that replaces missing body parts due to trauma, disease / accident. Prosthesis are mainly intended for restoring the normal functions of moving body part. Double pendulum structure is taken into consideration for modelling the system. In this system thigh and shank represents the pendulum links. Hip and knee joints create linkage between upper part of the body through thigh and shank. The muscles of both thigh and shank are initiated by external torques provided by stepper motor at hip and knee joints. The process variable to be controlled in the system is the position of both hip and knee. The work deals with the identification of system model along with the implementation of controllers like PID controller, Sliding mode controller (SMC) and Linear quadratic regulatory (LQR) controller and Linear quadratic tracking (LQT) controller. The results are compared.

Designing active flutter suppression for high-dimensional aeroelastic systems involving a control delay

Linear quadratic tracking control of unknown systems: A two-phase reinforcement learning method

The linear quadratic tracking control problem is studied for a class of discrete-time uncertain Markov jump linear systems with time-varying conditional distributions. The controller is designed under the assumption that it has no access to the true states of the Markov chain, but rather it depends on the Markov chain state estimates. To deal with uncertainty, the transition probabilities of Markov state estimates between the different operating modes of the system are considered to belong in an ambiguity set of some nominal transition probabilities. The estimation problem is solved via the one-step forward Viterbi algorithm, while the stochastic control problem is solved via minimax optimization theory. An optimal control policy with some desired robustness properties is designed, and a maximizing time-varying transition probability distribution is obtained. A numerical example is given to illustrate the applicability and effectiveness of the proposed approach.

Suboptimal linear quadratic tracking control for multi-agent systems

In this paper, the differences between decoupled and coupled control of bank-to-turn (BTT) missiles have been studied in detail from three aspects: stabilizability, linear quadratic regulator (LQR) control and tracking control. It is found that for some parameters the BTT missile can be controlled by a decoupled way, especially for stabilizability and LQR control. However, for some others, the BTT missile cannot be decoupled and a coupled control method is needed. Therefore, for a practical flight control system, the decouplability should be studied before giving a decoupled control method. In addition, in order to avoid the disadvantages of LQR control method, an improved H∞ controller design method is given. Combined with tracking control problem, a new H∞ control method is established for trajectory tracking.

This paper presents a landing controller for a fixed-wing aircraft during the landing phase, ensuring the aircraft reaches the touchdown point smoothly. The landing problem is converted to a finite-time linear quadratic tracking (LQT) problem in which an aircraft needs to track the desired landing path in the longitudinal-vertical plane while satisfying performance requirements and flight constraints. First, we design a smooth trajectory that meets flight performance requirements and constraints. Then, an optimal controller is designed to minimize the tracking error, while landing the aircraft within the desired time frame. For this purpose, a linearized model of an aircraft developed under the assumption of a small flight path angle and a constant approach speed is used. The resulting Differential Riccati equation is solved backward in time using the Dormand Prince algorithm. Simulation results show a satisfactory tracking performance and the finite-time convergence of tracking errors for different initial conditions of the flare-out phase of landing.

This article investigates the optimal tracking control problem for data-based stochastic discrete-time linear systems. An average off-policy Q-learning algorithm is proposed to solve the optimal control problem with random disturbances. Compared with the existing off-policy reinforcement learning (RL) algorithm, the proposed average off-policy Q-learning algorithm avoids the assumption of an initial stability control. First, a pole placement strategy is used to design an initial stable control for systems with unknown dynamics. Second, the initial stable control is used to design a data-based average off-policy Q-learning algorithm. Then, this algorithm is used to solve the stochastic linear quadratic tracking (LQT) problem, and a convergence proof of the algorithm is provided. Finally, numerical examples show that this algorithm outperforms other algorithms in a simulation.