Robust Policy Iteration of Uncertain Interconnected Systems With Imperfect Data

In this paper, the problem of the robust optimal control (ROC) for a class of nonlinear systems with uncertainties and external disturbances is studied. For this purpose, the State Dependent Riccati Equation (SDRE) is considered as a solution of the ROC. The proposed method is to transform a robust control problem into an optimal control problem, where the uncertainties and disturbances are reflected in the performance index. Hence, the performance index is selected as an energy function and using the proposed constraints in this paper the Lyapunov stability for this class of nonlinear systems is proved and the robust optimal controller is designed. In order to illustrate the high capability of the ROC method in optimal control of complex systems such as Hyper-chaotic system, the time response of state variables in the proposed method and the Global Robust Optimal Sliding Mode Control (GROSMC) method are compared together. The simulation results show that the time response of state variables in the ROC method is more sufficient than the GROSMC method.

In this paper, insulin injection process in which the level of glucose is monitored and corrected using optimal robust controller. The Bergman minimal model with unknown parameters was adopted to simulate a diabetes mellitus patient. An exogenous disturbance is considered as a meal effect on the patient. Optimal control approach to robust control design is suggested for glucose problem. The goal is to design a state feedback to stabilize the glucose system under uncertainty and disturbance. Linearized model of glucose system is considered with nonlinearity is lumped as a matched and mismatched uncertainties. Comparison in performance is made between the robust optimal controllers with classical optimal controller. The simulated results showed that the robust optimal controller outperforms the classical optimal controller for the considered cases. Also, the disturbance rejection capability of both controllers has been examined, and the results showed that the robust optimal controller has better rejection characteristics than its counterpart.

In this article, the optimal control scheme for ice-storage air conditioning (IAC) system is solved via a data-based adaptive dynamic programming (ADP) method. It is the first time that ADP is employed to design a self-learning scheme, which obtains the optimal control policy of IAC system. First, based on the data of the temperature, irradiance, and cooling load in an actual project, a prediction model of cooling load is built by a three-layer neural network with the performance verification. Second, the operation of the IAC system is analyzed. Third, a data-based ADP method is designed to realize a self-learning optimal control for the IAC system. Then, numerical results show that using the data-based optimal control method can reduce the operation costs. Finally, the comparison results show that the developed ADP method improves the system efficiency, minimizing the overall cost. Thus, the superiority of the developed algorithm is verified.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

The problem of synthesis of l/sub 1/ optimal robust controller for SISO plant under coprime factor perturbations and bounded external disturbance is considered. A geometric interpretation of l/sub 1/ optimal robust controller, standard l/sub 1/ optimal controller, and two other optimal robust controllers is presented. The existence of l/sub 1/ optimal robust linear controller is proved and an algorithm for approximate solution of the problem is proposed. The algorithm is reduced to approximate solution of finite family of standard l/sub 1/ optimization problems.

In this article, we propose an adaptive robust nonlinear optimal sliding mode control (SMC) using the optimal homotopy asymptotic method (RNOSC‐OHAM) for maximizing wind power capture. Because of the unstable nature of the wind and the presence of uncertainties and disturbances in the structure of the wind turbine, the optimal controller cannot provide robustness against uncertainties. Hence, we integrated this approach with a proportional–integral–derivative SMC (PID‐SMC) scheme to ensure the robustness of the system. The presented hybrid control input signal involves two components: a nonlinear optimal control law that provides the optimal performance of the nominal system and an adaptive PID‐SMC law. The adaptive PID‐SMC law employs a PID sliding surface along with adaptive mechanisms to create the robustness of the nonlinear wind turbine system. The nonlinear optimal control policy is designed by addressing the partial differential Hamilton–Jacobi–Bellman (HJB) equation. This equation is approximated using the OHAM. This combination strategy reduces the final time of the optimal control problems, provides desired responses and improves performance. Additionally, the RNOSC‐OHAM controller facilitates the safe performance of the wind turbine under uncertainties and maximizes the wind power captured. To evaluate the performance of RNOSC‐OHAM, the results of the presented algorithm are compared with some existing control schemes. The results indicate that the designed RNOSC‐OHAM controller is very rapid and needs few iterations and computational costs. Indeed, the present control scheme exhibits the best characteristics and has the fastest transient response. The RNOSC‐OHAM controller offers a proper balance between enhancing aerodynamic power capture and minimizing low‐speed shaft fluctuations using a small control input.

This paper aims at developing near optimal traffic signal control for multi-intersections in city. As a new optimization technique, adaptive dynamic programming (ADP) combines concepts of reinforcement learning and dynamic programming. ADP could learn continually from experience to achieve a near optimal control policy under varying conditions. However, without the cooperation among adjacent intersections, the near optimal control for each individual intersection can not guarantee a larger traffic area composing several intersections to be near optimal. This paper presents a new signal control method based on a model-free action-dependent ADP (ADHDP). This method can be used for cooperative control of multiple intersections. In every intersection, an ADHDP signal controller is adopted to adjust signal time according to an integrated unity parameter. The unity parameter is designed to consider not only the control performance in local intersection but also those in the neighbor intersections. Thus the designed controllers could achieve a set of near optimal control police for multi-intersections in a long run. Simulation results show that the trained controller achieves shorter average vehicular delay.

This paper investigates the robust optimal formation control for the position subsystem of quadrotor unmanned aerial vehicles (UAVs) subject to external disturbances and collision constraints. To prevent collision with both members of the formation and external obstacles, a collision avoidance potential function is constructed using relative position and velocity information. The basic bounded control input can ensure the stable flight and collision avoidance of a quadrotor UAV system. Based on the approximate dynamic programming (ADP) framework and two‐player zero‐sum differential game theory, the optimal controller is designed to further enhance the control performance of the system. The optimal value function is approximated by a single layer neural network, which avoids solving complex nonlinear Hamilton‐JacobiIsaac (HJI) equation. The stability of the closed loop system is proved. The effectiveness of the robust optimal formation controller based on learning is validated through MATLAB simulation in two distinct scenarios.

<p style='text-indent:20px;'>The imperfect sinusoidal flux distribution, cogging torque, and current measurement errors can cause periodic torque ripple in the permanent magnet synchronous motor (PMSM). These ripples are reflected in the periodic oscillation of the motor speed and torque, causing vibration at low speeds and noise at high speeds. As a high-precision tracking application, ripple degrades the application performance of PMSM. In this paper, an adaptive dynamic programming (ADP) scheme is proposed to reduce the periodic torque ripples. An optimal controller is designed by iterative control algorithm using robust adaptive dynamic programming theory and strategic iterative technique. ADP is combined with the existing Proportional-Integral (PI) current controller and generates compensated reference current iteratively from cycle to cycle so as to minimize the mean square torque error. As a result, an optimization problem is constructed and an optimal controller is obtained. The simulation results show that the robust adaptive dynamic programming achieves lower torque ripple and shorter dynamic adjustment time during steady-state operation, thus meeting the requirements of steady speed state and the dynamic performance of the regulation system.

Rigid spacecraft robust optimal attitude stabilization under actuator misalignments

In this paper, a new robust controller is proposed to improve the motion control performance of an autonomous four‐wheel steering vehicle. The vehicle is subject to time‐varying uncertainties caused by parameter perturbation and unmodeled disturbance. First of all, the vehicle motion control problem is modeled as a constraint‐following problem with the goal to drive the vehicle system to follow the given constraints. For safety reasons, inequality constraints are imposed on the lateral displacement of the vehicle. Next, the diffeomorphism mapping method is used to deal with the inequality constraints in the vehicle motion control, and the constrained lateral displacement space is mapped to an unbounded space. On this basis, a new multi‐parameter robust constraint‐following control is designed. Based on Lyapunov stability theory, the approximate constraint tracking performance of the path tracking system is proved. In order to make a trade‐off between the system performance and control cost, a multi‐parameter optimization problem is established, and the optimal robust controller is obtained. Last, the main theoretical results are verified by the Carsim‐Simulink co‐simulations.

Optimal control of interconnected systems with time-correlated noises: Application to vehicle platoon

In this paper, an infinite horizon optimal robust guaranteed cost control scheme of a class of continuous-time uncertain nonlinear systems is established based on adaptive dynamic programming. The main idea lies in that the optimal robust guaranteed cost control problem can be transformed into an optimal control problem. Actually, the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to help solving the modified Hamilton-Jacobi-Bellman equation corresponding to the nominal system. Then, an additional stabilizing term is introduced to reinforce the updating process of the weight vector and reduce the requirement of an initial stabilizing control. An example is provided to illustrate the effectiveness of the present control approach.

In this paper, a learning-based predictive control (LPC) scheme is proposed for adaptive optimal control of discrete-time nonlinear systems under stochastic disturbances. The proposed LPC scheme is different from conventional model predictive control (MPC), which uses open-loop optimization or simplified closed-loop optimal control techniques in each horizon. In LPC, the control task in each horizon is formulated as a closed-loop nonlinear optimal control problem and a finite-horizon iterative reinforcement learning (RL) algorithm is developed to obtain the closed-loop optimal/suboptimal solutions. Therefore, in LPC, RL and adaptive dynamic programming (ADP) are used as a new class of closed-loop learning-based optimization techniques for nonlinear predictive control with stochastic disturbances. Moreover, LPC also decomposes the infinite-horizon optimal control problem in previous RL and ADP methods into a series of finite horizon problems, so that the computational costs are reduced and the learning efficiency can be improved. Convergence of the finite-horizon iterative RL algorithm in each prediction horizon and the Lyapunov stability of the closed-loop control system are proved. Moreover, by using successive policy updates between adjoint time horizons, LPC also has lower computational costs than conventional MPC which has independent optimization procedures between two different prediction horizons. Simulation results illustrate that compared with conventional nonlinear MPC as well as ADP, the proposed LPC scheme can obtain a better performance both in terms of policy optimality and computational efficiency.

9 - Optimal and robust control system design