Data-driven optimal trajectory tracking control of autonomous surface vessels via reinforcement learning

The optimal output tracking control problem for nonlinear large-scale dynamic systems is considered. A series-based approximate approach for designing the optimal controller is proposed with respect to quadratic performance indexes. By using the approach, the high order, coupling, nonlinear two-point boundary value problem is transformed into a sequence of linear decoupling two-point boundary value problems. By intercepting the frontal finite terms of the solution series, we obtained a suboptimal tracking control law. A simulation example illustrates the efficiency of the optimal output tracking control law designed.

Learning-based optimal cooperative control using edge-based triggering strategy

In this paper, the nonlinear optimal control problem is investigated for spacecraft formation flying with collision avoidance. Based on a nonlinear model of formation flying, two optimal tracking control laws are proposed to ensure the formation pattern converges to the predetermined configuration. The first optimal control law which combines Lyapunov optimizing control with a trajectory-following optimization technique is developed to solve the finite-time nonlinear optimal control problem. For the second controller, an extended [Formula: see text]-[Formula: see text] method is applied to design a closed-form feedback control scheme for nonlinear control problem of spacecraft formation flying with non-standard cost functions. By taking into account the flying safety, a repulsive control scheme is incorporated in the optimal controller to ensure collision avoidance and minimize the performance index. Finally, a numerical example is performed to demonstrate the effectiveness of the proposed approaches.

Design of an intelligent optimal neural network-based tracking controller for nonholonomic mobile robot systems

This paper investigates the application of optimal control to the experimental flight of a Single Coaxial Rotor (SCR) Unmanned Aerial Vehicle (UAV) conducted in the indoor flight facility of the National Defense Academy (NDA) of Japan. The optimal control problem is prescribed as a minimum-length obstacle avoidance maneuver of the SCR UAV and it is solved using pseudospectral (PS) optimal control theory. The optimal trajectory is computed offline as a kinematic path-planning problem and then provided to the real UAV system as reference input commands. While only preliminary studies have been conducted at NDA, the results provide nominal tracking performance and validate the feasibility of the approach.

This paper develops an approximate design procedure of optimal tracking controllers for a class of nonlinear systems with persistent disturbances in the infinite time domain. By using the successive approximation approach (SAA), the two-point boundary value (TPBV) problem, which is derived from the optimal control theory, is transformed into solving a sequence of linear TPBV problems. The solution sequence of the linear TPBV problems uniformly converges to the solution of the original optimal tracking control (OTC) problem. A feedforward and feedback optimal tracking control (FFOTC) law is derived from a Riccati equation, matrix equations and a sequence of adjoint vectors. The existence and uniqueness conditions of the FFOTC are given. At last, a design algorithm of solving the approximate FFOTC law is proposed, and simulation results demonstrate the validity of the algorithm.

This paper considers the near-optimal tracking control problem for discrete-time systems with delayed input. Using a variable transformation, the system with delayed input is transformed into a non-delayed system, and the quadratic performance index of the optimal tracking control is transformed into a relevant format. The optimal tracking control law is constructed by the solution of a Riccati matrix equation and a Stein matrix equation. A reduced-order observer is constructed to solve the physically realizable problem of the feedforward compensator and a near-optimal tracking control is obtained. Simulation results demonstrate the effectiveness of the optimal tracking control law.

The paper presents a reinforcement learning approach for optimal tracking control of switched systems with application to a grid-tied hybrid generation system. To enhance interaction with the irregular environment, reference trajectory is learned via controller from states to optimal control. The main issue is to solve the optimal tracking control problem for a hybrid generation system consisting of multiple switched subsystems, and reinforcement learning can seek the globally optimal solution well without knowing accurate system dynamics. The investigated learning algorithm is used to generate an optimum map based on the learned ultimate value without knowledge of system parameters and obtains the optimal control law via deriving of algebraic Riccati equation (ARE) with unnecessary knowing of command generator dynamics. The optimal control solution can converge the online learning algorithm well based on policy iteration as verification in the simulation.

Optimal tracking control with zero steady-state error for time-delay systems with sinusoidal disturbances

The optimal output tracking control(OOTC) problem is considered for continuous linear time-invariable systems, which include control delays and external sinusoidal disturbances. We first transform the original system into a new system without time-delay, then construct a new state vector, which containing a external system’s vector. The external system described the aim which we want to track. Following the law of internal model principle, we transform the original and external systems into a new augmented system. Finally we get an optimal control law according to the optimal performance index, and prove the method is effective by emulation.

In this paper, a comparison is addressed between two methods, that is, the optimal tracking control methods of unknown nonlinear systems with and without constrained input. Firstly, the optimal tracking problem for a class of affine nonlinear system is formulated. The tracking cost functions are also defined, both for the two methods. The two methods are proved to be equivalent as the actuator bound is large enough. Integral reinforcement learning (IRL) algorithm is employed to solve the optimal tracking problem by using only system data. To facilitate the implementation of the IRL algorithm, the actor-critic neural network technique and the least squares method are employed in approximating the unknown weights iteratively. In the simulation, a detailed comparison is given to demonstrate the relationship between the two methods in the aspects of control input and tracking cost value.

With the increasing deployment of Unmanned Aerial Vehicle (UAV) swarms in uncertain and dynamically changing environments, optimal cooperative control has become essential for ensuring robust and efficient system coordination. To this end, this paper designs a data-driven optimal bipartite containment tracking control scheme for multi-UAV systems under compound uncertainties. A novel Dynamic Iteration Regulation Strategy (DIRS) is proposed, which enables real-time adjustment of the learning iteration step according to the task-specific demands. Compared with conventional fixed-step data-driven algorithms, the DIRS provides greater flexibility and computational efficiency, allowing for better trade-offs between the performance and cost. First, the optimal bipartite containment tracking control problem is formulated, and the associated coupled Hamilton–Jacobi–Bellman (HJB) equations are established. Then, a data-driven iterative policy learning algorithm equipped with the DIRS is developed to solve the optimal control law online. The stability and convergence of the proposed control scheme are rigorously analyzed. Furthermore, the control law is approximated via the neural network framework without requiring full knowledge of the model. Finally, numerical simulations are provided to demonstrate the effectiveness and robustness of the proposed DIRS-based optimal containment tracking scheme for multi-UAV systems, which can reduce the number of iterations by 88.27% compared to that for the conventional algorithm.

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming ( ADP ) technique is utilized to implement the optimal parallel tracking control using a critic neural network ( NN ) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded ( UUB ) . Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.

Modeling uncertainties are a major concern in vehicle path tracking control. As a practical engineering system, the uncertainties in vehicle lateral dynamics can be time-varying while bounded and have certain distributions wherein. The fuzzy set theory can effectively describe system uncertainties in terms of boundary and distribution. Contrary to fuzzy logic-based approaches, this article puts forward an explicit multiparameter optimal robust control law to ensure the uniform boundedness and ultimate uniform boundedness of the closed-loop path tracking dynamical system. Then, the tracking performance as well as the control cost is quantified as cost functions using fuzzy set theories. Finally, an optimization problem is established in the content of cooperative game to seek the optimal values for the tunable parameters. Simulations are conducted using CarSim and Simulink under double lane change and serpentine driving conditions. The results show that the proposed robust optimal control exhibits superior tracking performance.

A robust optimal output tracking control method for a midwater trawl system is investigated based on T-S fuzzy nonlinear model. A simplified nonlinear mathematical model is first employed to represent a midwater trawl system, and then a T-S fuzzy model is adopted to approximate the nonlinear system. Since the strong nonlinearities and the external disturbance of the trawling system, a mixed H2/H∞ fuzzy output tracking control strategy via T-S fuzzy system is proposed to regulate the trawl depth to follow a desired trajectory. The trawl depth can be regulated by adjusting the winch velocity automatically and the tracking error can be minimized according to the robust optimal criterion. In order to validate the proposed control method, a computer simulation is conducted. The simulation results indicate that the proposed fuzzy robust optimal controller make the trawl net rapidly follow the desired trajectory under the model uncertainties and the external disturbance caused by wave and current.