• Home
  • Search
  • A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking
  • Cite Icon6
  • https://doi.org/10.3390/machines10070496Copy DOI Icon

A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking

Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The unmanned aerial vehicle (UAV) trajectory tracking control algorithm based on deep reinforcement learning is generally inefficient for training in an unknown environment, and the convergence is unstable. Aiming at this situation, a Markov decision process (MDP) model for UAV trajectory tracking is established, and a state-compensated deep deterministic policy gradient (CDDPG) algorithm is proposed. An additional neural network (C-Net) whose input is compensation state and output is compensation action is added to the network model of a deep deterministic policy gradient (DDPG) algorithm to assist in network exploration training. It combined the action output of the DDPG network with compensated output of the C-Net as the output action to interact with the environment, enabling the UAV to rapidly track dynamic targets in the most accurate continuous and smooth way possible. In addition, random noise is added on the basis of the generated behavior to realize a certain range of exploration and make the action value estimation more accurate. The OpenAI Gym tool is used to verify the proposed method, and the simulation results show that: (1) The proposed method can significantly improve the training efficiency by adding a compensation network and effectively improve the accuracy and convergence stability; (2) Under the same computer configuration, the computational cost of the proposed algorithm is basically the same as that of the QAC algorithm (Actor-critic algorithm based on behavioral value Q) and the DDPG algorithm; (3) During the training process, with the same tracking accuracy, the learning efficiency is about 70% higher than that of QAC and DDPG; (4) During the simulation tracking experiment, under the same training time, the tracking error of the proposed method after stabilization is about 50% lower than that of QAC and DDPG.

Loading PDF

Similar Papers
  • PDF
  • Research Article
  • Citations38

UAV maneuvering decision -making algorithm based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

  • Dec 07, 2021
  • Journal of Artificial Intelligence and Technology
  • Shuangxia Bai +5
  • Research Article

Closed-Loop Control of Droplet Quality Based on Curriculum Deep Deterministic Policy Gradient Algorithm

  • Jan 01, 2025
  • IEEE Access
  • Yunyun Shen +5
  • Research Article
  • Citations1

Multi-Microgrid Energy Trading Strategy Based on Multi-Agent Deep Deterministic Policy Gradient Algorithm

  • Jan 01, 2024
  • Applied Mathematics and Nonlinear Sciences
  • Genhong Qi +5
  • Research Article
  • Citations77

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

  • Jan 12, 2021
  • Chinese Journal of Aeronautics
  • Zijian Hu +4
  • Research Article

Voltage equalization control strategy for series battery formation based on Deep Deterministic Policy Gradient algorithm

  • Jun 01, 2026
  • Journal of Physics: Conference Series
  • Hao Liu +2
  • PDF
  • Book Chapter
  • Citations2

Intelligent Robotic Arm Path Planning (IRAP2) Framework to Improve Work Safety in Human-Robot Collaboration (HRC) Workspace Using Deep Deterministic Policy Gradient (DDPG) Algorithm

  • Oct 13, 2022
  • Xiangqian Wu +5
  • Conference Article
  • Citations4

Real-Time Optimal Dispatch of Microgrid Based on Deep Deterministic Policy Gradient Algorithm

  • Jul 01, 2021
  • Weidong Chen +2
  • PDF
  • Research Article
  • Citations7

Deep Deterministic Policy Gradient Algorithm Based on Convolutional Block Attention for Autonomous Driving

  • Jun 12, 2021
  • Symmetry
  • Yanliang Jin +3
  • Research Article
  • Citations128

Morphing control of a new bionic morphing UAV with deep reinforcement learning

  • May 28, 2019
  • Aerospace Science and Technology
  • Dan Xu +3
  • Research Article
  • Citations26

Robot grasping method optimization using improved deep deterministic policy gradient algorithm of deep reinforcement learning

  • Feb 01, 2021
  • Review of Scientific Instruments
  • Hongxu Zhang +3
  • Research Article
  • Citations20

Collaborative Control of Vehicle Platoon Based on Deep Reinforcement Learning

  • Oct 01, 2024
  • IEEE Transactions on Vehicular Technology
  • Jianzhong Chen +4
  • Book Chapter
  • Citations2

An Improved DDPG Algorithm with Barrier Function for Lane-Change Decision-Making of Intelligent Vehicles

  • Jan 01, 2021
  • Tianshuo Feng +3
  • Research Article
  • Citations20

A deep deterministic policy gradient algorithm based on averaged state-action estimation

  • May 11, 2022
  • Computers and Electrical Engineering
  • Jian Xu +2
  • PDF
  • Research Article
  • Citations20

Enhanced Deep Deterministic Policy Gradient Algorithm Using Grey Wolf Optimizer for Continuous Control Tasks

  • Jan 01, 2023
  • IEEE Access
  • Ebrahim Hamid Hasan Sumiea +6
  • Research Article
  • Citations6

Fast-Charging Optimization Method for Lithium-Ion Battery Packs Based on Deep Deterministic Policy Gradient Algorithm

  • May 17, 2025
  • Batteries
  • Zhi Zhang +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.