Three Dimensional Path Planning System for Unmanned Aerial Vehicles Based on Reinforcement Learning Algorithm

In complex dynamic environments, robot path planning faces challenges in multi‐objective optimization, such as path length, smoothness and obstacle avoidance capability. To address this, this paper proposes an improved sparrow search algorithm based on chaotic initialization and the golden positive cosine strategy for multi‐objective path planning. Diverse initial populations are generated through chaotic mapping to enhance global search capability and avoid falling into local optima. The golden positive cosine strategy optimizes individual position updates to accelerate convergence and ensure path smoothness. Results demonstrate that the proposed method outperforms sparrow search algorithm (SSA), with improvements of 21.1% in path length, 16.3% in smoothness and 14.2% in obstacle avoidance capability. After achieving global path optimization, the improved dynamic window approach (IDWA) is employed for real‐time obstacle avoidance in dynamic environments, dynamically adjusting the window size based on robot speed, obstacle density and target distance to adaptively expand or shrink the search space, thereby improving obstacle avoidance flexibility and efficiency. Simulation results show that the proposed method surpasses SSA in terms of path length, smoothness, obstacle avoidance capability and computational efficiency in both static and dynamic environments.

This study proposes an enhanced multi-agent swarm control algorithm (EN-MASCA) to solve the problem of efficient patrolling of drone swarms in complex durian orchard environments. It introduces a virtual navigator model to dynamically adjust the patrol path of the drone swarm and perform obstacle avoidance and path optimization in real time according to environmental changes. Different from traditional algorithms that only rely on fixed path planning, the virtual navigator model significantly improves the flexibility and stability of the drone swarm in complex environments. It also applies deep reinforcement learning algorithms to path planning and obstacle avoidance of drone swarms for the first time, improving the algorithm’s adaptability and optimization capabilities by learning dynamic information in complex environments. This innovation significantly improves the applicability of existing methods in complex terrain and dynamic obstacle environments. Finally, it incorporates the simulation characteristics of biological swarm behavior, and on this basis, comprehensively optimizes the flight path, obstacle avoidance and swarm stability of the drone swarm. By improving control strategies and parameter design, it improves the trajectory consistency and mission completion efficiency of the UAV swarm during flight. In the experimental part, this study verified in detail the advantages of the EN-MASCA algorithm in terms of flight trajectory, flight stability, cluster consistency and task completion efficiency by constructing a six-degree-of-freedom UAV motion simulation model and real environment simulation. It provides an efficient and intelligent solution for collaborative patrol operations of drones in durian orchards, which has important practical application value and promotion prospects.

This paper proposes a hybrid Unmanned Aerial Vehicle (UAV) path planning method that combines the Rapidly-exploring Random Tree (RRT) algorithm with Proximal Policy Optimization (PPO). The proposed method aims to enhance the efficiency and adaptability of UAV path planning in complex and dynamic environments. The RRT algorithm excels at quickly generating a feasible path from a start point to a goal point. However, the quality of its paths is often suboptimal, and it lacks adaptability in dynamic settings. In contrast, PPO, a deep reinforcement learning algorithm, optimizes paths through iterative policy updates, enabling the UAV to adapt to environmental changes. Our approach first employs RRT to generate an initial path, which is subsequently refined by PPO to improve smoothness and adaptability. Experimental results demonstrate that the RRT-PPO hybrid method performs favorably in terms of path length, computational time, and obstacle avoidance capability, effectively improving the task completion efficiency of UAVs in complex environments.

In this paper, path planning and sorting of packages for Omnidirectional-Wheel conveyor are presented using Reinforcement Learning (RL). Q-learning, Double Q-learning, Deep Q-learning, and the Double Deep Q-learning algorithms are investigated. The RL algorithms enable the conveyor to self-learn the packages path and sort them without using conventional control or path planning theories. The RL algorithms are used for two different case studies on conveyors structures with different numbers of cells to compare and evaluate their performances in large- and small-scale sized structures. To explore the proposed methods response to external environment effects, two types of collisions between multiple packages were considered, the proposed RL algorithms showed their ability to resolve both types successfully. Comparative study between multiple RL algorithms for path planning showed that the Q-learning and Double Q-learning algorithms had outperformed their Deep learning versions for path planning in the two case studies. Furthermore, the proposed RL methods are compared experimentally to classic control and path planning theories using a hardware prototype for one of the presented case studies. The hardware experimental results showed that the proposed RL methods were as successful as the conventional methods in path planning and sorting in much less processing time. Two types of sorting scenarios (Type I and II) were tested for same package type and for multiple ones. For Type I sorting the Q-learning algorithm performed better than the Q-learning with weights approach, achieving better mean and minimum rewards while maximum rewards remain the same for both techniques. As for Type II sorting, only the Q-learning with weights approach was able to achieve it and converge in a reasonable time.

Aiming at the problem of multi-objective path planning (MOPP) and dynamic obstacle avoidance of manipulator in dynamic environment, this paper proposes a solution based on hierarchical reinforcement learning (HRL) framework. Traditional path planning methods have some problems in dynamic scenes, such as poor real-time performance, difficult to balance multi-objective conflicts and insufficient adaptability to environmental changes. Therefore, this paper designs a two-tier architecture including global path planning layer and local obstacle avoidance layer, which is trained by Proximal policy optimization (PPO) and Soft Actor-Critic (SAC) algorithms respectively, and achieves collaborative optimization through an efficient information exchange mechanism between layers. At the same time, a multi-objective reward function based on dynamic weight adjustment strategy is introduced. Fuzzy logic is used to adaptively balance the relationship among path length, obstacle avoidance safety and energy consumption according to environmental complexity. Combined with Long Short-Term Memory (LSTM), the trajectory of obstacles is predicted, and the potential field method is further introduced to modify the obstacle avoidance reward, which improves the real-time response ability and robustness of the algorithm in dynamic environment. The experimental results show that the HRL-SAC-PPO method proposed in this paper shows superior performance in both static and dynamic scenarios. In the static scene, the success rate of this method reaches 100%, the average path length is shortened to 2.13m, no collision occurs, and the energy consumption is reduced to 1.12 kJ, which shows a good multi-objective optimization effect. In the dynamic scene, the trajectory error of obstacles predicted by LSTM is only 4.2%, and the safe distance between the robot arm and obstacles is improved by 35%, which significantly enhances the reliability of obstacle avoidance. In addition, the average decision delay of this method is only 11.3ms, and the peak delay is 23ms, which is much lower than that of the contrast algorithm, showing stronger real-time response ability. The ablation experiment further verified the key role of LSTM trajectory prediction, dynamic weight adjustment and layered structure on the overall performance. In the welding task verification of the real UR5 manipulator, the success rate of the system in dynamic environment is 95.3%, the average path length is 3.41m, and the maximum joint acceleration is only 0.87 radian/s², which is far below the safety threshold, indicating that the algorithm has good stability and obstacle avoidance ability in practical application. The comprehensive performance comparison shows that this method performs well in different industrial scenarios, and has stronger environmental adaptability and comprehensive path planning ability.

With the accelerated development of smart cities, the concept of a “smart industrial park” in which unmanned ground vehicles (UGVs) have wide application has entered the industrial field of vision. When faced with multiple tasks and heterogeneous tasks, the task execution efficiency of a single UGV is inefficient, thus the task planning research under multi-UGV cooperation has become more urgent. In this paper, under the anti-collision cooperation mechanism for multi-UGV path planning, an improved algorithm with optimized-weighted-speedy Q-learning (OWS Q-learning) is proposed. The slow convergence speed of the Q-learning algorithm is overcome to a certain extent by changing the update mode of the Q function. By improving the selection mode of learning rate and the selection strategy of action, the relationship between exploration and utilization is balanced, and the learning efficiency of multi-agent in complex environments is improved. The simulation experiments in static environment show that the designed anti-collision coordination mechanism effectively solves the coordination problem of multiple UGVs in the same scenario. In the same experimental scenario, compared with the Q-learning algorithm and other reinforcement learning algorithms, only the OWS Q-learning algorithm achieves the convergence effect, and the OWS Q-learning algorithm has the shortest collision-free path for UGVS and the least time to complete the planning. Compared with the Q-learning algorithm, the calculation time of the OWS Q-learning algorithm in the three experimental scenarios is improved by 53.93%, 67.21%, and 53.53%, respectively. This effectively improves the intelligent development of UGV in smart parks.

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments. As a computational approach to learning through interaction with the environment, reinforcement learning algorithms have been widely used for intelligent robot control, especially in the field of autonomous mobile robots. However, the learning process is slow and cumbersome. For practical applications, rapid rates of convergence are required. Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning, a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments. The state chain is built during the searching process. After one action is chosen and the reward is received, the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning. With the increasing number of Q-values updated after one action, the number of actual steps for convergence decreases and thus, the learning time decreases, where a step is a state transition. Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments. The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time, compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.

Automatic object searching is one of the essential skills for mobile robots to operate in unstructured and dynamical environments. It requires a robot to be endowed with object identification, obstacle avoidance, path planning, and navigation abilities. In this paper, a generic framework for automatic object searching is proposed. It has obstacle avoidance capability in unstructured and dynamical environments assisted by a behaviour learning algorithm using deep belief networks (DBN). As soon as a target object is recognized by a vision-based method, a bug-based path planning algorithm will be triggered for the robot in order to approach the target. Compared to state-of-art systems where laser sensors and cameras are widely used together for object identification, obstacle avoidance, path planning and navigation, only a single low cost RGB-D camera is used in our system to perform the above tasks. The proposed system has been implemented and tested in two indoor environments including a laboratory and a pantry of an office. The experimental results demonstrate the effectiveness of the proposed framework.

Path planning and obstacle avoidance are pivotal for intelligent unmanned aerial vehicle (UAV) systems in various domains, such as postdisaster rescue, target detection, and wildlife conservation. Currently, reinforcement learning (RL) has become increasingly popular in UAV decision-making. However, the RL approaches confront the challenges of partial observation and large state space when searching for random targets through continuous actions. This paper proposes a representation enhancement-based proximal policy optimization (RE-PPO) framework to address these issues. The representation enhancement (RE) module consists of observation memory improvement (OMI) and dynamic relative position-attitude reshaping (DRPAR). OMI reduces collision under partially observable conditions by separately extracting perception features and state features through an embedding network and feeding the extracted features to a gated recurrent unit (GRU) to enhance observation memory. DRPAR compresses the state space when modeling continuous actions by transforming movement trajectories of different episodes from an absolute coordinate system into different local coordinate systems to utilize similarity. In addition, three step-wise reward functions are formulated to avoid sparsity and facilitate model convergence. We evaluate the proposed method in three 3D scenarios to demonstrate its effectiveness. Compared to other methods, our method achieves a faster convergence during training and demonstrates a higher success rate and a lower rate of timeout and collision during inference. Our method can significantly enhance the autonomy and intelligence of UAV systems under partially observable conditions and provide a reasonable solution for UAV decision-making under uncertainties.

According to the problem of obstacles avoidance path planning in mobile robot system, this article uses the Q-learning algorithm in reinforcement learning to solve it. The original Q-learning algorithm has the problem of low learning efficiency, then an improved algorithm is proposed that adds a layer of learning process based on it. So that the mobile robot can find obstacles and target positions as soon as possible, which accelerates the efficiency of path planning and improves the efficiency. Finally, the path planning is established by a grid method on Python, and the comparison between the original algorithm, the improved algorithm proves that the learning efficiency of the algorithm is improved.

In the task of path planning, the unmanned surface vessels (USV) are required to reach the destination while avoiding obstacle. However, it is difficult for USV to prioritize the two sub-target tasks of destination arrival and obstacle avoidance in a complex dynamic environment using traditional path planning methods. Based on the above considerations, a path planning method for USVs based on strategy integration is proposed. Firstly, the path planning scenario containing dynamic obstacles and two sub-target scenarios are set up. In order to strengthen its ability to reach the destination and avoid dynamic obstacles, the dynamic obstacles and the destination are removed separately in the two sub-target scenarios. Furthermore, the model for strategy selection is trained on action decisions in sub-target scenarios through double deep Q-Network with prioritized experience replay. Finally, a framework for strategy integration is designed to optimize strategy selection for USV. The simulation results show that the proposed method achieves a good overall performance and a higher path planning success rate than the traditional path planning method.

Robot path planning is a cornerstone of precision agriculture, enabling safe and efficient operations for agricultural robots. However, complex field environments—characterized by static and dynamic obstacles, dense vegetation, and unstructured terrain—pose significant challenges to effective path planning. Conventional methods, such as A*, Dijkstra, and rapidly exploring random tree (RRT), exhibit limitations in efficiency and adaptability to dynamic conditions. To address these challenges, this study introduces AgriPath, a robust multi-objective path planning framework that integrates an improved convolutional neural network (CNN), an improved A* algorithm, and an improved whale optimization algorithm (IWOA) to optimize pathfinding, convergence efficiency, and obstacle avoidance in complex agricultural settings. Key innovations include an improved CNN leveraging causal convolution and multi-head self-attention mechanisms to improve temporal modeling for short-term trajectory prediction, augmented by Gaussian perturbations to enhance initial solution diversity; an improved A* algorithm incorporating dynamic heuristic functions based on Normalized Difference Vegetation Index (NDVI), combined with Kalman filtering, to bolster global path adaptability; IWOA employing non-linear convergence factors and differential evolution mechanisms to dynamically balance path length, smoothness, and planning time; and an improved Douglas–Peucker algorithm paired with cubic B-spline smoothing and navigation command modules to ensure path simplification and real-time execution. Experiments conducted in the Modern Agricultural Demonstration Zone at Chengdu, Sichuan Province, China, across simple, moderate, and complex scenarios, demonstrate that AgriPath outperforms advanced algorithms—SBREA*, Ant Colony A*, Orchard A*, and Greedy A*—in path length, smoothness, planning time, and dynamic obstacle avoidance success rate, indicative of superior multi-objective optimization balance. This study significantly enhances the efficiency and robustness of agricultural robot path planning, offering a more adaptive solution for autonomous navigation in precision agriculture while providing new theoretical and practical directions for the field of path planning.

With the wide application of unmanned vehicles (UV) in a complex environment, the research on path optimization and obstacle detection system has gradually become an important research part in the field of unmanned vehicle (UV) systems. The basic concept of path planning methods has always been a basic and difficult problem, especially in complex environments, in the effect of the dynamic environment, the safety, smoothness, and real-time requirement. To perform the task in a different environment and follow the exact movement and find the accurate path as well as remove the minimum human hurdle. In this research paper to find the shortest path with the avoidance of human obstacles with optimization techniques with meta-heuristic algorithms such as A*, fuzzy logic, GA, etc. we compare A* and fuzzy logic algorithms with the terms of processing time and path length with obstacle detection for high traffic areas. This research paper was practically implemented by the MATLAB software to simulate UV traversing from starting to endpoints.

With the rapid development of intelligent information technology, human–computer interaction for unmanned aerial vehicle (UAV) systems has gradually developed from single-mode interaction to multi-modal interaction. Due to the problems of information ambiguity and operational redundancy in multi-modal interaction, and the inability to effectively realize the coupling of human–computer interaction of different modalities, a priority strategy for multi-modal interaction for UAV systems is proposed. Living cells or tissues, whether in a static or active state, will produce regular electrical phenomena called bioelectricity. Human–computer interaction intention recognition technology based on bioelectrical signals can significantly enhance the real-time performance and flexibility of human–computer interaction and has great development potential. The characteristics of human–computer interaction intention recognition methods based on bioelectrical signals, such as electroencephalograms (EEGs), electromyograms (EMGs), electrooculograms (EOGs), and other conventional methods are introduced and analyzed, and then the basic characteristics of single-mode interaction and multi-modal interaction are compared. Furthermore, the priority strategies for UAV systems under different interaction modes are studied based on the analysis of task value evaluation parameters. The cumulative value, urgency, and task complexity are comprehensively analyzed and evaluated, and a set of priority strategies for multi-modal interaction is integrated. Through the simulation and verification of the multi-modal human–computer interaction software of the UAV system, it is determined that the proposed priority strategy conforms to the operation intention of the user, which improves the interaction efficiency and the reliability. In noisy environments, the UAV system control can be combined with the complementary information of various interaction methods to achieve precise control, which has obvious advantages.

This paper presents a method based on an enhanced Q-learning algorithm to improve the efficiency and safety of UAV path planning in fire detection tasks. Addressing the shortcomings of the traditional Q-learning algorithm in complex and high-risk environments, this study incorporates a dynamic greedy strategy and a risk weighting mechanism to improve convergence speed and planning safety. Experimental results show that the improved Q-learning algorithm outperforms the classical version in convergence speed, average cumulative reward and obstacle avoidance rate. demonstrating stronger adaptability in fire detection scenarios with obstacles and high-risk regions. This method efficiently determines the optimal drone path from the starting point to the target area while minimizing the likelihood of crossing high-risk zones, thereby improving the safety and efficiency of fire detection. The research findings provide strong support for the path planning problem in drone-based fire detection and offer a theoretical basis for future related research and applications.