Path Planning for Autonomous Robots using Quantum Reinforcement Learning

A digital twin describes the virtual representation of a real process. This twin is constantly updated with real data and can thus control and adapt the real model. Designing suitable digital twins for path planning of autonomous robots or drones is often challenging due to the large number of different dynamic environments and multi-task and agent systems. However, common path algorithms are often limited to two tasks and to finding shortest paths. In real applications, not only a short path but also the width of the passage with a path as centered as possible are crucial, since robotic systems are not ideal and require recalibration frequently. In this work, so-called homotopic shrinking is used to generate the digital twin, which can be used to extract all possible path proposals including their passage widths for 2D and 3D environments and multiple tasks and robots. The erosion of the environment is controlled by constraints such that the task stations, the robot or drone positions, and the topology of the environment are considered. Such a deterministic path algorithm can flexibly respond to changing environmental conditions and consider multiple tasks simultaneously for path generation. A distinctive feature of these paths is the central orientation to the non-passable areas, which can have significant benefits for worker and patient safety. The method is tested on 2D and 3D maps with different tasks, obstacles, and multiple robots. For example, the robust generation of the digital twin for a maze and also the dynamic adaptation in case of sudden changes in the environment is covered. This variety of use cases and the comparison with alternative methods result in significant advantages, such as high robustness, consideration of multiple targets, and high safety distances to obstacles and areas that cannot be traversed. Finally, it was shown that the environment for the digital twin can be reduced to reasonable paths by constrained shrinking, both for real 2D maps and for complex virtual 2D and 3D maps.

Aiming at the problem of multi-objective path planning (MOPP) and dynamic obstacle avoidance of manipulator in dynamic environment, this paper proposes a solution based on hierarchical reinforcement learning (HRL) framework. Traditional path planning methods have some problems in dynamic scenes, such as poor real-time performance, difficult to balance multi-objective conflicts and insufficient adaptability to environmental changes. Therefore, this paper designs a two-tier architecture including global path planning layer and local obstacle avoidance layer, which is trained by Proximal policy optimization (PPO) and Soft Actor-Critic (SAC) algorithms respectively, and achieves collaborative optimization through an efficient information exchange mechanism between layers. At the same time, a multi-objective reward function based on dynamic weight adjustment strategy is introduced. Fuzzy logic is used to adaptively balance the relationship among path length, obstacle avoidance safety and energy consumption according to environmental complexity. Combined with Long Short-Term Memory (LSTM), the trajectory of obstacles is predicted, and the potential field method is further introduced to modify the obstacle avoidance reward, which improves the real-time response ability and robustness of the algorithm in dynamic environment. The experimental results show that the HRL-SAC-PPO method proposed in this paper shows superior performance in both static and dynamic scenarios. In the static scene, the success rate of this method reaches 100%, the average path length is shortened to 2.13m, no collision occurs, and the energy consumption is reduced to 1.12 kJ, which shows a good multi-objective optimization effect. In the dynamic scene, the trajectory error of obstacles predicted by LSTM is only 4.2%, and the safe distance between the robot arm and obstacles is improved by 35%, which significantly enhances the reliability of obstacle avoidance. In addition, the average decision delay of this method is only 11.3ms, and the peak delay is 23ms, which is much lower than that of the contrast algorithm, showing stronger real-time response ability. The ablation experiment further verified the key role of LSTM trajectory prediction, dynamic weight adjustment and layered structure on the overall performance. In the welding task verification of the real UR5 manipulator, the success rate of the system in dynamic environment is 95.3%, the average path length is 3.41m, and the maximum joint acceleration is only 0.87 radian/s², which is far below the safety threshold, indicating that the algorithm has good stability and obstacle avoidance ability in practical application. The comprehensive performance comparison shows that this method performs well in different industrial scenarios, and has stronger environmental adaptability and comprehensive path planning ability.

Path planning aims to enable autonomous robots to navigate safely and efficiently from a starting point to a target point in challenging and dynamic environments. Path planning in robotics is highly significant and still an ongoing subject of research. The increasing use of robots in various applications such as industrial automation, service robotics, and autonomous vehicles has brought forth the need for reliable and efficient path planning algorithms. The inherent capability of Voronoi diagrams to partition space based on proximity makes them an effective framework for research in path planning. Ant colony optimization, a bio-inspired optimization technique, is based on the foraging behavior of ants and is commonly employed to address the traveling salesman problem and various other combinatorial optimization problems. A hybrid method was adopted in this study by combining a Voronoi diagram and an ant colony algorithm. To create paths for the robot where it can stay as far away from obstacles as possible, a Voronoi diagram was utilized. Additionally, to find the shortest path from the starting point to the destination among these paths, ant colony optimization was employed. The main contribution of the study lies in the combination of the Voronoi diagram for obstacle avoidance and ant colony optimization for finding the optimal path. The combination of these techniques makes an effective contribution to robotic path planning by focusing on ensuring safety by avoiding obstacles while optimizing the shortest path. Experimental studies show that the hybrid method produces successful results for the desired purpose.

Classical reinforcement learning (CRL) has been widely applied in neuroscience and psychology; however, quantum reinforcement learning (QRL), which shows superior performance in computer simulations, has never been empirically tested on human decision-making. Moreover, all current successful quantum models for human cognition lack connections to neuroscience. Here we studied whether QRL can properly explain value-based decision-making. We compared 2 QRL and 12 CRL models by using behavioural and functional magnetic resonance imaging data from healthy and cigarette-smoking subjects performing the Iowa Gambling Task. In all groups, the QRL models performed well when compared with the best CRL models and further revealed the representation of quantum-like internal-state-related variables in the medial frontal gyrus in both healthy subjects and smokers, suggesting that value-based decision-making can be illustrated by QRL at both the behavioural and neural levels.

This study reviews the progress in the application of deep reinforcement learning in the field of path planning and control for mobile robots. Traditional methods exhibit poor adaptability in complex and dynamic environments, struggling to handle static and dynamic obstacles as well as environmental uncertainties. Deep reinforcement learning enables robots to autonomously interact with the environment and learn optimal strategies, significantly enhancing the performance of path planning and control. The article provides a detailed analysis of the limitations of traditional methods, such as high computational complexity, susceptibility to local optima, and lack of adaptability. It also introduces the advantages of deep reinforcement learning algorithms (e.g., DDPG, SAC) and their improved variants (e.g., APF-DDPG, AM-LSTM-SAC). These algorithms demonstrate excellent experimental results in various environments through multi-algorithm fusion strategies. The paper further discusses the setup of simulation environments, experimental configurations, and result analysis, summarizing existing achievements and shortcomings while providing an outlook on future research directions. This work offers valuable insights for the further development of mobile robot technology."

To address the limitations of the Deep Deterministic Policy Gradient (DDPG) in robot path planning, we propose an improved DDPG method that integrates kinematic analysis and D* algorithm, termed D*-KDDPG. Firstly, the current work promotes the reward function of DDPG to account for the robot’s kinematic characteristics and environment perception ability. Secondly, informed by the global path information provided by the D* algorithm, DDPG successfully avoids getting trapped in local optima within complex environments. Finally, a comprehensive set of simulation experiments is carried out to investigate the effectiveness of D*-KDDPG within various environments. Simulation results indicate that D*-KDDPG completes strategy learning within only 26.7% of the training steps required by the original DDPG, retrieving enhanced navigation performance and promoting safety. D*-KDDPG outperforms D*-DWA with better obstacle avoidance performance in dynamic environments. Despite a 1.8% longer path, D*-KDDPG reduces navigation time by 16.2%, increases safety distance by 72.1%, and produces smoother paths.

Automatically constructing smooth paths that satisfy a formal specification is a challenging problem. Existing methods struggle to scale to long horizon specifications and challenging environments. We present a method that uses abstraction, model checking, and convex optimization to solve for a smooth Bézier spline that is guaranteed to satisfy a Linear Temporal Logic specification. Our approach uses a coarse abstraction to avoid the state explosion of other abstraction based methods, and successfully avoids the computational challenges of directly optimizing the non-convex temporal logic semantics. We prove our method is sound and complete and demonstrate a significant computational advantage relative to state of the art approaches. Generating such smooth paths has natural applications in path planning for autonomous robots, and we demonstrate the applicability of our method on path planning for a quadrotor.

This paper presents a method of trajectory planning and design for Unmanned Aerial Vehicle (UAV) based on the A∗ algorithm. Using grids to process the trajectory path planning of UAV under the environment with presence of obstacles, and then find the shortest path from the initial point to the target point based on rasterized environment using A∗ algorithm. The simulation of trajectory planning is realized in the Microsoft Visual C++ 6.0 developing environment. Through the simulation of trajectory planning, we can prove that UAV can find the shortest path from the initial point to the target point based on A∗ algorithm.

Robot path planning is a cornerstone of precision agriculture, enabling safe and efficient operations for agricultural robots. However, complex field environments—characterized by static and dynamic obstacles, dense vegetation, and unstructured terrain—pose significant challenges to effective path planning. Conventional methods, such as A*, Dijkstra, and rapidly exploring random tree (RRT), exhibit limitations in efficiency and adaptability to dynamic conditions. To address these challenges, this study introduces AgriPath, a robust multi-objective path planning framework that integrates an improved convolutional neural network (CNN), an improved A* algorithm, and an improved whale optimization algorithm (IWOA) to optimize pathfinding, convergence efficiency, and obstacle avoidance in complex agricultural settings. Key innovations include an improved CNN leveraging causal convolution and multi-head self-attention mechanisms to improve temporal modeling for short-term trajectory prediction, augmented by Gaussian perturbations to enhance initial solution diversity; an improved A* algorithm incorporating dynamic heuristic functions based on Normalized Difference Vegetation Index (NDVI), combined with Kalman filtering, to bolster global path adaptability; IWOA employing non-linear convergence factors and differential evolution mechanisms to dynamically balance path length, smoothness, and planning time; and an improved Douglas–Peucker algorithm paired with cubic B-spline smoothing and navigation command modules to ensure path simplification and real-time execution. Experiments conducted in the Modern Agricultural Demonstration Zone at Chengdu, Sichuan Province, China, across simple, moderate, and complex scenarios, demonstrate that AgriPath outperforms advanced algorithms—SBREA*, Ant Colony A*, Orchard A*, and Greedy A*—in path length, smoothness, planning time, and dynamic obstacle avoidance success rate, indicative of superior multi-objective optimization balance. This study significantly enhances the efficiency and robustness of agricultural robot path planning, offering a more adaptive solution for autonomous navigation in precision agriculture while providing new theoretical and practical directions for the field of path planning.

In complex, semi-structured orchard environments, mobile inspection robots often suffer from excessive turning points, low search efficiency, limited trajectory-tracking accuracy, and poor adaptability to dynamic obstacles. To address these issues, this study proposes an integrated autonomous navigation method that employs an improved A* algorithm for global path planning, a Fuzzy-Weighted Dynamic Window Approach (FW-DWA) for local path optimization, and a model predictive control (MPC)-based trajectory-tracking controller. First, a dynamic heuristic-weight adjustment strategy is introduced into the conventional A* algorithm, in which a correction factor adaptively tunes the heuristic weight; a two-stage node optimization procedure then removes hazardous and redundant nodes to improve path smoothness and safety. Second, the FW-DWA, grounded in fuzzy control theory, uses goal distance and obstacle distance to update the weights of the heading, clearance, and velocity evaluation functions in real time, thereby enhancing obstacle avoidance in dynamic environments. Finally, a discrete kinematic model is established to design the MPC Controller, which achieves high-precision tracking through receding-horizon optimization and feedback correction. Experiments conducted in real orchards demonstrate that the proposed method reduces path length by 5.79%, shortens planning time by 3.64%, and increases the minimum safety distance by 50%. Comparative results further show that the MPC Controller attains a mean position error of 0.032 m and a mean heading error of 3.14°, clearly outperforming a conventional Proportional–Integral–Derivative (PID) controller. These findings provide an effective solution for reliable autonomous navigation of orchard inspection robots and offer a valuable reference for smart agricultural robotics applications.

In complex dynamic environments, robot path planning faces challenges in multi‐objective optimization, such as path length, smoothness and obstacle avoidance capability. To address this, this paper proposes an improved sparrow search algorithm based on chaotic initialization and the golden positive cosine strategy for multi‐objective path planning. Diverse initial populations are generated through chaotic mapping to enhance global search capability and avoid falling into local optima. The golden positive cosine strategy optimizes individual position updates to accelerate convergence and ensure path smoothness. Results demonstrate that the proposed method outperforms sparrow search algorithm (SSA), with improvements of 21.1% in path length, 16.3% in smoothness and 14.2% in obstacle avoidance capability. After achieving global path optimization, the improved dynamic window approach (IDWA) is employed for real‐time obstacle avoidance in dynamic environments, dynamically adjusting the window size based on robot speed, obstacle density and target distance to adaptively expand or shrink the search space, thereby improving obstacle avoidance flexibility and efficiency. Simulation results show that the proposed method surpasses SSA in terms of path length, smoothness, obstacle avoidance capability and computational efficiency in both static and dynamic environments.

Rapid-exploration Random Tree (RRT) is an efficient algorithm to search non-convex and high-dimensional spaces via randomly constructing spatial filling trees. This algorithm has been widely used in autonomous robot path planning. However, the basic RRT algorithm has some shortcomings. In order to improve the defects of low search efficiency and poor path quality of the RRT algorithm, this paper proposes an A* based RRT path planning algorithm with the advantages of completeness and optimality of the A* algorithm and fast extensibility of the RRT algorithm. During the procedure of random node sampling of the RRT algorithm, A* path is used to formulate the sampling strategy. Meanwhile, the constraint of the path turning angle is added to the nearest neighboring search of the RRT algorithm, which can enhance the rationality of the search tree node selection and improve the obtained path quality. Simulation experiments have been performed to verify the effectiveness of the proposed method for unmanned aerial vehicle path planning.

With the popularity of service robots in dynamic environments such as hospitals and shopping malls, traditional path planning algorithms have problems such as inflection points of planning paths and difficulty in effectively avoiding dynamic obstacles. In response to these problems, this paper proposes an optimization algorithm based on A* and dynamic window method (DWA). First, in the global path planning, the number of turning points is reduced by deleting redundant turning points. Secondly, by establishing the motion model of the obstacle, the range of the dynamic obstacle is updated to the static environment, and the dynamic obstacle avoidance ability of the DWA algorithm is improved. Finally, the optimized A* algorithm is combined with the DWA algorithm to realize the path planning of the service robot in a dynamic environment. Experimental results show that the algorithm reduces the global path length without disturbing dynamic obstacles, the path is smoother, and the dynamic obstacle avoidance ability is more reliable.

Background: Path planning technology has a wide range of applications in many fields. Applications in the field of high technology include: autonomous and touchless action of robots; obstacle avoidance and defense flight of drones; cruise missiles to avoid radar search, anti- bouncing attacks, and to complete the task of sudden defense and demolition. Applications in daily life include: GPS navigation; road planning based on GIS system; urban road network planning and navigation. Applications in the field of decision-making management include: vehicle problem (VRP) in logistics management and similar resource allocation problems in resource management. Routing problems in the field of communication technology. Any planning problem that can be topologized as a point and line network can basically be solved by the method of path planning. Different intelligent algorithms have different characteristics, and their application scope and fields are also different, so it is of great significance for the development of path planning technology to study the path planning intelligent algorithms from the characteristics of the algorithms themselves and their applications. Objective: Analyze the advantages and disadvantages of various types of planning algorithms, look forward to the future development trend of mobile robot path planning, and provide certain ideas for the research of mobile robot path planning. Methods: Search journals, patents, conferences, and papers related to mobile robot path planning, and summarize and analyze the advantages and disadvantages of various planning algorithms. Results: Based on the research results of many scholars, this study summarizes different mobile robot path planning methods. It is divided into four categories: traditional planning, intelligent search, artificial intelligence and local obstacle avoidance. This paper introduces and analyzes the latest research results of these types, including their design ideas, advantages and disadvantages, and improvement measures. The research methods adopted are analyzed in order to maximize the advantages of each algorithm and expand the application field of robot path planning to provide ideas and references. Conclusion: This paper provides guidance for the design and optimization of robot path planning. Finally, this paper summarizes the future development trend of robot path planning, and looks forward to the future development trend and key areas of robot path planning.

Autonomous mobile robot encompasses modules such as perception, path planning, decision-making, and control. Among these modules, path planning serves as a prerequisite for mobile robots to accomplish tasks. Enhancing path planning capability of mobile robots can effectively save costs, reduce energy consumption, and improve work efficiency. The primary slime mold algorithm (SMA) exhibits characteristics such as a reduced number of parameters, strong robustness, and a relatively high level of exploratory ability. SMA performs well in path planning for mobile robots. However, it is prone to local optimization and lacks dynamic obstacle avoidance, making it less effective in real-world settings. This paper presents an enhanced SMA (ESMA) path-planning algorithm for mobile robots. The ESMA algorithm incorporates adaptive techniques to enhance global search capabilities and integrates an artificial potential field to improve dynamic obstacle avoidance. Compared to the SMA algorithm, the SMA-AGDE algorithm, which combines the Adaptive Guided Differential Evolution algorithm, and the Lévy Flight-Rotation SMA (LRSMA) algorithm, resulted in an average reduction in the minimum path length of (3.92%, 8.93%, 2.73%), along with corresponding reductions in path minimum values and processing times. Experiments show ESMA can find shortest collision-free paths for mobile robots in both static and dynamic environments.