Research on multi-objective path planning and dynamic obstacle avoidance algorithm of manipulator based on reinforcement learning

The construction of a practical nanorobot is a definite futuristic reality. However development of a Nanorobot is associated with a multitude of challenges and limitations related mainly to its control and behavior aspects in different dynamic work environments. In this chapter a novel Nanorobot movement control algorithms in dynamic environment has being introduced. To avoid obstacles during movement trajectory, swarm intelligence based approach and for sensing, path planning, obstacle avoidance and target detection of nano-robot Quorum and RFID based approach has been utilized. Major proportion of work has been done on Microsoft Robotics Developer Studio 2008 (MSRD, 2008) along with two kinds of programming environment VPL and SPL. The chapter also includes a description of how nanorobotics technology has been applied in medical field. In this context, a non invasive method to treat a brain blood clot using nanorobots has being explained. Various other aspects of application of nanorobots has also been briefly mentioned.

In order to solve the “minimum trap” of artificial potential field method and the limitation of traditional path planning algorithm in dynamic obstacle environment, a path planning control algorithm based on improved artificial potential field method is proposed. Firstly, a virtual potential field detection circle model (VPFDCM) with adjustable radius is proposed to detect the “minimum trap” formed by the repulsion field of obstacles in advance. And the motion model of unmanned vehicle is established. Combined with the improved reinforcement learning algorithm based on Long Short-Term Memory(LSTM), the radius of virtual potential field detection circle is adjusted to achieve effective avoidance of dynamic obstacles, The reliable online collision free path planning of unmanned vehicle in semi closed dynamic obstacle environment is realized. Finally, the reliability and robustness of the algorithm are verified by MATLAB simulation. The simulation results show that the improved artificial potential field method can effectively solve the problem of unmanned vehicle falling into the “minimum trap” and improve the reliability of unmanned vehicle movement. Compared with the traditional artificial potential field method, the improved artificial potential field method can achieve more than 90% success rate in obstacle avoidance.

In response to the problem of low efficiency and weak obstacle avoidance ability in finding the optimal or suboptimal path for 3D path planning of drones in complex dynamic environments, this paper uses Q-learning algorithm to complete the 3D path planning of drones, aiming to improve their path planning and obstacle avoidance capabilities. Firstly, by constructing a three-dimensional gridded environment model, the system calculates the reward for each state under the influence of natural environment and obstacles, and then guides the drone to avoid obstacles and find the optimal path. The system uses the ε - greedy strategy for exploration and learning, optimizing decisions by continuously updating the Q-table value table. The experimental results show that the drone has a success rate of 93.3% in obstacle avoidance in complex and multi obstacle scenes. Moreover, in terms of average path length, the Q-learning algorithm has shortened it by approximately 20.00%, 11.45%, and 40.39% compared to ant colony algorithm, A* algorithm, and RRT algorithm, respectively. In dynamic wind speed environments, the Q-learning algorithm reduces the path length by about 4% to 11% compared to other algorithms, further demonstrating its effectiveness and advantages in complex environments.

Mobile robots face collision risk avoidance challenges in dynamic environments, necessitating that we address the safety and adaptability shortcomings of traditional navigation methods. Traditional methods rely on predefined rules, making it difficult to achieve flexible, safe, and real-time obstacle avoidance in complex, dynamic environments. To address this issue, a risk-aware, dynamic, adaptive regulation barrier policy optimization (RADAR-BPO) method is proposed, combining proximal policy optimization (PPO) with the control barrier function (CBF). RADAR-BPO generates exploratory actions using PPO and constructs a real-time safety filter using the CBF. This method uses quadratic programming to minimize risky actions, thereby ensuring safe obstacle avoidance while maintaining navigation efficiency. Testing of three phased learning environments in the ROS Gazebo simulation environment demonstrated that the proposed method achieves an obstacle avoidance success rate of nearly 90% in complex, dynamic, multi-obstacle environments and improves the overall mission success rate, validating its robustness and effectiveness in complex dynamic scenarios.

This paper proposes a hybrid Unmanned Aerial Vehicle (UAV) path planning method that combines the Rapidly-exploring Random Tree (RRT) algorithm with Proximal Policy Optimization (PPO). The proposed method aims to enhance the efficiency and adaptability of UAV path planning in complex and dynamic environments. The RRT algorithm excels at quickly generating a feasible path from a start point to a goal point. However, the quality of its paths is often suboptimal, and it lacks adaptability in dynamic settings. In contrast, PPO, a deep reinforcement learning algorithm, optimizes paths through iterative policy updates, enabling the UAV to adapt to environmental changes. Our approach first employs RRT to generate an initial path, which is subsequently refined by PPO to improve smoothness and adaptability. Experimental results demonstrate that the RRT-PPO hybrid method performs favorably in terms of path length, computational time, and obstacle avoidance capability, effectively improving the task completion efficiency of UAVs in complex environments.

With the development and widely use of unmanned aerial vehicles (UAVs) in recent years, the development of efficient path planning methods for automation has become crucial. Obstacle avoidance and path planning are the key components of UAV path planning. This article provides an overview of obstacle avoidance and path planning techniques for UAVs based on the artificial potential field method (APF method). This article begins with the explaining the principles of artificial potential field on this basis discusses its advantages and limitations. The article then summarizes the improvement strategies proposed by previous researchers to address issues like local minimum values and unreachable targets, such as introducing a new repulsive potential energy function, combining APF with other planning methods, and utilizing flow functions. Furthermore, it presents examples of the application and the performance of usage of these techniques in both static and dynamic environments. Based on this, the prospects and developing trend of UAV obstacle avoidance methods based on artificial potential field are foresee, such as combined with DRL and deep learning.

Group animation technology is widely used in the fields of virtual reality, intelligent simulation and game development, the core of which lies in simulating the autonomous movement and interaction behaviour of large-scale intelligent bodies in complex environments. However, there is still room for optimising the motion coordination and environment adaptability of intelligent bodies. In order to improve the performance of group animation in complex environments, a group animation algorithm combining the firefly algorithm and the leader-follower mechanism is designed. By combining global optimisation and local cooperative control, the path planning and motion control of the intelligent body are optimised. The experimental results show that in a small-scale scene with a dataset size of 50, the path smoothness of the proposed algorithm under the study is 0.94, the average deviation of the intelligent body is reduced to 0.92m, and the growth rate of the path length is 8.30%. In a 50 × 25 smart body and obstacle environment, the algorithm’s path smoothness is improved to 0.91, the average deviation of the smart body is reduced to 1.36m, and the path growth rate is reduced to 10.7%. And the success rate of obstacle avoidance of the algorithm is still maintained at 0.92 when the size of the dataset is increased to 600.The study shows that the algorithm has better performance in path planning and obstacle avoidance in complex environments. The proposed algorithm provides an efficient, stable and adaptable solution for group animation control in virtual reality.

Greenhouse climate control is essential for optimizing crop growth while minimizing resource consumption in controlled environment agriculture. Traditional rule-based and fixed-action strategies often struggle to achieve a balance between these objectives. This paper proposes a reinforcement learning (RL) based framework for greenhouse climate control, integrating deep learning models to predict both crop growth and resource consumption. The framework enables an RL agent to optimize greenhouse control setpoints dynamically, maximizing crop yield while ensuring sustainable resource usage. The proposed system incorporates a Multi-Layer Perceptron (MLP) model to predict internal greenhouse climate conditions, a Long Short-Term Memory (LSTM) model for crop parameter estimation, and a separate LSTM model for forecasting daily resource consumption. These models collectively simulate a greenhouse environment where an RL agent learns to regulate temperature, CO2 concentration, and irrigation levels by interacting with the virtual environment. A custom reward function is designed to guide the agent, considering key crop parameters; stem elongation, stem thickness, and cumulative trusses; alongside resource consumption metrics, including heating, electricity, CO2, and irrigation costs. To enhance the adaptability of the RL agent, a feature-selection mechanism identified the most influential climate and control features, reducing observation complexity and accelerating convergence. Retraining under stochastic weather conditions strengthened robustness to dynamic environments, enabling the agent to consistently outperform fixed-action strategies. Evaluation revealed a stable Pareto frontier between yield and resource consumption, confirming that the framework accurately captured the productivity and sustainability trade-off and remained robust across varying reward-weight settings. Comparative analysis of multiple RL algorithms; Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3) demonstrated that TD3 outperforms other algorithms, achieving the highest cumulative rewards and reaching optimal policies faster. Experimental evaluations demonstrate that the proposed TD3 RL-based greenhouse control system achieves higher crop yield growth rates while optimizing resource usage, outperforming conventional greenhouse control strategies. This study presents a novel data-driven, adaptive greenhouse management approach, bridging the gap between crop growth modeling and autonomous climate control, contributing to sustainable and intelligent agricultural practices.

To address the issues of low efficiency, insufficient coverage, and high energy consumption in wheat sowing path planning for large-scale irregular farmland, this study proposes an improved hybrid particle swarm optimization algorithm (TLG-PSO) for autonomous operational path planning. Building upon the standard PSO, the proposed method introduces a Tent chaotic mapping initialization mechanism, a Logistic-based dynamic inertia weight adjustment strategy, and adaptive Gaussian perturbation optimization to achieve precise control of the agricultural machinery’s driving orientation angle. A comprehensive path planning model is constructed with the objectives of minimizing the effective operation path length, reducing turning frequency, and maximizing coverage rate. Furthermore, cubic Bézier curves are employed for path smoothing, effectively controlling path curvature and ensuring the safety and stability of agricultural operations. The simulation experiment results demonstrate that the TLG-PSO algorithm achieved exceptional full-coverage operation performance across four categories of typical test fields. Compared to conventional fixed-direction path planning strategies, the algorithm reduced average total path length by 6228 m, improved coverage rate by 1.31%, achieved average labor savings of 96.32%, and decreased energy consumption by 6.45%. In large-scale comprehensive testing encompassing 1–27 field plots, the proposed algorithm reduced average total path length by 8472 m (a 5.45% decrease) and achieved average energy savings of 44.21 kW (a 5.48% reduction rate). Comparative experiments with mainstream intelligent optimization algorithms, including GA, ACO, PSO, BreedPSO, and SecPSO, revealed that TLG-PSO reduced path length by 0.16%–0.74% and decreased energy consumption by 0.53%–2.47%. It is worth noting that for large-scale field operations spanning hundreds of acres, even an approximately 1% path reduction translates to substantial fuel and operational time savings, which holds significant practical implications for large-scale agricultural production. Furthermore, TLG-PSO demonstrated exceptional performance in terms of algorithm convergence speed and computational efficiency. The improved TLG-PSO algorithm provides a feasible and efficient solution for autonomous operation of large-scale agricultural machinery.

In complex dynamic environments, robot path planning faces challenges in multi‐objective optimization, such as path length, smoothness and obstacle avoidance capability. To address this, this paper proposes an improved sparrow search algorithm based on chaotic initialization and the golden positive cosine strategy for multi‐objective path planning. Diverse initial populations are generated through chaotic mapping to enhance global search capability and avoid falling into local optima. The golden positive cosine strategy optimizes individual position updates to accelerate convergence and ensure path smoothness. Results demonstrate that the proposed method outperforms sparrow search algorithm (SSA), with improvements of 21.1% in path length, 16.3% in smoothness and 14.2% in obstacle avoidance capability. After achieving global path optimization, the improved dynamic window approach (IDWA) is employed for real‐time obstacle avoidance in dynamic environments, dynamically adjusting the window size based on robot speed, obstacle density and target distance to adaptively expand or shrink the search space, thereby improving obstacle avoidance flexibility and efficiency. Simulation results show that the proposed method surpasses SSA in terms of path length, smoothness, obstacle avoidance capability and computational efficiency in both static and dynamic environments.

Efficiency of manipulator path planning and obstacle avoidance are crucial technologies in the field of manipulator research. Conventional manipulator path planning algorithms have deficiencies such as low efficiency and unstable planned paths. To address this, this paper proposes an improved joint space three-dimensional Astar algorithm with pre-planning strategy. Firstly, we present a pre-planning strategy to generate new start point and target point that can reduce computational load. Next, we conduct joint space three-dimensional Astar path planning, which enables us to circumvent a multitude of forward and inverse kinematics computations as well as singularities, thereby diminishing the amounts of high-dimensional calculations. In addition to enabling the manipulator end-effector to avoid obstacles, we also achieved the obstacle avoidance for manipulator link. In simulation and experiments, the effectiveness of proposed path planning algorithm is validated and in contrast to the traditional Astar, RRT, and Dijkstra algorithms, the path generated exhibits a stronger tendency towards optimality.

Efficiency in manipulator path planning and obstacle avoidance represents a crucial technological challenge in manipulator research. Traditional path planning methods have certain limitations when dealing with the obstacle avoidance for links of manipulator, such as large calculational amount or unstable planned paths. To address this, this paper presents an improved joint space Astar algorithm for a 6 degree of freedom (DOF) manipulator with a pre-planning strategy. Firstly, we propose a pre-planning strategy that can refresh start positions and target positions to reduce computational load. Next, we perform a three-dimensional Astar path planning method in joint space, which avoids numerous inverse kinematics computations and singularities, and reduces the amount of high-dimensional calculations. In addition to the obstacle avoidance for manipulator end-effector, we also realize the obstacle avoidance for manipulator link. In simulation and experiments, the effectiveness of the proposed path planning algorithm is validated and the path tends to be optimal compared to the conventional Astar, RRT and Dijkstra algorithms.

Given a map and a target position, a robot must perform path planning in order to determine the best route from its current position to the target position. Three algorithms for path planning are presented: Dijkstra’s shortest path algorithm for a grid of cells, an algorithm for continuous maps and the A\(^{*}\) algorithm, an improvement of Dijkstra’s algorithm that uses heuristic functions. The chapter concludes with a description of the integration of path planning with obstacle avoidance.

Robot path planning is a cornerstone of precision agriculture, enabling safe and efficient operations for agricultural robots. However, complex field environments—characterized by static and dynamic obstacles, dense vegetation, and unstructured terrain—pose significant challenges to effective path planning. Conventional methods, such as A*, Dijkstra, and rapidly exploring random tree (RRT), exhibit limitations in efficiency and adaptability to dynamic conditions. To address these challenges, this study introduces AgriPath, a robust multi-objective path planning framework that integrates an improved convolutional neural network (CNN), an improved A* algorithm, and an improved whale optimization algorithm (IWOA) to optimize pathfinding, convergence efficiency, and obstacle avoidance in complex agricultural settings. Key innovations include an improved CNN leveraging causal convolution and multi-head self-attention mechanisms to improve temporal modeling for short-term trajectory prediction, augmented by Gaussian perturbations to enhance initial solution diversity; an improved A* algorithm incorporating dynamic heuristic functions based on Normalized Difference Vegetation Index (NDVI), combined with Kalman filtering, to bolster global path adaptability; IWOA employing non-linear convergence factors and differential evolution mechanisms to dynamically balance path length, smoothness, and planning time; and an improved Douglas–Peucker algorithm paired with cubic B-spline smoothing and navigation command modules to ensure path simplification and real-time execution. Experiments conducted in the Modern Agricultural Demonstration Zone at Chengdu, Sichuan Province, China, across simple, moderate, and complex scenarios, demonstrate that AgriPath outperforms advanced algorithms—SBREA*, Ant Colony A*, Orchard A*, and Greedy A*—in path length, smoothness, planning time, and dynamic obstacle avoidance success rate, indicative of superior multi-objective optimization balance. This study significantly enhances the efficiency and robustness of agricultural robot path planning, offering a more adaptive solution for autonomous navigation in precision agriculture while providing new theoretical and practical directions for the field of path planning.

Among the shortcomings of the A* algorithm, for example, there are many search nodes in path planning, and the calculation time is long. This article proposes a three-neighbor search A* algorithm combined with artificial potential fields to optimize the path planning problem of mobile robots. The algorithm integrates and improves the partial artificial potential field and the A* algorithm to address irregular obstacles in the forward direction. The artificial potential field guides the mobile robot to move forward quickly. The A* algorithm of the three-neighbor search method performs accurate obstacle avoidance. The current pose vector of the mobile robot is constructed during obstacle avoidance, the search range is narrowed to less than three neighbors, and repeated searches are avoided. In the matrix laboratory environment, grid maps with different obstacle ratios are compared with the A* algorithm. The experimental results show that the proposed improved algorithm avoids concave obstacle traps and shortens the path length, thus reducing the search time and the number of search nodes. The average path length is shortened by 5.58%, the path search time is shortened by 77.05%, and the number of path nodes is reduced by 88.85%. The experimental results fully show that the improved A* algorithm is effective and feasible and can provide optimal results.