Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments. As a computational approach to learning through interaction with the environment, reinforcement learning algorithms have been widely used for intelligent robot control, especially in the field of autonomous mobile robots. However, the learning process is slow and cumbersome. For practical applications, rapid rates of convergence are required. Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning, a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments. The state chain is built during the searching process. After one action is chosen and the reward is received, the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning. With the increasing number of Q-values updated after one action, the number of actual steps for convergence decreases and thus, the learning time decreases, where a step is a state transition. Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments. The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time, compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.

Dynamic Window Approach (DWA) is widely used in the local path planning of mobile robot. DWA algorithm has the advantages of simple planning algorithm and good realtime performance. However, the DWA exists unreachable target point, unreasonable path planning in the dense obstacles and it cannot ensure both speed and security. Therefore, an online parameter self-adaptive DWA algorithm is proposed to automatically adjust the weight of the objective function on the basis of the size of alternative velocity space and the distance between the mobile robot to the target point. By adapting to the dynamic changes of the environment, the optimal running speed and reasonable path of mobile robot can be obtained. The simulation verified that the mobile robot can pass through the dense obstacles area and reach the target point safely and reliably. And the problems that the robot might move around the dense obstacles and result in an unsmooth path are avoided. and the operational efficiency of the mobile robot is ensured simultaneously.

To address the critical limitations of conventional Grey Wolf Optimization (GWO) in path planning scenarios—including insufficient exploration capability during the initial phase, proneness to local optima entrapment, and inherent deficiency in dynamic obstacle avoidance—this paper proposes a multi-strategy enhanced GWO algorithm. Firstly, the Piecewise chaotic mapping is applied to initialize the Grey Wolf population, enhancing the initial population quality. Secondly, the linear convergence factor is modified to a nonlinear one to balance the algorithm’s global and local search capabilities. Thirdly, Evolutionary Population Dynamics (EPD) is incorporated to enhance the algorithm’s ability to escape local optima, and dynamic weights are used to improve convergence speed and accuracy. Finally, the algorithm is integrated with the Improved Dynamic Window Approach (IDWA) to enhance path smoothness and perform dynamic obstacle avoidance. The proposed algorithm is named PAGWO-IDWA. The results demonstrate that, compared to traditional GWO, PAGWO-IDWA reduces the path length, number of turns, and running time by 9.58%, 33.16%, and 30.31%, respectively. PAGWO-IDWA not only overcomes the limitations of traditional GWO but also enables effective path planning in dynamic environments, generating paths that are both safe and smooth, thus validating the effectiveness of the algorithm.

PurposeThe purpose of this paper is to solve the problem that mobile robots are still based on reactive collision avoidance in unknown dynamic environments leading to a lack of interaction with obstacles and limiting the comprehensive performance of mobile robots. A dynamic window approach with multiple interaction strategies (DWA-MIS) is proposed to solve this problem.Design/methodology/approachThe algorithm firstly classifies the moving obstacle movement intention, based on which a rule function is designed to incorporate positive incentives to motivate the robot to make correct avoidance actions. Then, the evaluation mechanism is improved by considering the time cost and future information of the environment to increase the motion states. Finally, the optimal objective function is designed based on genetic algorithm to adapt to different environments with time-varying multiparameter optimization.FindingsFaced with obstacles in different states, the mobile robot can choose a suitable interaction strategy, which solves the limitations of the original DWA evaluation function and avoids the defects of reactive collision avoidance. Simulation results show that the algorithm can efficiently adapt to unknown dynamic environments, has less path length and iterations and has a high comprehensive performance.Originality/valueA DWA-MIS is proposed, which increases the interaction capability between mobile robots and obstacles by improving the evaluation function mechanism and broadens the navigation strategy of DWA at a lower computational cost. After real machine verification, the algorithm has a high comprehensive performance based on real environment and provides a new idea for local path planning methods.

Foresighted navigation is an essential skill for robots to rise from rigid factory floor installations to much more versatile mobile robots that partake in our everyday environment. The current state of the art that provides this mobility to some extent is the Dynamic Window Approach combined with a global start-to-target path planner. However, neither the Dynamic Window Approach nor the path planner are equipped to predict the motion of other objects in the environment. We propose a change in the Dynamic Window Approach—a dynamic collision model—that is capable of predicting future collisions with the environment by also taking into account the motion of other objects. We show in simulated experiments that our new way of computing the Dynamic Window Approach significantly reduces the number of collisions in a dynamic setting with nonholonomic vehicles while still being computationally efficient.

Multi-robot path planning faces challenges such as conflict avoidance, collaboration, and dynamic environments. This paper proposes a multi-robot path planning algorithm that integrates the improved nutcracker optimization algorithm with the improved dynamic window approach. To address the nutcracker algorithm’s sensitivity to initial conditions and slow convergence, a population initialization strategy is introduced for more diverse initial populations. Additionally, a simplified path node strategy is also designed to shorten paths and reduce steering times. By incorporating a dynamic inertia weight factor , the balance between global exploration and local optimization is improved. To address the limitations of the dynamic window approach, which is unable to avoid dynamic obstacles instantly and is prone to falling into local optimal solutions, the target distance subfunction, the path evaluation subfunction and the deviation from danger zone subfunction are added to the evaluation function. Finally, the two algorithms were fused together and we conducted four experiments to validate the performance of the MANOA, IDWA, and MANOA-IDWA algorithms, as well as the application of MANOA-IDWA in multi-robot path planning. Results show that MANOA-IDWA significantly increases path planning success rates in dynamic environments, producing shorter and smoother paths, thus enhancing the safety and stability of multi-robot operations.

Autonomous service robots are gaining popularity as they find application in different sectors such as logistics, hospitality, medical, space exploration and military. It is widely seen that autonomous service robots help increase efficiency, reduce cost and ensure safer operations. Automating material handling operations in highly dynamic indoor environments is risky and involves use of complex algorithms. However, continuously advancing technologies and machine programming tend to overcome these challenges. The three essential functions of an autonomous robot are to identify its instantaneous location in three-dimensional space, generate the environment map and decide an optimal path for locomotion. Simultaneous Localization and Mapping (SLAM) is a widely used algorithm for navigation in unfamiliar environments. This chapter presents a cost-effective autonomous ground robot for automating material handling, transportation and other repetitive tasks in dynamically changing surroundings. The robot is programmed to map the indoor environment and localize itself in the workspace with the help of SLAM and Adaptive Monte Carlo Localization (AMCL). In addition, the robot can manoeuvre through the indoor workspace by avoiding static and dynamic obstacles. This work employs the GMapping open-source package in the Robot Operating System (ROS) for performing SLAM with the aid of a laser range finder, wheel odometry and inertial data. The GMapping algorithm uses a probabilistic approach to construct a map and estimate the robot’s location in two-dimensional space. In addition, this chapter presents an autonomous navigation feature with the help of the AMCL method for localizing the robot within an uncertain environment and making the robot move to its goal using move base actions in ROS. The path planning of the mobile robot is classified into global and local plans. The global plan is responsible for planning the path from the start point to the endpoint. The local plan is responsible for planning the actual movement of the robot by avoiding any dynamic obstacles not mapped earlier. This work uses the Dijistra algorithm for the global and the dynamic window approach (DWA) algorithm for the local path planning. The optimization algorithm minimizes the error between the global and local paths. The simulation study of the proposed algorithm is performed in a virtual environment called Gazebo in ROS, which is nearly similar to a real-time indoor workspace. Further, a robot prototype is fabricated, and its performance is evaluated in an unknown environment. The robot prototype is equipped with brushed DC motors with optical encoder feedback for wheel odometry (ODOM), Light Detection and Ranging (LiDAR), and Inertial Measurement Unit (IMU) sensors controlled using a Jetson Nano. The simulation and experimental results showcase the robot’s successful two-dimensional localization and navigation in the factory environment. The current system has the potential for indoor assistance during material transfer and logistic applications. Further upgrades to the system can make it usable for Automated Storage and Retrieval System (ASRS) applications in warehouses/factories.

A new complete coverage path planning algorithm based on rolling path planning, dynamic heuristic searching and directly filling is proposed to improve efficiency of path planning for mobile robot in unknown environments. The biological inspired neural network was used to model the environment of mobile robot. After the characteristic of local environment around robot was identified on line, the dynamic heuristic planning method combing static searching and following the boundary of obstacle is applied and generated path is much shorter and less turning. Using virtual obstacle and directly filling algorithm, the continuously covered area in U shape obstacle is obtained in unknown environments. The effectiveness of the proposed algorithm is validated by simulation in unknown complicated environments.

This paper proposes a local path planning method based on virtual manipulators and dynamic window approach (VMDWA) for a wheeled mobile robot. In the conventional researches such as the dynamic window approach (DWA), the mobile robot follows the desired path and avoids obstacles in the dynamic and static environment. However, DWA calculates the predicted path assuming the constant velocities. From these predicted paths, the best path is selected using the cost function. Since the DWA assumes the constant value of velocities, multiple candidates of paths cannot generate flexible movements. Therefore, it may not be possible to generate the path that does not collide with obstacles. To solve these problems, this paper proposes the local path planning method based on virtual manipulators and dynamic window approach (VMDWA). In the proposed method, virtual manipulators are taken into account at the calculation of the path planning based on DWA. Therefore, it is possible to use the variable velocities values for the predictive paths. VMDWA selects the optimal path from the several calculations of predictive paths. The simulation results confirmed the effectiveness of the VMDWA.

The study of path-planning algorithms is crucial for an electric logistics vehicle to reach its target point quickly and safely. In light of this, this work suggests a novel path-planning technique based on the improved A-star (A*) fusion dynamic window approach (DWA). First, compared to the A* algorithm, the upgraded A* algorithm not only avoids the obstruction border but also removes unnecessary nodes and minimizes turning angles. Then, the DWA algorithm is fused with the enhanced A* algorithm to achieve dynamic obstacle avoidance. In addition to RVIZ of ROS, MATLAB simulates and verifies the upgraded A* algorithm and the A* fused DWA. The MATLAB simulation results demonstrate that the approach based on the enhanced A* algorithm combined with DWA not only shortens the path by 4.56% when compared to the A* algorithm but also smooths the path and has dynamic obstacle-avoidance capabilities. The path length is cut by 8.99% and the search time is cut by 16.26% when compared to the DWA. The findings demonstrate that the enhanced method in this study successfully addresses the issues that the A* algorithm’s path is not smooth, dynamic obstacle avoidance cannot be performed, and DWA cannot be both globally optimal.

This article proposes, a novel obstacle avoidance algorithm for a mobile robot based on finite memory filtering (FMF) in unknown dynamic environments. To overcome the limitations of the existing dynamic window approach (DWA), we propose a new version of the DWA, called the finite distribution estimation-based dynamic window approach (FDEDWA), which is an algorithm that avoids dynamic obstacles through estimating the overall distribution of obstacles. FDEDWA estimates the distribution of obstacles through the FMF, and predicts the future distribution of obstacles. The FMF is derived to minimize the effect of the measurement noise through the Frobenius norm, and covariance matrix adaptation evolution strategy. The estimated information is used to derive the control input for the robust mobile robot navigation effectively. FDEDWA allows for the fast perception of the dynamic environment, and superior estimation performance, and the mobile robot can be controlled by a more optimal path while maintaining real-time performance. To demonstrate the performance of the proposed algorithm, simulations, and experiments were carried out under dynamic environments by comparing the latest dynamic window for dynamic obstacle, and the existing DWA.

Many applications in mobile robotics require the safe execution of a real-time motion to a goal location through completely unknown environments. In this context, the dynamic window approach (DWA) is a well-known solution which is safe by construction —assuming reliable sensory information— and has shown to perform very efficiently in many experimental setups. Nevertheless, the approach is not free of shortcomings. Examples where DWA fails to attain the goal configuration due to the local minima problem can be easily found. This limitation, however, has been overcome by many researches following a common framework which essentially provides the strategy with a deliberative layer. Based on a model of the environment, the deliberative layer of these approaches computes the shortest collision-free path to the goal point being, afterwards, this path followed by DWA. In unknown environments, nevertheless, such a model is not initially available and has to be progressively built by means of the local information supplied by the robot sensors. Under these circumstances, the path obtained by the deliberative layer may repeatedly and radically change during navigation due to the model updates, which usually results in high-suboptimal final trajectories. This paper proposes an extension to DWA without the local minima problem that is able to produce reasonable good paths in unknown scenarios with a minimal computational cost. The convergence of the proposed strategy is proven from a geometric point of view.

In complex dynamic environments, robot path planning faces challenges in multi‐objective optimization, such as path length, smoothness and obstacle avoidance capability. To address this, this paper proposes an improved sparrow search algorithm based on chaotic initialization and the golden positive cosine strategy for multi‐objective path planning. Diverse initial populations are generated through chaotic mapping to enhance global search capability and avoid falling into local optima. The golden positive cosine strategy optimizes individual position updates to accelerate convergence and ensure path smoothness. Results demonstrate that the proposed method outperforms sparrow search algorithm (SSA), with improvements of 21.1% in path length, 16.3% in smoothness and 14.2% in obstacle avoidance capability. After achieving global path optimization, the improved dynamic window approach (IDWA) is employed for real‐time obstacle avoidance in dynamic environments, dynamically adjusting the window size based on robot speed, obstacle density and target distance to adaptively expand or shrink the search space, thereby improving obstacle avoidance flexibility and efficiency. Simulation results show that the proposed method surpasses SSA in terms of path length, smoothness, obstacle avoidance capability and computational efficiency in both static and dynamic environments.

The traditional Rapidly-exploring Random Tree Star (RRT*) suffers from the low path generation efficiency, numerous invalid exploration points, and unsuitability for navigation in unknown dynamic environments. In this article, we propose a dynamic path planning scheme by combining the improved RRT* and the improved dynamic window approach (DWA). For pregenerating an initial path, we use the artificial potential field (APF) method to expand new nodes. The adaptive dynamic step-size is introduced for accelerating the optimal path searching. Moreover, the improved ant colony algorithm is used to perform multiobjective optimization on the generated initial path. When unknown obstacles appear in the path, the improved DWA is developed for obstacle avoidance. Finally, the proposed method is validated by simulation and experiment in both of the static and dynamic environments. In particular, the simulation results show that, compared with some existing methods, our algorithm can generate a higher-quality initial path in the static environment and avoid unknown dynamic obstacles effectively in the dynamic environment. Furthermore, we implement our algorithm in a mobile robot to verify the correctness and effectiveness in the practical scenario.

To address issues such as insufficient real-time performance, poor obstacle avoidance flexibility, and slow emergency stop response in path planning for transport robots operating within complex, dynamic hospital environments, this paper proposes a hybrid path planning strategy that integrates an improved A* algorithm with the Dynamic Window Algorithm (DWA). This strategy is implemented within the ROS2 framework. First, the traditional $\mathbf{A}^{*}$ algorithm is optimized by introducing a dynamic weighted Manhattan distance <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</sup> as the heuristic function. This incorporates path smoothness and safety distance penalty terms, enhancing the rationality and efficiency of the global path. Second, at the local planning layer, a multi-modal perception fusion dynamic window algorithm is designed. This method dynamically embeds real-time evaluations of LiDAR, visual semantic information, and UWB positioning data into the evaluation function, enabling adaptive obstacle avoidance for diverse obstacles such as pedestrian flows, IV stands, and mobile hospital beds. Experiments demonstrate that the improved algorithm reduces planning time by approximately 35% while maintaining near-optimal path length. Emergency stop response time is $\boldsymbol{\leqslant} 180 \mathrm{~ms}$, and dynamic obstacle avoidance success rate increases to 98.7%, effectively supporting high-frequency, high-priority medication transport tasks in medical settings.