Robot Path Planning Based on Improved Reinforcement Learning

This paper presents a path planning algorithm for autonomous navigation of nonholonomic mobile robots in complex environment. The irregular contour of obstacles is represented by segments. The goal of the robot is to move towards a known target while avoiding obstacles. The velocity constraints, kinematic robot model and nonholonomic constraint are considered in the problem. The optimal path planning problem is formulated as a constrained receding horizon planning problem and the trajectory is obtained by solving an optimal control problem with constraints. Local minima are avoided by choosing intermediate objectives based on the real time environment.

In this paper, we propose a new path planning algorithm based on the probabilistic roadmaps method (PRM), in order to effectively solve the autonomous path planning of mobile robots in complex environments with multiple narrow channels. The improved PRM algorithm mainly improves the density and distribution of sampling points in the narrow channel, through a combination of the learning process of the PRM algorithm and the APF algorithm. We also shortened the required time and path length by optimizing the query process. The first key technology to improve the PRM algorithm involves optimizing the number and distribution of free points and collision-free lines in the free workspace. To ensure full visibility of the narrow channel, we extend the obstacles through the diagonal distance of the mobile robot while ignoring the safety distance. Considering the safety distance during movement, we re-classify the all sampling points obtained by the quasi-random sampling principle into three categories: free points, obstacle points, and adjacent points. Next, we transform obstacle points into the free points of the narrow channel by combining the APF algorithm and the characteristics of the narrow channel, increasing the density of sampling points in the narrow space. Then, we include potential energy judgment into the construction process of collision-free lines shortening the required time and reduce collisions with obstacles. Optimizing the query process of the PRM algorithm is the second key technology. To reduce the required time in the query process, we adapt the bidirectional A* algorithm to query these local paths and obtain an effective path to the target point. We also combine the path pruning technology with the potential energy function to obtain a short path without collisions. Finally, the experimental results demonstrate that the new PRM path planning technology can improve the density of free points in narrow spaces and achieve an optimized, collision-free path in complex environments with multiple narrow channels.

Path planning is a fundamental task in autonomous mobile robot navigation and one of the most computationally intensive tasks. In this work, a parallel version of the bacterial potential field (BPF) method for path planning in mobile robots is presented. The BPF is a hybrid algorithm, which makes use of a bacterial evolutionary algorithm (BEA) with the artificial potential field (APF) method, to take advantage of intelligent and classical methods. The parallel bacterial potential field (parallel-BPF) algorithm is implemented on a graphics processing unit (GPU) to speed up the path planning computation in mobile robot navigation. Simulation results to validate the analysis and implementation are provided; the experiments were specially designed to show the effectiveness and the efficiency of the parallel-BPF algorithm.

<p align="left">Path planning is an essential task for the navigation of Autonomous Mobile Robot. This is one of the basic problems in robotics. Path planning algorithms are classified as global or local, depending on the knowledge of surrounding environment. In local path planning, the environment is unknown to the robot, and sensors are used to detect the obstacles and to avoid collision. Bug algorithms are one of the frequently used path planning algorithms where a mobile robot moves to the target by detecting the nearest obstacle and avoiding it with limited information about the environment. This proposed Critical-PointBug algorithm, is a new Bug algorithm for path planning of mobile robots. This algorithm tries to minimize traversal of obstacle border by searching few important points on the boundary of obstacle area as a rotation point to goal and end with a complete path from source to goal.</p>

Efficient navigation planning for mobile robots in complex environments is a challenging problem. In this paper we consider the path planning problem for mobile robots with adjustable relative wheel positions, which further increase the navigation capabilities. In particular we account for changes of these relative wheel positions during planning time, thus fully leveraging the capabilities of the robot. Whereas these additional degrees of freedom increase flexibility, they introduce a more challenging planning problem. The approach proposed in this paper is built upon a search-based planner. We describe how to flexibly integrate joint angle changes in the path planning process and furthermore propose a representation of the robot configuration that substantially reduces the computational burden. In addition, we introduce search guidance heuristics that are particularly useful in environments in which a robot is required to pass over obstacles, such as on agricultural fields. An extensive evaluation on simulated and real-world data with our BoniRob agricultural robot demonstrates the efficiency of our approach.

Background: Path planning technology has a wide range of applications in many fields. Applications in the field of high technology include: autonomous and touchless action of robots; obstacle avoidance and defense flight of drones; cruise missiles to avoid radar search, anti- bouncing attacks, and to complete the task of sudden defense and demolition. Applications in daily life include: GPS navigation; road planning based on GIS system; urban road network planning and navigation. Applications in the field of decision-making management include: vehicle problem (VRP) in logistics management and similar resource allocation problems in resource management. Routing problems in the field of communication technology. Any planning problem that can be topologized as a point and line network can basically be solved by the method of path planning. Different intelligent algorithms have different characteristics, and their application scope and fields are also different, so it is of great significance for the development of path planning technology to study the path planning intelligent algorithms from the characteristics of the algorithms themselves and their applications. Objective: Analyze the advantages and disadvantages of various types of planning algorithms, look forward to the future development trend of mobile robot path planning, and provide certain ideas for the research of mobile robot path planning. Methods: Search journals, patents, conferences, and papers related to mobile robot path planning, and summarize and analyze the advantages and disadvantages of various planning algorithms. Results: Based on the research results of many scholars, this study summarizes different mobile robot path planning methods. It is divided into four categories: traditional planning, intelligent search, artificial intelligence and local obstacle avoidance. This paper introduces and analyzes the latest research results of these types, including their design ideas, advantages and disadvantages, and improvement measures. The research methods adopted are analyzed in order to maximize the advantages of each algorithm and expand the application field of robot path planning to provide ideas and references. Conclusion: This paper provides guidance for the design and optimization of robot path planning. Finally, this paper summarizes the future development trend of robot path planning, and looks forward to the future development trend and key areas of robot path planning.

PurposeThe purpose of this paper is to put forward a rolling assistant robot with two rolling modes, and the multi-mode rolling motion strategy with path planning algorithm, which is suitable to this multi-mode mobile robot, is proposed based on chessboard-shaped grid division (CGD).Design/methodology/approachBased on the kinematic analysis and motion properties of the mobile parallel robot, the motion strategy based on CGD path planning algorithm of a mobile robot with two rolling modes moving to a target position is divided into two parts, which are local self-motion planning and global path planning. In the first part, the mobile parallel robot can move by switching and combining the two rolling modes; and in the second part, the specific algorithm of the global path planning is proposed according to the CGD of the moving ground.FindingsThe assistant robot, which is a novel 4-RSR mobile parallel robot (where R denotes a revolute joint and S denotes a spherical joint) integrating operation and rolling locomotion (Watt linkage rolling mode and 6R linkage rolling mode), can work as a moving spotlight or worktable. A series of simulation and prototype experiment results are presented to verify the CGD path planning strategy of the robot, and the performance of the path planning experiments in simulations and practices shows the validation of the path planning analysis.Originality/valueThe work presented in this paper is a further exploration to apply parallel mechanisms with two rolling modes to the field of assistant rolling robots by proposing the CGD path planning strategy. It is also a new attempt to use the specific path planning algorithm in the field of mobile robots for operating tasks.

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments. As a computational approach to learning through interaction with the environment, reinforcement learning algorithms have been widely used for intelligent robot control, especially in the field of autonomous mobile robots. However, the learning process is slow and cumbersome. For practical applications, rapid rates of convergence are required. Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning, a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments. The state chain is built during the searching process. After one action is chosen and the reward is received, the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning. With the increasing number of Q-values updated after one action, the number of actual steps for convergence decreases and thus, the learning time decreases, where a step is a state transition. Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments. The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time, compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.

-By using the two methods of Lyapunov exponent and power spectra analysis, it is tested and verified that Chaotic phenomenon exists in time sequence of distance information that autonomous mobile robot has got from sensors between robots and obstacles. The complexity of path planning for autonomous mobile robot through sensors is explained under dynamic environments by use of chaotic phenomenon. The existence of chaotic phenomenon makes the path planning for autonomous mobile robot under dynamic environments NP-hard and also lays the foundation for revealing why navigation of autonomous mobile robot under dynamic environments is NP-hard.

Aiming at disadvantages of particle swarm optimization in the path planning of mobile robots, such as low convergence accuracy and easy maturity, this paper proposes an improved particle swarm optimization algorithm based on differential evolution. First, the concept of corporate governance is introduced, adding adaptive adjustment weights and acceleration coefficients to improve the traditional particle swarm optimization and increase the algorithm convergence speed. Then, in order to improve the performance of the differential evolution algorithm, the size of the mutation is controlled by adding adaptive parameters. Moreover, a “high-intensity training” mode is developed to use the improved differential evolution algorithm to intensively train the global optimal position of the particle swarm optimization, which can improve the search precision of the algorithm. Finally, the mathematical model for robot path planning is devised as a two-objective optimization with two indices, i.e., the path length and the degree of danger to optimize the path planning. The proposed algorithm is applied to different experiments for path planning simulation tests. The results demonstrate the feasibility and effectiveness of it in solving a mobile robot path-planning problem.

Path planning is one of the most important and challenging problems of mobile robot. It is one of the keys that will make the mobile robots fully autonomous. In this paper, we summarized the application of soft computing approaches in path planning for mobile robot. Finally the future works of path planning for mobile robots are prospected.

The path of the mobile robot is a multi-objective optimization problem. The traditional ant colony algorithm is introduced into the path planning of the mobile robot. Due to the complexity of the robot path, the large search space and the low efficiency, it is easy to fall into the local optimum and even deadlock and other defeats. In order to enhance the efficiency and accuracy of path planning, a path planning algorithm of mobile robot based on ant colony algorithm is proposed. The path planning of mobile robot is simulated under grid environment. The experimental results show that the improved algorithm can improve the efficiency of path planning for mobile robot, safely avoid obstacles and find all the optimal solutions in the shortest time.

In order to make the robot obtain the optimal action directly from the original visual perception without any hand-crafted features and features matching, a novel end-to-end path planning method-mobile robot path planning using deep reinforcement learning is proposed. Firstly, a deep Q-network (DQN) is designed and trained to approximate the mobile robot state-action value function. Then, the Q value corresponding to each possible mobile robot action (i.e., turn left, turn right, forward) is determined by the well trained DQN, here, the input of the DQN is the original RGB image (image pixels) captured from the environment without any hand-crafted features and features matching; Finally, the current optimal mobile robot action is selected by the action selection strategy. Mobile robot reach to the goal point while avoiding obstacles ultimately. 30 times path planning experiments are conducted in the seekavoid_arena_01 environment on DeepMind Lab platform. The experimental results show that our deep reinforcement learning based robot path planning method is an effective end-to-end mobile robot path planning method.

The research of mobile robot has always been the focus of academia. In the specific academic content, robot path planning and general positioning technology are two of the more important academic research in mobile robot technology. Mobile planning technology is divided into global path planning and local environment path planning. This paper describes the current situation and development of the robot path planning. In addition, this paper also describes the research progress of mobile robot positioning technology based on map. Through the research of the above two technologies of mobile robot, we can get the corresponding conclusions.

Path planning for a mobile robot is a vital task for navigating in a complex environment. However, the path planning problem is challenging because of its non-deterministic polynomial-time (NP) character. In this research, the authors have proposed the Multi-objective Adaptive Ant Colony Optimization for path planning of mobile robot in a static object environment on the grid. The heuristic information function in the conventional ant colony optimization is modified according to the A* algorithm, which helps in mitigating the slow convergence of the traditional algorithm. It is because the ants will choose the nodes which are closer to the goal position. Additionally, an objective function is formulated as a multiple objective problem by incorporating (a) Path length, (b) Safety factor, and (c) Energy consumption. Simulation results show that the proposed modification helps to achieve the optimal path in a quicker time i.e., 1.3 times faster than the traditional counterpart.