Research on Q-ELM algorithm in robot path planning

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments. As a computational approach to learning through interaction with the environment, reinforcement learning algorithms have been widely used for intelligent robot control, especially in the field of autonomous mobile robots. However, the learning process is slow and cumbersome. For practical applications, rapid rates of convergence are required. Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning, a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments. The state chain is built during the searching process. After one action is chosen and the reward is received, the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning. With the increasing number of Q-values updated after one action, the number of actual steps for convergence decreases and thus, the learning time decreases, where a step is a state transition. Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments. The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time, compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.

An improved TD3 algorithm is studied for the low success rate and slow learning speed of TD3(Twin Delayed Deep Deterministic Policy Gradients) algorithm in mobile robot path planning. Prioritized experience replay is added and dynamic delay update strategy is designed which can reduce the impact of value estimation errors to improve the success rate of path planning, and reduce training time. Based on the ROS melodic operating system and the Gazebo simulation software, the Turtlebot3 robot model and simulation experimental environment are established. The effectiveness of the improved TD3 algorithm is verified by comparative experiments. For the path planning task of the mobile robot, the simulation results show that compared with TD3 algorithm, the success rate of the improved TD3 algorithm is increased by 15.6%, and the training navigation time is shortened by 20%. It indicates that the improved TD3 algorithm achieves better performance in the path planning of mobile robot with continuous action space.

Background: Path planning technology has a wide range of applications in many fields. Applications in the field of high technology include: autonomous and touchless action of robots; obstacle avoidance and defense flight of drones; cruise missiles to avoid radar search, anti- bouncing attacks, and to complete the task of sudden defense and demolition. Applications in daily life include: GPS navigation; road planning based on GIS system; urban road network planning and navigation. Applications in the field of decision-making management include: vehicle problem (VRP) in logistics management and similar resource allocation problems in resource management. Routing problems in the field of communication technology. Any planning problem that can be topologized as a point and line network can basically be solved by the method of path planning. Different intelligent algorithms have different characteristics, and their application scope and fields are also different, so it is of great significance for the development of path planning technology to study the path planning intelligent algorithms from the characteristics of the algorithms themselves and their applications. Objective: Analyze the advantages and disadvantages of various types of planning algorithms, look forward to the future development trend of mobile robot path planning, and provide certain ideas for the research of mobile robot path planning. Methods: Search journals, patents, conferences, and papers related to mobile robot path planning, and summarize and analyze the advantages and disadvantages of various planning algorithms. Results: Based on the research results of many scholars, this study summarizes different mobile robot path planning methods. It is divided into four categories: traditional planning, intelligent search, artificial intelligence and local obstacle avoidance. This paper introduces and analyzes the latest research results of these types, including their design ideas, advantages and disadvantages, and improvement measures. The research methods adopted are analyzed in order to maximize the advantages of each algorithm and expand the application field of robot path planning to provide ideas and references. Conclusion: This paper provides guidance for the design and optimization of robot path planning. Finally, this paper summarizes the future development trend of robot path planning, and looks forward to the future development trend and key areas of robot path planning.

Mobile robots, as a new type of automated equipment, are widely utilized in intelligent manufacturing production lines. This study focuses on addressing issues such as slow convergence speed, low solution accuracy, and susceptibility to local optima in the traditional whale optimization algorithm (WOA) for mobile robot path planning in intelligent manufacturing production lines. This paper presents a multi-population WOA method that integrates orthogonal learning. This algorithm effectively avoids the impact of local optima through orthogonal learning-based information exchange, enhances convergence speed through reverse learning strategies, and improves solution accuracy via nonlinear convergence factors and adaptive weighting strategies. Simulation results demonstrate significant advantages in search performance for path planning with the improved algorithm, validating its effectiveness and superiority in enhancing intelligent manufacturing efficiency. This research offers a more efficient and accurate solution to the path planning problem, which holds great significance in enhancing the efficiency and accuracy of path planning for mobile robots.

Aiming at the problem of robot path planning in complex maps, an algorithm of robot path planning based on triangular grid graph is proposed. Firstly, a weighted undirected loop graph and a feasible domain of nodes are obtained by discretizing the triangular mesh map. Next, the Dijkstra search algorithm is applied to find the feasible shortest path from an initial to a final configuration. Finally, The Douglas-Peucker algorithm is applied to remove duplicate and redundant nodes in the feasible path, and the waypoint are extracted. The final path is a curve that is obtained by connecting the several extracted waypoint. The proposed algorithm is tested for various maps. Compared with probabilistic roadmap method, the experimental results show that the proposed method can overcome the shortcomings of the random sampling method. Furthermore, the experimental result of triangular mesh map method used in two labyrinth maps show that the triangular mesh map method can solve the robot path planning problem in complex map very well, and it is an excellent algorithm for robot path planning.

Recently, genetic algorithms (GAs) have attracted great interest owing to efficiency and flexibility against complex robot path planning problems. To accelerate the convergence speed, preceding researches adapted conventional GAs by using problem-specific techniques. However, such approaches increase computational burden and algorithmic complexity, resulting in subsequent additional problems. In this paper, we used Population-Based Incremental Learning (PBIL) algorithm for robot path planning as a probabilistic evolutionary approach. In addition to PBIL, we also proposed the probabilistic model of nodes and the edge bank to generate promising paths. The experimental results demonstrate that the proposed method gave markedly better performance than its conventional counter-parts(GA,kGA,fGA) in terms of success rates and the quality of obtained paths.

Path planning is an important area of mobile robot research, and the ant colony optimization algorithm is essential for analyzing path planning. However, the current ant colony optimization algorithm applied to the path planning of mobile robots still has some limitations, including early blind search, slow convergence speed, and more turns. To overcome these problems, an improved ant colony optimization algorithm is proposed in this paper. In the improved algorithm, we introduce the idea of triangle inequality and a pseudo-random state transfer strategy to enhance the guidance of target points and improve the search efficiency and quality of the algorithm. In addition, we propose a pheromone update strategy based on the partition method with upper and lower limits on the pheromone concentration. This can not only improve the global search capability and convergence speed of the algorithm but also avoid the premature and stagnation phenomenon of the algorithm during the search. To prevent the ants from getting into a deadlock state, we introduce a backtracking mechanism to enable the ants to explore the solution space better. Finally, to verify the effectiveness of the proposed algorithm, the algorithm is compared with 11 existing methods for solving the robot path planning problem, including several ACO variants and two commonly used algorithms (A* algorithm and Dijkstra algorithm), and the experimental results show that the improved ACO algorithm can plan paths with faster convergence, shorter path lengths, and higher smoothness. Specifically, the algorithm produces the shortest path length with a standard deviation of zero while ensuring the most rapid convergence and the highest smoothness in the case of the shortest path in four different grid environments. These experimental results demonstrate the effectiveness of the proposed algorithm in path planning.

Evolutionary algorithms (EA) have been used to solve path planning of mobile robots successfully. However, accuracy and convergence are always topics. The cloud model which transforms qualitative concept into quantitative description, and vice versa, is adept in uncertainty modeling. It can be used in EA for evolving in a uniform and natural way. An EA embedded with cloud model (EACM) for path planning of mobile robots was proposed in this paper. The algorithm adopted floating-point coding and realized fast converging. Simulation results verified the efficiency of our algorithm.

In order to solve the problems of slow convergence speed, insufficient accuracy, and easily falling into the local optimum of the traditional whale optimization algorithm (WOA) in mobile robot path planning, an improved whale optimization algorithm (IWOA) combined with the bird navigation mechanism was proposed. Specific improvement measures include using logical chaos mapping to initialize the population to enhance the randomness and diversity of the initial solution, designing a nonlinear convergence factor to prevent the algorithm from prematurely entering the shrinking surround phase and extending the global search time, introducing an adaptive spiral shape constant to dynamically adjust the search range to balance exploration and development capabilities, optimizing the individual update strategy in combination with the bird navigation mechanism, and optimizing the algorithm through companion position information, thereby improving the stability and convergence speed of the algorithm. Path planning simulations were performed on 30 × 30 and 50 × 50 grid maps. The results show that compared with WOA, MSWOA, and GA, in the 30 × 30 map, the path length of IWOA is shortened by 3.23%, 7.16%, and 6.49%, respectively; in the 50 × 50 map, the path length is shortened by 4.88%, 4.53%, and 28.37%, respectively. This study shows that IWOA has significant advantages in the accuracy and efficiency of path planning, which verifies its feasibility and superiority.

With the advancement of automation technology, swarm intelligence algorithms are becoming increasingly crucial for mobile robot path planning. Therefore, an improved sparrow search algorithm (CSWSSA) is proposed to address the shortcomings of swarm intelligence algorithms in path planning, such as long planning time and suboptimal planned paths. Firstly, Utilizing Elite SPM Mapping to initialize CSWSSA to improve the individual of sparrow quality and diversity. Secondly, an improved sine cosine algorithm was proposed to increase the ability to search maps and balance the ability of extensive and fine search. Then, a tracking strategy and an adaptive T-distribution strategy are adopted at different position update points to further avoid the algorithm plunging into local minimum. Finally, in order to avoid sparrow individuals exceeding their boundaries, a boundary redistribution mechanism was designed. To test the optimization and optimization ability of the proposed CSWSSA for various functions, some extensive validations are performed on publicly available test functions. Furthermore, we conducted experimental verification on both simulated and real maps, and compared our proposed method with some widely used algorithms, and the results unanimously demonstrate the superiority of the proposed CSWSSA. Compared with other methods, the optimal path length and the average path length are shortened by 18.8% and 15.4% respectively. Moreover, CSWSSA has the best stability, with a value of 1.0255. The research results of this project will provide new ideas for path planning of intelligent optimization algorithms.

This paper studies the obstacle avoidance and navigation of mobile robots in unstructured environments. Because of the shortcomings of one single sensor, the system integrates the stereo vision sensor blumblebee2 with the lidar sensor to detect the information surrounding the mobile robot’s surroundings. And then through data fusion to get a more complete, more accurate scene distribution. Then use the improved system of ant colony optimization to increase the convergence speed and precision of the algorithm in robot path planning. Finally, the simulation experiment is carried out in the environment of Matlab and Visual Studio, and the physical experiment is carried out under the ROS platform. The experimental results show the feasibility and effectiveness of the proposed method.

According to the realistic duty of robot, it's necessary to find a short path to avoid obstacles and carry limited goods, and then the added constrains let the problem is more complex and more practical. The paper proposes a tabu-genetic algorithm for robot path planning. By utilizing the main frame of parallel search supplied by genetic algorithm and embed the individual serial search mode of tabu search algorithm, this method can enlarge search space. And in order to enhance optimized speed and quality, also improve the algorithm performance, fitness function and amendatory policy are designed. Besides, algorithm convergence is analyzed based on the model of Markov chains. The results of simulations show that the method proposed in this paper, which can find an optimal path quickly and improve convergence speed and solution quality greatly.

Many function-approaching methods such as neural network, fuzzy method are used in reinforcement learning methods for solving its huge problem space dimensions. This paper presents a novel ART2 neural network based reinforcement learning method (ART2-RL) to solve the space problem. Because of its adaptive resonance characteristic, ART2 neural network is used to process the space measurement of reinforcement learning and improve the learning speed. This paper also gives the reinforcement learning algorithm based on ART2. A simulation of path planning of mobile robot has been developed to prove the validity of ART2-RL. As the complexity of the simulation increased, the result shows that the number of collision between robot and obstacles is effectively decreased; the novel neural network model provides significant improvement in the space measurement of reinforcement learning.

Based on the artificial potential field algorithm and bacterial foraging algorithm, research method for obstacle avoidance and path planning of the mobile robot. Firstly, analysis method of bacterial foraging three operations: trends, replication, migration, establishment of bacterial foraging algorithm process; secondly establish the bacterial foraging algorithm model; and then analyze the artificial potential field algorithm and bacterial foraging the advantage and disadvantage of the algorithm, proposed a new hybrid algorithm has the global search ability and fast convergence. Using artificial potential field and bacterial foraging method on mobile robot global path planning and then compared with the mixture algorithm, based on the simulation results, the global path planning of mobile robot Simulation results show the feasibility and effectiveness of the hybrid algorithm.

For the issues of the ant colony algorithm (ACO) to solving the problems in mobile robot path planning, such as the slow optimization speed and the redundant paths in planning results, a high-precision improved ant colony algorithm (IPACO) with fast optimization and compound prediction mechanism is proposed. Firstly, aiming at maximizing the possibility of optimal node selection in the process of path planning, a composite optimal node prediction model is introduced to improve the state transition function. Secondly, a pheromone model with initialize the distribution and “reward or punishment” update mechanism is used to updates the global pheromone concentration directionally, which increases the pheromone concentration of excellent path nodes and the heuristic effect; Finally, a prediction-backward mechanism to deal with the “deadlock” problem in the ant colony search process is adopted in the IPACO algorithm, which enhance the success rate in the ACO algorithm path planning. Five groups of different environments are selected to compare and verify the performance of IPACO algorithm, ACO algorithm and three typical path planning algorithms. The experimental simulation results show that, compared with the ACO algorithm, the convergence speed and the planning path accuracy of the IPACO algorithm are improved by 57.69% and 12.86% respectively, and the convergence speed and the planning path accuracy are significantly improved; the optimal path length, optimization speed and stability of the IPACO algorithm are improved. Which verifies that the IPACO algorithm can effectively improve the environmental compatibility and stability of the ant colony algorithm path planning, and the effect is significantly improved.