- Research Article
10
- 10.1016/j.ast.2015.09.022
OARPER-MAFO algorithm for anti-ship missile path planning
- Sep 25, 2015
- Aerospace Science and Technology
- Gang Liu + 4 more +4
OARPER-MAFO algorithm for anti-ship missile path planning
Abstract This paper addresses multi-agent cooperative region exploration and path planning. For region exploration, a double probability matrix updating rule based on distance weight is proposed and its convergence is proved by us. Compared with traditional Bayes updating rules, this updating rule improves the convergence speed of the exploration probability matrix and is applicable to multiple environments, thus improving the robustness. Secondly, a multi-agent region exploration strategy based on distance weight is proposed. Compared with the traditional coverage control algorithm, this method improves the search efficiency, reduces the variance of the results thus improves the confidence of the data. For path planning, a multi-agent hierarchical Q-learning algorithm is proposed and we use a multi-agent collaborative process of region exploration and path planning to visualize the influence of region exploration’s result on path planning algorithm. The above algorithm is modeled and simulated based on the cooperative search and rescue task at sea, and the simulation results verify the above conclusions.KeywordsMulti-agentRegion explorationPath planningQ-learning
OARPER-MAFO algorithm for anti-ship missile path planning
OARPER-MAFO algorithm for anti-ship missile path planning
Deep Neural Network Enhanced Sampling-Based Path Planning in 3D Space
Robot path planning in 3D space is a challenging problem for its complex configuration. Sampling-based algorithms have gained great success in solving path planning problems in 3D space, but the quality of the initial path is not guaranteed and the convergence to the optimal solution is slow. To address these problems, in this article, we present a novel sampling-based path planning framework enhanced by the deep neural network (DNN) with applications to 3D space. In the proposed framework, we first train the DNN with a number of successful path planning cases in 3D space. Then the DNN is utilized to predict the promising region where the feasible path probably exists for a given path planning problem. This predicted promising region serves as a nonuniform sampling heuristic to bias the sampling process of the path planner. In this way, the path planner can focus on the promising region in the exploration and exploitation process so that the path planning speed gets accelerated. We conduct numerical simulations to evaluate the performance of the proposed algorithm and the results show that it can perform much better than conventional path planning algorithms. Furthermore, we also investigate the performance of different DNN architectures for path planning in 3D space. Note to Practitioners—In this work, we aim to provide an efficient learning-based method to accelerate the robot path planning process in 3D space. Conventional path planning algorithms need to perceive the environment first, and then implement a series of calculations such as collision checking and data storing to generate a feasible path. When facing complex and high-dimensional environments, they do not perform well. But the proposed neural network method in this article can predict the promising region where the feasible path exists for any given environment. This prediction result is used to guide the path planning process so that the algorithm performance can get significantly improved. Apart from sampling-based algorithms, the proposed neural network model can also be extended to other types of path planning algorithms.
Read moreTheoretical Improvements of Control Algorithms in Robot Path Planning: From Traditional Methods to Intelligent Algorithms
As robotics technologies evolve rapidly, path planning has emerged as a fundamental pillar underpinning autonomous navigation systems. Traditional algorithms such as A*, Dijkstra, and Bellman-Ford demonstrate competence in structured environments, yet exhibit notable limitations in dynamic and uncertain scenarios. In contrast, intelligent algorithmsincluding Genetic Algorithms (GA), Reinforcement Learning (RL), and hybrid strategiesprovide measurable advantages in adaptability, learning capacity, and robustness. This study centers on comparing traditional and intelligent control algorithms for path planning from a theoretical perspective, outlining their respective principles, performance traits, and application contexts. The findings suggest that intelligent algorithms, owing to their capacity for real-time learning, environmental adaptation, and heuristic generalization, consistently outperform traditional approaches in dynamic and unstructured operational contexts. The paper offers critical theoretical insights to guide future algorithm selection and optimization for robot navigation. Furthermore, the paper discusses real-world implementation challenges, such as computational complexity, sensor uncertainty, and algorithm convergence stability, to highlight the applicability of these techniques in complex systems.
Read moreThe optimization of path planning for multi-robot system using Boltzmann Policy based Q-learning algorithm
Path planning is a fundamental method in solving mazes or moving robots traversing through open fields with obstacles. Q-learning method is a model-independent reinforcement learning method, which can be utilized in path planning optimization for robots in a multi-robot collaboration system. However, the efficiency of the traditional Q-learning algorithm is relatively low because of the adopted random exploration policy. In this paper, a Boltzmann Policy based Q-learning algorithm is proposed and applied into the problem of path planning optimization of a Multi-robot system. The method is composed of two parts, which are Q-learning and Boltzmann policy. Q-learning is a grid-based algorithm that can solve the low-dimensional path planning problems. Boltzmann Policy adopts statistical probability and simulated annealing, so it can help to avoid trapping in local optimum and provide global optimal solution. Player/Stage is used to evaluate the performance, which shows that the proposed Q-learning algorithm based on the Boltzmann policy can remarkably improve the efficiency of the multi-robot System, reducing the number of explorations and converging the process.
Read moreOmnidirectional-Wheel Conveyor Path Planning and Sorting Using Reinforcement Learning Algorithms
In this paper, path planning and sorting of packages for Omnidirectional-Wheel conveyor are presented using Reinforcement Learning (RL). Q-learning, Double Q-learning, Deep Q-learning, and the Double Deep Q-learning algorithms are investigated. The RL algorithms enable the conveyor to self-learn the packages path and sort them without using conventional control or path planning theories. The RL algorithms are used for two different case studies on conveyors structures with different numbers of cells to compare and evaluate their performances in large- and small-scale sized structures. To explore the proposed methods response to external environment effects, two types of collisions between multiple packages were considered, the proposed RL algorithms showed their ability to resolve both types successfully. Comparative study between multiple RL algorithms for path planning showed that the Q-learning and Double Q-learning algorithms had outperformed their Deep learning versions for path planning in the two case studies. Furthermore, the proposed RL methods are compared experimentally to classic control and path planning theories using a hardware prototype for one of the presented case studies. The hardware experimental results showed that the proposed RL methods were as successful as the conventional methods in path planning and sorting in much less processing time. Two types of sorting scenarios (Type I and II) were tested for same package type and for multiple ones. For Type I sorting the Q-learning algorithm performed better than the Q-learning with weights approach, achieving better mean and minimum rewards while maximum rewards remain the same for both techniques. As for Type II sorting, only the Q-learning with weights approach was able to achieve it and converge in a reasonable time.
Read moreAgent Maze Path Planning Based on Simulated Annealing Q-Learning Algorithm
The problem of path exploration and planning of agents in unknown environments is a popular application problem in the field of reinforcement learning. In this paper, we propose an improved reinforcement learning algorithm called the Q-Learning algorithm for adaptive exploration based on simulated annealing (AE-SAQL). We apply the algorithm to the agent path planning problem, improve the setting of the reward function and add the feedback information of the environment. By simulating the Metropolis criterion in the annealing algorithm and adding an adaptive adjustment mechanism, the agent fully explores the environment and makes full use of the environmental information, solving the exploration-utilization dilemma during the algorithm and finally enabling the agent to reach the target location safely. Compared with the standard Q-Learning algorithm and SARSA algorithm, AE-SAQL achieves better.
Read moreA hierarchic approach for path planning in virtual reality
This work considers path-planning processes for manipulation tasks such as assembly, maintenance or disassembly in a virtual reality (VR) context. The approach consists in providing a collaborative system associating a user immersed in VR and an automatic path planning process. It is based on semantic, topological and geometric representations of the environment and the planning process is split in two phases: coarse and fine planning. The automatic planner suggests a path to the user and guides him trough a haptic device. The user can escape from the proposed solution if he wants to explore a possible better way. In this case, the interactive system detects the users intention and computes in real-time a new path starting from the users guess. Experiments illustrate the different aspects of the approach: multi-representation of the environment, path planning process, users intent prediction and control sharing.
Read moreA Reinforcement Learning Framework for Efficient Informative Sensing
Large-scale spatial data can be collected using mobile robots with sensing and navigation capabilities. Due to limited battery lifetime and scarcity of charging stations, it is important to plan informative paths so as to maximize the utility of data given a limited travel budget, which is known as the informative path planning (IPP) problem. IPP is NP-hard, and existing solutions suffer from high complexity or low optimality. In this paper, we present a novel IPP solution based on reinforcement learning (RL). The basic idea is to learn the structural characteristics of informative paths, so informative paths can be predicted. As such, when budgets change, we avoid solving the problem from scratch and thus path planning efficiency can be improved dramatically. Among the 20 path planning experiments in two areas, the proposed RL based solution achieves the best path utility in 15 experiments, compared with state-of-the-art algorithms. More importantly, the inference complexity is linear with respect to the budget (equivalently, the maximum number of steps in RL), which is lower than other solutions. Despite the NP-hardness, the path planning process can be finished within a few seconds in our experiments on two graphs of different sizes.
Read moreAutonomous navigation and adaptive path planning in dynamic greenhouse environments utilizing improved LeGO‐LOAM and OpenPlanner algorithms
The autonomous navigation of greenhouse robots depends on precise mapping, accurate localization information and a robust path planning strategy. However, the complex agricultural environment introduces significant challenges to robot perception and path planning. In this study, a hardware system designed exclusively for greenhouse agricultural environments is presented, employing multi‐sensor fusion to diminish the interference of complex environmental conditions. Furthermore, a robust autonomous navigation framework based on the improved lightweight and ground optimized lidar odometry and mapping (LeGO‐LOAM) and OpenPlanner has been proposed. In the perception phase, a relocalization module is integrated into the LeGO‐LOAM framework. Comprising two key steps—map matching and filtering optimization, it ensures a more precise pose relocalization. During the path planning process, ground structure and plant density are considered in our Enhanced OpenPlanner. Additionally, a hysteresis strategy is introduced to enhance the stability of system state transitions. The performance of the navigation system in this paper was evaluated in several complex greenhouse environments. The integration of the relocalization module significantly decreases the absolute pose error (APE) in the perception process, resulting in more accurate pose estimation and relocalization information. In our experiments, the APE was reduced by at least 24.42%. Moreover, our enhanced OpenPlanner exhibits the capability to plan safer trajectories and achieve more stable state transitions in the experiments. The results underscore the safety and robustness of our proposed approach, highlighting its promising application prospects in autonomous navigation for agricultural robots.
Read moreChaotic predator–prey biogeography-based optimization approach for UCAV path planning
Chaotic predator–prey biogeography-based optimization approach for UCAV path planning
Autonomous Robot KUKA YouBot Navigation Based on Path Planning and Traffic Signals Recognition
We present the successful demonstration of autonomous robot KUKA YouBot navigation based on path planning and traffic signals recognition. The integration of both capabilities path planning and traffic signals recognition was carried out, thanks to the integration among Robot Operating System, MATrix LABoratory software and Open Source Computer Vision Library working environments. The Robot Operating System allows the simulation of the autonomous robot navigation by using Gazebo and provides the implementation of the algorithms in simulated and real platforms. MATrix LABoratory software improves the communication tasks taking advantage of data processing tools in the path planning process. Finally, Open Source Computer Vision Library allows the traffic signals recognition by using the Scale-Invariant Feature Transform and Speeded-Up Robust Features algorithm. The integration of Robot Operating System, MATrix LABoratory software and Open Source Computer Vision Library is a promising approach to provide autonomous navigation capability in any mobile robot and in uncontrolled environments.
Read moreA new optimization-driven path planning method with probabilistic completeness for wheeled mobile robots
Wheeled mobile robots are widely utilized for environment-exploring tasks both on earth and in space. As a basis for global path planning tasks for wheeled mobile robots, in this study we propose a method for establishing an energy-based cost map. Then, we utilize an improved dual covariant Hamiltonian optimization for motion planning method, to perform point-to-region path planning in energy-based maps. The method is capable of efficiently handling high-dimensional path planning tasks with non-convex cost functions through applying a robust active set algorithm, that is, non-monotone gradient projection algorithm. To solve the problem that the path planning process is locked in weak minima or non-convergence, we propose a randomized variant of the improved dual covariant Hamiltonian optimization for motion planning based on simulated annealing and Hamiltonian Monte Carlo methods. The results of simulations demonstrate that the final paths generated can be time efficient, energy efficient and smooth. And the probabilistic completeness of the method is guaranteed.
Read moreGrid Path Planning for Mobile Robots with Improved Q-learning Algorithm
According to the problem of obstacles avoidance path planning in mobile robot system, this article uses the Q-learning algorithm in reinforcement learning to solve it. The original Q-learning algorithm has the problem of low learning efficiency, then an improved algorithm is proposed that adds a layer of learning process based on it. So that the mobile robot can find obstacles and target positions as soon as possible, which accelerates the efficiency of path planning and improves the efficiency. Finally, the path planning is established by a grid method on Python, and the comparison between the original algorithm, the improved algorithm proves that the learning efficiency of the algorithm is improved.
Read moreTrajectory optimization of multiple quad-rotor UAVs in collaborative assembling task
Trajectory optimization of multiple quad-rotor UAVs in collaborative assembling task
An AUV Path Planner for Large-scale Search and Rescue based on A* Algorithm
For large-scale search and rescue (SAR) tasks that require complete coverage of the workspace, it is important to increase the efficiency and obtained sensor data quality. A novel path planner named SAR-A* to this problem is introduced, which takes into account the sensor performance and practical prior information. Firstly, the workspace is decomposed into plenty of hexagonal cells which are treated as waypoints for A* algorithm. Target present probability is then modeled to Gaussian distribution and the performance of the side-scan sonar (SSS) is evaluated. The proposed path planner is validated in a complex terrain scenario which proves that the SAR-A* path planner can increase confidence in locating the target quickly, and is suitable for the large-scale SAR.
Read more