A welding manipulator path planning method combining reinforcement learning and intelligent optimisation algorithm

Area coverage is essential for robots used in cleaning, painting, and exploration applications. Reconfigurable robots have been introduced to solve the area coverage limitation of fixed-shape robots. The existing global coverage algorithms of reconfigurable robots are limited to consideration of a limited set of predefined shapes for the reconfiguration and do not consider the exact geometrical shape of obstacles. Therefore, degraded coverage performance could be observed from the existing methods. On the other hand, the coverage methods that consider reconfiguring beyond a limited set of predefined shapes are limited to local coverage. Furthermore, these methods only consider a single reconfiguration for the coverage. Therefore, this paper proposes a novel coverage method for a reconfigurable robot consisting of both global and local path planners. The global path planner uses boustrophedon motion combined with the A * algorithm. The optimum grid positioning that maximizes the global coverage is determined through a Genetic Algorithm (GA). The local coverage planner performs continuous reconfig-uration of the robot to adequately cover obstacle zones while navigating through narrow spaces without collisions. A GA is used to determine the reconfiguration parameters of the robot at each instance of the local coverage. Simulation results confirm that the proposed method is effective in performing both global and local coverage path planning for improving the area coverage performance.

In this research, a hybrid approach for path planning of autonomous ships that generates both global and local paths, respectively, is proposed. The global path is obtained via an improved artificial potential field (APF) method, which makes up for the shortcoming that the typical APF method easily falls into a local minimum. A modified velocity obstacle (VO) method that incorporates the closest point of approach (CPA) model and the International Regulations for Preventing Collisions at Sea (COLREGS), based on the typical VO method, can be used to get the local path. The contribution of this research is two-fold: (1) improvement of the typical APF and VO methods, making up for previous shortcomings, and integrated COLREGS rules and good seamanship, making the paths obtained more in line with navigation practice; (2) the research included global and local path planning, considering both the safety and maneuverability of the ship in the process of avoiding collision, and studied the whole process of avoiding collision in a relatively entirely way. A case study was then conducted to test the proposed approach in different situations. The results indicate that the proposed approach can find both global and local paths to avoid the target ship.

Reasonable welding path has a significant impact on welding efficiency, and a collision-free path should be considered first in the process of welding robot path planning. The shortest path length is considered as an optimization objective, and obstacle avoidance is considered as the constraint condition in this paper. First, a grid method is used as a modeling method after the optimization objective is analyzed. For local collision-free path planning, an ant colony algorithm is selected as the search strategy. Then, to overcome the shortcomings of the ant colony algorithm, a secondary optimization is presented to improve the optimization performance. Finally, the particle swarm optimization algorithm is used to realize global path planning. Simulation results show that the desired welding path can be obtained based on the optimization strategy.

Dynamic path planning for unmanned surface vehicle in complex offshore areas based on hybrid algorithm

Robot path planner based on deep reinforcement learning and the seeker optimization algorithm

A novel technique called laser simulator approach for visibility search graph-based path planning has been developed in this article to determine the optimum collision-free path in unknown environment. With such approach, it is possible to apply constraints on the mobile robot trajectory while navigating in complex terrains such as in factories and road environments, as the first work of its kind. The main advantage of this approach is the ability to be used for both global/local path planning in the presence of constraints and obstacles in unknown environments. The principle of the laser simulator approach with all possibilities and cases that could emerge during path planning is explained to determine the path from initial to destination positions in a two-dimensional map. In addition, a comparative study on the laser simulator approach, A* algorithm, Voronoi diagram with fast marching and PointBug algorithms was performed to show the benefits and drawbacks of the proposed approach. A case study on the utilization of the laser simulator in both global and local path planning has been applied in a road roundabout setting which is regarded as a complex environment for robot path planning. In global path planning, the path is generated within a grid map of the roundabout environment to select the path according to the respective road rules. It is also used to recognize the real roundabout from a sequence of images during local path planning in the real-world system. Results show that the performance of the proposed laser simulator approach in both global and local environments is achieved with low computational and path costs, in which the optimum path from the selected start position to the goal point is tracked accordingly in the presence of the obstacles.

The ant colony optimization (ACO) algorithm is improved and further integrated with the immune algorithm (IA) to address its problems, such as slow convergence, local optimum, and premature convergence in the path planning. An algorithm integrating IA and improved ant colony optimization (IACO) is, therefore, put forward to realize the optimal planning of global path for an unmanned surface vehicle (USV). First, the ACO algorithm was improved in three aspects, that is, generation of initial pheromones, transition probability, and update of pheromones. The proposed IA-IACO algorithm combined the advantages of IA and IACO, sped up the convergence, and enhanced the optimization capability and operational efficiency. Second, the IA-IACO algorithm was designed and applied in the global path planning of an unmanned surface vehicle, achieving great global optimization and convergence. Finally, a path smoothing algorithm was devised to achieve the implementable, economic, and stable path while guaranteeing the safe navigation of the USV. A simulation test was carried out to prove the effectiveness and superiority of the designed global path planning algorithm in the practical engineering.

In this paper, path planning and sorting of packages for Omnidirectional-Wheel conveyor are presented using Reinforcement Learning (RL). Q-learning, Double Q-learning, Deep Q-learning, and the Double Deep Q-learning algorithms are investigated. The RL algorithms enable the conveyor to self-learn the packages path and sort them without using conventional control or path planning theories. The RL algorithms are used for two different case studies on conveyors structures with different numbers of cells to compare and evaluate their performances in large- and small-scale sized structures. To explore the proposed methods response to external environment effects, two types of collisions between multiple packages were considered, the proposed RL algorithms showed their ability to resolve both types successfully. Comparative study between multiple RL algorithms for path planning showed that the Q-learning and Double Q-learning algorithms had outperformed their Deep learning versions for path planning in the two case studies. Furthermore, the proposed RL methods are compared experimentally to classic control and path planning theories using a hardware prototype for one of the presented case studies. The hardware experimental results showed that the proposed RL methods were as successful as the conventional methods in path planning and sorting in much less processing time. Two types of sorting scenarios (Type I and II) were tested for same package type and for multiple ones. For Type I sorting the Q-learning algorithm performed better than the Q-learning with weights approach, achieving better mean and minimum rewards while maximum rewards remain the same for both techniques. As for Type II sorting, only the Q-learning with weights approach was able to achieve it and converge in a reasonable time.

가상의 캐릭터가 이동하기 위해 경로를 계획하는 일은 컴퓨터 게임을 포함한 여러 응용 분야에서 필수적인 문제이다. 기존의 로봇 경로 계획과는 달리 하나 이상의 캐릭터가 그룹 이동을 하는 경우가 많으며 이 경우에는 이동 경로의 최단 여부보다는 자연스러운 그룹 이동 등의 질적인 면이 강조된다. 본 논문에서는 리더로 정해진 단일 캐릭터에 대해 정적인 환경에서의 전역 경로를 계획하고 이 경로를 따라 그룹이 이동하며 동적 장애물을 피하도록 지역 경로를 계획하는 2단계 경로 계획 방법을 제안한다. 그룹이 이동할 수 있는 공간은 리더를 중심으로한 정방형 격자 배열을 이용하여 확보하고 이를 격자창이라고 부른다. 멤버 캐릭터들은 격자창 공간 안에서 리더에 대한 상대적인 위치를 잡아 이동하며 격자창이 이동할 수 있는 경로를 계획하기 위해 정적 환경을 격자창에 대한 형태 공간으로 치환하여 로드맵을 구한다. 지역 경로 계획 단계에서는 리더에 대해 인공 포텐셜 필드를 이용하여 동적 장애물을 회피하도록 하고 나머지 멤버들에 대해서는 격자창 내에서 리더에 대한 상대적인 위치를 확보하는 방법으로 연산량을 줄인다. 제안하는 경로 계획 방법을 구현하기 위한 효율적인 알고리즘을 소개하며 정적 환경에서 계획된 경로를 따라 그룹이 이동하면서 동적 장애물에 대해 효과적으로 대처하는 것을 시뮬레이션을 통해 보여 준다. Path planning is an essential problem to make virtual characters navigate in many applications including computer games. In many cases, multiple characters move in a group and qualitative aspects of planned paths are emphasized rather than optimality unlike Robotics. In this paper, we propose a two-level path planning algorithm in which the global path is planned for a single character specified as a leader and then the local path is planned to avoid dynamic obstacles while the group following this path. The space for group movement is achieved in the form of square grid array called a grid window. Member characters are located relatively to the leader within a space and moved. The static environment is reduced to the configuration space of this grid window to generate a roadmap on which a grid window can move. In local path planning, only the leader avoids dynamic obstacles by using an artificial potential field and the rest of members are located relatively to the leader in the grid window, which reduces computational load. Efficient algorithms to implement the proposed planning methods are introduced. The simulation results show that a group can handle with dynamic obstacles effectively while moving along the planned path for a static environment.

As robotics technologies evolve rapidly, path planning has emerged as a fundamental pillar underpinning autonomous navigation systems. Traditional algorithms such as A*, Dijkstra, and Bellman-Ford demonstrate competence in structured environments, yet exhibit notable limitations in dynamic and uncertain scenarios. In contrast, intelligent algorithmsincluding Genetic Algorithms (GA), Reinforcement Learning (RL), and hybrid strategiesprovide measurable advantages in adaptability, learning capacity, and robustness. This study centers on comparing traditional and intelligent control algorithms for path planning from a theoretical perspective, outlining their respective principles, performance traits, and application contexts. The findings suggest that intelligent algorithms, owing to their capacity for real-time learning, environmental adaptation, and heuristic generalization, consistently outperform traditional approaches in dynamic and unstructured operational contexts. The paper offers critical theoretical insights to guide future algorithm selection and optimization for robot navigation. Furthermore, the paper discusses real-world implementation challenges, such as computational complexity, sensor uncertainty, and algorithm convergence stability, to highlight the applicability of these techniques in complex systems.

This paper investigates the error accumulation problem in centralized training and decentralized execution (CTDE) policy‐based multi‐agent reinforcement learning (MARL) algorithms, which arises from local observation inaccuracies. To address this issue, we propose a novel MARL algorithm that incorporates imitation learning using local observations. Firstly, by analysing the multi‐agent proximal policy optimization algorithm and examining the problems arising when global states are replaced with local observations, it is proved that insufficient observations can lead to information loss, thereby introducing errors of advantage function, and it is demonstrated that the generalized advantage estimation method accumulates errors during the training process. Then, imitation learning is introduced and a novel training framework that combines reinforcement learning and imitation learning is proposed. During the reinforcement learning phase, an MARL agent trained with global observations acts as an expert. Subsequently, imitation learning is applied to train another agent that mimics the expert's decisions using only local observations. Finally, the effectiveness of this algorithm is verified in some commonly used multi‐agent environments, which demonstrates its superior performance compared to traditional multi‐agent reinforcement learning algorithms.

Intelligent parking systems have been recognized as a core technological intervention for resolving parking garage shortages and advancing traffic safety. Nevertheless, it remains challenging to generate a smooth, accurate, and optimal parking trajectory when employing conventional intelligent path optimization algorithms. Hence, building upon a newly designed optimization model for intelligent vehicle parking path planning, this study develops an improved immune moth-flame optimization algorithm (IIMFO). Specifically, aiming at the shortest path length and smooth enough trajectory, we leverage a cubic spline interpolation-driven path planning model to resolve the complex automatic parking trajectory optimization problem. To significantly strengthen the optimization effect, we introduce immune concentration selection, nonlinear decaying adaptive inertia weight adjustments, and elite opposition-based learning mechanisms to improve the immune moth-flame algorithm. Based on the evaluation results of the test functions, as well as the simulation and semi-automatic experiments of the real-world scenario of intelligent vehicle parking path optimization, the results indicate that the improved strategy can achieve better parking trajectories.

Path planning is an important area of mobile robot research, and the ant colony optimization algorithm is essential for analyzing path planning. However, the current ant colony optimization algorithm applied to the path planning of mobile robots still has some limitations, including early blind search, slow convergence speed, and more turns. To overcome these problems, an improved ant colony optimization algorithm is proposed in this paper. In the improved algorithm, we introduce the idea of triangle inequality and a pseudo-random state transfer strategy to enhance the guidance of target points and improve the search efficiency and quality of the algorithm. In addition, we propose a pheromone update strategy based on the partition method with upper and lower limits on the pheromone concentration. This can not only improve the global search capability and convergence speed of the algorithm but also avoid the premature and stagnation phenomenon of the algorithm during the search. To prevent the ants from getting into a deadlock state, we introduce a backtracking mechanism to enable the ants to explore the solution space better. Finally, to verify the effectiveness of the proposed algorithm, the algorithm is compared with 11 existing methods for solving the robot path planning problem, including several ACO variants and two commonly used algorithms (A* algorithm and Dijkstra algorithm), and the experimental results show that the improved ACO algorithm can plan paths with faster convergence, shorter path lengths, and higher smoothness. Specifically, the algorithm produces the shortest path length with a standard deviation of zero while ensuring the most rapid convergence and the highest smoothness in the case of the shortest path in four different grid environments. These experimental results demonstrate the effectiveness of the proposed algorithm in path planning.

Path planning for Autonomous Underwater Vehicle(AUV) is challenging for the reason that the dynamic system of AUV is under-actuated and the main sensor called sonar suffers from a limit detection range. This paper proposed a reinforcement learning (RL) based path planning method for the high speed AUV considering multi constrains. Firstly, an actor-critic based RL algorithm is designed including the constrains of sonar's detection distance, detect angle and AUV's angular rate. Secondly, an improved reward function is proposed considering total path length, angle change and the position of obstacles and target area. Finally, a priority experience replay(PER) technology is applied to RL algorithm to speed the training process. Simulation results show that the improved algorithm plan the reasonable path with mentioned constrains. Compared with conventional method in simulation, the improved algorithm finishes the path planning task with faster convergence speed, shorter path length and smaller cost about angle action.

Inverse transient radiation analysis in one-dimensional participating slab using improved Ant Colony Optimization algorithms