Path Planning for Moving Robots in an Unknown Dynamic Area Using RND-Based Deep Reinforcement Learning

COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs

Autonomous area covering robots are being increasingly deployed in residential and commercial settings for a variety of purposes. These robots usually employ universal area covering algorithms to cover indoor environments. The performance of such algorithms heavily depends on room geometry as well as obstacle location, and often suffers from significant overlap leading to inordinately long coverage time, especially in realistic unknown environments with dynamic obstacles. Hence, deploying smarter algorithms that adapt to the environment can improve the performance significantly. In this study, we explore deep reinforcement learning (RL) algorithms for efficient coverage in unknown environments with multiple dynamic obstacles. Through experiments in grid-based environments and Gazebo simulator, we demonstrate the superior performance of RL based coverage algorithms in environments with dynamic obstacles. The performance of RL based algorithm is compared with the BA* algorithm with dynamic re-planning to demonstrate the advantages of the former over one-shot algorithms. Further, by employing transfer learning the trained RL agent learns to cover unseen unknown environments with minimal additional sample complexity. Importantly, we show that RL agents trained in smaller environments can be deployed for coverage in larger unknown environments with marginal additional sample complexity.

Abstract—This paper presents GAMYA, a cost-effective autonomous navigation vehicle developed using the Robot Operating System 2 (ROS2) framework. The system integrates a Raspberry Pi 4 as the central processing unit, an Arduino Uno R3 for motor control, and a 2D LiDAR sensor for environmental perception. GAMYA is designed to perform real-time mapping, localization, and obstacle avoidance through the implementation of the ROS2 Navigation2 stack and SLAM Toolbox. A differential drive mechanism is employed, and odometry is computed using encoder feedback to support accurate navigation. The system is tested in a simulated environment using Gazebo, where it successfully navigates dynamic and static obstacles. Results demonstrate reliable autonomous performance with minimal latency, showcasing the potential of using low-cost components to develop robust robotic platforms suitable for academic and research applications. Keywords—Autonomous Navigation, ROS2, LiDAR, SLAM, Gazebo Simulation, obstacle avoidance, Raspberry Pi, Arduino, Mobile Robot

The ability of automated vehicles to avoid obstacles on the road is one of the most critical functionalities, as it directly impacts passenger safety. The trajectory planning functionality plays a key role in this task, as it uses data from sensors to identify obstacles and plan a free obstacle path. Thus, this paper proposes a trajectory planning method based on a model predictive control (MPC) strategy that takes into account both static and dynamic obstacles. Additionally, a trajectory tracking controller based on a linear quadratic regulator (LQR) algorithm is designed to ensure the vehicle follows the planned path. To evaluate the proposed control solution for the lateral dynamics of the vehicle, a traffic scenario involving multiple obstacles (both static and dynamic) is used. The results show that the proposed solution effectively controls the vehicle's lateral dynamics and successfully avoids collisions with obstacles.

We present a reactive base control method that enables high performance mobile manipulation on-the-move in real-world environments with static and dynamic obstacles. Performing manipulation tasks while the mobile base remains in motion can significantly decrease the time required to perform multi-step tasks, as well as improve the gracefulness of the robot's motion. Existing approaches to manipulation on-the-move either ignore the obstacle avoidance problem or rely on the execution of planned trajectories, which is not suitable in environments with dynamic objects and obstacles. The presented controller addresses both of these deficiencies and demonstrates robust performance of pick-and-place tasks in dynamic environments. The performance is evaluated on several simulated and real-world tasks. On a mobile manipulation task with static obstacles, we outperform an existing method by 41% in terms of total task time. Further, we present real-world examples of our robot performing manipulation tasks on-the-move while avoiding a second autonomous robot in the workspace.

During the mission of the USV (Unmanned Surface Vehicle), there may be some static or dynamic obstacles on the pre-planned track that are not displayed on the ground station. To avoid these obstacles in time, a model of DCPA (distance to the closest point of approach) between the USV and the obstacle is proposed, which is used to plan the speed of the USV, and adaptively accelerate and decelerate can safely avoid dynamic obstacles. We solved the problem of target unreachable and heading change optimization by planning the heading at IAPF (improved artificial potential field) in the process of sailing, which can effectively avoid static obstacles and respond to collision avoidance in emergencies. The collision avoidance planning outputs a set of behavior vectors (speed, heading), and hands it over to the USV controller to control the navigation of the USV. In this study, an adaptive speed and heading planning method is used to improve the collision avoidance ability of the USV in the mixed scene of dynamic and static obstacles. Through multiple launching experiments, the actual collision avoidance effect of the method is verified, and the safety and reliability of the USV application requirements are met.

The article deals with the high-energy consumption of excessive anthropomorphic robots manipulators problem. Among the existing ways of solving this problem, solving the problems of energy-efficient traffic control methods of are chosen. Earlier on this topic, the authors developed mathematical methods for energy-efficient planning of the path of the anthropomorphic robot manipulator in the working area with a stationary obstacle. Earlier on this topic, the authors developed mathematical methods for energy-efficient motion path planning of the anthropomorphic robot manipulator in the working area with a stationary obstacle. The proposed methods have reduced computational complexity for real-time operation mode. The principle of the presented mathematical apparatus operation is to construct a virtual rectilinear energy-efficient path for moving the manipulator effector to a given position and then iteratively pushing the manipulator links out of the obstacle area to build the necessary movement route. The developed methods can be modified according to the necessary criteria of motion manipulator quality. This article presents the results of the new algorithm energy efficiency analysis that combines the previously described methods. A comparison with the selected analogue, the principle of which is based on the representation of objects in the form of oriented connected containers, is accomplished. The results of an experiments series on the simulation model implemented in MatLab show a reduction in the energy consumption of manipulators when performing targeted operations in the working area with or without an obstacle. A promising area of further research is the modification of the proposed mathematical apparatus for working in a zone with multiple stationary and dynamic obstacles, as well as the elimination of collisions when working with a human.

Autonomous service robots are gaining popularity as they find application in different sectors such as logistics, hospitality, medical, space exploration and military. It is widely seen that autonomous service robots help increase efficiency, reduce cost and ensure safer operations. Automating material handling operations in highly dynamic indoor environments is risky and involves use of complex algorithms. However, continuously advancing technologies and machine programming tend to overcome these challenges. The three essential functions of an autonomous robot are to identify its instantaneous location in three-dimensional space, generate the environment map and decide an optimal path for locomotion. Simultaneous Localization and Mapping (SLAM) is a widely used algorithm for navigation in unfamiliar environments. This chapter presents a cost-effective autonomous ground robot for automating material handling, transportation and other repetitive tasks in dynamically changing surroundings. The robot is programmed to map the indoor environment and localize itself in the workspace with the help of SLAM and Adaptive Monte Carlo Localization (AMCL). In addition, the robot can manoeuvre through the indoor workspace by avoiding static and dynamic obstacles. This work employs the GMapping open-source package in the Robot Operating System (ROS) for performing SLAM with the aid of a laser range finder, wheel odometry and inertial data. The GMapping algorithm uses a probabilistic approach to construct a map and estimate the robot’s location in two-dimensional space. In addition, this chapter presents an autonomous navigation feature with the help of the AMCL method for localizing the robot within an uncertain environment and making the robot move to its goal using move base actions in ROS. The path planning of the mobile robot is classified into global and local plans. The global plan is responsible for planning the path from the start point to the endpoint. The local plan is responsible for planning the actual movement of the robot by avoiding any dynamic obstacles not mapped earlier. This work uses the Dijistra algorithm for the global and the dynamic window approach (DWA) algorithm for the local path planning. The optimization algorithm minimizes the error between the global and local paths. The simulation study of the proposed algorithm is performed in a virtual environment called Gazebo in ROS, which is nearly similar to a real-time indoor workspace. Further, a robot prototype is fabricated, and its performance is evaluated in an unknown environment. The robot prototype is equipped with brushed DC motors with optical encoder feedback for wheel odometry (ODOM), Light Detection and Ranging (LiDAR), and Inertial Measurement Unit (IMU) sensors controlled using a Jetson Nano. The simulation and experimental results showcase the robot’s successful two-dimensional localization and navigation in the factory environment. The current system has the potential for indoor assistance during material transfer and logistic applications. Further upgrades to the system can make it usable for Automated Storage and Retrieval System (ASRS) applications in warehouses/factories.

We present a planning algorithm for a mobile robot in unknown indoor dynamic environments. The algorithm takes into account the robot's own dynamics and the dynamic obstacles’ future behavior in a safety hierarchy. It also uses an appropriate time horizon for planning and respects the timing constraints on various modules of the planner that are imposed by interleaving of planning and execution. We provide a formulation of this safety hierarchy in a single step via a composite costmap. The planning algorithm has been implemented in Player/Stage. We have run extensive simulations and compared the performance of the full safety hierarchy with various permutations of safety levels, as well as with some baseline algorithms over three different performance measures. We show that our safety hierarchy performs statistically significantly better across most performance measures and most simulation scenarios.

An energy efficient IoD static and dynamic collision avoidance approach based on gradient optimization

This paper presents the design of a variable potential field for dynamic obstacle avoidance by mobile robots. In this scheme, the RPF (Repulsive Potential Field) for obstacle avoidance has a different magnitude in each direction based on the angle between a goal and an obstacle, unlike a conventional RPF in which the magnitude is the same in each direction. Conventional potential field designs for path planning use scaling parameters of the potential field as fixed values. Therefore, the local minimum problem occurs because of which the robot cannot reach its destination. This is because of the obstacle in the approach to the destination due to repulsive forces in all directions around the potential field. Similarly, studies on existing potential field designs for avoiding dynamic obstacles have used methods such as fixed obstacles by designing only the location of the dynamic obstacles in the current step. Therefore, a collision is inevitable when the dynamic obstacle moves faster than the robot. Thus, in this study, the scaling parameters of the RPF were designed using variable values depending on the locations of the obstacle and destination. In addition, a new field design was developed for obstacle avoidance considering the speeds of dynamic obstacles. The experimental results proved that a potential field could be effectively designed for avoiding dynamic obstacles and those near the destination.

In recent decades, the Timed Elastic Band (TEB) algorithm is widely used for the AGV local path panning because of its convenient and efficiency. However, it may make a local detour when encountering a curve turn and cause excessive energy consumption. To solve this problem, this paper proposed an improved TEB algorithm to make the AGV walk along the wall when turning, which shortens the planning time and saves energy. Experiments were implemented in the Rviz visualization tool platform of the robot operating system (ROS). Simulated experiment results reflect that an amount of 5% reduction in the planning time has been achieved and the velocity curve implies that the operation was relatively smooth. Practical experiment results demonstrate the effectiveness and feasibility of the proposed method that the robots can avoid obstacles smoothly in the unknown static and dynamic obstacle environment.

Background: With the development of robotics, more and more robots are being used in industrial production. However, as the production environment surrounding the robots becomes increasingly complex, there is a need for more intelligent industrial robots. For improving the intelligence of industrial robots, the most important thing is to ensure that they can carry out safe industrial production activities in a complex production environment. Therefore, studying the autonomous dynamic obstacle avoidance path planning of industrial robots in complex environments is of great significance for improving the intelligence of industrial robots and the application of human-machine collaboration. Objective: The main purpose of this paper is to improve the traditional artificial potential field method. It aims to improve the disadvantages of the traditional artificial potential field method, such as falling into the local minimum and failing to reach the target. Secondly, the background difference method, which is based on binocular vision and Kalman filtering algorithm, is used, and the environmental map containing the static and dynamic obstacles is obtained. After obtaining the information on the position of static and dynamic obstacles, the robot arm can make good use of the improved artificial potential field method to plan its own trajectory, thus realizing the dynamic obstacle avoidance of the robot arm in a complex environment. Methods: This paper proposes an improved artificial potential field method. First of all, in order to solve the problem of not being able to achieve the goal, the method of modifying the repulsion field function, as proposed by Wang Huili and others, has been cited. In the traditional repulsion function, the relative distance between the robot and the target is introduced. Due to this, the target point is always the minimum point of the potential field. It is necessary to ensure that the robot can reach the target position smoothly. For the local minimum problem, by adding a gravitational increase factor β, the gravitational force received is greater than the repulsive force, thereby breaking the balance of the resultant force. Finally, for the traditional artificial potential field method that does not consider the path planning problem in the presence of dynamic obstacles, the velocity vector of the dynamic obstacle is brought into the potential field function to improve the traditional artificial potential field method. Results: The robot easily falls into the local optimum during path planning, and the improved artificial potential field method overcomes this shortcoming. Therefore, the manipulator can perform autonomous dynamic obstacle avoidance path planning in an environment with dynamic obstacles, finally reaching the target point safely. Conclusions: The industrial production field faces an increasing demand for intelligent industrial robots. This paper improves on the traditional artificial potential field method based on binocular vision so that the mechanical arm can avoid dynamic obstacles autonomously in the presence of dynamic obstacles, thereby improving the degree of intelligence of the robotic arm, enabling the technology to meet the needs of future industrial development and contribute to the development of future industrial robot technology.

In this paper an approach for 360 degree multi sensor fusion for static and dynamic obstacles is presented. The perception of static and dynamic obstacles is achieved by combining the advantages of model based object tracking and an occupancy map. For the model based object tracking a novel multi reference point tracking system, called best knowledge model, is introduced. The best knowledge model allows to track and describe objects with respect to a best suitable reference point. It is explained how the object tracking and the occupancy map closely interact and benefit from each other. Experimental results of the 360 degree multi sensor fusion system from an automotive test vehicle are shown.

A flexible operation of multiple robotic manipulators operating in a dynamic environment requires online trajectory planning to ensure collision-free trajectories. In this work, we propose a real-time capable motion control algorithm, based on nonlinear model predictive control, which accounts for static and dynamic obstacles. The proposed algorithm is realized in a distributed scheme, where each robot optimizes its own trajectory with respect to the related objective and constraints. We propose a novel approach for collision avoidance between multiple robotic manipulators, where each robot accounts for the predicted movement of the neighboring robots. Additionally, we propose a method to reliably detect and resolve deadlocks occurring in a setup of multiple robotic manipulators. We validate our approach on pick and place scenarios involving multiple robotic manipulators operating in a common workspace in a realistic simulation environment set up in Gazebo. The robots are controlled using the Robot Operating System. Our approach scales up to 4 manipulators and computes a path for each robot in a simultaneous pick and place operation in 94% of all investigated cases without deadlock detection and 100 % of cases with the proposed deadlock resolution algorithm. In contrast, the investigated conventional path planners, such as PRM, PRM*, CHOMP and RRT-Connect, successfully plan a trajectory in at most 54% of all investigated cases for a simultaneous operation of 4 robotic manipulators hindering their application in setups of multiple manipulators.