An extended navigation framework for autonomous mobile robot in dynamic environments using reinforcement learning algorithm

Recently, embodied cognition for robotics has been discussed, and various types of artificial neural networks are applied for behavior learning of mobile robots in unknown and dynamic environments. In this research, the behavioral learning based on a spiking neural network to realize high adaptability of a mobile robot is proposed. The robot learn the forward relationship from sensory inputs to motor outputs as well as the predictive relationship from motor outputs to the sensory inputs. However, the behavioral leaning capability of the robot depends strongly on the network structure. Therefore, a VEGA to acquire the network structure suitable to the changing environment is applied. Finally, the effectiveness of the proposed method through experimental results on behavioral learning for collision avoidance and target tracing behavior is discussed.

Moving object velocity measurement in real-time is necessary to achieve obstacle avoidance and navigation for mobile robot in dynamic environment. A velocity measurement method based on machine vision using Kinect camera is proposed. Firstly, a marker fixed on moving object is designed, which consisted five circles with certain position relation, and then the moving object marker position in Kinect coordinate system was precisely detected according to the topological relationship of object marker using image processing algorithm. Then, the coordinate in Kinect system was transformed to world coordinate system. Finally, the velocity information was calculated with the distance in certain sampling time. Theoretical analysis and experimental results show that this method can be used to measure the velocity of moving object in both linear motion, situ rotation and synthetic curvilinear motion form, no matter the mobile robot is in the motion or stationary situation. It has a wide application range with real-time computing and high accuracy.

This article focuses on achieving efficient and safe navigation for robots in dynamic and unpredictable environments. We propose a hierarchical path planning framework that integrates global path planning with local dynamic obstacle avoidance. This framework aims to quickly plan a collision-free, shortest, and safest path for the robot while adapting the navigation path according to uncertainties in the operating environment. Global path planning is conducted using the improved gray wolf optimization algorithm (IGWO), trajectory tracking is achieved through a pure pursuit control algorithm, and a dynamic switching mechanism based on deep reinforcement learning (DRL) significantly enhances the navigation performance of the robotic system. The effectiveness of this approach has been verified through simulations and experiments. Simulation results indicate that in global path planning, the IGWO algorithm achieves faster convergence compared to algorithms, such as GWO-MP and GWO-CS. The planned path lengths are reduced by approximately 4.01% and 2.27%, respectively, and the fitness values are decreased by 4.78% and 1.9%, demonstrating superior path planning performance. For local dynamic obstacle avoidance, both single-robot and multirobot systems successfully avoided obstacles in multiple experiments. Finally, physical experiments conducted in various complex scenarios show that both single-robot and multirobot systems can effectively execute global planning and respond to unexpected obstacles. These results further demonstrate the method’s wide applicability and robust performance across diverse complex environments.

Mobile robots are mostly used in many industrial and military applications. Researches in the path planning is one of the most important aspects in mobile robot researches. Path planning for a mobile robot need to find a collision free path through the robot's environment with obstacles from the specified start location to a desired(target) goal location while satisfying certain optimization conditions. Existing path planning methods like graphical methods such as visibility graph, the potential field and the cell decomposition are designed for static environments, in which there are stationary obstacles. In practical systems such as Marine Science Research, Robots in Industry, and military combat applications, robots usually face dynamic environments where both moving and stationary obstacles exist. This project proposes a heuristic based method to search the feasible initial path efficiently. The heuristic based method is then combined into the simulated annealing algorithm based approach for dynamic robot path planning. Therefore the quality of the solution is characterised by the length of the planned path and it is improved with the combined heuristic method in the simulated annealing based approach for both runtime and offline path planning.

In this research, we have already developed a multiple omnidirectional vision system (MOVIS) with three omnidirectional cameras and its calculation method for the measurement of the object position and the self-localization in an autonomous mobile robot. Furthermore, we propose the omnidirectional behavior control method for collision avoidance and object chasing motion by using fuzzy reasoning in an autonomous mobile robot with MOVIS. In this paper, we also report some experimental results to confirm the efficiency of our proposed method by using a soccer robot in dynamic environment.

Rapidly-exploring random trees(RRT) is an important approach in motion planning. However, the uniform sampling strategy in conventional RRT methods leads to a low sampling efficiency in some situations and the solution is sub-optimal. Besides, conventional methods usually do not apply to dynamic environments. This paper presents an asymptotical RRT based path planning method for mobile robots in dynamic environments. To increase the sampling efficiency, a heuristic sampling algorithm is proposed on the basis of the prior knowledge of the previous planning results and the potential field of the environment. In addition, a post-processing procedure is adopted to optimize the path in consideration of path length and safety. Besides, a replanning strategy is presented to deal with the moving obstacles in dynamic environments. The simulation result shows that the heuristic sampling method has a higher sampling efficiency and faster rate of convergency than uniform sampling, a feasible path can be generated in consideration of safety and smoothness.

Real-time nonparametric reactive navigation of mobile robots in dynamic environments

In this paper, an obstacle avoidance and target tracking method for both indoor and outdoor mobile robots in dynamic environment is presented, and it aims to enhance autonomous navigation capability of such robots. In the proposed method, image processing and machine-learning approaches are considered. Since obstacles have differences in color and texture and in order to identify non-navigable areas, a monocular onboard camera is used to capture the road lanes by dealing with an image processing technique. The position of the robot with respect to road lane center during navigation is calculated on the basis of a proposed fuzzy logic rules set. In order to provide fast and robust computation, the Haar cascade classifier-based machine-learning technique has been exploited to detect the different sizes and shapes of the obstacles faced by the robot during its movement from source to destination. The dynamic model of a four-wheel mobile robot is initially developed using bond graph theory and then, the proposed obstacle-avoidance strategy is applied. The effectiveness and performance of the proposed method are tested under various simulation and experimentation scenarios. The behavior of the mobile robot for detecting and avoiding static obstacle for single and double road lane change environment is analyzed and discussed.

Navigating non-holonomic mobile robots in dynamic environments is challenging because it requires computing at each instant, the space of collision free velocities, characterized by a set of highly non-linear and non-convex inequalities. Moreover, uncertainty in obstacle trajectories further increases the complexity of the problem, as it now becomes imperative to relate the space of collision free velocities to a confidence measure. In this paper, we present a novel perspective towards analyzing and solving probabilistic collision avoidance constraints based on our previous works on non-linear time scaling. In particular, we have shown earlier that a time scaled version of collision cone constraints can be solved in closed form and thus can be used to efficiently characterize the space of collision free velocities. In the current proposed work, we present a probabilistic version of time scaled collision cone constraints obtained by representing obstacle states through generic probability distributions. We present a novel reformulation of the probabilistic constraints into a family of deterministic algebraic constraints. The solution space of each member of the family can be derived in closed form and at the same time, can also be related to the lower bound on confidence measure through Cantelli's inequality. Thus, the proposed work represents a significant improvement over the current state of the art frameworks where probabilistic collision avoidance constraints are solved through exhaustive sampling in the state-control space. We also present a cost metric which serves as the basis for the construction of the various collision avoidance maneuvers based on factors like deviation from the current path, acceleration/de-acceleration capability of the robot, confidence of collision avoidance etc. We very briefly explain how the current robot state can be connected to the solution space of safe velocities in smooth time optimal fashion. Finally, the validity of the proposed formulation is exhibited through extensive numerical simulation results.

This paper presents a self-adapting approach to global level path planning in dynamic environments. The aim of this work is to minimize risk and delays in possible applications of mobile robots (e.g., in industrial processes). We introduce a hybrid system that uses case-based reasoning as well as grid-based maps for decision-making. Maps are used to suggest several alternative paths between specific start and goal point. The casebase stores these solutions and remembers their characteristics. Environment representation and casebase design are discussed. To solve the problem of exploration vs. exploitation, a decision-making strategy is proposed that is based on the irreversibility of decisions. Forgetting strategies are discussed and evaluated in the context of case-based maintenance. The adaptability of the system is evaluated in a domain based on real sensor data with simulated occupancy probabilities. Forgetting strategies and decision-making strategies are evaluated in simulated environments. Experiments show that a robot is able to adapt in dynamic environments and can learn to use paths that are less risky to follow.

Autonomous following is one of the key issues in mobile robotics with a wide range of applications in commercial, industrial and military areas. To achieve autonomous following, the robot needs not only to robustly detect and track the target, but also efficiently follow the target while avoiding obstacles. In this paper, we propose a method that combines a Line-of-sight sensor (a 2D laser range finder) and a Non-line-of-sight sensor (a low-cost Angle-of-Arrival (AOA) tag) to identify, track and follow the target person in dynamic environments. First, in order to get smooth and continuous moving trajectory of the target person, a Kalman Filter is used to fuse person tracking information from AOA data and laser data. Then, a real-time robot-centric rolling grid map is constructed using the laser data. On top of the rolling grid map, a target potential field is generated by using the fast marching method, and then a direction gradient field is created based on the target potential field. With the rolling grid map, the target potential field and the direction gradient field, an improved dynamic window algorithm, FMM-DWA, is proposed to control the robot to move towards the target. The proposed method considers not only the distance between the robot and the target, but also the difference between the heading of the robot and the reference direction provided by the direction gradient field, to avoid falling into the local optimum. To validate the performance of the proposed method, a series of person following experiments are carried out in complex dynamic environments. The experimental results show that the proposed following algorithm can effectively deal with the occlusion problem and robustly follow the target person while quickly avoiding the static and dynamic obstacles.

This paper proposes a potential field immune network (PFIN) for dynamic motion planning of mobile robots in an unknown environment with moving obstacles and a fixed/moving goal. The velocity obstacle method is utilized to determine the imminent collision obstacle. Subsequently, PFIN is implemented to guide robots to avoid collision with the most dangerous objects at every time instant. The overall response of the immune network is calculated with the aid of fuzzy logic and genetic algorithms. Simulation results are presented to verify the effectiveness of the proposed architecture in dynamic environments with single and multiple fixed/moving obstacles.

Coverage Path Planning (CPP) is a fundamental aspect of mobile robotics, enabling robots to navigate dynamic environments efficiently while ensuring thorough coverage of all areas of interest and avoiding obstacles. CPP is pivotal in diverse applications, from everyday tasks like vacuum cleaning, lawn mowing, and window cleaning to specialized operations such as demining hazardous areas, autonomous underwater imaging, and inspecting complex structures. This survey reviews recent advancements in CPP, focusing on algorithms and methodologies tailored for dynamic environments. It highlights the challenges posed by environmental variability, obstacle dynamics, and real-time computational demands. By synthesizing insights from existing research, this survey aims to guide future developments in CPP, paving the way for smarter and more adaptive robotic systems capable of handling the complexities of real-world scenarios.

Navegación de Robots Móviles en Entornos no Estructurados utilizando Álgebra Lineal

Aimed to accuracy and convergence of Evolutionary Algorithms for path planning of robots in dynamic environment, this paper integrated cloud model into evolutionary algorithm to improved evolutionary operators. One-dimensional cloud operator was used evolution and mutation, at the same time the previous search results were used. Simulation results show that the method is effective in performance.