A Brief Insight into Nanorobotics

Aiming at the problem of multi-objective path planning (MOPP) and dynamic obstacle avoidance of manipulator in dynamic environment, this paper proposes a solution based on hierarchical reinforcement learning (HRL) framework. Traditional path planning methods have some problems in dynamic scenes, such as poor real-time performance, difficult to balance multi-objective conflicts and insufficient adaptability to environmental changes. Therefore, this paper designs a two-tier architecture including global path planning layer and local obstacle avoidance layer, which is trained by Proximal policy optimization (PPO) and Soft Actor-Critic (SAC) algorithms respectively, and achieves collaborative optimization through an efficient information exchange mechanism between layers. At the same time, a multi-objective reward function based on dynamic weight adjustment strategy is introduced. Fuzzy logic is used to adaptively balance the relationship among path length, obstacle avoidance safety and energy consumption according to environmental complexity. Combined with Long Short-Term Memory (LSTM), the trajectory of obstacles is predicted, and the potential field method is further introduced to modify the obstacle avoidance reward, which improves the real-time response ability and robustness of the algorithm in dynamic environment. The experimental results show that the HRL-SAC-PPO method proposed in this paper shows superior performance in both static and dynamic scenarios. In the static scene, the success rate of this method reaches 100%, the average path length is shortened to 2.13m, no collision occurs, and the energy consumption is reduced to 1.12 kJ, which shows a good multi-objective optimization effect. In the dynamic scene, the trajectory error of obstacles predicted by LSTM is only 4.2%, and the safe distance between the robot arm and obstacles is improved by 35%, which significantly enhances the reliability of obstacle avoidance. In addition, the average decision delay of this method is only 11.3ms, and the peak delay is 23ms, which is much lower than that of the contrast algorithm, showing stronger real-time response ability. The ablation experiment further verified the key role of LSTM trajectory prediction, dynamic weight adjustment and layered structure on the overall performance. In the welding task verification of the real UR5 manipulator, the success rate of the system in dynamic environment is 95.3%, the average path length is 3.41m, and the maximum joint acceleration is only 0.87 radian/s², which is far below the safety threshold, indicating that the algorithm has good stability and obstacle avoidance ability in practical application. The comprehensive performance comparison shows that this method performs well in different industrial scenarios, and has stronger environmental adaptability and comprehensive path planning ability.

Normal Distribution Transform (NDT) algorithm plays the role of detecting the relative pose of the vehicle in high-precision map. In this paper, a method of setting downsampling factor and grid factor for NDT relocation algorithm in dynamic environment is proposed, which can solve the problems of excessive NDT relocation error and location loss caused by dynamic objects accounting for 1 % to 35 % of the volume of scanning point cloud in vehicle environment. To simulate a real dynamic point cloud environment, the single-frame LiDAR point cloud space is voxelized into a mesh. Each grid is assigned a random number evenly distributed between 0 and 1. The threshold value for whether to add a Gaussian noise point is also set. Seven representative dynamic objects on the highway are selected. The threshold value of probability distribution function of Gaussian noise object needs to be set. Then the volume content of the dynamic object in the single frame point cloud space is calculated according to the set threshold value by using the definite integral. By changing the content and volume of dynamic obstacles in a dynamic environment, the effects of the downsampling factor and the grid factor on the accuracy of the repositioning trajectory are obtained. The resampling and mesh coefficients are optimized based on the analysis of the repositioning trajectory accuracy. The results show that When the current sampling factor is fixed, the grid factor of the NDT algorithm is proportional to the RMSE factor. When the NDT grid factor is fixed and the down sampling factor is equal to the side length of the obstacle, the NDT relocation accuracy is the highest and reaches the local optimum.

Reinforcement learning is an unsupervised learning algorithm, where learning is based upon feedback from the environment. Prior research has proposed cognitive (e.g., Instance-based Learning or IBL) and statistical (Q-learning) reinforcement learning algorithms. However, an evaluation of these algorithms in a single dynamic environment has not been explored. In this paper, a comparison between the statistical Q-learning algorithm and the cognitive IBL algorithm is presented. A well-known environment, “Frozen Lake,” is used to train, generalize, and scale Q-learning and IBL algorithms. For generalizing, the Q-learning and IBL agents were trained on one version of the Frozen Lake and tested on a permuted version of the same environment. For scaling, the two algorithms were tested on a larger version of the Frozen Lake environment. Results revealed that the IBL algorithm used less training time and generalized better to different environment variants. The IBL algorithm was also able to show scalability by retaining its superior performance in the larger environment. These results indicate that the IBL algorithm could be proposed as an alternative to the standard reinforcement learning algorithms based on dynamic programming such as Q-learning. The inclusion of human factors (such as memory) in the IBL algorithm makes it suitable for robust learning in complex and dynamic environments.

A spy search mechanism for memetic algorithm in dynamic environments

Dynamic optimization problems are receiving increasing attention from evolutionary algorithm community due to their general occurrences in the real world inspired by the permutation in group theory, a permutation-based dual scheme is introduced into genetic algorithm in order to improve its capability of addressing dynamic order-coding problems. Furthermore, a learning strategy is also developed to tune up the effect of dual scheme. The experimental results over dynamic flow shop problems could indicate the efficiency of the new dual scheme for genetic algorithm in dynamic environments.

One of the goals of naturalistic studies of human decision making is to reveal the cognitive loads or task difficulties imposed on the decision maker in real work environments. Fixation errors or cognitive lockups have been reported as a unique type of performance failure in dynamic work environments, and are thus particularly valuable to the understanding of the challenges and difficulties confronting practitioners in dynamic environments. In this paper, we present the analysis of fixation errors during real-life trauma patient resuscitation. The analysis elicits two factors, both rooted in the inherent complexity of the domain, that contributed to the occurrence of fixation errors: unreliable monitoring devices and delayed feedback. The former induces the behavior of preferring confirmatory information, partly for redundancy checks. The latter may create a false sense of system stability and divert attention away from the correct diagnosis.

This paper introduced the basic principles of the ant algorithm. On this basis, the method of dynamic mobile robot path planning based on ant algorithm is also proposed. Finally, the algorithm is certificated by Visual C++ simulation. It has a certain sense to solve the optimization problem that ant algorithm is applied to the mobile robot path planning.

With the development and widely use of unmanned aerial vehicles (UAVs) in recent years, the development of efficient path planning methods for automation has become crucial. Obstacle avoidance and path planning are the key components of UAV path planning. This article provides an overview of obstacle avoidance and path planning techniques for UAVs based on the artificial potential field method (APF method). This article begins with the explaining the principles of artificial potential field on this basis discusses its advantages and limitations. The article then summarizes the improvement strategies proposed by previous researchers to address issues like local minimum values and unreachable targets, such as introducing a new repulsive potential energy function, combining APF with other planning methods, and utilizing flow functions. Furthermore, it presents examples of the application and the performance of usage of these techniques in both static and dynamic environments. Based on this, the prospects and developing trend of UAV obstacle avoidance methods based on artificial potential field are foresee, such as combined with DRL and deep learning.

In complex dynamic environments, robot path planning faces challenges in multi‐objective optimization, such as path length, smoothness and obstacle avoidance capability. To address this, this paper proposes an improved sparrow search algorithm based on chaotic initialization and the golden positive cosine strategy for multi‐objective path planning. Diverse initial populations are generated through chaotic mapping to enhance global search capability and avoid falling into local optima. The golden positive cosine strategy optimizes individual position updates to accelerate convergence and ensure path smoothness. Results demonstrate that the proposed method outperforms sparrow search algorithm (SSA), with improvements of 21.1% in path length, 16.3% in smoothness and 14.2% in obstacle avoidance capability. After achieving global path optimization, the improved dynamic window approach (IDWA) is employed for real‐time obstacle avoidance in dynamic environments, dynamically adjusting the window size based on robot speed, obstacle density and target distance to adaptively expand or shrink the search space, thereby improving obstacle avoidance flexibility and efficiency. Simulation results show that the proposed method surpasses SSA in terms of path length, smoothness, obstacle avoidance capability and computational efficiency in both static and dynamic environments.

ABSTRACT-This research work focuses on the application of Quantum Reinforcement Learning (QRL) to improve path planning for autonomous robots. Traditional path planning techniques are prone to failure in handling the complexity of uncertain and dynamic environments, but QRL leverages the power of quantum computing to exploit the principles of superposition and parallelism to improve decision-making and optimization processes. By integrating QRL with existing reinforcement learning techniques, the proposed technique quickens the pace of learning processes and improves the quality of path planning to enable robots to navigate around obstacles more effectively with shorter travel time and lower energy costs. Simulation results show that QRL outperforms traditional techniques in convergence speed and optimality of path solutions and offers a promising solution for practical autonomous navigation. Keywords: Dynamic Environment, Path Planning, Autonomous Navigation, Obstacles avoidance.

This article focuses on achieving efficient and safe navigation for robots in dynamic and unpredictable environments. We propose a hierarchical path planning framework that integrates global path planning with local dynamic obstacle avoidance. This framework aims to quickly plan a collision-free, shortest, and safest path for the robot while adapting the navigation path according to uncertainties in the operating environment. Global path planning is conducted using the improved gray wolf optimization algorithm (IGWO), trajectory tracking is achieved through a pure pursuit control algorithm, and a dynamic switching mechanism based on deep reinforcement learning (DRL) significantly enhances the navigation performance of the robotic system. The effectiveness of this approach has been verified through simulations and experiments. Simulation results indicate that in global path planning, the IGWO algorithm achieves faster convergence compared to algorithms, such as GWO-MP and GWO-CS. The planned path lengths are reduced by approximately 4.01% and 2.27%, respectively, and the fitness values are decreased by 4.78% and 1.9%, demonstrating superior path planning performance. For local dynamic obstacle avoidance, both single-robot and multirobot systems successfully avoided obstacles in multiple experiments. Finally, physical experiments conducted in various complex scenarios show that both single-robot and multirobot systems can effectively execute global planning and respond to unexpected obstacles. These results further demonstrate the method’s wide applicability and robust performance across diverse complex environments.

Abstract: Path planning is a way to define the motion of an autonomous surface vehicle (ASV) in any existing obstacle environment to enable the vehicle's movement by setting directions to avoid that can react to the obstacles in the vehicle's path. A good, planned path perceives the environment to the extent of uncertainty and tries to build or adapt its change in the path of motion. Efficient path planning algorithms are needed to alleviate deficiencies, which are to be modified using the deterministic path that leads the ASV to reach a goal or a desired location while finding an optimal solution has become a challenge in the field of optimization along with a collision-free path, making path planning a critical thinker. The traditional algorithms have a lot of training and computation, making it difficult in a realistic environment. This review paper explores the different techniques available in path planning and collision avoidance of ASV in a dynamic environment. The objective of good path planning and collision avoidance for a dynamic environment is compared effectively with the existing obstacle’s movement of different vehicles. Different path planning technical approaches are compared with their performance and collision avoidance for unmanned vehicles in marine environments by early researchers. This paper gives us a clear idea for developing an effective path planning technique to overcome marine accidents in the dynamic ocean environment while choosing the shortest, obstacle-free path for Autonomous Surface Vehicles that can reduce risk and enhance the safety of unmanned vehicle movement in a harsh ocean environment.

Decentralized multi-agent trajectory planning (MATP) can enhance the efficiency of multi-robot systems thanks to high scalability and short computation time. However, it may lead to deadlock or livelock in obstacle-rich environments. To tackle this challenge, this paper presents a decentralized MATP algorithm for a quadrotor swarm that ensures convergence to a goal in maze-like environments. The proposed method guides the agents to their goal using the waypoints generated by a grid-based multi-agent path planning (MAPF) algorithm. Additionally, we introduce subgoal optimization to prevent deadlock while the agents follow the waypoints. The proposed algorithm guarantees that the agents converge to their goal if there is no dynamic obstacle and the agents are connected through a fully connected network. Moreover, it ensures deadlock-free even when the agent has a limited communication range. For dynamic obstacle avoidance, we revise the grid-based MAPF to prioritize collision avoidance when the agents encounter dynamic obstacles in a narrow corridor. In simulation, the proposed algorithm achieves a 100% success rate in static environments and shows a higher success rate and shorter flight time compared to most state-of-the-art baseline algorithms in dynamic environments. We validate the safety and robustness of the proposed work through the experiment with 10 quadrotors and one pedestrian in a maze-like environment.

Optimal maintenance over a finite time horizon for a system under imperfect inspection and dynamic working environment

Robot navigation in dynamic unknown environments is a challenging issue in the field of autonomous mobile robot control. This paper presents a hybrid robust method for navigating an industrial robot in an environment that contains dynamic obstacles. The objectives are to find the shortest path, to minimize the energy consumption of robot, to make the smoothness of the generated paths and to tackle dynamic obstacles. Robots employed in industrial environments demand considerable autonomy and require high level of accuracy and manoeuvrability at the same time. Besides, no collision is tolerable along the way. A single-objective optimization method based on path criteria fails to satisfy all of the requirements. This paper proposes a hybrid algorithm including the whale optimization algorithm (WOA) for path planning, a learnable function approximation network for making smoothness of the generated paths and a fuzzy logic controller to avoid obstacle collision. In this algorithm, WOA optimizes the best path to be taken from the start to goal position. Once a sequence of points is candidate and segments of path are merged, a radial basis function is trained to provide a smooth movement path in the dynamic environment while trying to maximize the safety margin. To further improve the safety of navigation, a fuzzy-based obstacle avoidance algorithm is executed when the robot is placed in the vicinity of an obstacle. Fuzzy decisions are made based on values of distance information. The proposed hybrid method for path planning and obstacle avoidance issues was implemented and evaluated in dynamic environments including specific shaped obstacles. A GUI-based simulation platform was designed in Matlab environment for testing the proposed algorithm. Implementation results indicate that the proposed algorithm has yielded in smooth non-marginal goal-directed navigation with acceptable performance metrics. Meanwhile, collisions to dynamic obstacles were adaptively and non-rigidly avoided. Such a model-free hybrid algorithm for path planning and obstacle avoidance can improve autonomy in industrial operation and decrease computational complexity.