Application of deep reinforcement learning in mobile robot path planning

Path planning refers to that the mobile robot can obtain the surrounding environment information and its own state information through the sensor carried by itself, which can avoid obstacles and move towards the target point. Deep reinforcement learning consists of two parts: reinforcement learning and deep learning, mainly used to deal with perception and decision-making problems, has become an important research branch in the field of artificial intelligence. This paper first introduces the basic knowledge of deep learning and reinforcement learning. Then, the research status of deep reinforcement learning algorithm based on value function and strategy gradient in path planning is described, and the application research of deep reinforcement learning in computer game, video game and autonomous navigation is described. Finally, I made a brief summary and outlook on the algorithms and applications of deep reinforcement learning.

This study reviews the progress in the application of deep reinforcement learning in the field of path planning and control for mobile robots. Traditional methods exhibit poor adaptability in complex and dynamic environments, struggling to handle static and dynamic obstacles as well as environmental uncertainties. Deep reinforcement learning enables robots to autonomously interact with the environment and learn optimal strategies, significantly enhancing the performance of path planning and control. The article provides a detailed analysis of the limitations of traditional methods, such as high computational complexity, susceptibility to local optima, and lack of adaptability. It also introduces the advantages of deep reinforcement learning algorithms (e.g., DDPG, SAC) and their improved variants (e.g., APF-DDPG, AM-LSTM-SAC). These algorithms demonstrate excellent experimental results in various environments through multi-algorithm fusion strategies. The paper further discusses the setup of simulation environments, experimental configurations, and result analysis, summarizing existing achievements and shortcomings while providing an outlook on future research directions. This work offers valuable insights for the further development of mobile robot technology."

Irrigation decision-making using Reinforcement Learning (RL) performs well in changing environment, but easily falls into sub-optimal solutions with high-dimensional data. Deep Reinforcement Learning (DRL) has fused RL with Deep Learning (DL) and excels at learning adaptive and long-term irrigation strategies directly from high-dimensional environment data. This paper systematically reviews the applications of DRL in irrigation optimization, covering both pre-trained environments based on crop growth simulators and dynamic environments driven by real-time sensors. We discussed the strengths of classic DRL algorithms, including their ability to handle dynamic and non-linear environments, and reviewed their performance in irrigation multi-objective optimization and decision-making. In addition, we identified constraints in applying DRL in irrigation decision making, which include data scarcity, poor model interpretability, and difficulties in field deployment. It shows DRL can provide a powerful framework for adaptive irrigation, but is constrained by the gap between simulation and real-world complexity. To address these limitations, we discussed approaches in future work, such as developing multi-objective DRL algorithms. These approaches will improve DRL modeling outcomes and provide a technological foundation for smart agriculture and sustainable resource management. • Review the application of Deep Reinforcement Learning (DRL) in agricultural irrigation. • Analyze the performance of DRL algorithms in irrigation decision-making. • Compare DRL models based on different environment in irrigation optimization. • Discuss the further work to improve DRL performance in irrigation optimization.

In this paper, we propose a novel deep reinforcement learning (DRL) method for optimal path planning for mobile robots using dynamic programming (DP)-based data collection. The proposed method can overcome the slow learning process and improve training data quality inherently in DRL algorithms. The main idea of our approach is as follows. First, we mapped the dynamic programming method to typical optimal path planning problems for mobile robots, and created a new efficient DP-based method to find an exact, analytical, optimal solution for the path planning problem. Then, we used high-quality training data gathered using the DP method for DRL, which greatly improves training data quality and learning efficiency. Next, we established a two-stage reinforcement learning method where, prior to the DRL, we employed extreme learning machines (ELM) to initialize the parameters of actor and critic neural networks to a near-optimal solution in order to significantly improve the learning performance. Finally, we illustrated our method using some typical path planning tasks. The experimental results show that our DRL method can converge much easier and faster than other methods. The resulting action neural network is able to successfully guide robots from any start position in the environment to the goal position while following the optimal path and avoiding collision with obstacles.

Locating microseismic sources in time is a challenging problem in microseismic monitoring. In order to improve the accuracy and efficiency of locating sources, this paper presents a method for locating microseismic sources using deep reinforcement learning. We first construct and train a convolutional autoencoder to preprocess the seismic records in the microseismic waveform database. Then, the problem of locating the source is described as a Markov decision process for the application of deep reinforcement learning. We decompose the task of locating the source into three subtasks and design the critical elements of deep reinforcement learning. Three agents independently learn optimal policies for their respective subtasks in the framework of a deep Q-network (DQN) and jointly determine the precise location of the microseismic source. Finally, we evaluate the proposed method using synthetic data generated from the Marmousi model and the 3D velocity model. The experiment results indicate that the proposed method can locate microseismic sources efficiently and accurately.

Background: Path planning technology has a wide range of applications in many fields. Applications in the field of high technology include: autonomous and touchless action of robots; obstacle avoidance and defense flight of drones; cruise missiles to avoid radar search, anti- bouncing attacks, and to complete the task of sudden defense and demolition. Applications in daily life include: GPS navigation; road planning based on GIS system; urban road network planning and navigation. Applications in the field of decision-making management include: vehicle problem (VRP) in logistics management and similar resource allocation problems in resource management. Routing problems in the field of communication technology. Any planning problem that can be topologized as a point and line network can basically be solved by the method of path planning. Different intelligent algorithms have different characteristics, and their application scope and fields are also different, so it is of great significance for the development of path planning technology to study the path planning intelligent algorithms from the characteristics of the algorithms themselves and their applications. Objective: Analyze the advantages and disadvantages of various types of planning algorithms, look forward to the future development trend of mobile robot path planning, and provide certain ideas for the research of mobile robot path planning. Methods: Search journals, patents, conferences, and papers related to mobile robot path planning, and summarize and analyze the advantages and disadvantages of various planning algorithms. Results: Based on the research results of many scholars, this study summarizes different mobile robot path planning methods. It is divided into four categories: traditional planning, intelligent search, artificial intelligence and local obstacle avoidance. This paper introduces and analyzes the latest research results of these types, including their design ideas, advantages and disadvantages, and improvement measures. The research methods adopted are analyzed in order to maximize the advantages of each algorithm and expand the application field of robot path planning to provide ideas and references. Conclusion: This paper provides guidance for the design and optimization of robot path planning. Finally, this paper summarizes the future development trend of robot path planning, and looks forward to the future development trend and key areas of robot path planning.

Cooperative robots became a very important research topic due to their capability to accomplish tasks fast and more efficiently. In order for cooperative robots to accomplish their tasks, they need to be able to plan their paths in the environment without any collision. Path planning for cooperative robots with deep reinforcement learning is a new research topic in robotics and artificial intelligence. Path planning via deep reinforcement learning is an end to end method. The robot directly receives the data from the sensor (usually high dimensional images) and generates an optimal policy that plans a safe path to the target. In this paper, we first present an overview of reinforcement learning, deep learning, and deep reinforcement learning. Then, we introduce methods used for path planning. Finally, we discuss the challenges of deep reinforcement learning path planning for cooperative robots.

Deep reinforcement learning for the optimal placement of cryptocurrency limit orders

Deep reinforcement learning is one of the most important branches in the field of artificial intelligence. It has strong high-latitude data processing capabilities and is mainly used to solve control and decision-making problems. Many problems in the field of unmanned aerial vehicle(UAV) involve control and decision-making, so deep reinforcement learning has a wide range of applications in the field of UAVs. First, this article introduces the principles of classic reinforcement learning and extends to the principles of deep reinforcement learning. Secondly, this article describes the control and decision-making problems in the field of UAVs, and further introduces the application of deep reinforcement learning to these problems. Finally, this article looks forward to the future development direction of deep reinforcement learning in this field, and provides references for future related research.

PurposeThe purpose of this paper is to put forward a rolling assistant robot with two rolling modes, and the multi-mode rolling motion strategy with path planning algorithm, which is suitable to this multi-mode mobile robot, is proposed based on chessboard-shaped grid division (CGD).Design/methodology/approachBased on the kinematic analysis and motion properties of the mobile parallel robot, the motion strategy based on CGD path planning algorithm of a mobile robot with two rolling modes moving to a target position is divided into two parts, which are local self-motion planning and global path planning. In the first part, the mobile parallel robot can move by switching and combining the two rolling modes; and in the second part, the specific algorithm of the global path planning is proposed according to the CGD of the moving ground.FindingsThe assistant robot, which is a novel 4-RSR mobile parallel robot (where R denotes a revolute joint and S denotes a spherical joint) integrating operation and rolling locomotion (Watt linkage rolling mode and 6R linkage rolling mode), can work as a moving spotlight or worktable. A series of simulation and prototype experiment results are presented to verify the CGD path planning strategy of the robot, and the performance of the path planning experiments in simulations and practices shows the validation of the path planning analysis.Originality/valueThe work presented in this paper is a further exploration to apply parallel mechanisms with two rolling modes to the field of assistant rolling robots by proposing the CGD path planning strategy. It is also a new attempt to use the specific path planning algorithm in the field of mobile robots for operating tasks.

With the development of deep space exploration technology, thermal control systems for space telescopes are becoming increasingly complex, leading to the key parameters of conventional thermal control systems are difficult to adjust online automatically. To achieve these adjustments, this paper provided detailed verification of the application of deep reinforcement learning to space telescope thermal control from three perspectives: thermophysical modeling, intelligent sensing-based radiator, and online self-tuning of thermal control parameters. This paper presents a high-speed and high-precision thermophysical modeling strategy in matlab/simulink with better computational efficiency than conventional approaches. And an intelligent sensing-based radiator is proposed that can realize autonomous regulation of the radiating cold plate by sensing the external space environment and the thermal load inside the spacecraft. A strategy for online self-tuning of the thermal control parameters based on deep reinforcement learning is also proposed. Theoretical and experimental results show that deep reinforcement learning thermal control (DRLPID) can achieve temperature control accuracy of 0.05 °C. The steady-state errors in the simulations were reduced by 22.7%, 37.4%, and 47.4% when compared with the reinforcement learning proportional–integral–derivative (PID), the neural network PID, and the fuzzy PID, respectively. The experimental steady-state errors were reduced by 20.4%, 32.5%, and 42.7%, respectively.

Path planning in an unknown environment is a basic task for mobile robots to complete tasks. As a typical deep reinforcement learning, deep Q-network (DQN) algorithm has gained wide popularity in path planning tasks due to its self-learning and adaptability to complex environment. However, most of path planning algorithms based on DQN spend plenty of time for model training and the learned model policy depends only on the information observed by sensors. It will cause poor generalization capability for the new task and time waste for model retraining. Therefore, a new deep reinforcement learning method combining DQN with prior knowledge is proposed to reduce training time and enhance generalization capability. In this method, a fuzzy logic controller is designed to avoid the obstacles and help the robot avoid blind exploration for reducing the training time. A target-driven approach is used to address the lack of generalization, in which the learned policy depends on the fusion of observed information and target information. Extensive experiments show that the proposed algorithm converges faster than DQN algorithm in path planning tasks and the target can be reached without retraining when the path planning task changes.

Path planning is a key task in mobile robots, and the application of Deep Q Network (DQN) algorithm for mobile robot path planning has become a hotspot and challenge in current research. In order to solve the obstacle avoidance limitations faced by the DQN algorithm in indoor robot path planning, this paper proposes a solution based on an improved DQN algorithm. In view of the low learning efficiency of the DQN algorithm, the Duel DQN structure is introduced to enhance the performance and combined with a Prioritized Experience Replay (PER) mechanism to ensure the stability of the robot during the learning process. In addition, the idea of Munchausen Deep Q Network (M-DQN) is incorporated to guide the robot to learn the optimal policy more effectively. Based on the above improvements, the PER-D2MQN algorithm is proposed in this paper. In order to validate the effectiveness of the proposed algorithm, we conducted multidimensional simulation comparison experiments of the PER-D2MQN algorithm with DQN, Duel DQN, and the existing methodology PMR-DQN in the Gazebo simulation environment and examined the cumulative and average rewards for reaching the goal point, the number of convergent execution steps, and the time consumed by the robot in reaching the goal point. The simulation results show that the PER-D2MQN algorithm obtains the highest reward in both static and complex environments, exhibits the best convergence, and finds the goal point with the lowest average number of steps and the shortest elapsed time.

With the continuous progress and development of robotics, mobile robots have been widely used in many different fields. In the power grid, mobile robots are used to inspect electrical equipment, which greatly reduce the investment of manpower and material resources. However, in many cases, mobile robots need to work in a constantly changing and complex environment. Because they cannot obtain environmental information in time, it is often difficult to make path planning. In response to this problem, this paper proposes a path planning method for mobile robots based on improved reinforcement learning. This method establishes a grid environment model, defines the return value through the number of steps of the robot, and then proposes a changing action selection strategy for the balance between the robot’s exploration and utilization of the environment in reinforcement learning, so that the exploration factor dynamically change with the increase of the robot’s exploration degree of the environment, thus speeding up the convergence speed of the learning algorithm. Simulation results show that this method can realize autonomous navigation and path planning of mobile robots in complex environments. Compared with traditional algorithms, it greatly reduces the number of iterations.

Robot path planner based on deep reinforcement learning and the seeker optimization algorithm