Memory-based hierarchical task and motion planning

This study is devoted to the challenges of motion planning for mobile robots with smart machine vision systems. Motion planning for mobile robots in the environment with obstacles is a problem to deal with when creating robots suitable for operation in real-world conditions. The solutions found today are predominantly private, and are highly specialized, which prevents judging of how successful they are in solving the problem of effective motion planning. Solutions with a narrow application field already exist and are being already developed for a long time, however, no major breakthrough has been observed yet. Only a systematic improvement in the characteristics of such systems can be noted. The purpose of this study: develop and investigate a motion planning algorithm for a mobile robot with a smart machine vision system. The research subject for this article is a motion planning algorithm for a mobile robot with a smart machine vision system. This study provides a review of domestic and foreign mobile robots that solve the motion planning problem in a known environment with unknown obstacles. The following navigation methods are considered for mobile robots: local, global, individual. In the course of work and research, a mobile robot prototype has been built, capable of recognizing obstacles of regular geometric shapes, as well as plan and correct the movement path. Environment objects are identified and classified as obstacles by means of digital image processing methods and algorithms. Distance to the obstacle and relative angle are calculated by photogrammetry methods, image quality is improved by linear contrast enhancement and optimal linear filtering using the Wiener-Hopf equation. Virtual tools, related to mobile robot motion algorithm testing, have been reviewed, which led us to selecting Webots software package for prototype testing. Testing results allowed us to make the following conclusions. The mobile robot has successfully identified the obstacle, planned a path in accordance with the obstacle avoidance algorithm, and continued moving to the destination. Conclusions have been drawn regarding the concluded research.

This study is devoted to the challenges of motion planning for mobile robots with smart machine vision systems. Motion planning for mobile robots in the environment with obstacles is a problem to deal with when creating robots suitable for operation in real-world conditions. The solutions found today are predominantly private, and are highly specialized, which prevents judging of how successful they are in solving the problem of effective motion planning. The purpose of this study: develop and investigate a motion planning algorithm for a mobile robot with a smart machine vision system. The research subject for this article is a motion planning algorithm for a mobile robot with a smart machine vision system. This study provides a review of domestic and foreign mobile robots that solve the motion planning problem in a known environment with unknown obstacles. The following navigation methods are considered for mobile robots: local, global, individual. In the course of work and research, a mobile robot prototype has been built, capable of recognizing obstacles of regular geometric shapes, as well as plan and correct the movement path. Environment objects are identified and classified as obstacles by means of digital image processing methods and algorithms. Distance to the obstacle and relative angle are calculated by photogrammetry methods, image quality is improved by linear contrast enhancement and optimal linear filtering using the Wiener-Hopf equation. Virtual tools, related to mobile robot motion algorithm testing, have been reviewed, which led us to selecting Webots software package for prototype testing. Conclusions have been drawn regarding the concluded research.

During task execution by a space manipulator, many factors need to be considered, such as joint motion range, external environment interference, running speed and task time. Studying the motion planning strategy under multi-constraint conditions is important for optimizing the task flow of the manipulator and improving the planning autonomy. Aiming at this, this paper proposes a method based on a hierarchical task network which has the ability to connect non-adjacent nodes of topology when generating the best path; it also carries out the control simulation of traditional path planning of a manipulator for typical tasks and the latter two working conditions using an optimal task planning method. The results show that this method is effective for realizing the Cartesian spatial task planning of a manipulator under multiple constraints and has important on-orbit application significance for improving the continuous autonomy of task completion.

In a human–robot coexisting environment, reaching the target place efficiently and safely is pivotal for a mobile service robot. In this paper, a Risk-based Dual-Tree Rapidly exploring Random Tree (Risk-DTRRT) algorithm is proposed for the robot motion planning in a dynamic environment, which provides a homotopy optimal trajectory on the basis of a heuristic trajectory. A dual-tree framework consisting of an RRT tree and a rewired tree is proposed for the trajectory searching. The RRT tree is a time-based tree, considering the future trajectory predictions of the pedestrians, and this tree is utilized to generate a heuristic trajectory. However, the heuristic trajectory is usually nonoptimal. Then, a line-of-sight (LoS) control checking algorithm is proposed to detect whether two time-based nodes can be rewired with the least cost. On the basis of the LoS control checking algorithm, a tree rewiring algorithm is proposed to optimize the heuristic trajectory. The tree generated in the tree rewiring process is called the rewired tree. The trajectory generated by the Risk-DTRRT algorithm proves to be optimal in the homotopy class of the heuristic trajectory. The navigation run time and the lengths of the planned trajectories are selected to demonstrate the effectiveness of the proposed algorithm. The experimental results in both simulation studies and real-world implementations reveal that our proposed method achieves convincing performance in both static and dynamic environments. Note to Practitioners —This paper is motivated by planning optimized trajectories for the mobile service robots in dynamic environments with pedestrians. In this area, the sampling-based motion planning algorithms have been widely used for their high efficiency and robustness. However, the real-time optimality of the motion planning cannot be guaranteed due to the challenges caused by the moving pedestrians. In this paper, we propose a dual-tree framework to solve this problem. First, a classic Rapidly exploring Random Tree (RRT) is constructed to generate a heuristic trajectory. Then, instead of reconnecting the nodes on the heuristic trajectory directly, a rewired tree is built to optimize the heuristic trajectory. This proposed dual-tree framework can fully exploit the information of the RRT tree and ensure the completeness of the motion planning. The proposed motion planning algorithm also considers the constraints of the nonholonomic mobile robots, and it can be applied in most mobile service robots to improve their motion planning quality.

We present a learning-based mapless motion planner by taking the sparse 10-dimensional range findings and the target position with respect to the mobile robot coordinate frame as input and the continuous steering commands as output. Traditional motion planners for mobile ground robots with a laser range sensor mostly depend on the obstacle map of the navigation environment where both the highly precise laser sensor and the obstacle map building work of the environment are indispensable. We show that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations. The trained planner can be directly applied in unseen virtual and real environments. The experiments show that the proposed mapless motion planner can navigate the nonholonomic mobile robot to the desired targets without colliding with any obstacles.

The construction of a smart autonomous mobile robot system that can operate in an anonymous environment can be difficult. Building an autonomous mobile robot requires collecting data from sensors and cameras in order to recognize the surrounding environment, get knowledge of existing landmarks, and figure out its position and orientation. Many research studies have been carried out in recent years providing algorithms for achieving autonomous motion of mobile robot systems and object detection. The most common platform utilized for robot motion design is ROS (Robot Operating System) and for object detection YOLO (You Only Look Once) is a common choice. Additionally, the autonomous system needs motion planning in a systematic way to avoid obstacles by using LIDAR or other sensing methods generating the feedback signals required for control of motion around occupied spaces. One of the most effective and versatile feedback systems for mobile robot control can be constructed using visual feedback, which relies on a high-resolution camera that processes the image in real-time and the result is fed back to the control system for decision making. Visual feedback can be employed alone without sensors for feedback control of autonomous mobile robots, and it can be used for multiple tasks like path planning, obstacle avoidance, object detection, object recognition in addition to motion control. In this research, results of design and construction of an autonomous mobile rescue robot for finding victims through video data are presented. It is aimed to find victims via applying video as a data collection and feedback generation tool in the autonomous mobile robot control system. The visual feedback and YOLO v3 are used not only for finding victims but also for motion control. Our autonomous mobile robot is designed to move smoothly to the victim's direction and avoids obstacles by using ROS Global Planner. The robot is tested in multiple scenarios in different locations, various obstacle and victim numbers, and satisfactory results with good performances are obtained.

Stability analysis and motion planning strategies with fuzzy coordinator and learning for mobile base robotic part macro/micro-assembly tasks

PurposeThe purpose of this paper is to put forward a rolling assistant robot with two rolling modes, and the multi-mode rolling motion strategy with path planning algorithm, which is suitable to this multi-mode mobile robot, is proposed based on chessboard-shaped grid division (CGD).Design/methodology/approachBased on the kinematic analysis and motion properties of the mobile parallel robot, the motion strategy based on CGD path planning algorithm of a mobile robot with two rolling modes moving to a target position is divided into two parts, which are local self-motion planning and global path planning. In the first part, the mobile parallel robot can move by switching and combining the two rolling modes; and in the second part, the specific algorithm of the global path planning is proposed according to the CGD of the moving ground.FindingsThe assistant robot, which is a novel 4-RSR mobile parallel robot (where R denotes a revolute joint and S denotes a spherical joint) integrating operation and rolling locomotion (Watt linkage rolling mode and 6R linkage rolling mode), can work as a moving spotlight or worktable. A series of simulation and prototype experiment results are presented to verify the CGD path planning strategy of the robot, and the performance of the path planning experiments in simulations and practices shows the validation of the path planning analysis.Originality/valueThe work presented in this paper is a further exploration to apply parallel mechanisms with two rolling modes to the field of assistant rolling robots by proposing the CGD path planning strategy. It is also a new attempt to use the specific path planning algorithm in the field of mobile robots for operating tasks.

This paper presents an adaptive approach based on the Reinforcement Learning (RL) method to manipulate and transport parts and also assemble 3-D structures in a moderately constrained and dynamic environment using a quad-rotor. Nowadays, complex construction tasks using mobile robots are characterized by two fundamental problems such as task planning and motion planning. However, to obtain the task and path planning that define a specific sequence of operations for construction of a given structure is generally very complex. In this context, we propose and investigate a system in which an aerial robot learns the assembly and construction tasks of multiple 3-D structures. This process involves the learning of the sequence of maneuvers of a vehicle, the assembly sequence of the parts and also the correct types of structural elements for each assembly point of the structure. A heuristic search algorithm is used in the learning process to find the optimal path for the quad-rotor so that its navigation through the dynamic environment is performed. The experimental results show that a 3-D structure can be built using the task planning approach derived from a learning algorithm combined with a heuristic search method.

Motion planning remains a fundamental problem in robotics. Sampling-based algorithms use randomization to allow efficient solutions to this complex problem. As mobile robots and autonomous vehicles become more prevalent in everyday life, motion planning must be applied to increasingly challenging scenarios. Safety has become a paramount concern in motion planning for ensuring robotic applications enrich human lives. To date, many motion planning techniques to increase safety in the face of uncertain and dynamic environments have been developed. This dissertation first addresses distributional safety of Rapidly-Exploring Random Trees (RRT) through our algorithm W-Safe RRT. To acknowledge distributional uncertainty and poor modeling, W-Safe RRT uses the Wasserstein metric to provide a probabilistic bound on the distributional distance between a robot and obstacles. Human-interpretable environmental agent classification allows online safety margin adaptation. We propose and analyze an integrating region method for online classification that increases actor labeling accuracy based on behavioral feature values when compared to state of the art methods. The method performs class assignments based on local maximum likelihood in a created behavioral feature-space, allowing a notion of classification uncertainty. Model-based methods with safety guarantees can quickly become computationally in tractable, especially with multiple agents, higher dimensions, and plentiful unknowns. Sampling based algorithms have been parallelized for computation with multi-core computers and GPUs. We consider the use of quantum algorithms and computers for sampling-based motion planning for the first time. Quantum computing performs operations on superpositions of states and can solve certain problems much more efficiently than classical computers, but introduces previously unseen challenges. With Quantum-RRT, we recast the motion planning problem into a database-search structure and use Quantum Amplitude Amplification to find reachable states in the database with a quadratic performance increase over classical methods. We address two error sources with this method: quantum measurement and quantum oracle errors. We then extend this method to Parallel Quantum-RRT, which uses a manager-worker architecture with multiple parallel quantum workers to increase database search efficiency. We compare algorithm architectures and characterize probabilities of multiple workers finding solutions. Lastly, we test in simulation the quantum algorithms against classical versions in a wide variety of scenarios, concluding that a similar parallelization improvement is to be found in the quantum case as was found in the parallelization of classical RRT.

One of the ultimate goals of mobile robotics research is to build robots that can safely carry out missions in hazardous and populated environments. Most of today's commercial mobile devices scale poorly along this dimension. Their motion planning relies on accurate, static models of the environments, and therefore they often fail their mission if humans or other unpredictable obstacles block their path. To build autonomous mobile robots one has to build systems that can perceive their environments, react to unforeseen circumstances, and plan dynamically in order to achieve their mission. Thus, the objective of the motion planning and control problem is to find collision-free trajectories, in static or dynamic environments containing some obstacles, between a start and a goal configuration. It has attracted much research in recent years. In this context the term control has a broad meaning that includes many different controls, such as low-level motor control, and behaviour control, where behaviour represents many complicated tasks, like obstacle avoidance and goal seeking. This article describes an intelligent motion planning and navigation system for omnidirectional mobile robots based on fuzzy logic. Owing to its simplicity and hence its short response time, the fuzzy navigator is especially suitable for on-line applications with strong real-time requirements. On-line planning is an on-going activity. The planner receives a continuous flow of information about occurring events and generates new commands in response to the incoming events, while previously planned motions are being executed. The fuzzy-rule-base of the proposed system combines the repelling influence, which is related to the distance and the angle between the robot and nearby obstacles, with the attracting influence produced by the distance and the angular difference between the actual direction and position of the robot and the final configuration, to generate actuating commands for the mobile platform. It can be considered as an on-line local navigation method for omnidirectional mobile robots for the generation of instantaneous collision-free motions. This reactive system is especially suitable for real-time applications. The use of fuzzy logic leads to a transparent system which can be tuned by hand or by a set of learning rules. Furthermore, this approach allows obstacle avoidance and navigation in dynamic environments. The functioning of the fuzzy motion planner with respect to omnidirectional mobile robots and results of simulated experiments are presented.

Task and Motion Planning for Apple Harvesting Robot

The mobile robots are equipped with sensitive audio visual sensors, usually microphone arrays and video cameras. They are the main sources of audio visual information to perform suitable mobile robots navigation tasks, modeling the human audio visual perception. The results from the audio and visual perception algorithms are widely used, separate or in conjunction (audio visual perception) in the mobile robots navigation, for example to control mobile robots motion in applications like people and objects tracking, surveillance systems, etc. The effectiveness and precision of the audio visual perception methods in the mobile robots navigation can be enhanced combining audio visual perception with audio visual attention. Sufficient relative knowledge exists, describing the phenomena of human audio and visual attention. Such approaches are usually based on a lot of physiological, psychological, medical and technical experimental investigations relating the human audio and visual attention, with the human audio and visual perception with the leading role of the brain activity. Of course, the results from these investigations are very important, but not sufficient for the mobile robots audio visual attention modeling, mainly because of brain missing in mobile robots audio visual perception systems. Therefore, in this chapter is proposed to use the existing definitions and models for human audio and visual attention, adapting them to the models of mobile robots audio and visual attention and combining with the results from the mobile robots audio and visual perception in the mobile robots navigation tasks.

In this paper, we propose a motion-planning method of multiple mobile robots for cooperative transportation of a large object in a three-dimensional environment. This task has various kinds of problems, such as obstacle avoidance and stable manipulation. All of these problems cannot be solved at once, since it would result in a dramatic increase of the computational time. Accordingly, we divided the motion planner into a global path planner and a local manipulation planner, designed them, and integrated them. The aim was to integrate a gross motion planner and a fine motion planner. Concerning the global path planner, we reduced the dimensions of the configuration space (C-space) using the feature of transportation by mobile robots. We used the potential field to find the solution by searching in this smaller-dimension reconstructed C-space. In the global path planner, the constraints of the object manipulation are considered as the cost function and the heuristic function in the A/sup */ search. For the local manipulation planner, we developed a manipulation technique, which is suitable for mobile robots by position control. We computed the conditions in which the object becomes unstable during manipulation and generated each robot's motion, considering the robots' motion errors and indefinite factors from the planning stage. We verified the effectiveness of our proposed motion planning method through simulations.

Solutions to robotic manipulation problems can be substantially improved through integrated task and motion planning. Existing approaches typically focus on satisfaction, finding a feasible solution, instead of optimization. We formulate large-scale robotic manipulation problems as multi-level optimization, incorporating task, action, and motion planning. We develop an integrated planning approach for solving this optimization problem and generating a combined motion plan for a robot to optimize a task-level objective. This approach utilizes a combinatorial search algorithm for task planning and incrementally exploits information from lower-level optimization to improve the high-level task plan. Empirical results show that this integrated approach not only significantly outperforms a traditional top-down approach in solution quality, but also avoids infeasible lower-level motion plans.