Planning on FPGAs

In daily life, manipulating objects is one of the most frequent behaviors which is often difficult for the elderly and the disabled to complete due to physical factors. With the aging trend of society and the development of robot technology, it is a trend to use the escort robot to assist people in manipulating objects. Therefore, the object recognition and operation of the escort robot is one of the key research directions of robot technology. It contains many key technologies, such as object recognition technology, motion planning and control technology and communication technology, which determine the ability of escort robot to interact with the environment to provide high quality service. In this paper, an object recognition and operation system for escort robot is developed which includes object recognition module, motion planning and control module and communication module. In the object recognition module, an improved SURF algorithm is designed to match the feature points and recognize the objects, which has a high matching accuracy. Motion planning and control module based on Moveit realizes the planning and execution of actions independently. In the communication module, dedicated communication protocols are designed and applied to ensure that each module can work independently, stably and efficiently. In addition, the system is tested and verified in this paper.

The motion planning task for a mobile robot aims to generate a free-collision path from an initial point to a target point. This task may be highly complex because it requires a complete knowledge of the robot's environment. In this paper an option-based hierarchical learning approach is proposed to this problem in which basic behaviors are applied in order to accomplish the robot motion planning task. Each behavior is independently learned by the robot in the learning phase. Afterward, the robot learns to coordinate these basic behaviors to resolve the motion planning task. The application of the learning approach is validated with robot motion planning tasks in simulation as well as in an experimental environment. The results show a solution to the motion planning problem that can be highly successful in new unknown environments.

Real-time motion planning and control of multi-joint robot manipulators are studied using neural networks. The proposed neural network approach consists of two modules: one for real-time collision-free motion planning, the other for real-time fine control of the robot manipulators. The motion planning module is a biologically inspired, parallel connected, topologically organised neural network, where each neuron is characterised by a shunting equation, whose state space is the robot configuration joint space. This module is capable of planning a real-time optimal path for robot manipulators through the dynamic activity landscape of the neural network. The motion control module involves of a feedforward neural network together with a traditional PD feedback loop. It is capable of achieving real-time fine motion control of robot manipulators under significant uncertainties and without any prior knowledge of the robot dynamics. This module can quickly compensate sudden changes in the robot dynamics. The real-time fine control of robot manipulators is achieved through the online learning of the neural network. Both the motion planning and motion control modules are computationally efficient, and their global stabilities are proved using Lyapunov stability theory.

In this paper, the problem of motion planning for a mobile robot is addressed in which motion planning scenarios such as reachability, invariance, surveillance and sequencing missions are formulated with Linear Temporal Logic (LTL). First, by using the idea of invariant set theory, a novel algorithm is proposed for checking the satisfaction of temporal logic planning specification in the offline phase. Then, an optimal control strategy is designed in the presence of bounded additive disturbances to meet the given temporal logic specification and system constraints. Illustrative case studies for a robot with a double integrator model are presented to demonstrate the applicability of the proposed approach in motion planning tasks.

Continuous-time trajectories are useful for reasoning about robot motion in a wide range of tasks. Sparse Gaussian processes (GPs) can be used as a non-parametric representation for trajectory distributions that enables fast trajectory optimization by sparse GP regression. However, most previous approaches that utilize sparse GPs for trajectory optimization are limited by the fact that the robot state is represented in vector space. In this paper, we first extend previous works to consider the state on general matrix Lie groups by applying a constant-velocity prior and defining locally linear GPs. Next, we discuss how sparse GPs on Lie groups provide a unified continuous-time framework for trajectory optimization for solving a number of robotics problems including state estimation and motion planning. Finally, we demonstrate and evaluate our approach on several different estimation and motion planning tasks with both synthetic and real-world experiments.

Internal computational models of physical bodies are fundamental to the ability of robots and animals alike to plan and control their actions. These "self-models" allow robots to consider outcomes of multiple possible future actions without trying them out in physical reality. Recent progress in fully data-driven self-modeling has enabled machines to learn their own forward kinematics directly from task-agnostic interaction data. However, forward kinematic models can only predict limited aspects of the morphology, such as the position of end effectors or velocity of joints and masses. A key challenge is to model the entire morphology and kinematics without prior knowledge of what aspects of the morphology will be relevant to future tasks. Here, we propose that instead of directly modeling forward kinematics, a more useful form of self-modeling is one that could answer space occupancy queries, conditioned on the robot's state. Such query-driven self-models are continuous in the spatial domain, memory efficient, fully differentiable, and kinematic aware and can be used across a broader range of tasks. In physical experiments, we demonstrate how a visual self-model is accurate to about 1% of the workspace, enabling the robot to perform various motion planning and control tasks. Visual self-modeling can also allow the robot to detect, localize, and recover from real-world damage, leading to improved machine resiliency.

Task and Motion Planning for Apple Harvesting Robot

Abstract. This paper addresses the motion planning problem in non-holonomic robotic systems. The system's kinematics and dynamics are represented as a control affine system with outputs. The problem is defined in terms of the end-point map of this system, using the endogenous configuration space approach. Special attention is paid to the multiple-task motion planning problem, i.e. a problem that beyond the proper motion planning task includes a number of additional tasks. For multiple-task motion planning two strategies have been proposed, called the egalitarian approach and the prioritarian approach. Also, two computational strategies have been launched of solving the motion planning problem: the parametric and the non-parametric. The motion planning and computational strategies have been applied to a motion planning problem of the trident snake robot. Performance of the motion planning algorithms is illustrated with computer simulations.

Today, a paradigm in the manufacturing industry has been shifted from mass production to small and custom production. To adapt to today’s industrial expectations, this paper presents position-based visual servoing of multiple robotic manipulators. The presented visual servoing consists of two modules: task planning and motion planning. The task planning module manipulates each robotic arm not to collide with each other by distributing a task to them in a systematic manner. The motion planning module employs the MoveIt framework to solve inverse kinematics for each robotic arm so that it moves each end-effector’s pose to the desired pose to complete the task. In addition, the module controls a robotic hand to grasp or release a target object. The presented visual servoing has been tested and verified in the Gazebo simulator.

Solutions to robotic manipulation problems can be substantially improved through integrated task and motion planning. Existing approaches typically focus on satisfaction, finding a feasible solution, instead of optimization. We formulate large-scale robotic manipulation problems as multi-level optimization, incorporating task, action, and motion planning. We develop an integrated planning approach for solving this optimization problem and generating a combined motion plan for a robot to optimize a task-level objective. This approach utilizes a combinatorial search algorithm for task planning and incrementally exploits information from lower-level optimization to improve the high-level task plan. Empirical results show that this integrated approach not only significantly outperforms a traditional top-down approach in solution quality, but also avoids infeasible lower-level motion plans.

SUMMARYThis paper presents a task-priority motion planning algorithm for underactuated robotic systems. The motion planning algorithm combines two features: the idea of the task-priority control of redundant manipulators and the endogenous configuration space approach. This combination results in the algorithm which solves the primary motion planning task simultaneously with one or more secondary tasks ordered in accordance with decreasing priorities. The performance of the task-priority motion planning algorithm has been illustrated with computer simulations of the motion planning problem for a container ship.

A central challenging problem in humanoid robotics is to plan and execute dynamic tasks in dynamic environments. Given that the environment is known, sampling-based online motion planners are best suited for handling changing environments. However, without learning strategies, each task still has to be planned from scratch, preventing these algorithms from getting closer to realtime performance. This paper proposes a novel learning-based motion planning algorithm for addressing this issue. Our algorithm, called the attractor guided planner (AGP), extends existing motion planners in two simple but important ways. First, it extracts significant attractor points from successful paths in order to reuse them as guiding landmarks during the planning of new similar tasks. Second, it relies on a task comparison metric for deciding when previous solutions should be reused for guiding the planning of new tasks. The task comparison metric takes into account the task specification and as well environment features which are relevant to the query. Several experiments are presented with different humanoid reaching examples in the presence of randomly moving obstacles. Our results show that the AGP greatly improves both the planning time and solution quality, when comparing to traditional sampling-based motion planners.

Large language models (LLMs) have opened up new possibilities for intelligent agents, endowing them with human-like thinking and cognitive abilities. In this work, we delve into the potential of large language models (LLMs) in autonomous driving (AD). We introduce DriveMLM, an LLM-based AD framework that can perform close-loop autonomous driving in realistic simulators. To this end, (1) we bridge the gap between the language decisions and the vehicle control commands by standardizing the decision states according to the off-the-shelf motion planning module. (2) We employ a multimodal LLM (MLLM) to model the behavior planning module of a module AD system, which uses driving rules, user commands, and inputs from various sensors (e.g., camera, LiDAR) as input and makes driving decisions and provide explanations. This model can plug-and-play in existing AD systems such as Autopilot and Apollo for close-loop driving. (3) We design an effective data engine to collect a dataset that includes decision state and corresponding explanation annotation for model training and evaluation. We conduct extensive experiments and show that replacing the decision-making modules of the Autopilot and Apollo with DriveMLM resulted in significant improvements of 3.2 and 4.7 points on the CARLA Town05 Long, respectively, demonstrating the effectiveness of our model. We hope this work can serve as a baseline for autonomous driving with LLMs.

Deterministic generative adversarial imitation learning

The need for combined task and motion planning (CTAMP) in robotics is well known as robotic technologies become more mature. The goal of CTAMP is to determine a proper sequence of a robot’s actions based on symbolic and geometric reasoning. Because of the fundamental difference in symbolic and geometric reasoning, a CTAMP system often requires an interface module between the two reasoning modules. We propose a CTAMP system in which a symbolic action sequence is generated in task planning, and each action is verified geometrically in motion planning using the off-the-shelf planners and reasoners. The approach is that a set of action models is defined with PDDL in the interface module (action library) and the required information to each planner is automatically provided by the interface module. The proposed method was successfully implemented in three simulated experiments that involve manipulation tasks. According to our findings, the proposed method is effective in responding to changes in the environment and uncertainty with errors in recognition of the environment and the robot motion control.