Integrating Motion Planning in Vision Language Action Agents

We present an optimization-based motion planning algorithm to compute a smooth, collision-free trajectory for a manipulator used to transfer a liquid from a source to a target container. We take into account fluid dynamics constraints as part of the trajectory computation. In order to avoid the high complexity of exact fluid simulation, we introduce a simplified dynamics model based on physically inspired approximations and system identification. Our optimization approach can incorporate various other constraints such as collision avoidance with obstacles, kinematic and dynamics constraints of the manipulator, and fluid dynamics characteristics. We demonstrate the performance of our planner on different benchmarks corresponding to various obstacles and container shapes. We also evaluate its accuracy by validating the motion plan using an accurate but computationally costly Navier-Stokes fluid simulation.

We describe some recent work on randomized motion planning algorithms and consider the problem of motion planning for systems with both kinematic and dynamic constraints. Such a problem is often referred to as kinodynamic motion planning. A rapidly-exploring random tree (RRT) is used for the motion planning of a blimp system, and some techniques for improving the performance of the planner are proposed. Based on a dynamic model of a blimp, dynamic constraints are introduced into the algorithm design and simulations are conducted for trajectory generation.

A formulation that makes possible the integration of collision prediction and avoidance stages for mobile robots moving in general terrains containing moving obstacles is presented. A dynamic model of the mobile robot and the dynamic constraints are derived. Collision avoidance is guaranteed if the distance between the robot and a moving obstacle is nonzero. A nominal trajectory is assumed to be known from off-line planning. The main idea is to change the velocity along the nominal trajectory so that collisions are avoided. A feedback control is developed and local asymptotic stability is proved if the velocity of the moving obstacle is bounded. Simulation results verify the value of the proposed strategy. >

An integrated collision prediction and avoidance scheme for mobile robots in non-stationary environments

Motion planning for autonomous vehicles in dense, complex traffic scenarios remains a significant challenge. The key issues lie in the trajectory safety and the path-speed coupling. To address these problems, this paper proposes a Safety-Guaranteed Trajectory Planning algorithm (SGTP) using a gap-oriented spatio-temporal corridor. Firstly, the candidate lanes for the ego vehicle are determined based on road topology and navigation information. Then, considering the uncertainty of surrounding traffic, the risk of each surrounding vehicle is assessed, and their occupied areas are safety-guaranteed according to a calculated risk field, enabling the construction of a dynamic spatio-temporal gap distribution map. By evaluating the cost of gap transitions, a tree-search strategy determines the optimal gap sequence. Subsequently, a spatio-temporal safety corridor is generated around the target gap sequence. Within this corridor, a reference trajectory is initialized using an Intelligent Driver Model integrated with Dynamic Programming (IDM-DP), and a safe, feasible, and smooth trajectory is optimized under the corridor constraints. Finally, SGTP is extensively validated on the nuPlan benchmark dataset. Experimental results demonstrate its outstanding performance in collision avoidance and driving efficiency. Compared to existing methods, SGTP achieves significant improvements in overall performance and safety metrics, highlighting its effectiveness for reliable motion planning in complex, interaction-intensive traffic environments.

In a collaborative scenario, the communication between humans and robots is a fundamental aspect to achieve good efficiency and ergonomics in the task execution. A lot of research has been made related to enabling a robot system to understand and predict human behaviour, allowing the robot to adapt its motion to avoid collisions with human workers. Assuming the production task has a high degree of variability, the robot’s movements can be difficult to predict, leading to a feeling of anxiety in the worker when the robot changes its trajectory and approaches since the worker has no information about the planned movement of the robot. Additionally, without information about the robot’s movement, the human worker cannot effectively plan own activity without forcing the robot to constantly replan its movement. We propose a novel approach to communicating the robot’s intentions to a human worker. The improvement to the collaboration is presented by introducing haptic feedback devices, whose task is to notify the human worker about the currently planned robot’s trajectory and changes in its status. In order to verify the effectiveness of the developed human-machine interface in the conditions of a shared collaborative workspace, a user study was designed and conducted among 16 participants, whose objective was to accurately recognise the goal position of the robot during its movement. Data collected during the experiment included both objective and subjective parameters. Statistically significant results of the experiment indicated that all the participants could improve their task completion time by over 45% and generally were more subjectively satisfied when completing the task with equipped haptic feedback devices. The results also suggest the usefulness of the developed notification system since it improved users’ awareness about the motion plan of the robot.

Future self-driving cars and current ones with advanced driver assistance systems are expected to interact with other traffic participants, which often are multiple other vehicles. To facilitate the motion planning of the autonomously controlled vehicle in collision avoidance, individual object vehicles with closeness in positions and velocities can be grouped as a single extended moving object. However, due to uncertainties from sensor imperfections and environmental disturbances, the collision avoidance conditions are often expressed as difficult to resolve probabilistic constraints in the motion planning problem. In this paper, we propose a constraint tightening method to transform the probabilistic collision avoidance condition for a vehicle group or an extended object into a deterministic form. This is done via a conservative closed-form transformation of the bivariate integral in the collision probability density function and subsequent computable approximation with logistic functions. Detailed numerical experiments are included to illustrate the workings and the performance of the proposed approach. This method can be incorporated in existing motion planning methods.

A novel model predictive artificial potential field based ship motion planning method considering COLREGs for complex encounter scenarios

Motion planning of underwater vehicles is regarded as a promising technique to make up the flexibility deficiency of underwater sensor networks (USNs). Nonetheless, the unique characteristics of underwater channel and environment make it challenging to achieve the above mission. This article is concerned with a communication-efficient and collision-free motion planning issue for underwater vehicles in fading channel and obstacle environment. We first develop a model-based integral reinforcement learning (IRL) estimator to predict the stochastic signal-to-noise ratio (SNR). With the estimated SNR, an integrated optimization problem for the codesign of communication efficiency and motion planning is constructed, in which the underwater vehicle dynamics, communication capacity, collision avoidance, and position control are all considered. In order to tackle this problem, a model-free IRL algorithm is designed to drive underwater vehicles to the desired position points while maximizing the communication capacity and avoiding the collision. It is worth mentioning that, the proposed motion planning solution in this article considers a realistic underwater communication channel, as well as a realistic dynamic model for underwater vehicles. Finally, simulation and experimental results are demonstrated to verify the effectiveness of the proposed approach.

Nowadays, multirotors are playing important roles in abundant types of missions. During these missions, entering confined and narrow tunnels that are barely accessible to humans is desirable yet extremely challenging for multirotors. The restricted space and significant ego airflow disturbances induce control issues at both fast and slow flight speeds, meanwhile bringing about problems in state estimation and perception. Thus, a smooth trajectory at the proper speed is necessary for safe tunnel flights. To address these challenges, in this letter, a complete autonomous aerial system that can achieve smooth flights through tunnels with dimensions as narrow as to 0.6 m is presented. The system contains a motion planner that generates smooth mini-jerk trajectories along the tunnel center lines, which are extracted according to the map and Euclidean distance field (EDF), and its practical speed range is obtained through computational fluid dynamics (CFD) and flight data analyses. Extensive flight experiments on the quadrotor are conducted inside multiple narrow tunnels to validate the planning framework as well as the robustness of the whole system.

Computationally efficient task execution is very important for autonomous mobile robots endowed with limited on-board computational resources. Most robot control approaches assume a fixed state and action representation, and use a single algorithm to map states to actions. However, not all situations in a given task require equally complex algorithms and equally detailed state and action representations. The main motivation for this work is a desire to reduce the computational footprint of performing a task by allowing the robot to run simpler algorithms whenever possible, and resort to a more complex algorithm only when needed. We contribute the Multi-Resolution Task Execution (MRTE) algorithm that utilizes human feedback to learn a mapping from a given state to an appropriate detail resolution consisting of a state and action representation, and an algorithm providing a mapping from states to actions at that resolution. The robot learns a policy from human demonstration to switch between different detail resolutions as needed while favoring lower detail resolutions to reduce computational cost of task execution. We then present the Model Plus Correction (M+C) algorithm to improve the performance of an algorithm using corrective human feedback without modifying the algorithm itself. Finally, we introduce the Multi-Resolution Model Plus Correction (MRM+C) algorithm as a combination of MRTE and M+C. MRM+C learns how to select an appropriate detail resolution to operate at in a given state from human demonstration. Furthermore, it allows the teacher to provide corrective demonstration at different detail resolutions to improve overall task execution performance. We provide formal definitions of MRTE, M+C, and MRM+C algorithms, and show how they relate to general robot control problem and Learning from Demonstration (LfD) approach. We present experimental results de-monstrating the effectiveness of proposed methods on a goal-directed humanoid obstacle avoidance task.

We present a new representation for task and motion planning that uses constraints to capture both continuous and discrete phenomena in a unified framework. We show that we can decide if a feasible plan exists for a given problem instance using only polynomial space if the constraints are semialgebraic and all actions have uniform stratified accessibility, a technical condition closely related to both controllability and to the existence of a symbolic representation of a planning domain. We show that there cannot exist an algorithm that solves the more general problem of deciding if a plan exists for an instance with arbitrary semialgebraic constraints. Finally, we show that our formalism is universal, in the sense that every deterministic robotic planning problem can be well-approximated within our formalism. Together, these results imply task and motion planning is PSPACE-complete.

Kinodynamic motion planning is pivotal in advancing robotics, en- abling autonomous systems to navigate dynamic environments effectively while adhering to both kinematic and dynamic constraints. This study delves into the efficacy of tree sampling-based planners, namely the Rapidly- exploring Random Tree (RRT), Rapidly-exploring Random Tree Star (RRT*), and Dominance Informed Region Trees (DIRT), in kinodynamic motion planning. Through a comparative analysis focusing on both fully informed and uninformed versions of these algorithms, I explore their performance in environments with dynamic constraints. Special emphasis is placed on the integration of learned controls, aiming to enhance maneuver planning. My research reveals significant differences in success rates, iterations, and path costs among the algorithms, underscoring DIRTs superiority under certain conditions and the beneficial impact of learned controls. These findings contribute valuable insights into the selection and optimization of motion planning algorithms, paving the way for more efficient and adapt- able autonomous systems.

Recently, it is expected to develop an autonomous robot working in a living environment. Motion planning is very important for an autonomous mobile robot, but kinematic constraints and dynamic constraints are generally solved separately in conventional motion planning. In this research, a simpler way of deciding the control input for an autonomous mobile robot, which is described by a highly nonlinear model in a multi-dimensional space, is proposed by solving kinematic constraints and dynamic constraints simultaneously, under the kinodynamic motion planning based on a harmonic potential field (HPF). In this paper, a quadrotor that is very attractive as an aerial robot due to its high maneuverability, is assumed to be a controlled object, and it is aimed at guiding the quadrotor to an arbitrary target point while avoiding obstacles. Then, three types of controllers for kinodynamic motion planning based on an HPF, which were developed for point mass control, are extended to be applied to the quadrotor. The extended controllers are compared each other in trajectory, amplitude of attitude, and arrival time through some flight simulations. Moreover, multiple gains included in the proposed controllers are optimized automatically by using genetic algorithm (GA), in order to realize highly accurate control. Flight simulation in an untrained environment, which is different from the environment that the gains were selected by GA, is also conducted, and it is confirmed that the quadrotor can move onto the target point with the same gains while avoiding obstacles.

Collision avoidance is very important problem in the domain of multi-robot interaction. In this paper we propose a new approach of collision avoidance in the context of the optimal control system synthesis problem definition with minimal information available. It is assumed that robots have a certain scope within which they can interact with static and dynamic phase constraints. A group of robots is considered to be homogeneous, and control system unit for reaching terminal states already available to robots. The control system which is responsible for collision avoidance is only activated when the nearest neighbor is located in the scope of the considered robot. The first important feature of this work is the fact that the collision avoidance between two robots is reciprocal with joint control system, without assigning priorities. Another key feature of this work is the complete absence of information about the environment and the current state of other robots at given time. Robots only share information with nearest neighbors if they locate in the scope of each other. We also present a computational experiment with mobile robots as control objects. A multilayer perceptron was used to approximate the control function. Weights of the perceptron were optimized in unsupervised paradigm by an algorithm belonging to the evolutionary strategies class. At the beginning of each epoch we generate a sample of collision scenarios for optimization, while the quality criterion of the achieved weights at the end of epoch is evaluated on a fixed test sample. Experimental results demonstrate strong ability of the optimized multilayer perceptron to map the relative state of two mobile robots to controls in order to avoid collisions.