Learned Critical Probabilistic Roadmaps for Robotic Motion Planning

We discuss the optimality and computational efficiency of sampling based motion planning (SBMP), which calculates dynamically precise and approximately optimal state transitions using arbitrarily selected nonlinear control system models. To ensure high optimality and computational efficiency, SBMP requires an approximately uniform state sampling function, though the non-linearity of the system models does not allow a perfect function. We propose linear prediction-based uniform state sampling (LPUSS) that samples approximately uniform state points while ensuring a dynamically correct state transition profile with a small calculation cost. LPUSS samples a state by using the given non-linear control system model after determining the input values by using a local linear transition model. We developed a mechanical motion planning system using LPUSS, articulated body algorithm, and parallel computing techniques. To validate LPUSS, we conducted experiments on double, triple, and sixtuple inverted pendulum models. LPUSS showed better optimality and computational efficiency with the double and triple inverted pendulum models, compared with randomized kinodynamic planning (RKP), which is based on rapid random tree (RRT), and our previously proposed rapid semi-optimal motion-planning method in which state sampling is based on uniform inputs. In particular, compared with our previous method, LPUSS was respectively 130 times and 3,000 times faster on double and triple inverted pendulum models under the condition of the same optimality. LPUSS found an approximately optimal swing up motion for the sixtuple inverted pendulum model within 40 minutes. According to our survey, there is no other optimization method that is applicable to higher than quadruple inverted pendulum models with standard constraints.

In this paper, we discuss the field of sampling-based motion planning. In contrast to methods that construct boundary representations of configuration space obstacles, samplingbased methods use only information from a collision detector as they search the configuration space. The simplicity of this approach, along with increases in computation power and the development of efficient collision detection algorithms, has resulted in the introduction of a number of powerful motion planning algorithms, capable of solving challenging problems with many degrees of freedom. First, we trace how sampling-based motion planning has developed. We then discuss a variety of important issues for sampling-based motion planning, including uniform and regular sampling, topological issues, and search philosophies. Finally, we address important issues regarding the role of randomization in sampling-based motion planning.

Sampling-based Motion Planners (SMPs) have become increasingly popular as they provide collision-free path solutions regardless of obstacle geometry in a given environment. However, their computational complexity increases significantly with the dimensionality of the motion planning problem. Adaptive sampling is one of the ways to speed up SMPs by sampling a particular region of a configuration space that is more likely to contain an optimal path solution. Although there are a wide variety of algorithms for adaptive sampling, they rely on hand-crafted heuristics; furthermore, their performance decreases significantly in high-dimensional spaces. In this paper, we present a neural network-based adaptive sampler for motion planning called Deep Sampling-based Motion Planner (DeepSMP). DeepSMP generates samples for SMPs and enhances their overall speed significantly while exhibiting efficient scalability to higher-dimensional problems. DeepSMP's neural architecture comprises of a Contractive AutoEncoder which encodes given workspaces directly from a raw point cloud data, and a Dropout-based stochastic deep feedforward neural network which takes the workspace encoding, start and goal configuration, and iteratively generates feasible samples for SMPs to compute end-to-end collision-free optimal paths. DeepSMP is not only consistently computationally efficient in all tested environments but has also shown remarkable generalization to completely unseen environments. We evaluate DeepSMP on multiple planning problems including planning of a point-mass robot, rigid-body, 6-link robotic manipulator in various 2D and 3D environments. The results show that on average our method is at least 7 times faster in point-mass and rigid-body case and about 28 times faster in 6-link robot case than the existing state-of-the-art.

To compute collision-free and dynamically-feasibile trajectories that satisfy high-level specifications given in a planning-domain definition language, this paper proposes to combine sampling-based motion planning with symbolic action planning. The proposed approach, Sampling-based Motion and Symbolic Action Planner (SMAP), leverages from sampling-based motion planning the underlying idea of searching for a solution trajectory by selectively sampling and exploring the continuous space of collision-free and dynamically-feasible motions. Drawing from AI, SMAP uses symbolic action planning to identify actions and regions of the continuous space that sampling-based motion planning can further explore to significantly advance the search. The planning layers interact with each-other through estimates on the utility of each action, which are computed based on information gathered during the search. Simulation experiments with dynamical models of vehicles carrying out tasks given by high-level STRIPS specifications provide promising initial validation, showing that SMAP efficiently solves challenging problems.

For scaling up reinforcement learning to large and complex problems, we propose an approach to partition the larger state space into multiple smaller state spaces based the critical states for decomposing learning task. During learning process, we record every training episode, and eliminate the state loops existed in it. We find some states have high probability (even to 1) appeared in all these acyclic episodes. We call these states critical states. That means, if agent wants to reach the goal state, then it will have high probability to pass these critical states according to the learned experience. So the critical states can be used to partition the state space for accomplishing learning task by stages. We also prove that the optimal policy found in the partitioned smaller state space is equivalent to the optimal policy found in the original state space. The experiment comparisons between Q-learning and Q-learning with critical states demonstrate our approach more effective. The more important is that our approach brings the light of how agent can use its own experience to plan learning for better performance.

In the Sampling-based motion planning, the collision checking procedure called many times and this operation takes the most planning time. The time spend in the collision checking increases with the number of obstacles. This paper aims to reduce the number of calls of the collision checking procedure in environment containing high number of obstacles. Our approach shows that the complexity of collision checking does not depend on the number of obstacles. The idea is to perform a pre-procedure that localizes the obstacles by segmenting the workspace into a grid and save it in a matrix form. This information is used in the collision checking procedure. We have used the Probabilistic Roadmaps (PRM) motion planner with a punctual robot and polygonal obstacles in 2D environment. Simulation results show the efficiency of our algorithm in reducing the complexity of collision checking and minimizing the planning time.

Sampling-based motion planning (SBMP) is a major trajectory planning approach in autonomous driving given its high efficiency in practice. As the core of SBMP schemes, sampling strategy holds the key to whether a smooth and collision-free trajectory can be found in real-time. Although some bias sampling strategies have been explored in the literature to accelerate SBMP, the trajectory generated under existing bias sampling strategies may lead to sharp lane changing. To address this issue, we propose a new learning framework for SBMP. Specifically, we develop a novel automatic labeling scheme and a 2-Stage prediction model to improve the accuracy in predicting the intention of surrounding vehicles. We then develop an imitation learning scheme to generate sample points based on the experience of human drivers. Using the prediction results, we design a new bias sampling strategy to accelerate the SBMP algorithm by strategically selecting necessary sample points that can generate a smooth and collision-free trajectory and avoid sharp lane changing. Data-driven experiments show that the proposed sampling strategy outperforms existing sampling strategies, in terms of the computing time, traveling time, and smoothness of the trajectory. The results also show that our scheme is even better than human drivers.

We present an optimization-based motion planner for medical steerable needles that explicitly considers motion and sensing uncertainty while guiding the needle to a target in 3D anatomy. Motion planning for needle steering is challenging because the needle is a nonholonomic and underactuated system, the needle's motion may be perturbed during insertion due to unmodeled needle/tissue interactions, and medical sensing modalities such as ultrasound imaging and x-ray projection imaging typically provide only noisy and partial state information. To account for these uncertainties, we introduce a motion planner that computes a trajectory and corresponding linear controller in the belief space - the space of distributions over the state space. We formulate the needle steering motion planning problem as a partially observable Markov decision process (POMDP) that approximates belief states as Gaussians. We then compute a locally optimal trajectory and corresponding controller that minimize in belief space a cost function that considers avoidance of obstacles, penalties for unsafe control inputs, and target acquisition accuracy. We apply the motion planner to simulated scenarios and show that local optimization in belief space enables us to compute higher quality plans compared to planning solely in the needle's state space.

Motion planning with imperfect state information is a crucial capability for autonomous robots to operate reliably in uncertain and dynamic environments. Partially observable Markov decision processes (POMDPs) provide a principled general framework for planning under uncertainty. Using probabilistic sampling, point-based POMDP solvers have drastically improved the speed of POMDP planning, enabling us to handle moderately complex robotic tasks. However, robot motion planning tasks with long time horizons remains a severe obstacle for even the fastest point-based POMDP solvers today. This paper proposes Milestone Guided Sampling (MiGS), a new point-based POMDP solver, which exploits state space information to reduce effective planning horizons. MiGS samples a set of points, called milestones, from a robot’s state space and constructs a simplified representation of the state space from the sampled milestones. It then uses this representation of the state space to guide sampling in the belief space and tries to capture the essential features of the belief space with a small number of sampled points. Preliminary results are very promising. We tested MiGS in simulation on several difficult POMDPs that model distinct robotic tasks with long time horizons in both 2-D and 3-D environments. These POMDPs are impossible to solve with the fastest point-based solvers today, but MiGS solved them in a few minutes.

Real-time motion planning and control of multi-joint robot manipulators are studied using neural networks. The proposed neural network approach consists of two modules: one for real-time collision-free motion planning, the other for real-time fine control of the robot manipulators. The motion planning module is a biologically inspired, parallel connected, topologically organised neural network, where each neuron is characterised by a shunting equation, whose state space is the robot configuration joint space. This module is capable of planning a real-time optimal path for robot manipulators through the dynamic activity landscape of the neural network. The motion control module involves of a feedforward neural network together with a traditional PD feedback loop. It is capable of achieving real-time fine motion control of robot manipulators under significant uncertainties and without any prior knowledge of the robot dynamics. This module can quickly compensate sudden changes in the robot dynamics. The real-time fine control of robot manipulators is achieved through the online learning of the neural network. Both the motion planning and motion control modules are computationally efficient, and their global stabilities are proved using Lyapunov stability theory.

State space is one of the key concepts of system theory. In general, a state space is introduced into a system description without examining its specific physical meaning. It is known, however, that if we select a suitable state space representation, it becomes easier for us to understand or to manipulate the property of a system. Thus, although it may not be meaningful to try to arrive at a physical meaning of the state space, it is important to find a way where a choice of a state space can be made in accordance with the system of interest, taking into account the theoretical meaning of that choice. In this paper, the choice of a state space is discussed in terms of the concepts that general systems theory have provided and the canonical forms of linear control theory are interpreted using the state space representation. Four realizations of a single-input and single-output discrete basic linear system are constructed, whose state spaces are derived from system properties. such as reachability, past-det...

Previous studies on Sound Event Localization and Detection (SELD) have primarily focused on CNN- and Transformer-based designs. While CNNs possess local receptive fields, making it difficult to capture global dependencies over long sequences, Transformers excel at modeling long-range dependencies but have limited sensitivity to local time–frequency features. Recently, the VMamba architecture, built upon the Visual State Space (VSS) model, has shown great promise in handling long sequences, yet it remains limited in modeling local spatial details. To address this issue, we propose a novel state space model with an attention-enhanced feature fusion mechanism, termed FFMamba, which balances both local spatial modeling and long-range dependency capture. At a fine-grained level, we design two key modules: the Multi-Scale Fusion Visual State Space (MSFVSS) module and the Wavelet Transform-Enhanced Downsampling (WTED) module. Specifically, the MSFVSS module integrates a Multi-Scale Fusion (MSF) component into the VSS framework, enhancing its ability to capture both long-range temporal dependencies and detailed local spatial information. Meanwhile, the WTED module employs a dual-branch design to fuse spatial and frequency domain features, improving the richness of feature representations. Comparative experiments were conducted on the DCASE2021 Task 3 and DCASE2022 Task 3 datasets. The results demonstrate that the proposed FFMamba model outperforms recent approaches in capturing long-range temporal dependencies and effectively integrating multi-scale audio features. In addition, ablation studies confirmed the effectiveness of the MSFVSS and WTED modules.

Various analytical techniques are available for modeling multivariate time series data. One such approach is the State Space Model, which can be employed to model this type of data. In this study, the data to be analyzed are data on the Indonesia Rupiah (IDR) exchange rate (ExR) against the US Dollar (USD), oil and gas exports (OGE), money supply (MS) and non-oil and gas exports (non-OGE) from January 2008 to December 2019. The aim of this study is to identify the most suitable state space model for the given data. In this research, the state space method will be applied to multivariate time series data, with the state space represented in the Vector Autoregressive (VAR) model to explore the interrelationships among groups of observed variables. The VAR model is a statistical technique used to analyze the relationships between variables in the dataset, employing the Granger Causality Test. The state space model is utilized to model and forecast multiple interconnected time series, where the variables exhibit dynamic interactions and to examine additional unobserved variables in the time series data. Based on the analysis results and the minimum value of the Akaike Information Criterion (AIC), the optimal VAR model identified is the VAR (6) model. The results of forecasting values using the state space model show that the predicted values and the real values for the state space model are very closed to each other.

Impact of control representations on efficiency of local nonholonomic motion planning In this paper various control representations selected from a family of harmonic controls were examined for the task of locally optimal motion planning of nonholonomic systems. To avoid dependence of results either on a particular system or a current point in a state space, considerations were carried out in a sub-space of a formal Lie algebra associated with a family of controlled systems. Analytical and simulation results are presented for two inputs and three dimensional state space and some hints for higher dimensional state spaces were given. Results of the paper are important for designers of motion planning algorithms not only to preserve controllability of the systems but also to optimize their motion.

Motion planning for robotic manipulation of clothing is a challenging problem as clothing articles have high-dimensional configuration spaces and are computationally expensive to simulate. We present an algorithm for robotic cloth folding that, given a sequence of desired folds, outputs a motion plan consisting of a sequence of primitives for a robot to fold the cloth. Previous work on cloth folding does not take into account the constraints of the robot, and thus produces plans which are often infeasible given the kinematics of robots like the Willow Garage PR2. In this paper we introduce a class of motion primitives that start and end in a subset of the cloth’s state space. To find a sequence of primitives that achieves all required folds, the algorithm takes advantage of the partial ordering inherent in folding, and produces a time-optimal motion plan (given the set of primitives) for the robot if one exists. We describe experiments with a general purpose mobile robotic platform, the PR2, folding articles that require dragging and base motion in addition to folding. Our experiments show that (1) many articles of clothing conform well enough to the assumptions made in our model and (2) this approach allows our robot to perform a wide variety of folds on articles of various sizes and shapes.