Periodic DMP formulation for Quaternion Trajectories

Dynamic movement primitives (DMPs) are a learning from demonstration (LfD) method that imitates movement primitives. Recently, reinforcement learning (RL) has been used to modify DMPs. In this regard, most methods have concentrated on improving DMPs with RL after DMPs are learned with supervised learning. In this way, there is no guarantee that the DMPs stay close to demonstrations while modified. To address this problem, in this study, a framework is proposed for learning DMPs with deep RL, which can be further used to modify DMPs. To do so, first, DMPs are modeled in the framework of deep RL. Then, after the reward function for learning is defined, they are learned with deep RL. The deep RL algorithm used in the present study is the twin-delayed deep deterministic policy gradient (TD3) algorithm. Our proposed algorithm is evaluated with various demonstrations, and a case study of a pick-and-place task is done. The results show that our method can learn all demonstrations with high accuracy.

Addressing the limitations of traditional Dynamic Movement Primitives (DMP) obstacle avoidance methods-which rely on dynamic coupling terms leading to complex algorithmic structures, sluggish responses, and difficulties in autonomous obstacle avoidance-this study innovatively proposes a robotic arm obstacle avoidance trajectory planning framework that integrates DMP with the Informed RRT* algorithm. This framework uses the Informed RRT algorithm to do local trajectory replanning in areas near obstacles. It keeps the main dynamic features of DMP and creates obstacle-avoidance trajectories that are then smoothly added to the original DMP trajectory. This approach eliminates the need for complex dynamic coupling terms, significantly reducing algorithmic complexity. Its local replanning mechanism not only enables autonomous obstacle avoidance and effectively suppresses trajectory oscillations but also substantially enhances real-time response speed. The main contributions of this study include 1) proposing a decoupled DMP obstacle avoidance method based on local trajectory replanning combined with Informed RRT*, simplifying the algorithm structure; 2) the adopted local replanning strategy significantly shortens response time and improves obstacle avoidance efficiency. The effectiveness of the proposed algorithm was validated through two sets of comparative experiments (against the RRT method) conducted on a collaborative robot platform. Experimental results demonstrate that the algorithm reduces response time for obstacle avoidance by 33.2% and increases success rate by 19.5%, fully proving its potential and applicability in specific industrial scenarios.

Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g. demonstrated by a human. DMPs are at their core second order autonomous differential equations. However, feedforward controls of robots with flexible joints are known to require reference trajectories up to the fourth derivative of position. Consequently, classical DMPs are mechanically not compatible with flexible joint robots. In this paper, we propose an extension of DMPs by introducing FlexDMPs. This concept retains the structural properties and benefits of classical DMPs but generates trajectories up to the fourth derivative that can theoretically be tracked ideally (i.e. with zero tracking error) by flexible joint robots. The concept is demonstrated on a high fidelity simulation model of an industrial robot and in experimental results on a collaborative manipulator.

The paper addresses the problem of learning internal task-specific dynamic models for a reaching task. Using task-specific dynamic models is crucial for achieving both high tracking accuracy and compliant behaviour, which improves safety concerns while working in unstructured environment or with humans. The proposed approach uses programming by demonstration to learn new task-related movements encoded as Compliant Movement Primitives (CMPs). CMPs are a combination of position trajectories encoded in a form of Dynamic Movement Primitives (DMPs) and corresponding task-specific Torque Primitives (TPs) encoded as a linear combination of kernel functions. Unlike the DMPs, TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability we propose a novel method which autonomously learns task-specific TPs, based on a given kinematic trajectory in DMPs.

In order to properly function in real-world environments, the gait of a humanoid robot must be able to adapt to new situations as well as to deal with unexpected perturbations. A promising research direction is the modular generation of movements that results from the combination of a set of basic primitives. Humans seem to be able to adapt their motion in an effortless way and efficient way. In this work we present a methodology to extract a single demonstration signal from a human teacher that can be used in a biped robot to walk in an efficient way and quickly adapt to new situations. Instead of using a normal human gait, the demonstrator walks on a gait we called "robot-like". It is our belief that the retargeting task will be more efficient and easier by using this methodology. The acquired signal is then used to train a set of Dynamic Movement Primitives (DMP) that directly retrieves the trajectories of the end effector in the Cartesian space. In previous works we have used this methodology with efficiency in several tasks, e.g., walking over irregularities, overcoming an obstacle. Now, by combining rhythmic DMP with discrete DMP, we show that the adaptability performance to new situations that require precise foot placement is increased.

Forestry cranes are an important tool for safe and efficient timber harvesting with forestry machines. However, their complex manual control often led to inefficiencies and excessive energy usage, due to the many joysticks and buttons that must be used in a precise sequence to perform efficient movements. To address this, the industry is increasingly turning to partial automation, making manual control more intuitive for the operator and, consequently, achieving improvements in energy efficiency. This article introduces a novel approach to energy-optimal motion planning that can be used along with a feedback control system to automate crane motions, taking over portions of the operator’s work. Our method combines dynamic movement primitives (DMPs) and an energy-optimization algorithm. DMPs is a machine learning technique for motion planning based on human demonstrations, while the optimization algorithm exploits the crane’s redundancy to find energy-optimal trajectories. Simulation results show that DMPs can replicate human-like controlled motions with a 25% reduction in energy consumption. However, our energy optimization algorithm shows improvements of over 40%, providing substantial energy savings and a promising pathway towards environmentally friendly partially automated machines.

Maintaining the balance is always a challenge issue for bipedal humanoid robots in dealing with various locomotive tasks to serve human society, especially when the real environment the robot worked within exhibits to be very complex. Unlike plenty of previous successful approaches on humanoid balancing or push recovery, in this research, the Dynamical Movement Primitives (DMP) is employed to model several typical bio-inspired strategies. As humanoid balancing or push recovery could be regarded as a problem of how a robot to get back to its ongoing behavior when a break happens on account of external force or uneven terrain etc., the DMP model becomes an alternative ideal choice due to its promising nature of attractor. Meanwhile, the DMP composed of a set of differential equations provides a compact parameterized representation in modelling a motion strategy, and thus leads to a strategy model that is suitable to be fulfilled with machine learning techniques. In this research, the learning process for those bio-inspired strategies modeled with DMP are completed by adopting the stochastic policy gradient reinforcement learning and imitation learning separately. Furthermore, with Gaussian Process(GP) regression, push recovery strategies are generalized taking the advantages of the invariance properties of the DMP model. As a consequence, an online adaptive push recovery control strategy is finally achieved. Experimental results on both simulated robot and a real bipedal humanoid robot PKU-HR5 demonstrate the presented approach is effective and promising.

Purpose This paper aims to solve the problem of the inability to apply learning methods for robot motion skills based on dynamic movement primitives (DMPs) in tasks with explicit environmental constraints, while ensuring the reliability of the robot system. Design/methodology/approach The authors propose a novel DMP that takes into account environmental constraints to enhance the generality of the robot motion skill learning method. First, based on the real-time state of the robot and environmental constraints, the task space is divided into different regions and different control strategies are used in each region. Second, to ensure the effectiveness of the generalized skills (trajectories), the control barrier function is extended to DMP to enforce constraint conditions. Finally, a skill modeling and learning algorithm flow is proposed that takes into account environmental constraints within DMPs. Findings By designing numerical simulation and prototype demonstration experiments to study skill learning and generalization under constrained environments. The experimental results demonstrate that the proposed method is capable of generating motion skills that satisfy environmental constraints. It ensures that robots remain in a safe position throughout the execution of generation skills, thereby avoiding any adverse impact on the surrounding environment. Originality/value This paper explores further applications of generalized motion skill learning methods on robots, enhancing the efficiency of robot operations in constrained environments, particularly in non-point-constrained environments. The improved methods are applicable to different types of robots.

Encoding desired motions into dynamic movement primitives (DMPs) is a common way for generating compact task representations that are able to handle sensor-based goal adaptations. At the same time, a robot should not only express adaptive motion capabilities at planning level, but use also contact wrench feedback in the adaptation and learning process of the DMP. Despite first approaches exist in this direction, no fully integrated approach has been proposed so far. In this paper, we introduce a new class of admittance-coupled DMPs that addresses environmental changes by including contact wrench feedback dynamics into the DMP formalism. Moreover, a novel iterative learning approach is devised that is based on monitoring the overall system passivity analysis in terms of reference power tracking. Simulations and experimental results with the Kuka LWR robot maintaining a non-rigid contact with the environment (wiping a surface) are shown for supporting the validity of our approach.

In this paper, composite dynamic movement primitives (DMPs) based on radial basis function neural networks (RBFNNs) are investigated for robots’ skill learning from human demonstrations. The composite DMPs could encode the position and orientation manipulation skills simultaneously for human-to-robot skills transfer. As the robot manipulator is expected to perform tasks in unstructured and uncertain environments, it requires the manipulator to own the adaptive ability to adjust its behaviours to new situations and environments. Since the DMPs can adapt to uncertainties and perturbation, and spatial and temporal scaling, it has been successfully employed for various tasks, such as trajectory planning and obstacle avoidance. However, the existing skill model mainly focuses on position or orientation modelling separately; it is a common constraint in terms of position and orientation simultaneously in practice. Besides, the generalisation of the skill learning model based on DMPs is still hard to deal with dynamic tasks, e.g., reaching a moving target and obstacle avoidance. In this paper, we proposed a composite DMPs-based framework representing position and orientation simultaneously for robot skill acquisition and the neural networks technique is used to train the skill model. The effectiveness of the proposed approach is validated by simulation and experiments.

Nowadays, endowing robots with the capability to learn is an important goal for the robotics research community. An important part of this research is learning skills. Dynamic movement primitives (DMPs) is a very powerful model to conduct learning from demonstration for robot. In this paper, we have made a great improvement on Local weighted Regression(LWR) which is an original regression technique in DMPs. Specifically, we change the phase from integrating into time average and give an logistic function to make sure the final forcing term to be zero. Then, we can make better use of min-jerk criterion demonstrate the effect and efficient.

This paper addresses a significant challenge in achieving collaborative tasks; how can a robot or multiple robots, endowed with a library of pre-learned primitive movements, generate multiple simultaneous coordinated robotic movements, adapting and optimizing those in the library, to complete one collaborative task? This work can thus be seen as a follow-up to the work with a motion presented as dynamic movement primitive (DMP) that now considers collaborative tasks and the existence of multiple robots/manipulators. Specifically, we start with a simple task using one DMP and extend it to accommodate the coordinated execution of multiple DMPs in robots with multiple manipulators or-alternatively-multiple robots with a single manipulator. We investigate mechanisms to jointly optimize multiple DMPs to perform one task in a coordinated fashion. The joint trajectory is built from initial DMPs learned for a single manipulator, and its optimization must comply with task-specific constraints. We illustrate the application of our approach both in a simulated environment and in a simulated and real Baxter robot.

Movement primitives are a well-established approach to robot motion planning, as they offer a modular basis for robot motion planning. Dynamic movement primitives (DMPs) are a popular control framework based on nonlinear differential equations which, when scaled in time, produce a smooth kinematic movement plan. In this letter, we introduce an adjusted movement primitive framework which replaces the stable equilibria of DMPs with saddle points. These saddle points are linked together in stable heteroclinic channels (SHCs) which are in turn part of dynamical systems called stable heteroclinic networks. SHC-based movement primitives (SMPs) form a framework where the weights of the kernel functions have spatial significance in the task space. In this letter, we show that SMPs and DMPs perform comparably according to a kinematic-based cost function. We also show that SMP kernel weights follow a given trajectory when plotted in the task space. The plotted kernel weights provide an intuitive tool for managing the controller.

Kinesthetic teaching and Dynamic Movement Primitives (DMPs) enable fast and adaptable learning of robot tasks based on a human demonstration. A task encoded as a dynamic movement primitive can be reused with a different goal position, albeit with a resulting distortion in the approach trajectory with regards to the original task. While this is sufficient for some robotic applications, the accuracy requirements for assembly tasks in an industrial context, where tolerances are tight and workpieces are small, is much higher. In such a context it is also preferable to keep the number of demonstrations and of external sensors low. Our approach relies on a single demonstration and a single force-torque sensor at the robot tool. We make use of a Convolutional Neural Network (CNN) trained on the force-torque sensor data to segment the task into several movement primitives for the different phases: pickup - approach - insertion - retraction, allowing us to achieve better positional accuracy when generalizing the task primitives to new targets. To the best of our awareness, we are the first to utilize a CNN as a segmentation tool to improve the generalization performance of DMPs.

Movement generation, and especially generalization to unseen situations, plays an important role in robotics. Different types of movement generation methods exist such as spline based methods, dynamical system based methods, and methods based on Gaussian mixture models (GMMs). Using a large, new dataset on human manipulations, in this paper we provide a highly detailed comparison of five fundamentally different and widely used movement encoding and generation frameworks: dynamic movement primitives (DMPs), time based Gaussian mixture regression (tbGMR), stable estimator of dynamical systems (SEDS), Probabilistic Movement Primitives (ProMP) and Optimal Control Primitives (OCP). We compare these frameworks with respect to their movement encoding efficiency, reconstruction accuracy, and movement generalization capabilities. The new dataset consists of nine object manipulation actions performed by 12 humans: pick and place, put on top/take down, put inside/take out, hide/uncover, and push/pull with a total of 7,652 movement examples. Our analysis shows that for movement encoding and reconstruction DMPs and OCPs are the most efficient with respect to the number of parameters and reconstruction accuracy, if a sufficient number of kernels is used. In case of movement generalization to new start- and end-point situations, DMPs, OCPs and task parameterized GMM (TP-GMM, movement generalization framework based on tbGMR) lead to similar performance, which ProMPs only achieve when using many demonstrations for learning. All models outperform SEDS, which additionally proves to be difficult to fit. Furthermore we observe that TP-GMM and SEDS suffer from problems reaching the end-points of generalizations. These different quantitative results will help selecting the most appropriate models and designing trajectory representations in an improved task-dependent way in future robotic applications. • Comparison of five motion encoding frameworks on human manipulation action dataset. • Large dataset containing 7652 movement trajectories from 9 different action classes. • Comparison regarding encoding and generalization reveals model specific differences.