Optimize and Coordinate Multiple DMPs Under Constraints to Achieve a Collaborative Manipulation Task

Dynamic movement primitives (DMPs) are a learning from demonstration (LfD) method that imitates movement primitives. Recently, reinforcement learning (RL) has been used to modify DMPs. In this regard, most methods have concentrated on improving DMPs with RL after DMPs are learned with supervised learning. In this way, there is no guarantee that the DMPs stay close to demonstrations while modified. To address this problem, in this study, a framework is proposed for learning DMPs with deep RL, which can be further used to modify DMPs. To do so, first, DMPs are modeled in the framework of deep RL. Then, after the reward function for learning is defined, they are learned with deep RL. The deep RL algorithm used in the present study is the twin-delayed deep deterministic policy gradient (TD3) algorithm. Our proposed algorithm is evaluated with various demonstrations, and a case study of a pick-and-place task is done. The results show that our method can learn all demonstrations with high accuracy.

Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g. demonstrated by a human. DMPs are at their core second order autonomous differential equations. However, feedforward controls of robots with flexible joints are known to require reference trajectories up to the fourth derivative of position. Consequently, classical DMPs are mechanically not compatible with flexible joint robots. In this paper, we propose an extension of DMPs by introducing FlexDMPs. This concept retains the structural properties and benefits of classical DMPs but generates trajectories up to the fourth derivative that can theoretically be tracked ideally (i.e. with zero tracking error) by flexible joint robots. The concept is demonstrated on a high fidelity simulation model of an industrial robot and in experimental results on a collaborative manipulator.

Maintaining the balance is always a challenge issue for bipedal humanoid robots in dealing with various locomotive tasks to serve human society, especially when the real environment the robot worked within exhibits to be very complex. Unlike plenty of previous successful approaches on humanoid balancing or push recovery, in this research, the Dynamical Movement Primitives (DMP) is employed to model several typical bio-inspired strategies. As humanoid balancing or push recovery could be regarded as a problem of how a robot to get back to its ongoing behavior when a break happens on account of external force or uneven terrain etc., the DMP model becomes an alternative ideal choice due to its promising nature of attractor. Meanwhile, the DMP composed of a set of differential equations provides a compact parameterized representation in modelling a motion strategy, and thus leads to a strategy model that is suitable to be fulfilled with machine learning techniques. In this research, the learning process for those bio-inspired strategies modeled with DMP are completed by adopting the stochastic policy gradient reinforcement learning and imitation learning separately. Furthermore, with Gaussian Process(GP) regression, push recovery strategies are generalized taking the advantages of the invariance properties of the DMP model. As a consequence, an online adaptive push recovery control strategy is finally achieved. Experimental results on both simulated robot and a real bipedal humanoid robot PKU-HR5 demonstrate the presented approach is effective and promising.

Previous task complexity studies have suggested that learners produce more negotiation of meaning opportunities during complex tasks than simple tasks (Robinson, 2011). The present study builds on the existing task complexity literature by examining the impact of task complexity and pragmatic situational demands on the number of learning opportunities for request‐making expressions. Forty‐nine Korean junior high school learners of English as a foreign language were randomly assigned to participate in either simple‐ or complex‐ collaborative pair writing tasks, which included different levels of pragmatic situational demands. The interactions were transcribed and analyzed for the frequency of pragmatic related episodes (PREs; Kim & Taguchi, 2015) as well as the number of turns within each PRE. The results showed that task complexity played an important role in facilitating PREs by targeting sociopragmatic factors and not pragmalinguistic forms, regardless of the level of pragmatic task demands. Results are discussed in light of promoting interaction‐driven learning opportunities for pragmatics using collaborative tasks.

Addressing the limitations of traditional Dynamic Movement Primitives (DMP) obstacle avoidance methods-which rely on dynamic coupling terms leading to complex algorithmic structures, sluggish responses, and difficulties in autonomous obstacle avoidance-this study innovatively proposes a robotic arm obstacle avoidance trajectory planning framework that integrates DMP with the Informed RRT* algorithm. This framework uses the Informed RRT algorithm to do local trajectory replanning in areas near obstacles. It keeps the main dynamic features of DMP and creates obstacle-avoidance trajectories that are then smoothly added to the original DMP trajectory. This approach eliminates the need for complex dynamic coupling terms, significantly reducing algorithmic complexity. Its local replanning mechanism not only enables autonomous obstacle avoidance and effectively suppresses trajectory oscillations but also substantially enhances real-time response speed. The main contributions of this study include 1) proposing a decoupled DMP obstacle avoidance method based on local trajectory replanning combined with Informed RRT*, simplifying the algorithm structure; 2) the adopted local replanning strategy significantly shortens response time and improves obstacle avoidance efficiency. The effectiveness of the proposed algorithm was validated through two sets of comparative experiments (against the RRT method) conducted on a collaborative robot platform. Experimental results demonstrate that the algorithm reduces response time for obstacle avoidance by 33.2% and increases success rate by 19.5%, fully proving its potential and applicability in specific industrial scenarios.

The paper addresses the problem of learning internal task-specific dynamic models for a reaching task. Using task-specific dynamic models is crucial for achieving both high tracking accuracy and compliant behaviour, which improves safety concerns while working in unstructured environment or with humans. The proposed approach uses programming by demonstration to learn new task-related movements encoded as Compliant Movement Primitives (CMPs). CMPs are a combination of position trajectories encoded in a form of Dynamic Movement Primitives (DMPs) and corresponding task-specific Torque Primitives (TPs) encoded as a linear combination of kernel functions. Unlike the DMPs, TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability we propose a novel method which autonomously learns task-specific TPs, based on a given kinematic trajectory in DMPs.

In order to properly function in real-world environments, the gait of a humanoid robot must be able to adapt to new situations as well as to deal with unexpected perturbations. A promising research direction is the modular generation of movements that results from the combination of a set of basic primitives. Humans seem to be able to adapt their motion in an effortless way and efficient way. In this work we present a methodology to extract a single demonstration signal from a human teacher that can be used in a biped robot to walk in an efficient way and quickly adapt to new situations. Instead of using a normal human gait, the demonstrator walks on a gait we called "robot-like". It is our belief that the retargeting task will be more efficient and easier by using this methodology. The acquired signal is then used to train a set of Dynamic Movement Primitives (DMP) that directly retrieves the trajectories of the end effector in the Cartesian space. In previous works we have used this methodology with efficiency in several tasks, e.g., walking over irregularities, overcoming an obstacle. Now, by combining rhythmic DMP with discrete DMP, we show that the adaptability performance to new situations that require precise foot placement is increased.

Forestry cranes are an important tool for safe and efficient timber harvesting with forestry machines. However, their complex manual control often led to inefficiencies and excessive energy usage, due to the many joysticks and buttons that must be used in a precise sequence to perform efficient movements. To address this, the industry is increasingly turning to partial automation, making manual control more intuitive for the operator and, consequently, achieving improvements in energy efficiency. This article introduces a novel approach to energy-optimal motion planning that can be used along with a feedback control system to automate crane motions, taking over portions of the operator’s work. Our method combines dynamic movement primitives (DMPs) and an energy-optimization algorithm. DMPs is a machine learning technique for motion planning based on human demonstrations, while the optimization algorithm exploits the crane’s redundancy to find energy-optimal trajectories. Simulation results show that DMPs can replicate human-like controlled motions with a 25% reduction in energy consumption. However, our energy optimization algorithm shows improvements of over 40%, providing substantial energy savings and a promising pathway towards environmentally friendly partially automated machines.

Purpose This paper aims to solve the problem of the inability to apply learning methods for robot motion skills based on dynamic movement primitives (DMPs) in tasks with explicit environmental constraints, while ensuring the reliability of the robot system. Design/methodology/approach The authors propose a novel DMP that takes into account environmental constraints to enhance the generality of the robot motion skill learning method. First, based on the real-time state of the robot and environmental constraints, the task space is divided into different regions and different control strategies are used in each region. Second, to ensure the effectiveness of the generalized skills (trajectories), the control barrier function is extended to DMP to enforce constraint conditions. Finally, a skill modeling and learning algorithm flow is proposed that takes into account environmental constraints within DMPs. Findings By designing numerical simulation and prototype demonstration experiments to study skill learning and generalization under constrained environments. The experimental results demonstrate that the proposed method is capable of generating motion skills that satisfy environmental constraints. It ensures that robots remain in a safe position throughout the execution of generation skills, thereby avoiding any adverse impact on the surrounding environment. Originality/value This paper explores further applications of generalized motion skill learning methods on robots, enhancing the efficiency of robot operations in constrained environments, particularly in non-point-constrained environments. The improved methods are applicable to different types of robots.

Imitation learning techniques have been used as a way to transfer skills to robots. Among them, dynamic movement primitives (DMPs) have been widely exploited as an effective and an efficient technique to learn and reproduce complex discrete and periodic skills. While DMPs have been properly formulated for learning point-to-point movements for both translation and orientation, periodic ones are missing a formulation to learn the orientation. To address this gap, we propose a novel DMP formulation that enables encoding of periodic orientation trajectories. Within this formulation we develop two approaches: Riemannian metric-based projection approach and unit quaternion based periodic DMP. Both formulations exploit unit quaternions to represent the orientation. However, the first exploits the properties of Riemannian manifolds to work in the tangent space of the unit sphere. The second encodes directly the unit quaternion trajectory while guaranteeing the unitary norm of the generated quaternions. We validated the technical aspects of the proposed methods in simulation. Then we performed experiments on a real robot to execute daily tasks that involve periodic orientation changes (i.e., surface polishing/wiping and liquid mixing by shaking).

Encoding desired motions into dynamic movement primitives (DMPs) is a common way for generating compact task representations that are able to handle sensor-based goal adaptations. At the same time, a robot should not only express adaptive motion capabilities at planning level, but use also contact wrench feedback in the adaptation and learning process of the DMP. Despite first approaches exist in this direction, no fully integrated approach has been proposed so far. In this paper, we introduce a new class of admittance-coupled DMPs that addresses environmental changes by including contact wrench feedback dynamics into the DMP formalism. Moreover, a novel iterative learning approach is devised that is based on monitoring the overall system passivity analysis in terms of reference power tracking. Simulations and experimental results with the Kuka LWR robot maintaining a non-rigid contact with the environment (wiping a surface) are shown for supporting the validity of our approach.

In this paper, composite dynamic movement primitives (DMPs) based on radial basis function neural networks (RBFNNs) are investigated for robots’ skill learning from human demonstrations. The composite DMPs could encode the position and orientation manipulation skills simultaneously for human-to-robot skills transfer. As the robot manipulator is expected to perform tasks in unstructured and uncertain environments, it requires the manipulator to own the adaptive ability to adjust its behaviours to new situations and environments. Since the DMPs can adapt to uncertainties and perturbation, and spatial and temporal scaling, it has been successfully employed for various tasks, such as trajectory planning and obstacle avoidance. However, the existing skill model mainly focuses on position or orientation modelling separately; it is a common constraint in terms of position and orientation simultaneously in practice. Besides, the generalisation of the skill learning model based on DMPs is still hard to deal with dynamic tasks, e.g., reaching a moving target and obstacle avoidance. In this paper, we proposed a composite DMPs-based framework representing position and orientation simultaneously for robot skill acquisition and the neural networks technique is used to train the skill model. The effectiveness of the proposed approach is validated by simulation and experiments.

Nowadays, endowing robots with the capability to learn is an important goal for the robotics research community. An important part of this research is learning skills. Dynamic movement primitives (DMPs) is a very powerful model to conduct learning from demonstration for robot. In this paper, we have made a great improvement on Local weighted Regression(LWR) which is an original regression technique in DMPs. Specifically, we change the phase from integrating into time average and give an logistic function to make sure the final forcing term to be zero. Then, we can make better use of min-jerk criterion demonstrate the effect and efficient.

This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking. During robot learning demonstrations, dynamic movement primitives (DMPs) are used to model robotic motion. Each DMP consists of a set of dynamic systems that enhances the stability of the generated motion toward the goal. A Gaussian mixture model and Gaussian mixture regression are integrated to improve the learning performance of the DMP, such that more features of the skill can be extracted from multiple demonstrations. The motion generated from the learned model can be scaled in space and time. Besides, a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model. In this controller, a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. The experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed methods.

Movement primitives are a well-established approach to robot motion planning, as they offer a modular basis for robot motion planning. Dynamic movement primitives (DMPs) are a popular control framework based on nonlinear differential equations which, when scaled in time, produce a smooth kinematic movement plan. In this letter, we introduce an adjusted movement primitive framework which replaces the stable equilibria of DMPs with saddle points. These saddle points are linked together in stable heteroclinic channels (SHCs) which are in turn part of dynamical systems called stable heteroclinic networks. SHC-based movement primitives (SMPs) form a framework where the weights of the kernel functions have spatial significance in the task space. In this letter, we show that SMPs and DMPs perform comparably according to a kinematic-based cost function. We also show that SMP kernel weights follow a given trajectory when plotted in the task space. The plotted kernel weights provide an intuitive tool for managing the controller.