Comparison of motion encoding frameworks on human manipulation actions

This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking. During robot learning demonstrations, dynamic movement primitives (DMPs) are used to model robotic motion. Each DMP consists of a set of dynamic systems that enhances the stability of the generated motion toward the goal. A Gaussian mixture model and Gaussian mixture regression are integrated to improve the learning performance of the DMP, such that more features of the skill can be extracted from multiple demonstrations. The motion generated from the learned model can be scaled in space and time. Besides, a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model. In this controller, a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. The experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed methods.

Dynamic movement primitives (DMPs) are a learning from demonstration (LfD) method that imitates movement primitives. Recently, reinforcement learning (RL) has been used to modify DMPs. In this regard, most methods have concentrated on improving DMPs with RL after DMPs are learned with supervised learning. In this way, there is no guarantee that the DMPs stay close to demonstrations while modified. To address this problem, in this study, a framework is proposed for learning DMPs with deep RL, which can be further used to modify DMPs. To do so, first, DMPs are modeled in the framework of deep RL. Then, after the reward function for learning is defined, they are learned with deep RL. The deep RL algorithm used in the present study is the twin-delayed deep deterministic policy gradient (TD3) algorithm. Our proposed algorithm is evaluated with various demonstrations, and a case study of a pick-and-place task is done. The results show that our method can learn all demonstrations with high accuracy.

Movement primitives are a well-established approach to robot motion planning, as they offer a modular basis for robot motion planning. Dynamic movement primitives (DMPs) are a popular control framework based on nonlinear differential equations which, when scaled in time, produce a smooth kinematic movement plan. In this letter, we introduce an adjusted movement primitive framework which replaces the stable equilibria of DMPs with saddle points. These saddle points are linked together in stable heteroclinic channels (SHCs) which are in turn part of dynamical systems called stable heteroclinic networks. SHC-based movement primitives (SMPs) form a framework where the weights of the kernel functions have spatial significance in the task space. In this letter, we show that SMPs and DMPs perform comparably according to a kinematic-based cost function. We also show that SMP kernel weights follow a given trajectory when plotted in the task space. The plotted kernel weights provide an intuitive tool for managing the controller.

Learning from demonstration (LfD), which provides a natural way to transfer skills to robots, has been extensively researched for decades, and an army of methods and applications have been developed and investigated for learning an individual or low-level task. Nevertheless, learning long time sequential tasks is still very difficult as it involves task segmentation and sub-task clustering under an extremely large demonstration variance. Besides, the representation problem should be considered when doing segmentation. This paper presents a new unified framework to solve the problems of segmentation, clustering, and representation in a sequential task. The segmentation algorithm segments unstructured demonstrations into movement primitives (MPs). Then, the MPs are automatically clustered and labeled so that they can be reused in other tasks. Finally, the representation model is leveraged to encode and generalize the learned MPs in new contexts. To achieve the first goal, a change-point detection algorithm based on Bayesian inference is leveraged. It can segment unstructured demonstrations online with minimum prior knowledge requirements. By following the Gaussian distributed assumption in the segmentation model, MPs are encoded by Gaussians or Gaussian mixture models. Thus, the clustering of MPs is formulated as a clustering over cluster (CoC) problem. The Kullback–Leibler divergence is used to measure similarities between MPs, through which the MPs with smaller distance are clustered into the same group. To replay and generalize the task in novel contexts, we use task-parameterized regression models such as the Gaussian mixture regression. We implemented our framework on a sequential open-and-place task. The experiments demonstrate that the segmentation accuracy of our framework can reach 94.3% and the recognition accuracy can reach 97.1%. Comparisons with the state-of-the-art algorithm also indicate that our framework is superior or comparable to their results.

Learning from demonstrations (LfD) is gaining more popularity in robotics due to its promise of providing a human-friendly technique for teaching robots new skills by robotics-naive users. The two main approaches to LfD are dynamic motor primitives (DMP) which models demonstrated motions as dynamical systems with the advantage flexibility in changing the motion’s starting position, goal or speed and Gaussian Mixture Modelling/ Gaussian Mixture Regression (GMM/GMR) which represents demonstrated motions as mixtures of Gaussians with the advantage of keeping track of the correlations between different dimensions of learned motions and automatic extraction of motion variability along these dimensions. This paper introduces a third approach that relies on symbolization of demonstrated motions by extending the Symbolic Aggregate approXimation (SAX) to handle multiple dimensions of data. The proposed approach is shown through several real-world evaluations to be more resistant to confusing demonstrations that usually arise when action segmentation is automated. The paper also discusses a possible way to combine SAX based LfD withGMM/GMR in order to preserve the advantages of these two approaches while providing superior confusion resistance.

The paper addresses the problem of learning internal task-specific dynamic models for a reaching task. Using task-specific dynamic models is crucial for achieving both high tracking accuracy and compliant behaviour, which improves safety concerns while working in unstructured environment or with humans. The proposed approach uses programming by demonstration to learn new task-related movements encoded as Compliant Movement Primitives (CMPs). CMPs are a combination of position trajectories encoded in a form of Dynamic Movement Primitives (DMPs) and corresponding task-specific Torque Primitives (TPs) encoded as a linear combination of kernel functions. Unlike the DMPs, TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability we propose a novel method which autonomously learns task-specific TPs, based on a given kinematic trajectory in DMPs.

This paper proposes a robot automatic valve turning control strategy based on teaching learning, which consists of teaching, model learning and task repetition. The first stage is the teaching and learning stage. The robot learns motor skills by observing the human performing tasks. In order to accurately learn motor skills from demonstrations, data alignment is performed on the teaching data through Dynamic Time Warping (DTW). The second stage is the model construction and learning stage. The high-level learning strategy aims to learn motor skills from demonstrations through Dynamic Movement Primitives (DMP), using the statistical approach Gaussian Mixture Model and Gaussian Mixture Regression (GMM-GMR) to analyze the data from demonstrations. And the valve turning is repetition. To verify the effectiveness of the proposed control strategy, the experiment of the butterfly valve closing is performed. The results show that the robot is able to learn and reproduce the valve reaching and turning tasks. It completes the valve closing action by turning the valve for 7 turns.

Teaching by demonstration (TbD) is a powerful way to generalize the skills learned from human demonstrations to fulfill complex requirements. The mobile robot can learn new skills from interaction with human being in an intuitive way. In this paper, we propose a human-robot skill transfer system for a mobile robot that is instructed to follow a trajectory demonstrated by a human teacher wearing a motion capturing device while the Kinect sensor is recording the trajectory. With multi-modal sensor fusion, the position and velocity of the human teacher are enhanced for the correction and accuracy. A nonlinear system named a dynamic movement primitive (DMP) is modeled by the trajectories data. The Gaussian mixture model is applied for the appraisal of DMP, so as to model numerous trajectories through the teaching of a demonstration. Further, to achieve the accuracy of the trajectory tracking, a novel nonlinear model predictive control (MPC) approach is proposed for motion control. By comparison with other obstacle avoidance works, the results show improved performances in terms of the computing time and length of the trajectory.

Simple and complex behavior learning using behavior hidden Markov model and CobART

Kinesthetic teaching and Dynamic Movement Primitives (DMPs) enable fast and adaptable learning of robot tasks based on a human demonstration. A task encoded as a dynamic movement primitive can be reused with a different goal position, albeit with a resulting distortion in the approach trajectory with regards to the original task. While this is sufficient for some robotic applications, the accuracy requirements for assembly tasks in an industrial context, where tolerances are tight and workpieces are small, is much higher. In such a context it is also preferable to keep the number of demonstrations and of external sensors low. Our approach relies on a single demonstration and a single force-torque sensor at the robot tool. We make use of a Convolutional Neural Network (CNN) trained on the force-torque sensor data to segment the task into several movement primitives for the different phases: pickup - approach - insertion - retraction, allowing us to achieve better positional accuracy when generalizing the task primitives to new targets. To the best of our awareness, we are the first to utilize a CNN as a segmentation tool to improve the generalization performance of DMPs.

In order to properly function in real-world environments, the gait of a humanoid robot must be able to adapt to new situations as well as to deal with unexpected perturbations. A promising research direction is the modular generation of movements that results from the combination of a set of basic primitives. Humans seem to be able to adapt their motion in an effortless way and efficient way. In this work we present a methodology to extract a single demonstration signal from a human teacher that can be used in a biped robot to walk in an efficient way and quickly adapt to new situations. Instead of using a normal human gait, the demonstrator walks on a gait we called "robot-like". It is our belief that the retargeting task will be more efficient and easier by using this methodology. The acquired signal is then used to train a set of Dynamic Movement Primitives (DMP) that directly retrieves the trajectories of the end effector in the Cartesian space. In previous works we have used this methodology with efficiency in several tasks, e.g., walking over irregularities, overcoming an obstacle. Now, by combining rhythmic DMP with discrete DMP, we show that the adaptability performance to new situations that require precise foot placement is increased.

Addressing the limitations of traditional Dynamic Movement Primitives (DMP) obstacle avoidance methods-which rely on dynamic coupling terms leading to complex algorithmic structures, sluggish responses, and difficulties in autonomous obstacle avoidance-this study innovatively proposes a robotic arm obstacle avoidance trajectory planning framework that integrates DMP with the Informed RRT* algorithm. This framework uses the Informed RRT algorithm to do local trajectory replanning in areas near obstacles. It keeps the main dynamic features of DMP and creates obstacle-avoidance trajectories that are then smoothly added to the original DMP trajectory. This approach eliminates the need for complex dynamic coupling terms, significantly reducing algorithmic complexity. Its local replanning mechanism not only enables autonomous obstacle avoidance and effectively suppresses trajectory oscillations but also substantially enhances real-time response speed. The main contributions of this study include 1) proposing a decoupled DMP obstacle avoidance method based on local trajectory replanning combined with Informed RRT*, simplifying the algorithm structure; 2) the adopted local replanning strategy significantly shortens response time and improves obstacle avoidance efficiency. The effectiveness of the proposed algorithm was validated through two sets of comparative experiments (against the RRT method) conducted on a collaborative robot platform. Experimental results demonstrate that the algorithm reduces response time for obstacle avoidance by 33.2% and increases success rate by 19.5%, fully proving its potential and applicability in specific industrial scenarios.

Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g. demonstrated by a human. DMPs are at their core second order autonomous differential equations. However, feedforward controls of robots with flexible joints are known to require reference trajectories up to the fourth derivative of position. Consequently, classical DMPs are mechanically not compatible with flexible joint robots. In this paper, we propose an extension of DMPs by introducing FlexDMPs. This concept retains the structural properties and benefits of classical DMPs but generates trajectories up to the fourth derivative that can theoretically be tracked ideally (i.e. with zero tracking error) by flexible joint robots. The concept is demonstrated on a high fidelity simulation model of an industrial robot and in experimental results on a collaborative manipulator.

Recently, as a highly infectious disease of novel coronavirus (COVID-19) has swept the globe, more and more patients need to be isolated in the rooms of the hospitals, so how to deliver the meals or drugs to these infectious patients is the urgent work. It is a reliable and effective method to transport medical supplies or meals to patients using robots, but how to teach the robot to the destination and to enter the door like a human is an exciting task. In this paper, a novel human-like control framework for the mobile medical service robot is considered, where a Kinect sensor is used to manage human activity recognition to generate a designed teaching trajectory. Meanwhile, the learning technique of dynamic movement primitives (DMP) with the Gaussian mixture model (GMM) is applied to transfer the skill from humans to robots. A neural-based model predictive tracking controller is implemented to follow the teaching trajectory. Finally, some demonstrations are carried out in a hospital room to illustrate the superiority and effectiveness of the developed framework.

Dynamic movement primitives are one of key concepts for understanding dexterous and flexible movements of biological bodies. In the field of robotics engineering, simple types of nonlinear differential equations are used to generate movement primitives from demonstrations, but it remains unclear how nonlinear dynamics in the real brain can also generate movement primitives in biologically natural ways. The aim of this study is to investigate a possible role of nonlinear dynamics in random recurrent neural networks (RNNs) for skillful motor learning. We show that one-shot temporal patterns such arm reaching movements can be trained by a type of RNN-learning so-called FORCE-learning recently proposed by Sussillo and Abbott and a number of patterns are summarized as a manifold embedded in a space of synaptic weights of readout neurons. We also discuss how generalization of learning against untrained motor patterns can be achieved by identifying nonlinear coordinates (meta-parameters) on this manifold in a higher level of the central nervous system.