Robot Manipulation Skill Learning Based on Dynamic Movement Primitive

This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking. During robot learning demonstrations, dynamic movement primitives (DMPs) are used to model robotic motion. Each DMP consists of a set of dynamic systems that enhances the stability of the generated motion toward the goal. A Gaussian mixture model and Gaussian mixture regression are integrated to improve the learning performance of the DMP, such that more features of the skill can be extracted from multiple demonstrations. The motion generated from the learned model can be scaled in space and time. Besides, a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model. In this controller, a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. The experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed methods.

Learning from demonstrations (LfD) is gaining more popularity in robotics due to its promise of providing a human-friendly technique for teaching robots new skills by robotics-naive users. The two main approaches to LfD are dynamic motor primitives (DMP) which models demonstrated motions as dynamical systems with the advantage flexibility in changing the motion’s starting position, goal or speed and Gaussian Mixture Modelling/ Gaussian Mixture Regression (GMM/GMR) which represents demonstrated motions as mixtures of Gaussians with the advantage of keeping track of the correlations between different dimensions of learned motions and automatic extraction of motion variability along these dimensions. This paper introduces a third approach that relies on symbolization of demonstrated motions by extending the Symbolic Aggregate approXimation (SAX) to handle multiple dimensions of data. The proposed approach is shown through several real-world evaluations to be more resistant to confusing demonstrations that usually arise when action segmentation is automated. The paper also discusses a possible way to combine SAX based LfD withGMM/GMR in order to preserve the advantages of these two approaches while providing superior confusion resistance.

Movement generation, and especially generalization to unseen situations, plays an important role in robotics. Different types of movement generation methods exist such as spline based methods, dynamical system based methods, and methods based on Gaussian mixture models (GMMs). Using a large, new dataset on human manipulations, in this paper we provide a highly detailed comparison of five fundamentally different and widely used movement encoding and generation frameworks: dynamic movement primitives (DMPs), time based Gaussian mixture regression (tbGMR), stable estimator of dynamical systems (SEDS), Probabilistic Movement Primitives (ProMP) and Optimal Control Primitives (OCP). We compare these frameworks with respect to their movement encoding efficiency, reconstruction accuracy, and movement generalization capabilities. The new dataset consists of nine object manipulation actions performed by 12 humans: pick and place, put on top/take down, put inside/take out, hide/uncover, and push/pull with a total of 7,652 movement examples. Our analysis shows that for movement encoding and reconstruction DMPs and OCPs are the most efficient with respect to the number of parameters and reconstruction accuracy, if a sufficient number of kernels is used. In case of movement generalization to new start- and end-point situations, DMPs, OCPs and task parameterized GMM (TP-GMM, movement generalization framework based on tbGMR) lead to similar performance, which ProMPs only achieve when using many demonstrations for learning. All models outperform SEDS, which additionally proves to be difficult to fit. Furthermore we observe that TP-GMM and SEDS suffer from problems reaching the end-points of generalizations. These different quantitative results will help selecting the most appropriate models and designing trajectory representations in an improved task-dependent way in future robotic applications. • Comparison of five motion encoding frameworks on human manipulation action dataset. • Large dataset containing 7652 movement trajectories from 9 different action classes. • Comparison regarding encoding and generalization reveals model specific differences.

Purpose This paper aims to solve the problem of the inability to apply learning methods for robot motion skills based on dynamic movement primitives (DMPs) in tasks with explicit environmental constraints, while ensuring the reliability of the robot system. Design/methodology/approach The authors propose a novel DMP that takes into account environmental constraints to enhance the generality of the robot motion skill learning method. First, based on the real-time state of the robot and environmental constraints, the task space is divided into different regions and different control strategies are used in each region. Second, to ensure the effectiveness of the generalized skills (trajectories), the control barrier function is extended to DMP to enforce constraint conditions. Finally, a skill modeling and learning algorithm flow is proposed that takes into account environmental constraints within DMPs. Findings By designing numerical simulation and prototype demonstration experiments to study skill learning and generalization under constrained environments. The experimental results demonstrate that the proposed method is capable of generating motion skills that satisfy environmental constraints. It ensures that robots remain in a safe position throughout the execution of generation skills, thereby avoiding any adverse impact on the surrounding environment. Originality/value This paper explores further applications of generalized motion skill learning methods on robots, enhancing the efficiency of robot operations in constrained environments, particularly in non-point-constrained environments. The improved methods are applicable to different types of robots.

As global energy demand continues to grow, the inherent safety requirements for natural gas long-distance pipelines are becoming increasingly stringent. Therefore, accurately analyzing the trends in pipeline defects using multi-round internal inspection data is of great significance for enhancing pipeline inherent safety levels and reducing the risk of pipeline medium leakage. However, existing pipeline in-line inspection data alignment methods for long-distance multi-round pipeline data alignment suffer from cumbersome alignment procedures and low computational efficiency. This paper proposes an adaptive threshold dynamic time warping defect alignment method (Adaptive Dynamic Threshold-Dynamic Time Warping, ADTH-DTW) for rapidly matching multi-round in-line inspection data. A new multi-round in-line inspection data alignment framework based on valve-weld-defect is established. By integrating the DTW algorithm into each alignment stage, unnecessary manual effort is avoided, significantly improving data alignment efficiency. First, the ADTH method is used to clean redundant weld seam data in the in-line inspection data. By dynamically generating expected values and combining an intelligent point selection strategy, the method accurately identifies and removes interfering data. Additionally, valve chamber data is used to correct the overall mileage, providing a data foundation for subsequent defect alignment. Second, the dynamic time warping algorithm is used to align weld seam data and establish a data mapping table. Finally, relative displacement methods are employed to achieve defect matching. The validation results from three rounds of in-vehicle inspection data tested on-site indicate that the ADTH-DTW algorithm achieves an average 23.08% improvement in alignment accuracy compared to methods such as DTW, KL divergence, JS divergence, and linear interpolation, with computational efficiency nearly tripled. This effectively addresses the issue of incompatible computational efficiency and accuracy in existing data alignment algorithms, thereby enhancing the intrinsic safety level of natural gas long-distance pipelines.

Neuroanatomical Correlates of Motor Skill Learning: Inferences from Neuroimaging to Behavior

Learning motor tasks involves distinct physiological processes in the cerebellum (CB) and primary motor cortex (M1). Previous studies have shown that motor learning results in at least two important neurophysiological changes: modulation of cerebellar output mediated in-part by long-term depression of parallel fiber-Purkinje cell synapse and induction of long-term plasticity (LTP) in M1, leading to transient occlusion of additional LTP-like plasticity. However, little is known about the temporal dynamics of these two physiological mechanisms during motor skill learning. Here we use non-invasive brain stimulation to explore CB and M1 mechanisms during early and late motor skill learning in humans. We predicted that early skill acquisition would be proportional to cerebellar excitability (CBI) changes, whereas later stages of learning will result in M1 LTP-like plasticity modifications. We found that early, and not late into skill training, CBI changed. Whereas, occlusion of LTP-like plasticity over M1 occurred only during late, but not early training. These findings indicate a distinct temporal dissociation in the physiological role of the CB and M1 when learning a novel skill. Understanding the role and temporal dynamics of different brain regions during motor learning is critical to device optimal interventions to augment learning.

Maintaining the balance is always a challenge issue for bipedal humanoid robots in dealing with various locomotive tasks to serve human society, especially when the real environment the robot worked within exhibits to be very complex. Unlike plenty of previous successful approaches on humanoid balancing or push recovery, in this research, the Dynamical Movement Primitives (DMP) is employed to model several typical bio-inspired strategies. As humanoid balancing or push recovery could be regarded as a problem of how a robot to get back to its ongoing behavior when a break happens on account of external force or uneven terrain etc., the DMP model becomes an alternative ideal choice due to its promising nature of attractor. Meanwhile, the DMP composed of a set of differential equations provides a compact parameterized representation in modelling a motion strategy, and thus leads to a strategy model that is suitable to be fulfilled with machine learning techniques. In this research, the learning process for those bio-inspired strategies modeled with DMP are completed by adopting the stochastic policy gradient reinforcement learning and imitation learning separately. Furthermore, with Gaussian Process(GP) regression, push recovery strategies are generalized taking the advantages of the invariance properties of the DMP model. As a consequence, an online adaptive push recovery control strategy is finally achieved. Experimental results on both simulated robot and a real bipedal humanoid robot PKU-HR5 demonstrate the presented approach is effective and promising.

Dynamic movement primitives (DMPs) have proven to be an effective movement representation for motor skill learning. In this paper, we propose a new approach for training deep neural networks to synthesize dynamic movement primitives. The distinguishing property of our approach is that it can utilize a novel loss function that measures the physical distance between movement trajectories as opposed to measuring the distance between the parameters of DMPs that have no physical meaning. This was made possible by deriving differential equations that can be applied to compute the gradients of the proposed loss function, thus enabling an effective application of backpropagation to optimize the parameters of the underlying deep neural network. While the developed approach is applicable to any neural network architecture, it was evaluated on two different architectures based on encoder–decoder networks and convolutional neural networks. Our results show that the minimization of the proposed loss function leads to better results than when more conventional loss functions are used.

Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression

Teleoperating a robot for complex and intricate tasks demands a high mental workload from a human operator. Deploying multiple operators can mitigate this problem, but it can be also a costly solution. Learning from Demonstrations can reduce the human operator’s burden by learning repetitive teleoperation tasks. Yet, the demonstrations via teleoperation tend to be inconsistent compared to other modalities of human demonstrations. In order to handle less consistent and asynchronous demonstrations effectively, this paper proposes a learning scheme based on Dynamic Movement Primitives. In particular, a new Expectation Maximization algorithm which synchronizes and encodes demonstrations with high temporal and spatial variances is proposed. Furthermore, we discuss two shared teleoperation architectures, where, instead of multiple human operators, a learned artificial agent and a human operator share authority over a task while teleoperating cooperatively. The agent controls the more mundane and repetitive motion in the task whereas human takes charge of the more critical and uncertain motion. The proposed algorithm together with the two shared teleoperation architectures (human-synchronized and agent-synchronized shared teleoperation) has been tested and validated through simulation and experiments on 3 Degrees-of-Freedom Phantom-to-Phantom teleoperation. Conclusively, the both proposed shared teleoperation architectures have shown superior performance when compared with the human-only teleoperation for a peg-in-hole task.

Extended Gaussian mixture regression for forward and inverse analysis

Learning from demonstration (LfD), which provides a natural way to transfer skills to robots, has been extensively researched for decades, and an army of methods and applications have been developed and investigated for learning an individual or low-level task. Nevertheless, learning long time sequential tasks is still very difficult as it involves task segmentation and sub-task clustering under an extremely large demonstration variance. Besides, the representation problem should be considered when doing segmentation. This paper presents a new unified framework to solve the problems of segmentation, clustering, and representation in a sequential task. The segmentation algorithm segments unstructured demonstrations into movement primitives (MPs). Then, the MPs are automatically clustered and labeled so that they can be reused in other tasks. Finally, the representation model is leveraged to encode and generalize the learned MPs in new contexts. To achieve the first goal, a change-point detection algorithm based on Bayesian inference is leveraged. It can segment unstructured demonstrations online with minimum prior knowledge requirements. By following the Gaussian distributed assumption in the segmentation model, MPs are encoded by Gaussians or Gaussian mixture models. Thus, the clustering of MPs is formulated as a clustering over cluster (CoC) problem. The Kullback–Leibler divergence is used to measure similarities between MPs, through which the MPs with smaller distance are clustered into the same group. To replay and generalize the task in novel contexts, we use task-parameterized regression models such as the Gaussian mixture regression. We implemented our framework on a sequential open-and-place task. The experiments demonstrate that the segmentation accuracy of our framework can reach 94.3% and the recognition accuracy can reach 97.1%. Comparisons with the state-of-the-art algorithm also indicate that our framework is superior or comparable to their results.

ABSTRACTHuman motor skill acquisition is improved by performance feedback and coupling such feedback with extrinsic reward (such as money) can enhance skill learning. However, the neurophysiology underlying such behavioral effect is unclear. To bridge this gap, we assessed the effects of reward on multiple forms of motor plasticity during skill learning. Sixty-five healthy participants divided in three groups performed a pinch-grip skill task with sensory feedback only, sensory and reinforcement feedback or both feedback coupled with an extrinsic monetary reward during skill training. To probe motor plasticity, we applied transcranial magnetic stimulation on the left primary motor cortex at rest before, during and after training in the three groups. We evaluated the amplitude and variability of corticospinal output, GABA-ergic short-intracortical inhibition and use-dependent plasticity before training and at two time points during and after training. At the behavioral level, monetary reward accelerated skill learning. In parallel, corticospinal output became less variable early on during training in the presence of extrinsic reward. Interestingly, this effect was particularly pronounced for participants who were more sensitive to reward, as evaluated in an independent questionnaire. Other measures of motor excitability remained comparable across groups. These findings highlight that a mechanism underlying the benefit of reward on motor skill learning is the fine tuning of early-training resting-state corticospinal variability.SIGNIFICANCE STATEMENTSkill acquisition is enhanced in the presence of reward. Despite its potential clinical relevance for motor rehabilitation, the underlying neurophysiological mechanisms remain largely unexplored. Specifically, whether reward affects the plasticity of motor cortex in the context of skill learning is unclear. We show that reward reduces the variability of corticospinal output at an early stage during training and that this effect correlates with individual sensitivity to reward. Our results suggest that a key mechanism underlying the beneficial effect of reward on motor skill learning may be an increase in the stability of motor output in response to training during early stages of skill learning.

Robots are key to expanding the scope of space applications. The end-to-end training for robot vision-based detection and precision operations is challenging owing to constraints such as extreme environments and high computational overhead. This study proposes a lightweight integrated framework for grasp detection and imitation learning, named GD-IL; it comprises a grasp detection algorithm based on manipulability and Gaussian mixture model (manipulability–GMM), and a grasp trajectory generation algorithm based on a two-stage robot imitation learning algorithm (TS-RIL). In the manipulability–GMM algorithm, we apply GMM clustering and ellipse regression to the object point cloud, propose two judgment criteria to generate multiple candidate grasp bounding boxes for the robot, and use manipulability as a metric for selecting the optimal grasp bounding box. The stages of the TS-RIL algorithm are grasp trajectory learning and robot pose optimization. In the first stage, the robot grasp trajectory is characterized using a second-order dynamic movement primitive model and Gaussian mixture regression (GMM). By adjusting the function form of the forcing term, the robot closely approximates the target-grasping trajectory. In the second stage, a robot pose optimization model is built based on the derived pose error formula and manipulability metric. This model allows the robot to adjust its configuration in real time while grasping, thereby effectively avoiding singularities. Finally, an algorithm verification platform is developed based on a Robot Operating System and a series of comparative experiments are conducted in real-world scenarios. The experimental results demonstrate that GD-IL significantly improves the effectiveness and robustness of grasp detection and trajectory imitation learning, outperforming existing state-of-the-art methods in execution efficiency, manipulability, and success rate.