A robot motion skills method with explicit environmental constraints

In this paper, composite dynamic movement primitives (DMPs) based on radial basis function neural networks (RBFNNs) are investigated for robots’ skill learning from human demonstrations. The composite DMPs could encode the position and orientation manipulation skills simultaneously for human-to-robot skills transfer. As the robot manipulator is expected to perform tasks in unstructured and uncertain environments, it requires the manipulator to own the adaptive ability to adjust its behaviours to new situations and environments. Since the DMPs can adapt to uncertainties and perturbation, and spatial and temporal scaling, it has been successfully employed for various tasks, such as trajectory planning and obstacle avoidance. However, the existing skill model mainly focuses on position or orientation modelling separately; it is a common constraint in terms of position and orientation simultaneously in practice. Besides, the generalisation of the skill learning model based on DMPs is still hard to deal with dynamic tasks, e.g., reaching a moving target and obstacle avoidance. In this paper, we proposed a composite DMPs-based framework representing position and orientation simultaneously for robot skill acquisition and the neural networks technique is used to train the skill model. The effectiveness of the proposed approach is validated by simulation and experiments.

This paper proposes a robot automatic valve turning control strategy based on teaching learning, which consists of teaching, model learning and task repetition. The first stage is the teaching and learning stage. The robot learns motor skills by observing the human performing tasks. In order to accurately learn motor skills from demonstrations, data alignment is performed on the teaching data through Dynamic Time Warping (DTW). The second stage is the model construction and learning stage. The high-level learning strategy aims to learn motor skills from demonstrations through Dynamic Movement Primitives (DMP), using the statistical approach Gaussian Mixture Model and Gaussian Mixture Regression (GMM-GMR) to analyze the data from demonstrations. And the valve turning is repetition. To verify the effectiveness of the proposed control strategy, the experiment of the butterfly valve closing is performed. The results show that the robot is able to learn and reproduce the valve reaching and turning tasks. It completes the valve closing action by turning the valve for 7 turns.

Movement primitives are a well-established approach to robot motion planning, as they offer a modular basis for robot motion planning. Dynamic movement primitives (DMPs) are a popular control framework based on nonlinear differential equations which, when scaled in time, produce a smooth kinematic movement plan. In this letter, we introduce an adjusted movement primitive framework which replaces the stable equilibria of DMPs with saddle points. These saddle points are linked together in stable heteroclinic channels (SHCs) which are in turn part of dynamical systems called stable heteroclinic networks. SHC-based movement primitives (SMPs) form a framework where the weights of the kernel functions have spatial significance in the task space. In this letter, we show that SMPs and DMPs perform comparably according to a kinematic-based cost function. We also show that SMP kernel weights follow a given trajectory when plotted in the task space. The plotted kernel weights provide an intuitive tool for managing the controller.

An effective robot trajectory representation should encode all relevant aspects of the desired motion. For kinematic representations, this means that both the spatial course of the trajectory and its speed profile must be specified. The concept of dynamic movement primitives (DMP) provides a kinematic representation that fully specifies these two aspects of motion. They are, however, not separated from each other within the DMP representation. This can be problematic when movements with significant speed variations are compared within movement recognition and skill learning algorithms. In such comparisons it is often important to distinguish between the spatial and temporal aspects of motion. In this paper we propose a new representation based on dynamic movement primitives, where spatial and temporal aspects are well separated. We demonstrate the effectiveness of the proposed representation for statistical learning of robot skills and movement recognition and compare the performance with standard DMPs, where temporal and spatial aspects of motion are intertwined.

We propose a framework based on imitation learning and self-learning to enable robots to learn, improve, and generalize motor skills. The peg-in-hole task is important in manufacturing assembly work. Two motor skills for the peg-in-hole task are targeted: “hole search” and “peg insertion”. The robots learn initial motor skills from human demonstrations and then improve and/or generalize them through reinforcement learning (RL). An initial motor skill is represented as a concatenation of the parameters of a hidden Markov model (HMM) and a dynamic movement primitive (DMP) to classify input signals and generate motion trajectories. Reactions are classified as familiar or unfamiliar (i.e., modeled or not modeled), and initial motor skills are improved to solve familiar reactions and generalized to solve unfamiliar reactions. The proposed framework includes processes, algorithms, and reward functions that can be used for various motor skill types. To evaluate our framework, the motor skills were performed using an actual robotic arm and two reward functions for RL. To verify the learning and improving/generalizing processes, we successfully applied our framework to different shapes of pegs and holes. Moreover, the execution time steps and path optimization of RL were evaluated experimentally.

Dynamic movement primitives (DMPs) have proven to be an effective movement representation for motor skill learning. In this paper, we propose a new approach for training deep neural networks to synthesize dynamic movement primitives. The distinguishing property of our approach is that it can utilize a novel loss function that measures the physical distance between movement trajectories as opposed to measuring the distance between the parameters of DMPs that have no physical meaning. This was made possible by deriving differential equations that can be applied to compute the gradients of the proposed loss function, thus enabling an effective application of backpropagation to optimize the parameters of the underlying deep neural network. While the developed approach is applicable to any neural network architecture, it was evaluated on two different architectures based on encoder–decoder networks and convolutional neural networks. Our results show that the minimization of the proposed loss function leads to better results than when more conventional loss functions are used.

Human factors and human-centered design philosophy are highly desired in today's robotics applications such as human-robot interaction (HRI). Several studies showed that endowing robots of human-like interaction skills can not only make them more likeable but also improve their performance. In particular, skill transfer by imitation learning can increase the usability and acceptability of robots by users without computer programming skills. In fact, besides positional information, muscle stiffness of the human arm and contact force with the environment also play important roles in understanding and generating human-like manipulation behaviors for robots, e.g., in physical HRI and teleoperation. To this end, we present a novel robot learning framework based on dynamic movement primitives (DMPs), taking into consideration both the positional and contact force profiles for human-robot skills transferring. Distinguished from the conventional method involving only the motion information, the proposed framework combines two sets of DMPs, which are built to model the motion trajectory and the force variation of the robot manipulator, respectively. Thus, a hybrid force/motion control approach is taken to ensure the accurate tracking and reproduction of the desired positional and force motor skills. Meanwhile, in order to simplify the control system, a momentum-based force observer is applied to estimate the contact force instead of employing force sensors. To deploy the learned motion-force robot manipulation skills to a broader variety of tasks, the generalization of these DMP models in actual situations is also considered. Comparative experiments have been conducted using a Baxter robot to verify the effectiveness of the proposed learning framework on real-world scenarios like cleaning a table.

In this article, we present a general internal model (GIM) approach for motion skill learning at the elementary level and coordination level. In the past, internal models with two different configurations are used to describe the two classes of dynamic movement primitives (DMPs): discrete and rhythmic movement. In this work, we developed a unified internal model which can describe both classes of DMPs. In particular, a discrete movement can be modeled as a fraction of a rhythmic movement. The general internal model retains the temporal and spatial scalabilities which are defined as the ability to generate similar movement patterns directly by means of tuning some parameters of the internal model. The advantage of scalability lies in that the learning or training process can be avoided while dealing with similar tasks. Complex motions require movement coordinations, hence coordination of multiple internal models. In the general internal model approach, the coordination is implemented with appropriate phase shifts among multiple internal models. Further in the GIM, the phase shift can be achieved by means of adjusting the initial state values of internal models. Through two illustrative examples, we show that the human behavior patterns with single or multiple limbs can be easily learned and established by the GIM at elementary and coordination levels.

Dynamic movement primitives are one of key concepts for understanding dexterous and flexible movements of biological bodies. In the field of robotics engineering, simple types of nonlinear differential equations are used to generate movement primitives from demonstrations, but it remains unclear how nonlinear dynamics in the real brain can also generate movement primitives in biologically natural ways. The aim of this study is to investigate a possible role of nonlinear dynamics in random recurrent neural networks (RNNs) for skillful motor learning. We show that one-shot temporal patterns such arm reaching movements can be trained by a type of RNN-learning so-called FORCE-learning recently proposed by Sussillo and Abbott and a number of patterns are summarized as a manifold embedded in a space of synaptic weights of readout neurons. We also discuss how generalization of learning against untrained motor patterns can be achieved by identifying nonlinear coordinates (meta-parameters) on this manifold in a higher level of the central nervous system.

Motivation and learning skill are essential and mustbe owned by students to be successful in learning. But, the fact that are students with lack of learning motivation and learning skill. The aims from this research are: (1) to describing student’s learning motivation and learning skill which high achieving and underachieving, (2) to analyze student’s learning motivation and learning skill differences who high achieving and underachieving. This research using quantitative approach with comparative description type. This research conducted on students in VII and VIII grades at MTsN Salido Pesisir Selatan regency in 2013-2014 school year, with 258 students population, and 158 students as research sample using propotional random sampling techniques. Instrument for this research is questionnaire. Data analyzing for this research using descriptive statistic techniques and t-test using SPSS 16.00. Data analyzing revealed that (1) learning motivation students who is high achieving stand at high category, (2) learning motivation students who is underachieving stand at average category, (3) learning skill students who is high achieving stand at good category, (4) learning skill students who is underachieving stand at enough category, (5) there is a difference of learning motivation between students with high achieving and underachieving, (6) there is a difference of learning skill between students with high achieving and underachieving. The implication from this research to guidance and counseling is forming guidance counseling treatment programme that concern to students with underachieving in order to reach good learning result. .

Maintaining the balance is always a challenge issue for bipedal humanoid robots in dealing with various locomotive tasks to serve human society, especially when the real environment the robot worked within exhibits to be very complex. Unlike plenty of previous successful approaches on humanoid balancing or push recovery, in this research, the Dynamical Movement Primitives (DMP) is employed to model several typical bio-inspired strategies. As humanoid balancing or push recovery could be regarded as a problem of how a robot to get back to its ongoing behavior when a break happens on account of external force or uneven terrain etc., the DMP model becomes an alternative ideal choice due to its promising nature of attractor. Meanwhile, the DMP composed of a set of differential equations provides a compact parameterized representation in modelling a motion strategy, and thus leads to a strategy model that is suitable to be fulfilled with machine learning techniques. In this research, the learning process for those bio-inspired strategies modeled with DMP are completed by adopting the stochastic policy gradient reinforcement learning and imitation learning separately. Furthermore, with Gaussian Process(GP) regression, push recovery strategies are generalized taking the advantages of the invariance properties of the DMP model. As a consequence, an online adaptive push recovery control strategy is finally achieved. Experimental results on both simulated robot and a real bipedal humanoid robot PKU-HR5 demonstrate the presented approach is effective and promising.

This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking. During robot learning demonstrations, dynamic movement primitives (DMPs) are used to model robotic motion. Each DMP consists of a set of dynamic systems that enhances the stability of the generated motion toward the goal. A Gaussian mixture model and Gaussian mixture regression are integrated to improve the learning performance of the DMP, such that more features of the skill can be extracted from multiple demonstrations. The motion generated from the learned model can be scaled in space and time. Besides, a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model. In this controller, a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. The experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed methods.

ABSTRACTHuman motor skill acquisition is improved by performance feedback and coupling such feedback with extrinsic reward (such as money) can enhance skill learning. However, the neurophysiology underlying such behavioral effect is unclear. To bridge this gap, we assessed the effects of reward on multiple forms of motor plasticity during skill learning. Sixty-five healthy participants divided in three groups performed a pinch-grip skill task with sensory feedback only, sensory and reinforcement feedback or both feedback coupled with an extrinsic monetary reward during skill training. To probe motor plasticity, we applied transcranial magnetic stimulation on the left primary motor cortex at rest before, during and after training in the three groups. We evaluated the amplitude and variability of corticospinal output, GABA-ergic short-intracortical inhibition and use-dependent plasticity before training and at two time points during and after training. At the behavioral level, monetary reward accelerated skill learning. In parallel, corticospinal output became less variable early on during training in the presence of extrinsic reward. Interestingly, this effect was particularly pronounced for participants who were more sensitive to reward, as evaluated in an independent questionnaire. Other measures of motor excitability remained comparable across groups. These findings highlight that a mechanism underlying the benefit of reward on motor skill learning is the fine tuning of early-training resting-state corticospinal variability.SIGNIFICANCE STATEMENTSkill acquisition is enhanced in the presence of reward. Despite its potential clinical relevance for motor rehabilitation, the underlying neurophysiological mechanisms remain largely unexplored. Specifically, whether reward affects the plasticity of motor cortex in the context of skill learning is unclear. We show that reward reduces the variability of corticospinal output at an early stage during training and that this effect correlates with individual sensitivity to reward. Our results suggest that a key mechanism underlying the beneficial effect of reward on motor skill learning may be an increase in the stability of motor output in response to training during early stages of skill learning.

In robot skill learning, the higher accuracy of trajectory reproduction, the smaller difference between the trajectory reproduced by the robot and the expected trajectory, and the higher accuracy and reliability of the robot’s task execution. In robot skill learning methods based on dynamic movement primitives, the problems are found that the tail exponential decay of Gaussian kernel function and the number of shape parameters have a great influence on the accuracy of trajectory reproduction, a high precision trajectory learning method is proposed. A novel truncation processing method is proposed to eliminate the impact of tail exponential decay, and the optimization on the number of shape parameters is used to improve the approximation of the local gradient of the target forcing term, which are combined to improve trajectory reproduction accuracy. The principle of the proposed method is described in detail. The simulation and comparison experiments are performed to verify the effectiveness of the proposed method in improving trajectory reproduction accuracy. This paper makes contributions to the field of robot skill trajectory learning and provides a promising method for improving trajectory reproduction accuracy.

An essential aspect for making robots succeed in real-world environments is to give them the ability to robustly perform motions in continuously changing situations. Classical motion planning methods usually create plans for static environments. The direct execution of such plans in dynamic environments often becomes problematic. We present an approach that adapts motion plans by feeding changes of the environment into a transformation of the plan in task space. Furthermore, the progress in the plan is defined with a phase variable that is updated adaptively according to the actual task progress. This phase variable releases the strict time compliance that many motion planning methods bring along. The main benefit of our approach is the ability to do this adaptation in a computational efficient manner during the execution of the motion. Thus, the gap between the motion planning and motion execution stage is bridged by continuously transforming geometric and dynamic features of a reference plan to the current situation. We evaluate the performance of our approach by comparing it to alternative methods such as dynamic motion primitives and continuous replanning on several simulated benchmark tasks. Moreover, we demonstrate the real robot applicability on a PR2 robot platform.