Optimizing robot striking movement primitives with Iterative Learning Control

As robots become more capable in terms of hardware, and more complex tasks are considered, optimality starts playing a more important role in the design of algorithms implemented in these systems. Optimality is a guiding principle that directs the computation of feasible and efficient solutions to different robotics tasks. In control theory, this principle is implemented online as a set of efficient numerical optimization algorithms, that in addition to solving the task, purports to save a suitably defined effort or energy term. This thesis investigates trajectory generation, learning and control for dynamic tasks from the unifying point of view of optimization. As an application, we focus on Table Tennis, a chal- lenging task where robots are yet to outperform humans. We believe that the required dexterity and accuracy for this dynamical task hinges on the developments in online optimization and efficient learning algorithms. We consider trajectory generation for table tennis in the first part of the thesis. In highly dynamic tasks like table tennis that involve moving targets, planning is necessary to figure out when, where and how to intercept the target. Motion planning can be very challenging in robotic table tennis in particular, due to time constraints, dimension of the search space and joint limits. Conventional planning algorithms often rely on a fixed virtual hitting plane to construct robot striking trajectories. These algorithms, however, generate restrictive strokes and can result in unnatural strategies when compared with human playing. In this thesis, we introduce a new trajectory generation framework for robotic table tennis that does not involve a fixed hitting plane. A free-time optimal control approach is used to derive two different trajectory optimizers. The resulting two algorithms, Focused Player and Defensive Player, encode two different play- styles. We evaluate their performance in simulation and in our robot table tennis platform with a high speed cable-driven seven DOF robot arm. The algorithms return the balls with a higher probability to the opponent’s court when compared with a virtual hitting plane based method. Moreover, both can be run online and the trajectories can be corrected with new ball observations. In the second part of the thesis, we look at how such trajectories, computed on the kine- matics level, can be tracked accurately with learning control based approaches. Highly dynamic tasks like table tennis require large accelerations and precise tracking for successful perfor- mance. To track desired trajectories well, such tasks usually rely on accurate models and/or high gain feedback. While kinematic optimization allows for efficient representation and online generation of hitting trajectories, learning to track such dynamic movements with inaccurate models remains an open problem. In particular, stability issues surrounding the learning per- formance, in the iteration domain, can prevent the successful implementation of model based learning approaches. To achieve accurate tracking for these tasks in a stable and efficient way, we propose a new adaptive Iterative Learning Control algorithm that is implemented efficiently using a recursive approach. Moreover, covariance estimates of model matrices are used to ex- ercise caution during learning. We evaluate the performance of the proposed approach in our robotic table tennis platform, where we show how the performance of two Barrett WAMs can be optimized. Our implementation on the table tennis platform compares favorably with two state-of-the-art approaches. Finally, we discuss an alternative learning from demonstrations approach, where we learn sparse representations from demonstrated movements. Learning from demonstrations is an easy and intuitive way to show examples of successful behavior to a robot. However, the fact that humans optimize or take advantage of their body and not of the robot, usually called the embodiment problem in robotics, often prevents industrial robots from executing the task in a straightforward way. The shown movements often do not or cannot utilize the degrees of freedom of the robot efficiently, and typically suffer from excessive execution errors. In the last chapter, we show a new approach that can alleviate some of these difficulties by learning sparse representations of movement. Moreover, the number of learned parameters are independent of the degrees of freedom of the robot. Sparsity is a desirable feature for policy search Rein- forcement Learning algorithms that adapt the parameters of these movement primitives. By ranking the learned parameters on the Elastic Net path in terms of importance, we note that our approach could be potentially useful to combat the curse of dimensionality in robot learning applications. We show preliminary results on the real robot setup, including a successful table tennis serve using our new movement primitive representation. Throughout the thesis, we present and analyze in detail new control and learning algo- rithms. Efficient online optimization approaches are presented that can be used to solve not just table tennis problems, but they can be adapted to solve different dynamic tasks.

The repetitive tracking task for time‐varying systems (TVSs) with non‐repetitive time‐varying parameters at each trial, which is also called non‐repetitive TVSs, is realized in this article using iterative learning control (ILC). A machine learning (ML) based nominal model update mechanism, which utilizes the linear regression technique to update the nominal model at each ILC trial using only the current trial information, is proposed for non‐repetitive TVSs in order to enhance the ILC performance. Given that the ML mechanism forces the model uncertainties to remain within the ILC robust tolerance, an ILC update law is proposed to deal with non‐repetitive TVSs. How to tune parameters inside ML and ILC algorithms to achieve the desired aggregate performance is also provided. The robustness and reliability of the proposed method are verified by real experiments. Real data comparison with current state‐of‐the‐art methods demonstrates its superior control performance in terms of controlling precision. This article broadens ILC applications from time‐invariant systems to non‐repetitive TVSs, adopts ML regression technique to estimate non‐repetitive time‐varying parameters between two ILC trials and proposes a detailed parameter tuning mechanism to achieve desired performance, which are the main contributions.

Learning new motor tasks from physical interactions is an important goal for both robotics and machine learning. However, when moving beyond basic skills, most monolithic machine learning approaches fail to scale. For more complex skills, methods that are tailored for the domain of skill learning are needed. In this paper, we take the task of learning table tennis as an example and present a new framework that allows a robot to learn cooperative table tennis from physical interaction with a human. The robot first learns a set of elementary table tennis hitting movements from a human table tennis teacher by kinesthetic teach-in, which is compiled into a set of motor primitives represented by dynamical systems. The robot subsequently generalizes these movements to a wider range of situations using our mixture of motor primitives approach. The resulting policy enables the robot to select appropriate motor primitives as well as to generalize between them. Finally, the robot plays with a human table tennis partner and learns online to improve its behavior. We show that the resulting setup is capable of playing table tennis using an anthropomorphic robot arm.

While iterative learning control (ILC) and repetitive control (RC) have much ground in common, they fundamentally differ in the initial conditions at each repetition. This difference has lead to distinct analysis techniques, hereby clouding the interrelations between both control strategies. To facilitate the transfer of results, this paper presents a unified approach to ILC and RC. Both control problems are formulated in the trial domain using so-called system lifting. For a given system, the corresponding ILC and RC trial-domain models differ, and a thorough system theoretic analysis and comparison of these models is performed. To illustrate the value of a unified formulation of ILC and RC, the analysis of the most commonly used ILC and RC structures is harmonized. This analysis reveals central differences and similarities between various stability, monotonic convergence and steady-state performance conditions.

A conditional iterative learning control (ILC) is derived for uncertain non-linear systems to track repetitive trajectory under the alignment initial condition. The learning of the conditional ILC is performed, only if the tracking error is dominated by the input instead of the initial conditions. The major advantage of this method is that it can simultaneously handle non-parametric uncertainties and the alignment initial condition. A simulation example is presented to illustrate the performance and implementation of the conditional ILC.

In practical applications, the durations of operations in uncertain batch processes may fluctuate due to security considerations, physical constraints, and environmental changes. Regarding this issue, this paper addresses iterative learning control (ILC) for a class of uncertain single‐input single‐output (SISO) batch processes with nonuniform batch lengths. The aim is to develop an optimization‐based ILC scheme that guarantees robustness while meeting input constraints requirements. Two robust optimal ILC algorithms resting upon the modified error update model are proposed for the nonuniform batch length problem, where the linear matrix inequality (LMI) optimization techniques are employed to deal with polytopic and ellipsoidal model uncertainties. Hence, the ILC design requirements are reduced to a convex optimization method involving LMIs, which results in iterative input update signals at the end of each batch. Also, the theoretical analyses of the proposed algorithms are presented. Finally, two numerical cases for batch processes are simulated to testify the feasibility and performance of the proposed methods.

This paper proposes an optimal gait generation framework using virtual constraint and learning optimal control. In this method, firstly, we add a constraint by a virtual potential energy to prevent the robot from falling. Secondly, we execute iterative learning control (ILC) to generate an optimal feedforward input. Thirdly, we execute iterative feedback tuning (IFT) to mitigate the strength of the virtual constraint automatically according to the progress of learning control. Consequently, it is expected to generate an optimal gait without constraint eventually. Although existing ILC frameworks require a lot of experimental data under the same initial condition, the proposed method does not need to repeat experiments under the same initial condition because the virtual constraint restricts the motion of the robot to a symmetric trajectory. Furthermore, it does not require the precise knowledge of the plant system. Finally, some numerical simulations demonstrate the effectiveness of the proposed method.

External actuation in self-reconfigurable modular robots promises to allow modules to shrink down in size. Synchronous external motions promise to allow fast convergence and assembly times. XBot is a modular system that uses synchronous external actuation, but has a limited range of reachable configurations stemming from a single motion primitive of a module rotating about another. This paper proposes to extend the motion primitives by using moves with two modules swinging in a dynamic chain. The feasibility of these motion primitives is proven experimentally. A parameterization of the external actuation motion profiles is explored to define a space of physically valid motion profiles. The larger the space, the more robust the motion primitives will be to inexact initial conditions and to imprecision in the external actuation mechanisms. In addition, this paper proves that a configuration of XBot meta-modules can reach any configuration using just these motion primitives.

Fractional-order iterative learning control for fractional-order systems with initialization non-repeatability

The precise trajectory tracking is one of the most important topics in the climb and descent operations for aircraft in the dense terminal airspace. It facilitates the accurate execution of the planned trajectory, maintenance of safety separation, and improvement of airspace utilization. This paper aims to solve the aircraft trajectory tracking problem by proposing a class of modeling and learning‐based control methods. A nonlinear, time‐varying aircraft model is constructed under repetitive horizontal wind and non‐repetitive turbulence, which is further converted into a linear form through a virtual input transformation. By resorting to the repetitive execution of trajectory tracking tasks in aircraft climb and descent operations, an adaptive iterative learning control (ILC) method is developed. Then, the ILC convergence analysis is constructed based on a composite energy function. As a result, precise trajectory tracking can be achieved under both repetitive and non‐repetitive disturbances as the iteration number increases. A simulation based on the Continuous Climb Operation and Continuous Descent Operation procedures of Guangzhou Baiyun International Airport is conducted and the results verify the feasibility and efficacy of the proposed algorithm.

In this paper we present an approach to deal with trial-varying initial conditions in norm-optimal iterative learning control (ILC). Varying initial conditions generally degrade the performance of conventional learning algorithms. We therefore introduce a worst-case optimization problem that accounts for trial-varying of initial conditions. The optimization is then reformulated as a convex minimization problem, which can be solved efficiently to generate the control signal. We investigate the relationship between the proposed approach and classical norm-optimal ILC; where we find that our methodology is equivalent to classical norm-optimal ILC with trial-varying parameters. Finally, simulation results of the presented technique are given.

On reference governor in iterative learning control for dynamic systems with input saturation

An Iterative Predictive Learning Control Approach With Application to Energy Efficient Train Trajectory Tracking

Future unmanned aerial vehicle (UAV) missions are likely to include those for which the characteristic dimensions of the environment are similar in scale to the characteristic dimensions of the vehicle dynamics. This class of missions will require precision trajectory planning and tracking that utilizes the full agility of the vehicle to ensure safety and performance. Hybrid models consisting of a set motion primitives allow for the safe integration of aggressive maneuvers into a motion plan; however, planning precision is limited by the finite set of primitives. This paper presents an extension of existing hybrid modeling strategies to allow for a variable set of motion primitives that are tailored to the current mission-performance requirements. A randomized sampling-based planning approach is then adopted to plan trajectories using this modeling strategy. Performance of the resulting trajectory-planning system is demonstrated through a simulated example.

Precise tracking of highly nonlinear phase-shift full-bridge series resonant inverter via iterative learning control