Control of a Robot Arm Using Iterative Learning Algorithm with a Stopping Criterion

Bidirectional Human-Robot Learning: Imitation and Skill Improvement

Within impedance control framework, since force tracking accuracy is affected by the robotic tracking error, an adaptive force control scheme combined with an iterative learning algorithm is proposed in this paper. First, the unknown environment stiffness and location are observed in real time based on the actual contact force and robot position, and the adaptive reference trajectory is obtained by the estimated environment information. Then, position-based impedance control scheme is adopted to adjust the dynamical relationship between the end-effect or position and the contact force. Considering the effect of the position tracking error on the force tracking accuracy, an iterative learning control method is utilized to compensate the motion error effectively by the force error. The proposed method can effectively reduce the steady state force error by simply combining the indirectly adaptive reference trajectory generation technique with position tracking error regulation method. Stability and convergence conditions are presented with a Lyapunov function and frequency domain method. Simulations and experiments are conducted on a biaxial platform to demonstrate the effectiveness of the proposed method.

This paper discusses a complex-valued Hopfield associative memory with an iterative incremental learning algorithm. The mathematical proofs derive that the weight matrix is approximated as a weight matrix by the complex-valued pseudo inverse algorithm. Furthermore, the minimum number of iterations for the learning sequence is defined with maintaining the network stability. From the result of simulation experiment in terms of memory capacity and noise tolerance, the proposed model has the superior ability than the model with a complex-valued pseudo inverse learning algorithm.

Constrained Point-to-Point Iterative Learning Control

The authors show that a local iterative learning algorithm of Diederich and Opper (1987) converges for any set of patterns, including linearly dependent sets, and that the resulting coupling constants for a given set of patterns are the same as those given by the pseudoinverse rule for any maximal linearly independent subset of the given set. Thus, the matrix of couplings produced by the algorithm is the desired projection matrix onto the subspace spanned by the training set.

Over the last couple of decades the demand for high precision and enhanced performance of physical systems has been steadily increasing. This demand often results in miniaturization and complex design, thus increasing the need for complex nonlinear control methods. Some of the state of the art nonlinear methods are stymied by the requirement of full state information, model and parameter uncertainties, mathematical complexity, etc. For many scenarios it is nearly impossible to consider all the uncertainties during the design of a feedback controller. Additionally, while designing a modelbased nonlinear control there is no standard mechanism to incorporate performance measures. Some of the mentioned issues can be addressed by using online learning. Animals and humans have the ability to share, explore, act or respond, memorize the outcome and repeat the task to achieve a better outcome when they encounter the same or a similar scenario. This is called learning from interaction. One instance of this approach is reinforcement learning (RL). However, RL methods are hindered by the curse of dimensionality, non-interpretability and non-monotonic convergence of the learning algorithms. This can be attributed to the intrinsic characteristics of RL, as it is a modelfree approach and hence no standard mechanism exists to incorporate a priori model information. In this thesis, learning methods are proposed which explicitly use the available system knowledge. This can be seen as a new class of approaches that bridge model-based and model-free methods. These methods can address some of the hurdles mentioned earlier. For example, i) a prior system information can speed up the learning, ii) new control objectives can be achieved which otherwise would be extremely difficult to attain using only model-based methods, iii) physical meaning can be attributed to the learned controller. The developed approach is as follows: themodel of the given physical system is represented in the port-Hamiltonian (PH) form. For the system dynamics in PH form a passivity-based control (PBC) law is formulated, which often requires the solution to a set of partial differential equations (PDEs). Instead of finding an analytical solution, the PBC control law is parameterized using an unknown parameter vector. Then, by using a variation of the standard actor-critic learning algorithm, the unknown parameters can be learned online. Using the principles of stochastic approximation theory, a proof of convergence for the developed method is shown. The proposedmethods are evaluated for the stabilization and regulation ofmechanical and electro-mechanical systems. The simulation and experimental results show comparable learning curves. In the final part of the thesis a novel integral reinforcement learning approach is developed to solve for the optimal output tracking control problem for a set of linear heterogeneous multi-agent systems. Unlike existing methods, this approach does not need to solve either the output regulator equation or requires a p-copy of the leader’s dynamics in the agent’s control law. A detailed numerical evaluation has been conducted to show the feasibility of the developed method.

Learning fixed point patterns by recurrent networks

This paper, considering constrained inputs, focuses on designing the optimal controller for discrete-time $H_{\infty}$ tracking control problems. The neural-network-based iterative learning algorithm is deemed to be an excellent method to work. The iterative heuristic dynamic programming algorithm is employed for developing the formulated regulation of the tracking error. Strict convergence guarantees are given to support the optimality of the adaptive algorithm. A neural-network-based training scheme is adopted to implement the learning algorithm and stabilize the system in the optimal manner. Two numerical examples are provided to illustrate the applicability and good performance of the designed tracking control scheme.

In this paper, iterative learning identification algorithms are proposed to estimate the time-varying parameters in multi-input-single-output (MISO) Wiener nonlinear time-varying systems. The regression model of the Wiener system is built by using the polynomial expansion of the nonlinear inverse function. Then, two iterative learning algorithms, including iterative learning gradient identification and iterative learning least squares identification, are presented to estimate the time-varying parameters of the regression model. The convergence performance of the iterative learning identification algorithms is analyzed, and numerical simulations are provided to verify the effectiveness of the proposed algorithms.

In this thesis, we present a dynamical systems approach to adaptive controllers for locomotion control. The approach is based on a rigorous mathematical framework using nonlinear dynamical systems and is inspired by theories of self-organization. Nonlinear dynamical systems such as coupled oscillators are an interesting approach for the on-line generation of trajectories for robots with many degrees of freedom (e.g. legged locomotion). However, designing a nonlinear dynamical system to satisfy a given specification and goal is not an easy task, and, hitherto no methodology exists to approach this problem in a unified way. Nature presents us with satisfactory solutions for the coordination of many degrees of freedom. One central feature observed in biological subjects is the ability of the neural systems to exploit natural dynamics of the body to achieve efficient locomotion. In order to be able to exploit the body properties, adaptive mechanisms must be at work. Recent work has pointed out the importance of the mechanical system for efficient locomotion. Even more interestingly, such well suited mechanical systems do not need complicated control. Yet, in robotics, in most approaches, adaptive mechanisms are either missing or they are not based on a rigorous framework, i.e. they are based on heuristics and ad-hoc approaches. Over the last three decades there has been enormous progress in describing movement coordination with the help of Synergetic approaches. This has led to the formulation of a theoretical framework: the theory of dynamic patterns. This framework is mathematically rigorous and at the same time fully operational. However, it does not provide any guidelines for synthetic approaches as needed for the engineering of robots with many degrees of freedom, nor does it directly help to explain adaptive systems. We will show how we can extend the theoretical framework to build adaptive systems. For this purpose, we propose the use of multi-scale dynamical systems. The basic idea behind multi-scale dynamical systems is that a given dynamical system gets extended by additional slow dynamics of its parameters, i.e. some of the parameters become state variables. The advantages of the framework of multi-scale dynamical systems for adaptive controllers are 1) fully dynamic description, 2) no separation of learning algorithm and learning substrate, 3) no separation of learning trials or time windows, 4) mathematically rigorous, 5) low dimensional systems. However, in order to fully exploit the framework important questions have to be solved. Most importantly, methodologies for designing the feedback loops have to be found and important theoretical questions about stability and convergence properties of the devised systems have to be answered. In order to tackle this challenge, we first introduce an engineering view on designing nonlinear dynamical systems and especially oscillators. We will highlight the important differences and freedom that this engineering view introduces as opposed to a modeling one. We then apply this approach by first proposing a very simple adaptive toy-system, consisting of a dynamical system coupled to a spring-mass system. Due to its spring-mass dynamics, this system contains clear natural dynamics in the form of resonant frequencies. We propose a prototype adaptive multi-scale system, the adaptive frequency oscillator, which is able to adapt its intrinsic frequency to the resonant frequency of the body dynamics. After a small sidetrack to show that we can use adaptive frequency oscillators also for other applications than for adaptive controllers, namely for frequency analysis, we then come back to further investigation of the adaptive controller. We apply the same controller concept to a simple spring-mass hopper system. The spring-mass system consists of a body with two legs attached by rotational joints. The legs contain spring-damper elements. Finally, we present results of the implementation of the controller on a real robot, the experimental robot PUPPY II. This robot is a under-actuated robot with spring dynamics in the knee joints. It will be shown, that due to the appropriate simplification and concentration on relevant features in the toy-system the controller concepts works without a fundamental change on all systems from the toy system up to the real robot.

Traditional methods for assessing upper-limb functional outcomes in stroke patients often fail to estimate the number of trials required to achieve performance stability of a chosen kinematic metric. Limited non-model-based studies have attempted to tackle this issue. To bridge this gap, this study utilized an iterative learning algorithm (ILA) in MATLAB, employing linear models to represent the muscle dynamics and forearm extension of impaired patients. The reference task space trajectory was set as a straight-line point-point trajectory within a range of 0 - 0.2828m. By using the root mean square error (RMSE) as a metric for evaluating kinematic accuracy, a maximum kinematic deviation error of 0.01m was imposed with respect to the trajectory by the (ILA). Results indicate that over 16 trials, performance stability was obtained with improvement in deviation error from 0.0168m in the first trial to 0.0060 at sixteen trials. The result obtained is in line with similar non-model studies and our findings inform the potential of ILAs with linear models for estimation of trial numbers required to attain performance stability of a selected kinematic metric (i.e., kinematic accuracy).

The performance comparison of PID controllers for idle speed control internal combustion engine (ICE) is presented in this paper. There are three methods of PID tuning applied: heuristic tuning, self-tuning by pole placement, and also iterative learning algorithm (ILA). Prior to that, data collection is gained from four cylinder sparkignition engine modeling and used as the input-output data in the system identification part. The obtained transfer function from RLS technique is used in the development of the control system for the idle speed control by PID controllers. The main objective of the idle speed control is to maintain the idling speed at 650rpm and to reject as quickly as possible the extra load exerted on the engine system during idling mode. Comparing the three methods, it shows that PID tuning by ILA produced better results than heuristic tuning and pole placement.

Aiming at the problem of a great deal of blindly searching the proportional, differential, integral parameters in the process of designing PID controller, the new algorithm combining genetic algorithm with iterative learning algorithm is named as an iterative learning genetic algorithm (ILGA), which can be used to optimize three controller parameters, thus the optimal parameters can be achieved swiftly by virtue of less iterative learning times, and the design of the PID controller is simplified. As the simulation results shown, the effectiveness of the method is verified in optimizing the PID controller parameters

Objective To investigate the feasibility and safety of robotic-assisted laparoscopic pyeloplasty (RALP) in infant ureteropelvic junction obstruction. Methods Of 31 infants who underwent RALP between March 2017 and August 2017 were reviewed, including 25 boys and 6 girls. 22 infants were found that the hydronephrosis was in the left kidney and 9 in the right kidney. The average age of the infants was (3.5±0.7) months, ranged from 8 days to 6 months, including 7 infants less than 2 months. The average weight of the infants was (6.7±1.2) kg, with a minimum weight of 3.3 kg. The diagnosis and outcomes of ureteropelvic junction obstruction (UPJO) were reviewed on clinical and imaging data. The ultrasound showed that the mean anteroposterior diameter of hydronephrotic kidney was (4.01±0.56)cm. The split renal function was (37.1±3.2)% in infants who were more than 2 months. Patients with proximal defect were positioned in the lateral flank position with 50°-70° inclination and as close as possible to the edge of operating table. The trocars of robotic lens and No.1 robotic arm were placed on the middle of the umbilicus and the Pfannenstiel line respectively, and the trocar of No.2 robotic arm was placed on the supraumbilical symmetry point of the trocar No.1 robotic arm, and the auxiliary trocar was placed on the Pfannenstiel line outside the 3-4 cm of healthy side of the trocar of No.1 robotic arm. During operation, the ureteropelvic junction was dissected and tailored. Then the ureter was open longitudinally at the posterior lateral side and was anastomosed at the lowest part the renal pelvis. We assessed preoperative clinical data and outcomes, and analyzed the transition experience. Data are expressed as medians for continuous variables. Results All operations were successful with no serious intraoperative complication and traditional laparoscopic or open surgical conversion. The average operative time was 108 min, the average intraoperative blood loss was 9.5 ml. The average drain removal was 1.2 days, the average start oral of feeding was 3.2 hours days and the average postoperative hospital stay was 6.2 days. Postoperative pathological results showed smooth muscle hyperplasia, hypertrophy and fibrous tissue in the urothelium, accompanied by chronic inflammation of urothelial mucosa. Patients with Clavien Ⅰ and Ⅱ complications 22.6%(7/31) postoperatively were managed conservatively. Among them, there were 1 cases of flank pain, 1 cases of hematuria, 2 cases of urinary leakage, and 3 cases of urinary tract infection. Patients' symptoms disappeared by conservative treatment. No patient suffered Clavien Ⅲ and Ⅳ complication postoperatively. The renal pelvis anteroposterior diameter was 1.58 cm at three months postoperatively, which was significantly smaller than that of peroperation (P 0.05). Conclusions Severe hydronephrosis in neonates can be acted as an indication for exploratory surgery when the surgeon's ability, anesthesia, monitoring and management conditions are available. Robotic-assisted laparoscopic pyeloplasty is a safe and feasible option for the treatment of UPJO in infant with less trauma, rapid recovery and good cosmetic result. We suggest that RALP is potentially a recommended feasible treatment option for infant with UPJO. Key words: Robotic-assisted laparoscopic pyeloplasty; Infant; Hydronephrosis; Ureteropelvic junction obstruction

We propose two algorithms involving the relaxation of either the given Dirichlet data or the prescribed Neumann data on the over-specified boundary, in the case of the alternating iterative algorithm of ` 12 ` 12 `$12 `&12 `#12 `^12 `_12 `%12 `~12 *Kozlov91 applied to Cauchy problems for the modified Helmholtz equation. A convergence proof of these relaxation methods is given, along with a stopping criterion. The numerical results obtained using these procedures, in conjunction with the boundary element method (BEM), show the numerical stability, convergence, consistency and computational efficiency of the proposed methods.