Learner control in computer‐assisted learning*

This paper formulates a new approach to the classical learning/adaptive control. problem. Our approach is based on two key observations: (1( the inherent conflict between control and identification as they compete for the only available resource, namely the input to the plant; (2) when designing and optimizing the performance of a control system the current task, as well as the repertoire of other typical future tasks which the system may encounter during its life time, should be considered. Our approach is formulated for a general nonlinear time-varying plant; thus, unlike existing adaptive control theory, the theory for a linear time-invariant system evolves as a special case of the general case. The design for the full lifetime of the system creates a methodology that specifies what current actions should be taken in addition to the tracking of the current reference trajectory, at the expense of some performance degradation in the current task, so as to improve the performance of future tasks: this is the learning trade off. The conflicting objectives, namely, tracking versus learning and current task versus future tasks, are most naturally posed and partially solved in the domain of ‘multiple objective optimization theory’. We demonstrate for linear time-invariant plants with quadratic cost, that Pareto optimal learning adaptive controllers may be obtained by simple ‘out of loop’ mixing, where a scalar controls the tracking versus learning trade off in a reliable way.

This paper proposes an adaptive learning control scheme to solve high-precision velocity tracking problem for tank gun control servo systems. Lyapunov approach is used to design the learning controller, with alignment condition used to cope with initial problem of iterative learning control. Robust control technique and adaptive learning control technique are synthesized to handle nonlinear uncertainties and external disturbances. The unknown parameters are estimated according to the full saturation difference learning strategy. As the iteration number increases, the system state can accurately track the reference signal over the whole time interval, and all signal are guaranteed to be bounded.

In this work, path tracking control of autonomous vehicles (AVs) are studied under the framework of adaptive iterative learning control (AILC). In order to facilitate the controller design, the nonlinear vehicle dynamics are firstly transformed into a parametric form by utilizing proper variable transformations. Then a robust adaptive learning control approach is developed to achieve the path tracking tasks. Furthermore, to deal with the actuator constraints, an input-dependent auxiliary system is employed to work together with the proposed learning controller, which thus helps to reduce their influence to the control performance. The convergence analysis for the proposed method is also provided by virtue of the Lyapunov-like theory. It has been shown that with the proposed AILC approach, a good path tracking performance can be ensured in the presence of system uncertainties, exteranl disturbances and input constraints. A numerical example is illustrated to verify its effectiveness.

ABSTRACTA new type of saturated position (Proportional) and velocity (Differential) feedback (SPD servo-loop) induced by a quasi-natural potential is proposed for control of robot manipulators with all revolute-type joints. If the servo loop is composed of such SP-D feedback, then not only the original robot dynamics but also the residual error dynamics satisfy the passivity with appropriate choice of feedback gains. By virtue of the passivity, it is possible to show the existence of Lyapunov functions in a global sense for a variety of set-point, adaptive trajectory tracking, and learning controls. Proofs of convergence of tracking errors in model-based adaptive control and learning control are given on the basis of construction of such Lyapunov functions.

Considerable research has described students' deep and surface approaches to learning. Other research has described individuals' self-regulated learning and need for cognition. There is a need for research examining the relationships among these constructs. This study explored relationships among approaches to learning (deep, surface), need for cognition, and three types of control of learning (adaptive, inflexible, irresolute). Theory suggested similarities among the deep approach, need for cognition, and adaptive control (aspects of self-regulated learning); and among surface, inflexible, and irresolute control (aspects of an ineffective approach to learning). One-factor and two-factor models were proposed. Participants were 226 Canadian military college students. Participants completed the following questionnaires: the Study Process Questionnaire (Biggs, 1978), the Need for Cognition Scale (Cacioppo & Petty, 1982), and the Strategic Flexibility Questionnaire (Cantwell & Moore, 1996). Confirmatory factor analysis supported the identification of the six scale factors. Second order confirmatory factor analysis indicated three factors representing constructs underlying these factors. Neither the one- nor two-factor models accounted adequately for the data. Self-regulated learning was defined by measures of the deep approach to learning, need for cognition, and adaptive control of learning. The second factor divided into one factor consisting of irresolute control, the surface approach, and negative need for cognition; and another consisting of inflexible and negative adaptive control. Substantial relationships among scales support the need for further theory development.

In this paper, an adaptive iterative learning control (AILC) method combined with sliding mode technique is proposed to improve the force control performance for repeating tasks of fluidic muscle (FM) driven parallel manipulators. Different from the traditional iterative learning control method, the proposed AILC is to learn the controller time-varying parameters rather than to learn the control signals. Since the AILC is sensitive to non-repetitive disturbances, the sliding mode technique is introduced to enhance the robustness. Since no model information involved in the controller design, the proposed method is a complete data-driven method. Hence, the difficulty of obtaining accurate model is avoided. Simulation studies are performed on a two degrees of freedom FM driven parallel manipulator. Simulation results demonstrate that the proposed method can achieve high force tracking performance and robustness.

Adaptive iterative learning control is a popular approach to control parametric time-varying systems over finite time interval. However, initial resetting condition limits the application of a large amount of learning control algorithms. In this paper, we propose a suboptimal iterative learning control for a class of dynamic parametric systems to solve this problem. An error-tracking control strategy is used to deal with initial position problem, with suboptimal strategy used to accelerate convergence speed. As the iteration increases, the tracking error of the closed-loop system accurately follows the desired error trajectory over the entire time interval, and the system state tracks the reference signal perfectly on the pre-specified interval. The performance of the proposed suboptimal error-tracking learning control scheme is illustrated by an example involving both suboptimal errortracking algorithm and normal error-tracking algorithm.

Effectiveness of cognitive-load based adaptive instruction in genetics education

This study investigated the effects of two general approaches to the provision of learner control and of two types of incentive on achievement, use of options, time, and attitude using computer-delivered instruction. Posttest scores for the “FullMinus” treatment for learner control, where learners could selectively bypass elements of a full instructional program, were marginally higher than those for the “LeanPlus” treatment, where learners could opt to add elements to a core program (p=.052). The FullMinus treatment resulted in more positive attitudes while requiring no more time. Performance-contingent incentive groups had higher post-test scores than task-contingent groups (p<.05), with no greater investment of learner time and no negative effect on attitude. The findings relate directly to the design of instructional computer programs as well as to the design of future studies on learner control.

In this paper, an adaptive iterative learning control algorithm is designed based on extended state observer. Traditional robotic path tracking control has low control precision and poor control performance. A basic structure of the double joint manipulator is used for study. The system model is established, and a manipulator trajectory tracking control scheme is proposed. The specific algorithm principle and implementation method are given. The proposed algorithm consists of variable gain PD control, adaptive control, and extended state observer. The PD control can change the control gain adaptively because of the adaptive feedback loop to ensure the system stability. The extended state observer regards both the internal interference and the external random disturbance of the system. The state is used to compensate for the uncertainty of both. Experiment results show that the proposed algorithm improves the range and performance of the trajectory tracking error while improving the system convergence, and improves the control performance of the system.

A new paradigm for learning control of a generic nonlinear system is presented. The method is based on a special radial basis function network architecture and is adaptive as the input/output properties of the system are estimated and the learning control gain is computed online. The controlled system is described with a mapping between the feedforward control and an array of sampled output values over a given time interval. The network approximates the system input/output mapping as a function of the task parameter vector comprising the initial and a desired final system state. The approximation is affine and linear in the control input. For given network weights, this allows inverting the network approximation and an optimal control program. The network weights are updated depending on the results of the learning trials. The application of the algorithm to the control of an arbitrary planar motion of a two link arm demonstrates fast learning and the high accuracy of the method. >

As robots become more capable in terms of hardware, and more complex tasks are considered, optimality starts playing a more important role in the design of algorithms implemented in these systems. Optimality is a guiding principle that directs the computation of feasible and efficient solutions to different robotics tasks. In control theory, this principle is implemented online as a set of efficient numerical optimization algorithms, that in addition to solving the task, purports to save a suitably defined effort or energy term. This thesis investigates trajectory generation, learning and control for dynamic tasks from the unifying point of view of optimization. As an application, we focus on Table Tennis, a chal- lenging task where robots are yet to outperform humans. We believe that the required dexterity and accuracy for this dynamical task hinges on the developments in online optimization and efficient learning algorithms. We consider trajectory generation for table tennis in the first part of the thesis. In highly dynamic tasks like table tennis that involve moving targets, planning is necessary to figure out when, where and how to intercept the target. Motion planning can be very challenging in robotic table tennis in particular, due to time constraints, dimension of the search space and joint limits. Conventional planning algorithms often rely on a fixed virtual hitting plane to construct robot striking trajectories. These algorithms, however, generate restrictive strokes and can result in unnatural strategies when compared with human playing. In this thesis, we introduce a new trajectory generation framework for robotic table tennis that does not involve a fixed hitting plane. A free-time optimal control approach is used to derive two different trajectory optimizers. The resulting two algorithms, Focused Player and Defensive Player, encode two different play- styles. We evaluate their performance in simulation and in our robot table tennis platform with a high speed cable-driven seven DOF robot arm. The algorithms return the balls with a higher probability to the opponent’s court when compared with a virtual hitting plane based method. Moreover, both can be run online and the trajectories can be corrected with new ball observations. In the second part of the thesis, we look at how such trajectories, computed on the kine- matics level, can be tracked accurately with learning control based approaches. Highly dynamic tasks like table tennis require large accelerations and precise tracking for successful perfor- mance. To track desired trajectories well, such tasks usually rely on accurate models and/or high gain feedback. While kinematic optimization allows for efficient representation and online generation of hitting trajectories, learning to track such dynamic movements with inaccurate models remains an open problem. In particular, stability issues surrounding the learning per- formance, in the iteration domain, can prevent the successful implementation of model based learning approaches. To achieve accurate tracking for these tasks in a stable and efficient way, we propose a new adaptive Iterative Learning Control algorithm that is implemented efficiently using a recursive approach. Moreover, covariance estimates of model matrices are used to ex- ercise caution during learning. We evaluate the performance of the proposed approach in our robotic table tennis platform, where we show how the performance of two Barrett WAMs can be optimized. Our implementation on the table tennis platform compares favorably with two state-of-the-art approaches. Finally, we discuss an alternative learning from demonstrations approach, where we learn sparse representations from demonstrated movements. Learning from demonstrations is an easy and intuitive way to show examples of successful behavior to a robot. However, the fact that humans optimize or take advantage of their body and not of the robot, usually called the embodiment problem in robotics, often prevents industrial robots from executing the task in a straightforward way. The shown movements often do not or cannot utilize the degrees of freedom of the robot efficiently, and typically suffer from excessive execution errors. In the last chapter, we show a new approach that can alleviate some of these difficulties by learning sparse representations of movement. Moreover, the number of learned parameters are independent of the degrees of freedom of the robot. Sparsity is a desirable feature for policy search Rein- forcement Learning algorithms that adapt the parameters of these movement primitives. By ranking the learned parameters on the Elastic Net path in terms of importance, we note that our approach could be potentially useful to combat the curse of dimensionality in robot learning applications. We show preliminary results on the real robot setup, including a successful table tennis serve using our new movement primitive representation. Throughout the thesis, we present and analyze in detail new control and learning algo- rithms. Efficient online optimization approaches are presented that can be used to solve not just table tennis problems, but they can be adapted to solve different dynamic tasks.

This article tries to handle the alignment initial condition for contraction mapping based iterative learning control, such that the system can operate continuously without any initial condition reset. This goal is achieved for a class of nonlinear systems through the proposed conditional learning control, which has several advantages over the alternative method, adaptive learning control. The conditional learning control guarantees that sufficient knowledge can be learned to update the input and achieve perfect output tracking, despite the non-identical initial conditions. The sufficient conditions of either monotonic or strictly monotonic convergence of the input sequence, and the choice of learning gains are given. The performance of the proposed method is illustrated by simulated examples.

A neural network adaptive inverse controller structure is proposed to solve the problems that hypersonic aircraft with highly nonlinear and flight conditions of large span. OS-ELM algorithm is improved with RBF network for online learning and adaptive inverse controller for the on line training is used. Hypersonic aircraft with six degrees of freedom non-linear model is used to verify the performance of adaptive inverse controller. The controller based on flight conditions doing online learning in real time is shown with flight simulation in a typical condition. Aircraft orders tracking is achieved rapidly and steadyly and desired control effect is achieved.

This paper presents the problems of accurate identification and learning control of ocean surface ship in uncertain dynamical environments. Thanks to the universal approximation capabilities, radial basis function neural networks (NNs) are employed to approximate the unknown ocean surface ship dynamics. A stable adaptive NN tracking controller is first designed using backstepping and Lyapunov synthesis. Partial persistent excitation (PE) condition of some internal signals in the closed-loop system is satisfied during tracking control to a recurrent reference trajectory. Under the PE condition, the proposed adaptive NN controller is shown to be capable of accurate identification/learning of the uncertain ship dynamics in the stable control process. Subsequently, a novel NN learning control method which effectively utilizes the learned knowledge without re-adapting to the unknown ship dynamics is proposed to achieve closed-loop stability and improved control performance. Simulation studies are performed to demonstrate the effectiveness of the proposed method.