Gaussian Processes for Learning and Control: A Tutorial with Examples

SummaryIn this study, we present a reinforcement learning (RL)‐based flight control system design method to improve the transient response performance of a closed‐loop reference model (CRM) adaptive control system. The methodology, known as RL‐CRM, relies on the generation of a dynamic adaption strategy by implementing RL on the variable factor in the feedback path gain matrix of the reference model. An actor‐critic RL agent is designed using the performance‐driven reward functions and tracking error observations from the environment. In the training phase, a deep deterministic policy gradient algorithm is utilized to learn the time‐varying adaptation strategy of the design parameter in the reference model feedback gain matrix. The proposed control structure provides the possibility to learn numerous adaptation strategies across a wide range of flight and vehicle conditions instead of being driven by high‐fidelity simulators or flight testing and real flight operations. The performance of the proposed system was evaluated on an identified and verified mathematical model of an agile quadrotor platform. Monte‐Carlo simulations and worst case analysis were also performed over a benchmark helicopter example model. In comparison to the classical model reference adaptive control and CRM‐adaptive control system designs, the proposed RL‐CRM adaptive flight control system design improves the transient response performance on all associated metrics and provides the capability to operate over a wide range of parametric uncertainties.

This study proposes integrating Large Language Models (LLMs) into the adaptation mechanism of Model Reference Adaptive Control (MRAC) systems to improve their performance. MRAC is a widely used method for controlling systems that involve uncertainty and change over time; however, fixed-gain structures cause problems like slow convergence, overshoot, and instability in rapidly changing dynamics. To overcome these limitations, three different MRAC architectures are considered: classical MRAC, low-pass filter-based MRAC, and Error-Feedback (EF) MRAC. In the proposed approach, the GPT-4o-mini model optimizes adaptation gains and error compensation coefficients in real time using current data obtained from the control loop. The methods were tested on a DC motor, a mass–spring–damper system, and a wing rock aircraft model. Simulation results show that LLM-enhanced MRAC architectures not only improve control performance but also provide a flexible and scalable framework for adaptive control design. These findings open new research avenues at the intersection of control theory and natural language processing.

Adaptive control for hybrid dynamical systems with user-defined rate of convergence

This study explores the integration of the adaptive control approach with the Model Reference Adaptive Control (MRAC) method, with a view to examining the impact of this integration on the performance of dynamic systems. Adaptive control is highlighted as a versatile and robust methodology that enables systems to cope with uncertainties and adapt to rapidly changing conditions in their operating environments. As a branch of adaptive control, the MRAC method focuses on efficiently achieving desired performance objectives by adjusting system parameters based on a predefined reference model. The primary focus is on investigating how these methods can be applied to optimize the functionality of systems characterized by dynamic and variable nature, such as unmanned aerial vehicles. By addressing challenges related to uncertainties, disturbances, and adaptability, the integration of adaptive control with the MRAC method demonstrates significant potential in improving the robustness, responsiveness, and reliability of such systems. This research investigates the performance of adaptive techniques in enhancing the capabilities of modern control systems. The objective of the research is to assess the effectiveness of MRAC in terms of reference signal tracking. Simulation results demonstrate the ability of MRAC to maintain accurate tracking of the desired reference for key vehicle dynamics, including roll, pitch and yaw. The results demonstrate the robustness of the MRAC algorithm in attitude control, which may lead to support for sustainability issues such as energy consumption.

A New Model Reference Adaptive Aircraft Controller

Model reference adaptive fuzzy control (MRAFC) is proposed to control the active power filter (APF) and model the APF system caused by nonlinear characteristics, variable parameters and external interferences. T-S fuzzy theory is utilized on the approximation of nonlinear model and the current tracking control. APF system based on MRAFC can be guaranteed to be asymptotical stable. Simulation results are included to confirm the effectiveness of the proposed controller and comparisons with APF system based on model reference adaptive control (MRAC) demonstrate that MRAFC method is capable of reducing the modeling error and tracking reference current signal accurately. Moreover, APF based on the proposed method shows the dynamic and static control performance of stronger robustness, better suitability and higher accuracy than the one based on MRAC modus.

Robust Adaptive Control

Recently, a class of adaptive control schemes called L1 Adaptive Control (L1-AC) has been proposed and widely advertised in aerospace control for achieving fast and robust adaptation and better performance than the existing Model Reference Adaptive Control (MRAC) Schemes. The L1-AC scheme is designed mainly for plants with full state measurement even though the name L1-AC has been used as an umbrella name for more general classes of plants. In this paper, we show that the L1-AC for plants with measured states is simply a standard MRAC with a low pass filter inserted in front of the control input. The analysis of the scheme is almost identical to that of MRAC as the same Lyapunov function is used to establish stability. The motivation for using the filter is the fact that for this class of adaptive schemes i.e. MRAC for plants with full state measurement the tracking error can be made arbitrarily small during transient by increasing the adaptive gain. A high adaptive gain however makes the differential equation of the adaptive law or estimator very stiff and leads to numerical problems that cause high oscillations in the estimated parameters leading to loss of adaptivity and deviations from what the theoretical properties dictate. The L1-AC approach mistook these numerical oscillations as properties of the adaptive scheme and inserted an input low pass filter in order to filter them out. While the filter helps reduce the frequency of these oscillations in the control law the price paid is high. First the numerical instability does not go away and the estimated parameters continue to oscillate without converging to the true parameters even in the presence of sufficiently rich signals. Second, due to the filter the tracking error is no longer guaranteed to converge to zero and the transient bounds for the tracking error also depend on the filter. As a result, the tracking properties of the L1-AC scheme are worse than what a simple MRAC scheme can generate with adaptive gains that could be high but away from the region of numerical instability. In addition, the presence of the filter reduces the robust stability margins in the presence of unmodeled dynamics and provides literally no advantage as simple robust MRAC techniques can solve the same problem achieving much better properties. The authors of L1-AC often compare the properties of a numerically unstable MRAC due to extremely high adaptive gains something that is prohibited by robust adaptive control with those of the filtered MRAC aka L1-AC to show that L1-AC performs better. Such comparisons are not only misleading but do not reveal what causes what giving the reader a false impression of a new theory that results to new performance and robustness improvements. The use of filters in adaptive control is not new and they are used to improve the performance and robustness of certain adaptive control schemes without destroying their ideal tracking properties. In this paper we present such a scheme that guarantees good transient performance and robustness and reveals the trade off between transient performance and robust stability.

Trajectory tracking control is indispensable for a wheeled mobile robot to achieve successful navigation. The classical tracking control systems that are used in wheeled mobile robots do not compensate for the parameter uncertainties and external disturbances. For control strategy, this paper presents a novel hybrid approach, combining a neural network-based kinematic controller and a model reference adaptive control. The controller parameters are adaptively determined online using neural networks. The adaptively tuned kinematic controller ensures a fast convergence to the desired trajectory. The model reference adaptive controller retains the desired tracking performance when parameter and model uncertainties occur. The Lyapunov stability method is used to obtain the adaptive gains which guarantee the asymptotic stability of the error dynamics, where the error is the difference between the outputs of the reference model and the actual plant. The performance of the proposed controller is compared with that of the PID controller, kinematic controller, and adaptive dynamic controller using different performance analysis indices such as integral absolute error, integral squared error, and mean absolute error. Simulation studies demonstrate that the proposed controller achieves high tracking accuracy and fast convergence as compared to the PID, kinematic, and adaptive dynamic controllers considering parameter uncertainties and slip disturbances. The outcomes of the simulation studies also illustrate that the proposed controller achieves the best transient performance. Experiments using real-world tests based on a two-wheeled differential drive robot architecture have elucidated the feasibility of the developed controller regarding tracking accuracy, total control effort, and robustness against uncertainties.

Distributed model reference adaptive containment control of heterogeneous multi-agent systems with unknown uncertainties and directed topologies

The problem of rejecting unknown sinusoidal disturbances in the output of an unknown LTI system which is also corrupted by wide-band noise has received a lot of interest in the recent years. Most of the successful efforts proposed up to now assume that the model of the LTI system is stable and is known exactly. This paper shows how an overparameterized classical robust model-reference adaptive control (MRAC) scheme can handle the problem of simultaneous reference-model tracking and unknown disturbance suppression. The results are applicable to a plant whose dominant modeled part is unknown and possibly unstable, yet has stable zeros and a known relative degree. Some considerations are taken to avoid boosting broadband noise effects on the plant output and to improve robustness with respect to plant unstructured unmodeled dynamics. The architecture of the classical MRAC is employed together with overparameterization of the controller and an auxiliary control input for performance improvement. The performance and robustness properties of the scheme are discussed and numerical simulations are given to demonstrate the efficacy of the scheme.

The purpose of this chapter is to redesign the standard adaptive control schemes by using hybrid structure composed by Model Reference Adaptive Control (MRAC) or Adaptive Pole Placement Control (APPC) strategies, associated to Variable Structure (VS) schemes for achieving non-standard robust adaptive control strategies. The both control strategies is now on named VS-MRAC and VS-APPC. We start with the theoretical base of standard control strategies APPC and MRAC, discussing their structures, as how their parameters are identified by adaptive observers and their robustness properties for guaranteeing their stability. After that, we introduce the sliding mode control (variable structure) in each control scheme for simplifying their design procedure. These design procedure are based on stability analysis of each hybrid robust control scheme. With the definition of both hybrid control strategies, it is analyzed their behavior when controlling system plants with unmodeled disturbances and parameter variation. It is established how the adaptive laws compensates these unmodeled dynamics. Furthermore, by using simple systems examples it is realized a comparison study between the hybrid structures VS-APPC and VSMRAC and the standard schemes APPC and MRAC. As the hybrid structures use switching laws due to the sliding mode scheme, the effect of chattering is analyzed on the implementation and consequently effects on the digital control hardware where sampling times are limiting factor. For reducing these drawbacks it is also discussed possibilities which kind of modifications can employ. Finally, some practical considerations are discussed on an implementation on motor drive systems.

Real-Time Fuzzy Adaptive Control of the Temperature Plant

This paper considers the output feedback control of single input single output uncertain nonlinear systems without the matching condition. The approach is based on the control parametrization employed in classical model reference adaptive control (MRAC). The nonlinearities are allowed to depend not only on the plant output but also on the plant unmeasurable state. The variable structure model reference adaptive control (VS-MRAC) is redesigned with an appropriate state upper bound for the rejection of state dependent input disturbance. Global or semi-global exponential stability of the closed loop system with respect to a small residual set is achieved. A simplified version of VS-MRAC with constant modulation functions is applied to the nonlinear longitudinal dynamics of a Twin Otter aircraft for pitch pointing manoeuvre.

Partial-state feedback multivariable MRAC and reduced-order designs