Reinforcement Learning-Based Cooperative Optimal Output Regulation via Distributed Adaptive Internal Model.

This paper proposes a reinforcement learning approach to the output synchronization problem for heterogeneous leader-follower multi-agent systems, where the system dynamics of all agents are completely unknown. First, to solve the challenge caused by unknown dynamics of the leader, we develop an experience-replay learning method to estimate the leader's dynamics, which only uses the leader's past state and output information as training data. Second, based on the newly estimated leader's dynamics, we design an event-triggered observer for each follower to estimate the leader's state and output. Furthermore, the experience-replay learning method and the event-triggered leader observer are co-designed, which ensures the convergence and Zeno behavior exclusion. Subsequently, to free the followers from reliance on system dynamics, a data-driven adaptive dynamic programming (ADP) method is presented to iteratively derive the optimal control gains, based on which we design a policy iteration (PI) algorithm for output synchronization. Finally, the proposed algorithm's performance is validated through a simulation.

In this work, we investigate the cooperative output regulation problem of multi-agent uncertain nonlinear systems in lower triangular form with unknown leader. The cooperative output regulator construction is done based on a nonlinear internal model approach. Specifically, nonlinear internal models are first constructed, converting the cooperative output regulation problem into a cooperative stabilization problem of the augmented system, and then an integral input-to-state stable (iISS) based approach is proposed to solve the latter cooperative stabilization. It finally gives rise to a constructive state feedback design for cooperative nonlinear output regulation problem.

In this paper, we study the cooperative output regulation problem for a class of second-order linear uncertain multi-agent systems with unknown exosystem. We first present a distributed canonical internal model to convert the cooperative output regulation problem for the original plant into a stabilization problem for its augmented system. Then we solve the problem via a distributed state feedback controller by combining the distributed internal model approach and the adaptive control technique. Our design is illustrated by an example.

The discrete distributed observer is developed recently to solve the cooperative output regulation problem. The validation of the existing discrete distributed observer depends on the establishment of the communication network. In some practical applications, only the sensor network can be established among all agents. To handle such cases, we propose a sensory feedback-based discrete distributed observer, which depends on the relative output of neighboring agents. In contrast to existing results, the design of the discrete distributed observer and cooperative output regulation analysis should be conducted simultaneously. Resorting to the small-gain theorem, we develop a small-gain condition-based approach and a low-gain-based approach, respectively. By these two approaches, we demonstrate that the discrete distributed observer holds and the cooperative output regulation problem of heterogeneous discrete-time linear multiagent systems can be solved.

In this paper, we consider the cooperative output regulation problem of heterogeneous linear multi-agent systems (MASs) by event-triggered control. We first develop an event-triggering mechanism for leader-following consensus of homogeneous MASs. Then by proposing an internal reference model for each agent, a novel distributed event-triggered control scheme is developed to solve the cooperative output regulation problem of heterogeneous MASs. Furthermore, a novel self-triggered control scheme is also proposed, such that continuous monitoring of measurement errors can be avoided. The feasibility of both proposed control schemes is studied by excluding Zeno behavior for each agent. An example is finally provided to demonstrate the effectiveness of the control schemes.

Event-triggered robust cooperative output regulation for a class of linear multi-agent systems with an unknown exosystem

In this article, the event-triggered cooperative output regulation problem of T–S fuzzy multiagent systems under jointly connected switching directed topologies is investigated. A distributed sampled-data event-triggered scheme is proposed such that the estimator, which estimates the exosystem, transmits the data information to its neighbors under the switching directed topologies at its triggering instants. The continuous event detection and triggering Zeno behavior are avoided via aperiodic sampling. Then, the time-dependent Lyapunov functional approach together with the dwell time technique is utilized, such that the estimation errors converge to zero exponentially. Furthermore, the observer-based fuzzy controllers are developed to ensure the solvability of the cooperative output regulation. Besides, the advantage of the distributed sampled-data event-triggered scheme is that the data transmissions caused by triggerings and switchings are autocephalous, which avoids the situation that triggering must occur at the switching points. Finally, an example is given to show the validity of results.

This paper studies the cooperative output regulation problem for heterogeneous multi‐agent systems (MASs) subject to false data injection (FDI) attacks, which means the malicious cyber‐attackers may inject false data into the communication topology randomly. To ensure that each follower agent can track the leader agent accurately, we construct both an observer and a control law for each follower agent. Since the FDI attacks may exist in the communication network, a filter with an adaptive compensator, which can estimate attack signals dynamically, is equipped for each observer, such that the filter can neutralize attacks when malicious attackers launch attacks. Therefore, the cooperative output regulation can be achieved under the control law based on the observer no matter whether there exist attacks or not. Finally, a numerical example is provided for validating the theoretical results.

In this paper, we consider the cooperative output regulation problem for heterogeneous linear multi-agent systems in the presence of communication constraints. Under standard assumptions on the agents dynamics, we propose a distributed control algorithm relying on intermittent and asynchronous discrete-time information exchange that can be subject to unknown time-varying delays and information losses. We show that cooperative output regulation can be reached for arbitrary characteristics of the discrete-time communication process and under mild assumptions on the interconnection topology between agents. A numerical example is given to illustrate the effectiveness of our theoretical results.

While several robust cooperative output regulation approaches for heterogeneous systems have been proposed (with fixed-gain distributed controllers), the design of adaptive-gain distributed controllers becomes relevant in dealing with larger uncertainty than robust approaches. This letter addresses the adaptive cooperative output regulation problem for heterogeneous systems with unknown linear dynamics, where possibly large system uncertainty would make fixed-gain robust approaches not applicable. A passification method is adopted to design adaptive-gain distributed controllers solving the problem. The proposed method includes two steps: in the first step, a distributed observer of the exogenous signal is designed for those systems that cannot access the exosystem, and a reference model is designed whose output can converge to the exogenous signal; in the second step, command generator tracking is achieved via adaptive laws that make the output of each system converge to the output of the reference model, and thus to the exogenous signal. Stability analysis is provided via a Lyapunov approach, and a numerical example illustrates the effectiveness of the approach.

The cooperative output regulation of a linear multi-agent system can be viewed as a generalization of the leader-following consensus problem and was studied recently for the case where the system uncertain parameters vary in a sufficiently small neighborhood of their nominal value. This case was handled by the internal model design which converts the problem into a simultaneous eigenvalue placement problem of an augmented multi-agent system. In this paper, we further consider the cooperative robust output regulation problem for a class of minimum phase linear multi-agent systems in the sense that the controller allows the system uncertain parameters to vary in an arbitrarily prescribed compact subset. For this purpose, we introduce a new type of internal model that allows the cooperative robust output regulation problem of the given plant to be converted into a robust stabilization problem of an augmented multi-agent system. We then solve our problem by combining a simultaneous high-gain state feedback control technique and a distributed high-gain observer technique. A special case of our result leads to the solution of the leader-following robust consensus problem for a class of uncertain multi-agent systems. Copyright © 2013 John Wiley & Sons, Ltd.

This paper develops a distributed control framework to coordinate large-scale wind turbines for fairly sharing the power reserve request on a wind farm. A cooperative output regulation problem is formulated in a multi-agent framework, where the synchronization of turbine system outputs are established regarding the heterogeneity of turbine models. For this purpose, a homogeneous virtual system is created first and embedded into each turbine to generate the agreed trajectories across the wind farm in a distributed manner. The local power tracking of individual turbines is then enforced by the synthesized output regulator, whose control gains are determined by a variant of the Sylvester equation. The proposed approach issues the generator torque and blade pitch commands in a unified manner and is resistant to wind disturbances. The case studies are carried out through simulations and in a real laboratory-scale distributed communication network. The simulation results demonstrate the effectiveness of coordinated actions of wind turbines in response to the changing demands, under both constant and turbulent wind scenarios with the plug-and-play capability.

This paper studies the cooperative global robust output regulation problem for the nonlinear multi-agent systems in output feedback form with relative degree $r=2$ subject to arbitrarily large uncertainty. The robust control method alone cannot handle arbitrarily large uncertainty. By combining the robust and adaptive high gain technique, we develop a recursive procedure to construct a distributed output feedback control law to deal with the problem. The theoretical result of this paper is applied to the leader-following consensus problem of a family of hyper-chaotic Lorenz systems.

Survey/review study The Cooperative Output Regulation by the Distributed Observer Approach Youfeng Su 1, He Cai 2, and Jie Huang 3,* 1 College of Computer and Data Science, Fuzhou University, Fuzhou 350116, China 2 School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, China 3 Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China * Correspondence: jhuang@mae.cuhk.edu.hk Received: 7 November 2022 Accepted: 13 November 2022 Published: 22 December 2022 Abstract: The cooperative output regulation problem (CORP) is an extension of the leader-following consensus problem of multi-agent systems (MASs), and has been studied by two approaches, namely, the distributed observer (DO) approach and the distributed internal model (DIM) approach. The two approaches are, respectively, the extensions of the classical feedforward control approach and the classical internal model approach (for a single system) to the MASs. This paper overviews the CORP by the DO approach with the emphasis on linear MASs. After formulating the CORP, we present the evolution process of three types of DOs and the corresponding solutions to the CORP. Furthermore, some variants and extensions of the DO approach are also briefly surveyed for completeness.

Fully distributed dynamic event-triggered reliable COR of nonlinear MASs with communication link faults and actuator faults for directed graphs