Fault-Tolerant Optimal Consensus Control for Heterogeneous Multi-Agent Systems

This paper investigates the optimal couple-group consensus control (OCGCC) for heterogeneous multi-agent systems (HeMASs) with completely unknown dynamics. The agents in HeMASs are divided into two groups according to order differences. Meanwhile, heterogeneous systems are transformed into homogeneous ones by adding virtual velocities. Then, a novel data-driven distributed control protocol for HeMASs is proposed based on policy gradient reinforcement learning (RL). The proposed algorithm is implemented asynchronously, and is specifically designed to address the issue of computational imbalance caused by individual differences among participants. It achieves this by constructing an actor-critic (AC) framework. The system’s learning efficacy is optimized using offline data sets. The convergence and stability are ensured by applying functional analysis and the Lyapunov stability theory. Finally, the effectiveness of the proposed algorithm is confirmed by various simulation examples.

This article addresses the event-based adaptive optimal fault-tolerant consensus control problem for a class of nonlinear multiagent systems (MASs) under a directed graph. It can be solved that both control gains and actuator failure parameters are unknown in considered MASs, which enhances the practicability of optimal consensus control. First, a reinforcement learning-based distributed optimal control law is designed by constructing the identifier-critic-actor learning networks. Furthermore, by utilizing the distributed optimal control law as an auxiliary variable, an adaptive fault-tolerant controller is proposed to effectively compensate for actuator failures. Meanwhile, a co-design scheme is proposed for the construction of an event-triggered control input with the fault-tolerant property. It is proven that the designed control input ensures the boundedness of closed-loop systems through rigorous stability analysis. Finally, the effectiveness of the developed approach can be illustrated via simulations of numerical nonlinear MASs and a group of autonomous underwater vehicles.

This paper studies the multi-tracking control for high-order heterogeneous multi-agent systems under a directed acyclic graph. The authors first convert the multi-tracking control problem for multi-agent systems into the zero steady-state error control problem of some independent subsystems. Based on final-value theorem and Mason's rule, a sufficient condition for achieving the multi-tracking control is proposed. It explicitly reveals how the system dynamic and desired object dynamic jointly affect the multi-tracking control of heterogeneous multi-agent systems. Meanwhile, a new result on the steady-state error for multi-subnetworks is established. Moreover, the multi-tracking control for heterogeneous multi-agent systems with time delay is discussed. Some simulations are performed to illustrate the effectiveness of the theoretical results.

This paper deals with the optimal bipartite consensus control (OBCC) for heterogeneous multi-agent systems (MASs) with time-delay based on reinforcement learning (RL) scheme. MASs, denoted by communication network, is modeled as a signed graph, and the competitive (cooperative) relationship is represented by negative (positive) edges. Firstly, Heterogeneous MASs is depicted by cooperative and competitive (hereinafter referred to as coopetition) network, and local neighborhood bipartite consensus errors and their performance index function (PIF) are put forward to formulate OBCC problem. Secondly, a proper model reduction method is developed to tackle time-delay in coopetition network, and then MASs with time-delay can be transformed into a delay-free MASs. Thirdly, a RL algorithm is employed to solve the solution of Hamilton-Jacobi-Bellman (HJB) equations under OBCC laws, and substantially we adopt neural networks (NNs) to calculate the control policy and PIF online. Finally, the pivotal simulation validates availability of proposed algorithm.

This paper studies time-varying output formation control for linear heterogeneous multi-agent systems (MASs) under signed graphs, employing an adaptive dynamic event-based approach. The signed graph framework is used to model agent interactions, encompassing both cooperative and antagonistic behaviours. Firstly, a data-driven learning algorithm is given for the estimation of the unknown leader system matrix. Secondly, for a single leader MAS, a distributed adaptive dynamic event-based bipartite compensator is designed to estimate the leader's state. This triggering function utilises the internal dynamic variable to dynamically adjust the threshold, ensuring fewer triggering instances compared to traditional static thresholds. Thirdly, a controller integrating the compensator and observer is developed to solve the time-varying output formation control problem for heterogeneous MASs with signed graphs under a single leader. Subsequently, the findings are extended to multi-leader MASs, providing a solution for the formation control problem in such scenarios. Finally, several illustrative examples are provided to validate the theoretical results.

Adaptive output containment control of heterogeneous multi-agent systems with unknown leaders

This article studies the robust model predictive control of heterogeneous multi-agent systems with polytopic uncertainties and time delay. Moreover, some constraints on the amplitude of the control inputs are considered for each agent. The proposed distributed controllers are designed as state feedback. The goal is to design the feedback gains such that the multiagent system achieves consensus in the presence of time-delay and model uncertainties. For this purpose, an optimization problem is proposed, and by using the constrained robust model predictive control and the appropriate Lyapunov–Krasovskii functional, sufficient conditions are obtained to solve the optimization problem through solving the linear matrix inequalities. In this regard, a theorem is presented, and it is proved that the consensus errors converge to zero. Finally, the effectiveness of the proposed method is illustrated using numerical simulations.

This paper investigates the adaptive fault-tolerant formation control scheme for heterogeneous multi-agent systems consisting of unmanned aerial vehicles (UAVs) and unmanned surface vehicles (USVs) with actuator faults, parameter uncertainties and external disturbances under directed communication topology. Firstly, the dynamic models of UAVs and USVs are introduced, and a unified heterogeneous multi-agent system model with actuator faults is established. Then, a distributed fault-tolerant formation controller is proposed for the unified model of UAVs and USVs in the plane by using adaptive updating laws and radial basis function neural network. After that, a decentralized formation-tracking controller is designed for the altitude control system of UAVs. Based on the Lyapunov stability theory, it can be proved that the formation errors and tracking errors are uniformly ultimately bounded which means that the expected time-varying formation is achieved. Finally, a simulation study is given to demonstrate the effectiveness of the proposed scheme.

Purpose This paper proposes an adaptive distributed observer-based optimal control protocol designed to tackle the challenges posed by unavailable state information, without relying on global information, while ensuring optimal system performance in heterogeneous multi-agent systems. The purpose of this paper is to achieve distributed output tracking control under these constraints. Design/methodology/approach An adaptive distributed observer using single adaptive and local static gains estimates leader’s output. A feedback-feedforward control policy is designed for optimal tracking. Unlike conventional methods, this reduces communication load and broadens applications when leader’s state is unmeasurable. Findings A numerical simulation example related to autonomous ground vehicle is given to demonstrate that the proposed control algorithm can ensure output tracking in heterogeneous multi-agent systems. Research limitations/implications Although the proposed algorithm can effectively guarantee output tracking, further research is needed to design an optimal controller for model-free system dynamics in unknown environments. Practical implications The proposed method can be implemented for formation control of multi-robot systems since each robot can measure the relative output information, thereby effectively overcoming the limitation of unavailable system states. Originality/value This paper introduces a novel adaptive distributed observer-based optimal control protocol for heterogeneous multi-agent systems, uniquely relying on a single adaptive gain and local output information, eliminating the need for global network data. This reduces communication overhead and enhances applicability in scenarios where the leader’s state is unmeasurable. Unlike existing methods, it integrates optimal tracking control via the Bellman equation, considering transient performance. Its value lies in practical implementation for multi-robot formation control, though limitations include the need for further research into model-free systems in unknown environments.

In this article, a distributed adaptive optimal control method is proposed to solve the bipartite consensus problem for heterogeneous multi-agent systems. First, with a change in coordinates, the optimal bipartite consensus can be transformed into the optimal consensus problem and the optimal control law is also established. Second, a distributed state observer is designed for each agent to estimate the leader's state under the cooperate/antagonistic interaction, which is used to replace the unavailable leader's signal. Then, to find the optimal control solution adaptively, two online integral reinforcement learning algorithms, i.e., on-policy and off-policy, are developed. Based on the policy iteration in learning process, the algorithms proposed here utilize the state data of systems without requiring the complete knowledge of the leader's and agents' dynamics. It is proven that the observer is exponentially convergent, which guarantees the accuracy of solution in algorithms. Finally, two examples are given to show the validity of the proposed method.

ABSTRACTThis paper considers the controllability problem for both continuous- and discrete-time linear heterogeneous multi-agent systems with directed and weighted communication topology. First, two kinds of neighbour-based control protocols based on the distributed protocol of first-order and second-order multi-agent systems are proposed, under which it is shown that a heterogeneous multi-agent system is controllable if the underlying communication topology is controllable. Then, under special leader selection, the result shows that the controllability of a heterogeneous multi-agent system is solely decided by its communication topology graph. Furthermore, some necessary and/or sufficient conditions are derived for controllability of communication topology from algebraic and graphical perspectives. Finally, simulation examples are presented to demonstrate the effectiveness of the theoretical results.

Observer-based reinforcement learning for optimal fault-tolerant consensus control of nonlinear multi-agent systems via a dynamic event-triggered mechanism

The problem of resilient optimal bipartite tracking control for heterogeneous multi-agent systems with multiple targets under denial-of-service (DoS) attacks is investigated in this paper. A bipartite tracking mechanism is devised in which the agents track the targets under bipartite consensus control, which becomes non-autonomous. This is owing to DoS attacks, as the agents cannot obtain real-time information on the tracked targets and neighboring agents. Consequently, A target observer with a storage module has been developed for the efficient estimation of agent states and the storage of observed information as historical data. By recalling the historical data of the observed state, a new type of distributed resilient optimal controller is formulated, which can achieve the control objective in the case of communication blockage, while minimizing the performance index function of the system. Numerical simulations are performed to verify the proposed secure control design.

Containment control of heterogeneous multi-agent systems

In this paper, the problem of formation control of heterogeneous multi-agent systems is investigated. The heterogeneous multi-agent system is composed of first-order and second-order agents, where the second-order agents cannot obtain the velocity information. Firstly, without considering the input saturation, the distributed formation control protocols are designed for multi-agent systems with static leader and active leader respectively. By using Lyapunov stability theory and LaSalle's invariance principle, the protocols are proved to be effective. Then due to the actor saturation, the formation protocols for multi-agent systems with input saturation based on distributed observers and auxiliary systems are presented. Finally, the simulation results are given to illustrate the effectiveness of the proposed protocols.