Data-Based Optimal Couple-Group Consensus Control for Heterogeneous Multi-Agent Systems via Policy Gradient Reinforcement Learning

This study explores fault-tolerant consensus in leader–following heterogeneous multi-agent systems, focusing on actuator failures in uncrewed aerial vehicles (UAVs) and uncrewed ground vehicles (UGVs). An optimization-based fault-tolerant consensus algorithm is proposed. The algorithm utilizes the Euler–Lagrange formula to ensure system consistency under actuator failures, with the Lyapunov stability theory proving the asymptotic stability of the consistency error. The algorithm is applied to heterogeneous multi-agent systems of UAVs and UGVs to derive optimal fault-tolerant consensus control laws for each vehicle type. Simulation experiments give evidence for the feasibility of the proposed control strategy.

Learning-based optimal cooperative control using edge-based triggering strategy

This paper studies the multi-tracking control for high-order heterogeneous multi-agent systems under a directed acyclic graph. The authors first convert the multi-tracking control problem for multi-agent systems into the zero steady-state error control problem of some independent subsystems. Based on final-value theorem and Mason's rule, a sufficient condition for achieving the multi-tracking control is proposed. It explicitly reveals how the system dynamic and desired object dynamic jointly affect the multi-tracking control of heterogeneous multi-agent systems. Meanwhile, a new result on the steady-state error for multi-subnetworks is established. Moreover, the multi-tracking control for heterogeneous multi-agent systems with time delay is discussed. Some simulations are performed to illustrate the effectiveness of the theoretical results.

This paper studies time-varying output formation control for linear heterogeneous multi-agent systems (MASs) under signed graphs, employing an adaptive dynamic event-based approach. The signed graph framework is used to model agent interactions, encompassing both cooperative and antagonistic behaviours. Firstly, a data-driven learning algorithm is given for the estimation of the unknown leader system matrix. Secondly, for a single leader MAS, a distributed adaptive dynamic event-based bipartite compensator is designed to estimate the leader's state. This triggering function utilises the internal dynamic variable to dynamically adjust the threshold, ensuring fewer triggering instances compared to traditional static thresholds. Thirdly, a controller integrating the compensator and observer is developed to solve the time-varying output formation control problem for heterogeneous MASs with signed graphs under a single leader. Subsequently, the findings are extended to multi-leader MASs, providing a solution for the formation control problem in such scenarios. Finally, several illustrative examples are provided to validate the theoretical results.

Adaptive output containment control of heterogeneous multi-agent systems with unknown leaders

This article studies the robust model predictive control of heterogeneous multi-agent systems with polytopic uncertainties and time delay. Moreover, some constraints on the amplitude of the control inputs are considered for each agent. The proposed distributed controllers are designed as state feedback. The goal is to design the feedback gains such that the multiagent system achieves consensus in the presence of time-delay and model uncertainties. For this purpose, an optimization problem is proposed, and by using the constrained robust model predictive control and the appropriate Lyapunov–Krasovskii functional, sufficient conditions are obtained to solve the optimization problem through solving the linear matrix inequalities. In this regard, a theorem is presented, and it is proved that the consensus errors converge to zero. Finally, the effectiveness of the proposed method is illustrated using numerical simulations.

The problem of resilient optimal bipartite tracking control for heterogeneous multi-agent systems with multiple targets under denial-of-service (DoS) attacks is investigated in this paper. A bipartite tracking mechanism is devised in which the agents track the targets under bipartite consensus control, which becomes non-autonomous. This is owing to DoS attacks, as the agents cannot obtain real-time information on the tracked targets and neighboring agents. Consequently, A target observer with a storage module has been developed for the efficient estimation of agent states and the storage of observed information as historical data. By recalling the historical data of the observed state, a new type of distributed resilient optimal controller is formulated, which can achieve the control objective in the case of communication blockage, while minimizing the performance index function of the system. Numerical simulations are performed to verify the proposed secure control design.

Distributed Optimal Output Feedback Control of Heterogeneous Multi-agent Systems under a Directed Graph

In this article, a distributed adaptive optimal control method is proposed to solve the bipartite consensus problem for heterogeneous multi-agent systems. First, with a change in coordinates, the optimal bipartite consensus can be transformed into the optimal consensus problem and the optimal control law is also established. Second, a distributed state observer is designed for each agent to estimate the leader's state under the cooperate/antagonistic interaction, which is used to replace the unavailable leader's signal. Then, to find the optimal control solution adaptively, two online integral reinforcement learning algorithms, i.e., on-policy and off-policy, are developed. Based on the policy iteration in learning process, the algorithms proposed here utilize the state data of systems without requiring the complete knowledge of the leader's and agents' dynamics. It is proven that the observer is exponentially convergent, which guarantees the accuracy of solution in algorithms. Finally, two examples are given to show the validity of the proposed method.

ABSTRACTThis paper considers the controllability problem for both continuous- and discrete-time linear heterogeneous multi-agent systems with directed and weighted communication topology. First, two kinds of neighbour-based control protocols based on the distributed protocol of first-order and second-order multi-agent systems are proposed, under which it is shown that a heterogeneous multi-agent system is controllable if the underlying communication topology is controllable. Then, under special leader selection, the result shows that the controllability of a heterogeneous multi-agent system is solely decided by its communication topology graph. Furthermore, some necessary and/or sufficient conditions are derived for controllability of communication topology from algebraic and graphical perspectives. Finally, simulation examples are presented to demonstrate the effectiveness of the theoretical results.

This paper deals with the optimal bipartite consensus control (OBCC) for heterogeneous multi-agent systems (MASs) with time-delay based on reinforcement learning (RL) scheme. MASs, denoted by communication network, is modeled as a signed graph, and the competitive (cooperative) relationship is represented by negative (positive) edges. Firstly, Heterogeneous MASs is depicted by cooperative and competitive (hereinafter referred to as coopetition) network, and local neighborhood bipartite consensus errors and their performance index function (PIF) are put forward to formulate OBCC problem. Secondly, a proper model reduction method is developed to tackle time-delay in coopetition network, and then MASs with time-delay can be transformed into a delay-free MASs. Thirdly, a RL algorithm is employed to solve the solution of Hamilton-Jacobi-Bellman (HJB) equations under OBCC laws, and substantially we adopt neural networks (NNs) to calculate the control policy and PIF online. Finally, the pivotal simulation validates availability of proposed algorithm.

Containment control of heterogeneous multi-agent systems

In this paper, the problem of formation control of heterogeneous multi-agent systems is investigated. The heterogeneous multi-agent system is composed of first-order and second-order agents, where the second-order agents cannot obtain the velocity information. Firstly, without considering the input saturation, the distributed formation control protocols are designed for multi-agent systems with static leader and active leader respectively. By using Lyapunov stability theory and LaSalle's invariance principle, the protocols are proved to be effective. Then due to the actor saturation, the formation protocols for multi-agent systems with input saturation based on distributed observers and auxiliary systems are presented. Finally, the simulation results are given to illustrate the effectiveness of the proposed protocols.

Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning

The distributed tracking control problem for heterogeneous multi-agent systems with unknown system dynamics is investigated. The objective is to provide a data-driven distributed tracking control protocol that ensures tracking performance between agents and the leader. To this end, the concept of data informativity for matching conditions is introduced. Then, the data-based sufficient and necessary conditions to achieve state tracking are provided. Meanwhile, a data-driven parameterisation approach for designing the distributed tracking control protocol is given. Compared with previous results, the reference input is considered in the leader's dynamics, and the computational burden is reduced by solving a set of data-based equations and inequality constraints rather than iteration. Additionally, the developed results are still appropriate for handling the tracking control issue of the single linear system, and the current constraint that the reference system be stable is eased. Finally, two simulation examples are given to verify the proposed schemes' effectiveness.