Neural-Network-Based Adaptive Leader-Following Control for Multiagent Systems With Uncertainties

This paper proposes a reinforcement learning approach to the output synchronization problem for heterogeneous leader-follower multi-agent systems, where the system dynamics of all agents are completely unknown. First, to solve the challenge caused by unknown dynamics of the leader, we develop an experience-replay learning method to estimate the leader's dynamics, which only uses the leader's past state and output information as training data. Second, based on the newly estimated leader's dynamics, we design an event-triggered observer for each follower to estimate the leader's state and output. Furthermore, the experience-replay learning method and the event-triggered leader observer are co-designed, which ensures the convergence and Zeno behavior exclusion. Subsequently, to free the followers from reliance on system dynamics, a data-driven adaptive dynamic programming (ADP) method is presented to iteratively derive the optimal control gains, based on which we design a policy iteration (PI) algorithm for output synchronization. Finally, the proposed algorithm's performance is validated through a simulation.

This article focuses on the issue of scaled tracking consensus for discrete-time second-order multiagent systems under random packet dropouts, where the cases with a static leader and a dynamic leader are considered, respectively. The scaled tracking consensus means that all agents reach a consensus value determined by the leader but with different scales, and the phenomenon of packet dropout on each communication link is described as a Bernoulli variable independent of other communication links. By virtue of random environment-based scaled consensus algorithms, it is shown how to reconstruct the original system into augmented error systems with random coefficient matrices. With the kind assistance of substochastic matrix and super-stochastic matrix, sufficient conditions for the cases with a static leader and a dynamic leader are derived, respectively. Moreover, computer simulations are performed to demonstrate the dynamics of network agents under random packet dropouts.

An event-based intelligent cooperative control policy is developed in this paper for a class of second-order multiagent systems (MASs). The followers are subject to external disturbances and output constraints, in which the constraint ranges are asymmetric and time-varying. By using one-to-one nonlinear mapping technique to obtain equivalent unconstrained systems and designing a distributed observer to estimate the leader's states, a simplified unconstrained tracking control task is established for each follower. After that, event-driven distributed optimal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H_\infty$</tex-math></inline-formula> controllers are designed to mitigate the effects of external disturbances, which are obtained from a set of decoupled Hamilton-Jacobi-Isaacs (HJI) equations, where the proposed event-triggered strategy is dynamic. Aiming at solving the HJI equations more efficiently, an event-based adaptive dynamic programming (ADP) learning algorithm applying only critic neural networks (NNs) is developed. And the critic NN weights are convergent by using stored data under finite excitation condition. Finally, the validity of the developed control scheme is demonstrated by multiple single-link robot arm systems.

When applied to the consensus tracking of repetitive leader-follower multiagent systems (MASs), most of existing distributed iterative learning control (DILC) methods assume that the dynamics of agents are exactly known or up to the affine form. In this article, we study a more general case where the dynamics of agents are unknown, nonlinear, nonaffine, and heterogeneous, and the communication topologies can be iteration-varying. More specifically, we first apply the controller-based dynamic linearization method in the iteration domain to obtain a parametric learning controller using only the local input-output data collected from neighboring agents in a directed graph, and then propose a data-driven distributed adaptive iterative learning control (DAILC) method through the parameter-adaptive learning methods. We show that for each time instant, the tracking error is ultimately bounded in the iteration domain for both of the cases with iteration-invariant and iteration-varying communication topologies. The simulation results show that the proposed DAILC method has faster convergence speed, higher tracking accuracy, and more robust learning and tracking in comparison with a typical DAILC method.

In this paper, a decentralized control scheme for group formation and trajectory tracking of multi-agent systems is designed. In this scheme, each agent dynamics is represented by a single-input single-output linear systemand the information that each controller requires is only the output of the others agents. With the proposed decentralized control law, multiagent system tracks asymptotically the trajectory and formation of the agents, even in the presence of smooth changes of formations. Moreover, the agent dynamics (structure and dimension) are not restricted to be the same, as usually assumed in the literature. A simulation example is provided in order to illustrate the application of the proposed control.

Adaptive neural network tracking control for manipulators with uncertain kinematics, dynamics and actuator model

The cooperative tracking control problem for the multiagent systems with unknown dynamic models, state constraints, control input constraints, optimal performance index, and convergence rate constraints is investigated in this article. A novel design framework is proposed to cope with these constraints and requirements. More specifically, the mean value theorem is utilized to transform the control input constraint to the unconstrained form. The performance function combined with a barrier Lyapunov function is leveraged to achieve the guaranteed transient tracking performance, finite-time convergence, and state constraints. To satisfy the optimal performance index, while considering the unknown dynamics of agents, we employ the actor-critic neural network architecture to get the near-optimal solution. Our control scheme is completely model-free. The rigorous Lyapunov stability analyses show that the full-state constraints and control input constraints are always satisfied, and the cooperative tracking errors can be made as small as possible in a finite time at a desired decay rate by appropriately setting the parameters in the performance function and barrier Lyapunov function. Finally, the simulation and hardware tests are conducted to clarify the effectiveness of the proposed control strategy.

A fundamental problem in formation control is under what conditions one can design protocols for a multi-agent system to achieve desired formations. By examining the property of a class of formations that are exactly achieved in a linear multi-agent system, an interesting fact is observed that there is a connection between the achievability of desired formations and the consistency of time-varying linear equations. From the asymptotic consistency of time-varying linear equations, we propose the asymptotic consistency concept for desired formations and study its properties using the Moore-Penrose generalized inverse. With this concept, a result on achievability of desired formations is presented for linear multi-agent systems: all asymptotically consistent formations are achievable as long as the agent dynamics and communication network satisfy the controllability and connectivity conditions, respectively. We design state feedback protocols for asymptotically consistent formations using the weighted controllability Gramian and the Moore-Penrose generalized inverse. If further, the agent dynamics are observable, the observer-based protocols can be designed by the separation principle. Finally, a simulation example is given to illustrate the obtained results.

This paper addresses the leader–follower consensus control problem for a nonlinear multi-agent robot system with control input constraint and external disturbances. Robot system is one of the most important practical systems in the industry. Due to the presence of disturbances in most practical systems, this paper considers the issue of finite-time leader–follower consensus control of the nonlinear multi-agent robot system along with actuator saturation and bounded disturbance. The modified terminal sliding mode control method is suggested for the system which is able to guarantee the stability of the overall system and fast finite-time leader–follower consensus control. For two different scenarios, the simulation of multi-agent robot system has been performed. The results show the effectiveness of the proposed control method.

This paper studies time-varying output formation control for linear heterogeneous multi-agent systems (MASs) under signed graphs, employing an adaptive dynamic event-based approach. The signed graph framework is used to model agent interactions, encompassing both cooperative and antagonistic behaviours. Firstly, a data-driven learning algorithm is given for the estimation of the unknown leader system matrix. Secondly, for a single leader MAS, a distributed adaptive dynamic event-based bipartite compensator is designed to estimate the leader's state. This triggering function utilises the internal dynamic variable to dynamically adjust the threshold, ensuring fewer triggering instances compared to traditional static thresholds. Thirdly, a controller integrating the compensator and observer is developed to solve the time-varying output formation control problem for heterogeneous MASs with signed graphs under a single leader. Subsequently, the findings are extended to multi-leader MASs, providing a solution for the formation control problem in such scenarios. Finally, several illustrative examples are provided to validate the theoretical results.

ABSTRACTThis paper addresses the distributed consensus protocol design problem for linear multi-agent systems with directed graphs and external unmatched disturbances. Novel distributed adaptive consensus protocols are proposed to achieve leader–follower consensus for any directed graph containing a directed spanning tree with the leader as the root node and leaderless consensus for strongly connected directed graphs. It is pointed out that the adaptive protocols involve undesirable parameter drift phenomenon when bounded external disturbances exist. By using the σ modification technique, distributed robust adaptive consensus protocols are designed to guarantee the ultimate boundedness of both the consensus error and the adaptive coupling weights in the presence of external disturbances. All the adaptive protocols in this paper are fully distributed, relying on only the agent dynamics and the relative states of neighbouring agents.

Most optimal control method aim at finding a control law to minimize the corresponding cost function, and the key problem is to solve the Hamilton–Jacobi–Bellman equation (HJBE). However, the solution of HJBE is hard and even unavailable to obtain for nonlinear system and, thus, the optimal control method can not be developed. To address this problem, an adaptive neural inverse optimal control method is proposed for nonlinear systems with external disturbances and actuator failures in this article. The cost function in this article is defined based on the controller and, therefore, once the controller is obtained, the corresponding cost function is minimized. Accordingly, the directly solution of HJBE is avoided. In more detail, the auxiliary system is constructed to develop the auxiliary controller, based on which the actual optimal fault-tolerant controller can be further obtained. The uncertainties in system dynamics are compensated by neural networks, and the unknown actuator failures are handled by adaptive control gain. It is proved that nonlinear system is input-to-state stabilizable (ISS) and the proposed controller can minimize the cost function that puts penalty on tracking error and system input. Two simulation examples are provided to demonstrate the effectiveness of the proposed control method.

In this paper, we develop control algorithms for addressing second-order consensus problems in intrinsically second-order nonlinear multi-agent dynamical systems with switching topologies. The proposed adaptive controller applies a leader-following strategy in which all agents are assumed to be informed of the leader's state. Another feature of this work which specifically distinguishes it from others is that the proposed controller uses undirected and directed graphs to accommodate for a full range of possible graph information topologies without limitations of bidirectional communication.

In this article, a data-driven distributed control method is proposed to solve the cooperative optimal output regulation problem of leader-follower multiagent systems. Different from traditional studies on cooperative output regulation, a distributed adaptive internal model is originally developed, which includes a distributed internal model and a distributed observer to estimate the leader's dynamics. Without relying on the dynamics of multiagent systems, we have proposed two reinforcement learning algorithms, policy iteration and value iteration, to learn the optimal controller through online input and state data, and estimated values of the leader's state. By combining these methods, we have established a basis for connecting data-distributed control methods with adaptive dynamic programming approaches in general since these are the theoretical foundation from which they are built.

This paper presents an online recorded data-based composite neural finite-time control scheme for underactuated marine surface vessels (MSVs) subject to uncertain dynamics and time-varying external disturbances. The underactuation problem of the MSVs was solved by introducing the line-of-sight (LOS) method. The uncertain dynamics of MSVs are approximated by the composite neural networks (NNs). A modified prediction error signal is designed by virtue of online recorded data. The weight updating law of NN is driven by both tracking error and prediction error, introducing additional correction information to the weights of NN, thus improving the learning ability of the NN. Furthermore, disturbance observers can be devised to estimate the compound disturbances consisting of the approximation errors of NNs and external disturbances. Moreover, the smooth function is inserted into the design of the control scheme, and the finite-time composite neural trajectory tracking control of MSVs is achieved. The stability of the MSVs trajectory tracking closed-loop control system is guaranteed rigorously by the Lyapunov approach, and the tracking error will converge to the set of residuals around zero within a finite time. The simulation tests on an MSV verify the effectiveness of the proposed control scheme.