Dual ML-ADHDP method for heterogeneous discrete-time nonlinear multi-agent systems with unknown dynamics and time delay

This paper develops a new dual ML-ADHDP method to solve the optimal consensus problem (OCP) of a class of heterogeneous discrete-time nonlinear multi-agent systems (MASs) with unknown dynamics and time delay. A hierarchical and distributed control strategy is used to transform the original problem into nonlinear model reference adaptive control (MRAC) problems and an OCP of virtual linear MASs. For the nonlinear MRAC problems, a new multi-layer action-dependent heuristic dynamic programming (ML-ADHDP) method is developed to overcome the unknown dynamics and neural network estimation errors, which has higher control accuracy. In order to solve the OCP of virtual linear MASs and improve the convergence speed, a new multi-layer performance index is proposed. Then the ML-ADHDP method is used to solve the coupled Hamiltonian–Jacobi–Bellman equation and obtain the optimal virtual control. Theoretical analysis proves that the original MASs can achieve Nash equilibrium, and simulation results show that the developed dual ML-ADHDP method ensures better convergence speed and higher control accuracy of original MASs.