Communication-efficient hierarchical distributed optimization for multi-agent policy evaluation

•Previous decentralized algorithms for multi-agent policy evaluation problems only consider each agent homogeneously.•A hierarchical distributed algorithm that differentiates the roles of each of the agents over the network during the multi-agent policy evaluation process is proposed in this work.•The convergence rate of the proposed algorithm is analyzed and proved in the paper.•The experiment results of the proposed algorithm is demonstrated in comparison with other state-of-the-art algorithms.•The hierarchical structure of the proposed algorithm allows it to be applied to policy evaluation problems on directed graphs with non-symmetric and non-doubly-stochastic mixing matrices.