Towards Circular and Asymmetric Cooperation in a Multi-player Graph-based Iterated Prisoner's Dilemma

In collaborations involving multiple actors, it is well known that tensions between individual interest and global welfare can emerge: actors are personally incentivized to have selfish behavior whereas mutual cooperation may provide a better outcome for all. Known as social dilemmas, these cooperation paradigms have aroused renewed interest in solving social issues, particularly in environmental and energy issues. Hybrids methods with Reinforcement Learning (RL) policies and Tit-for-Tat (TFT) strategies have proven successful to identify fruitful collaboration in complex social dilemmas. However, there are also many situations, where cooperation cannot always be given back directly, and has instead to be carried out through one or more intermediary actor(s). This specificity ruins win-win approaches like TFT. To address this specificity, we introduce a Graph-based Iterated Prisoner's Dilemma: a N-player game in which the possible cooperation between players is modeled by a weighted directed graph. In addition to this new paradigm, we propose a graph-based TFT algorithm that we evaluate on multiple scenarios and compare to other algorithms. Our experiments show that leveraging a graph-based structure in the original TFT algorithm allows it to spread favor better collaboration synergies in most situations.