User Association and Power Allocation for User-Centric Smart-Duplex Networks via Deep Reinforcement Learning.

This paper considers smart-duplex (SD) powered user-centric ultra dense networks (UC-UDN), which shifts the conventional access point-centric paradigm to the user-centric one by de-cellular concept, to provide good quality-of-service for a large number of users via flexibly designing the user association, power allocation, and duplex mode. The maximization average ratio of satisfied users for the considered SD UC-UDN in the long-term time scale is firstly formulated as a Markov decision process (MDP) problem with large discrete action space. To reduce the action space, the user association and power allocation processes are modeled as a two-layer tree structure, and then selecting an action is equivalent to finding the path from root to one of the leaf nodes of the tree. A multi-agent tree-structured policy gradient (MATSPG) based deep reinforcement learning (DRL) algorithm is proposed to solve this problem by directly mapping the action space for user association and power allocation to the two layers of the tree, respectively, whose training is shown to be equivalent to the training of neural networks on two-layer paths. The time and space complexity for searching one action in the proposed MATSPG is also proved to be lower than other conventional DRL algorithms. Simulations show that the proposed MATSPG algorithm significantly improves the average ratio of the satisfied users than the conventional DRL methods in typical scenarios.