Deep Q-network Based Dynamic Power Allocation for Cell-Free Massive MIMO

Numerical optimization has been investigated for decades to address complex problems. Many effective methods, e.g., the weighted minimum mean square error (WMMSE) algorithm, have been developed for a large variety of applications in wireless communication systems. However, these methods often require high computational cost creating a serious gap between theoretical analysis and real-time processing. Recently data-driven methods have attracted a lot of attention due to their near-optimal performance with affordable computational cost. Deep Q-network (DQN) is one of the most promising optimization techniques for future wireless communication systems. In this paper, we investigate the DQN method to allocate the downlink transmission power in cell-free (CF) massive multiple-input multiple-output (MIMO) systems. We consider the sum spectral efficiency (SE) optimization problem for systems with mobile user equipment (UEs). The DQN is trained by the rewards of trial-and-error interactions with the environment over time. It takes as input the long-term fading information and it outputs the downlink transmission power values. The numerical results, obtained from a particular 3GPP scenario, show that the DQN outperforms WMMSE in terms of sum-SE and has a less execution time.