P-MCGS: Parallel Monte Carlo Acyclic Graph Search.

Recently, there have been great interests in Monte Carlo Tree Search (MCTS) in AI research. Although the sequential version of MCTS has been studied widely, its parallel counterpart still lacks systematic study. This leads us to the following question: emph{How to design efficient parallel Monte Carlo search algorithms that achieves linear speedup and has rigorous theoretical guarantee?} In this paper, we consider the Monte Carlo search over a more general acyclic one-root graph, named as Monte Carlo Graph Search (MCGS), which includes MCTS as a special case. We develop a parallel algorithm (P-MCGS) to assign multiple workers to investigate appropriate leaf nodes simultaneously. In addition, we also extend the idea to parallelize the widely used UCT algorithm and develop P-UCT. Our analysis shows that P-MCGS and P-UCT (The linear speedup for P-UCT is only shown empirically). algorithms achieve linear speedup in time and that the total sample complexity is comparable to its sequential counterpart. Furthermore, we also show that applying P-MCGS directly to the original acyclic graph outperforms applying P-MCGS or P-UCT to its expanded tree.