Optimizing VLSI Implementation with Reinforcement Learning - ICCAD Special Session Paper

Reinforcement learning (RL) has gained attention recently as an optimization algorithm for chip design. This method treats many chip design problems as Markov decision problems (MDPs), where design optimization objectives are converted into rewards given by the environment and design variables are converted into actions provided to the environment. Some recent examples include applications of RL to macro placement and standard cell layout routing. We believe RL can be applied to nearly all aspects of VLSI implementation flows, since many VLSI implementation problems are often NP-complete and state-of-art algorithms cannot be guaranteed to be optimal. With enough training data, it is possible to achieve better results with RL. In this paper we review recent advances in applying RL to VLSI implementation problems such as cell layout, synthesis, placement, routing and parameter tuning. We discuss the challenges of applying RL to VLSI implementation flows and propose future research directions for overcoming these challenges.