On-policy learning-based deep reinforcement learning assessment for building control efficiency and stability

Deep reinforcement learning (DRL) has been considered as a potential solution to efficiently control and manage building systems. However, broad assessment of DRL-based building control is still required to characterize their pros and cons in comparison with conventional rule-based feedback controls. In this paper, we assessed DRL-based controls with on-policy learning-based algorithms and continuous control actions for cooling control of large office buildings in the summer season to minimize whole-building energy use and occupant discomfort. We compared DRL-based control methods with two baseline control methods: (1) a pre-determined schedule with supply temperature and static pressure setpoints, and (2) advanced reset method that adjusts setpoints based on heuristic rules, i.e., ASHRAE Guideline 36. We also tested the DRL algorithms to evaluate their performances in multiple climate locations. We found that DRL-based control methods outperformed the baseline control methods in terms of energy savings while maintaining a thermal comfort. DRL reduced energy use between similar to 4%-22% on average compared to the baseline methods, depending on climate location. We also evaluated DRL-based control in terms of control stability and showed that DRL-based methods should consider hardware lifetimes in practical operations.