Energy Efficient Thermal Management of 5G Base Station Site Based on Reinforcement Learning.

The rapid development of Fifth Generation (5G) mobile communication system has resulted in a significant increase in energy consumption. Even with all the efforts made in terms of network architecture, system hardware design and device operation, its energy consumption and costs remain high. In order to control the operating environment within a reliable temperature range, the heating ventilation and air conditioning (HVAC) of 5G base station (BS) site consume a significant amount of energy for thermal management, and its operation still has great energy saving potential. This paper presents a three-stage approach of energy-efficient thermal management of 5G BS sites based on Q-learning and imitation learning. An imitation learning controller is proposed and a feature-controller library is constructed. Reliable initial control policies can be generated for new BS sites based on ensemble learning and rule-based constraints with this library. Furthermore, the optimal control policy for HVAC is learned using Q-Learning. This approach can be directly configured within the building baseband unit (BBU) and eliminates the requirement for additional sensors, facilitating practical engineering deployment. The application results show that the average energy cost of the HVAC system is reduced by 18.96% with the proposed approach, and the total cost of BS sites more than 10%.