Online and hard constrained adaptive dynamic programming algorithm for energy storage control in smart buildings

The high computational complexity caused by static optimization is the key factor to hinder the development of energy management systems. Adaptive dynamic programming (ADP) is an effective dynamic optimization method to break through the bottleneck. However, the hard constraints of energy system have not been fully considered due to the non-convexity and nonlinearity of the value function, which makes the theoretical analysis complicated and brings safe security problems in battery systems. In this article, a systematic online ADP control framework is proposed for smart buildings control to ensure hard constraints to be satisfied. The second-order local expansion at the current state is used to replace the nonlinear value function to simplify the theoretical analysis with the error of the reminder term. Based on the local property of value function, a method for the determination of adaptive parameters is first designed. It is proven that the solution of adaptive parameters not only prevents over-charged and over-discharged of the battery but also limits the charging and discharging power of the battery to be less than the rated power. In addition, long-short term memory (LSTM) neural networks, a type dynamic network with memory characteristics, are used for the implementation of the present algorithm instead of the static networks to help realize the algorithm online. Due to the hidden state of LSTM, the performance of the online algorithm is improved after running the energy system. Numerical results verify the effectiveness of the proposed online control algorithm.