Multi-Network Constrained Operational Optimization in Community Integrated Energy Systems: A Safe Reinforcement Learning Approach
CoRR(2024)
摘要
The integrated community energy system (ICES) has emerged as a promising
solution for enhancing the efficiency of the distribution system by effectively
coordinating multiple energy sources. However, the operational optimization of
ICES is hindered by the physical constraints of heterogeneous networks
including electricity, natural gas, and heat. These challenges are difficult to
address due to the non-linearity of network constraints and the high complexity
of multi-network coordination. This paper, therefore, proposes a novel Safe
Reinforcement Learning (SRL) algorithm to optimize the multi-network
constrained operation problem of ICES. Firstly, a comprehensive ICES model is
established considering integrated demand response (IDR), multiple energy
devices, and network constraints. The multi-network operational optimization
problem of ICES is then presented and reformulated as a constrained Markov
Decision Process (C-MDP) accounting for violating physical network constraints.
The proposed novel SRL algorithm, named Primal-Dual Twin Delayed Deep
Deterministic Policy Gradient (PD-TD3), solves the C-MDP by employing a
Lagrangian multiplier to penalize the multi-network constraint violation,
ensuring that violations are within a tolerated range and avoid
over-conservative strategy with a low reward at the same time. The proposed
algorithm accurately estimates the cumulative reward and cost of the training
process, thus achieving a fair balance between improving profits and reducing
constraint violations in a privacy-protected environment with only partial
information. A case study comparing the proposed algorithm with benchmark RL
algorithms demonstrates the computational performance in increasing total
profits and alleviating the network constraint violations.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要