Analysis of strategic interactions among distributed virtual alliances in electricity and carbon emission auction markets using risk-averse multi-agent reinforcement learning

The incorporation of carbon emission auction market (CEAM) and ancillary service market (ASM) is an emerging trading paradigm in active distribution network (ADN). Such regime not only promotes the elimination of carbon emission, but also facilitates the secure operation of power network, especially considering the participation of distributed virtual alliances (DVAs) consisting of renewable distributed generators (RDGs) with uncertain output. In this research, a bi-level bidding and market clearing dynamic programming model is developed for in-depth analysis of market participants' bidding strategies and market equilibrium. This model allows DVAs to modify their bidding strategies in the energy market (EM), ASM and CEAM based on the market clearing results and uncertainty of RDG output. Also, a new Meta-Learning based Win-or-Learn-Fast (MLWoLF-PHC) algorithm, which not only enables the fully distributed bidding strategy modification, but also performs well considering uncertainty as a risk-averse method, is proposed to solve this model. Its computational performance, the market equilibrium analysis, and the impact of CEAM on the converged market clearing price of EM and ASM would be thoroughly investigated and examined in the case studies.