Solving an energy resource management problem with a novel multi-objective evolutionary reinforcement learning method

Microgrids have become popular candidates for integrating diverse energy sources into the power grid as means of reducing fossil fuel usage. Energy Resource Management (ERM) is a type of Unit Commitment problem, where a player operates a microgrid with diverse renewable generators integrated with an external supplier. Calculating the economic dispatch of each committed unit on a planning horizon is an NP-hard problem, and therefore, finding an exact solution is difficult. This paper presents a multi-objective solution to the ERM problem from the perspective of battery operation and external supplier dispatch. First, a novel multi-objective decision problem modeling is proposed that considers three objectives: cost, greenhouse gas emissions, and battery degradation. This framework involves a learning agent that controls the depth of discharge of a Lithium-Ion battery. To address the proposed problem, a new multi-objective algorithm called Multi-Objective Evolutionary Policy Search (MEPS) is introduced. The proposed algorithm uses NeuroEvolution of Augmenting Topologies structure to evolve artificial neural networks for estimating action-preference values considering multi-objective rewards. The MEPS performance is evaluated on both standard and newly-proposed benchmark problems, using the hypervolume as the evaluation metric. When compared to standard deep reinforcement learning, results showed that MEPS provides cost-effective, environmentally friendly, and efficient energy storage management solutions. Furthermore, MEPS effectively solves the proposed ERM problem by finding neural networks with a small number of nodes and connections, which are suitable for use in embedded control systems. Overall, MEPS proved to be a promising multi-objective approach in the transition to clean energy resources.