Reinforcement-Learning-Based Tracking Control of Waste Water Treatment Process Under Realistic System Conditions and Control Performance Requirements

The tracking control of a wastewater treatment process (WWTP) is considered. The process is highly nonlinear, with strong coupling, difficult to model mathematically, and the operation is subject to unknown disturbances. We address this multivariable tracking control problem by applying the direct heuristic dynamic programming (dHDP)-based reinforcement learning control. The control goal is to track a desired reference of the dissolved oxygen (DO) concentration of the 5th aerobic zone ( $S_{O5}$ ) and nitrate concentration of the 2nd anoxic zone ( $S_{NO2}$ ) by manipulating the oxygen transfer coefficient of the 5th aerobic zone ( $K_{L}a_{5}$ ) and internal recycle flow rate ( $Q_{a}$ ). The dHDP aims at achieving a minimal accumulated WWTP tracking error while dealing with strong coupling between the $S_{O5}$ and $S_{NO2}$ and eliminating unknown disturbances in the process. The proposed dHDP approach devises an optimal control strategy entirely driven by WWTP process data as an online learning control method. We have conducted extensive and systematic simulations based on the well-known BSM1 platform of the WWTP controlled by dHDP to compare and contrast performances with other methods.