Improving Control of Energy Systems With Reinforcement Learning: Application to a Reversible Pump Turbine

Abstract

Pumped hydro storage power systems are crucial to account for grid instabilities by providing flexibility services. To further increase flexibility, the acceleration of switching between operating modes is necessary. This can be achieved through precise and automated process control with reinforcement learning (RL). Besides the benefits of RL, safety concerns inhibit industrial-scale applications for process control with RL. We present measures to increase the reliability and stability of RL algorithms to enable applications for the control of energy systems. We demonstrate the viability of our approach by applying it to the control of the pump start-up process of a reversible pump turbine. To train the RL algorithm, we use a simulation model that accurately represents the test rig of a pump turbine located at the laboratory of TU Wien. Our results show that RL is suitable for finding optimal control strategies that can compete with traditional approaches. However, finding the optimal policy still requires a lot of computational effort. Future research will focus on optimizing the RL framework and then transferring the results to the real machine unit at the test facility.