Reinforcement Learning in Hydropower: Pump Start-Up Control in a Simulated Environment

Abstract

Reinforcement learning (RL) offers a promising framework for controlling transient processes in hydraulic machines. In this study, we apply RL to the start-up control of a reversible pump-turbine in a simulation of a pumped storage power system. A physics-based simulation environment capturing the system’s key hydraulic and mechanical dynamics is developed to train a Proximal Policy Optimization agent. Compared to a conventional control approach, the RL-based policy yields a higher cumulative reward in simulation by significantly reducing start-up time and cumulative operational cost. The approach demonstrates the potential of RL in a simulated pump start-up case, highlighting its ability to autonomously discover efficient control strategies for highly dynamic processes, such as pump start-up, without relying on predefined procedural logic. Future work includes experimental validation on a laboratory-scale pump-turbine to further investigate the policy’s performance under real-world conditions.