Reinforcement learning-based real-time control of a reversible pump turbine for reduced response time in pumped hydropower

Abstract

This paper investigates reinforcement learning (RL) for real-time control of a reversible pump turbine in pumped hydropower systems. Although RL shows strong potential for adaptive control, its adoption in safety–critical energy systems remains limited by challenges in training, deployment, and trust. To address these, a digital twin-based framework enables safe offline training, structured validation, and controlled simulation-to-real transfer of a Proximal Policy Optimization agent. The learned policy is distilled into an interpretable rule-based surrogate for experimental deployment. Validation on a 400 kW laboratory-scale pump turbine shows that the RL-derived strategy reduces start-up time by over 30% compared to a conventional control approach, achieving faster synchronization and earlier target discharge. The results underline RL’s capacity to discover efficient, non-intuitive control strategies while revealing key limits in sim-to-real transfer due to model errors and delays. Overall, the work establishes a systematic methodology for integrating RL into safety–critical energy control—bridging adaptive learning with physically interpretable and verifiable operation.