HPAQT: Adaptive and Interpretable High-level SLO-aware Autoscaling with Reinforcement Learning

Abstract

Modern distributed applications rely on virtualized infrastructures to elastically meet their performance requirements. In this setting, autoscaling enables elastic adaptations at runtime. While allowing for the overcoming of the burden of adjusting the provisioned resources, autoscaling shifts the problem to the definition of an accurate and appropriate threshold, for example, a certain CPU usage, which is a difficult challenge to achieve. Furthermore, defining a priori a fixed value clashes with the dynamicity of modern applications and infrastructure, leading to inflexibility that can affect the quality of service over time. Finally, most autoscaling techniques rely on low, resource-level metrics, which, in complex scenarios, are difficult to gauge. In our paper, we propose HPAQT, a lightweight, stable, and reproducible RL mechanism that self-calibrates the autoscaling threshold to enforce composite, high-level objectives rather than fixed low-level metrics. HPAQT yields an easily interpretable, deployable, and effective policy. In experiments, HPAQT achieves 10× fewer violations than the reference Q-Threshold and beats the standard Kubernetes HPA, with less than 0.5% total violations in over 12 hours, thus demonstrating practical gains.