We build a sovereign default model to understand the implications of rising safe interest rates for countries with high public debt. When debt levels are below a critical threshold, countries respond to higher interest rates by reducing their debt due to a dominant substitution effect. For high debt levels, in contrast, the same rate rise triggers even more debt - and possibly a slow moving debt spiral - due to a dominant income effect. The seeds for a debt spiral are laid by a long phase of low interest rates: they imply that debt levels rise over time, making a future interest rate normalization more difficult. A successful interest rate normalization involves a credible path of rising interest rates, the speed of which must be intermediary: a too fast normalization leads to debt spirals, but a too slow one undermines incentives by the government to repay.