A series of adverse events has led to subnational debt becoming associated with crises. There is a rich theoretical, empirical, and policy literature on why and how subnational debt should be limited and whether fiscal rules and limits are effective in containing overborrowing and avoiding further crises. Yet, in practice, we observe the opposite. Central governments across the world subsidize subnational public debt in many ways. This paper documents this paradox and models the tension between the need to limit subnational borrowing and the need to incentivize local public goods provision. An analytically tractable model explores the tradeoffs between the hazards of subnational debt and the externalities of regional public goods. I show that a bailout policy can counterbalance the underprovision caused by no coordination. Moreover, welfare under debt mutualization can be higher than in a constrained social optimum. Whether this happens depends on the heterogeneities of regional income and preferences for the public good. I also bring the model to the data and estimate regional preferences for the public good as well as the interregional externality strength, for several OECD countries. This model can help to frame further academic and policy discussions about the optimality of subsidies and limits to subnational borrowing. It also speaks to the fiscal design of monetary unions.