In this paper we develop a multivariate credit risk model that accounts for joint defaults of banks and allows us to determine how much of banks' credit risk is systemic. We find that the US and the UK differ not only in the evolution of systemic risk, but also in their banks' systemic exposures. In both countries, however, systemic credit risk varies substantially over time, represents about half of total bank credit risk on average and leads to high risk premia. Furthermore, the results suggest that sovereign and bank systemic risk are closely interlinked in the UK.