Statistical Challenges of Stress Test Financial Stability Assessments

By Paul H. Kupiec

Abstract

Banking system stress tests are a key component of IMF/World Bank financial stability assessments. The focus on stress tests is predicated on prior belief that there is a strong contemporaneous correlation between bank performance and stress scenario macro-financial variables. Using US data, I use latent factor analysis to show that the presumed correlations are weak at best. This shortcoming is addressed by introducing a novel macro-financial variable to measure system-wide bank performance and using shrinkage estimators to reduce the risk of overfitting. Bespoke stress test models are constructed for the largest 200 banks in the US as of 2008 Q2 using alternative shrinkage estimators. Bank performance is forecast over the first three years of the financial crisis using actual economic conditions as the stress test scenario. Parsimonious models including the new banking macro-financial variable are shown to be most accurate, but none of the stress test models analyzed are accurate by any absolute standard. The deficit in forecast accuracy identified herein is likely to be magnified for stress tests forecasts derived from less comprehensive data.

Read the full PDF here.