Banks’ credit forecasts excel at rank ordering but miss the evolution of losses

Focusing on credit risk, we compare banks’ expected loss (EL) rates, collected confidentially by the Basel Committee on Banking Supervision from 2009 to 2022, and the corresponding actual loss (AL) rates. Consistent with the use of through-the-cycle risk estimates for regulatory purposes, EL rates rarely evolve in line with AL rates, which helps explain a large precautionary element in Basel III capital requirements. By contrast, the rank-order of EL rates across banks matches closely that of AL rates, in line with recent and forthcoming regulatory efforts to improve risk-measurement practices. We also find that EL rates are relatively conservative for banks with higher valuations and are more likely to be optimistic on the heels of higher bank profitability and financial overheating, as captured by the credit-to-GDP gap.

1. Banks’ resilience hinges on credit risk measurement and management

With borrowing and lending at the core of bank business models, credit assessments are essential for banks’ risk management. Credit losses featured prominently in the great financial crisis (GFC) (eg Claessens et al (2010)). In recent years, credit risk accounted for 60% to 80% of the capital requirements for internationally active banks (BCBS (2023), p 50–51). On the back of significant debt accumulation during the low-for-long era, debt service costs are rising as policy rate hikes feed into corporate bond yields and lending rates. Under plausible scenarios, the rise of these costs may drive credit losses up to GFC levels (BIS (2023)). Ultimately, accurate credit loss forecasts and/or regulatory conservatism – ie larger precautionary elements in the mapping from the forecasts to capital requirements – need to ensure enough resources for absorbing credit losses.

2. Confidential supervisory data sheds light on banks’ credit forecasts

We assess the accuracy of banks’ credit loss forecasts – ie expected loss (EL) rates. We juxtapose these EL rates with banks’ actual loss (AL) rates and identify drivers of the discrepancies. At the heart of the exercise are exclusive confidential supervisory data, collected by the Basel Committee on Banking Supervision (BCBS). These data contain one-year EL rates, as reported by 65 internationally active banks to supervisors from end‑2008 to end-2022. Combining EL rates on non-defaulted exposures with vendor accounting data on AL rates, we answer two questions:

1. How well do EL rates capture the evolution of AL rates?
2. Does the rank-order of EL rates across banks align with that of AL rates?

3. Findings: two faces of banks’ credit loss forecasts

The performance of EL rates differs starkly between the time and cross-section dimensions.

EL rates generally fail to capture the time profile of AL rates. Concretely, the correlation of year‑to‑year changes in the two series is statistically significant for only 15% of the banks, reflecting divergencies as regards spikes and trends (Graph 1). This implies that a conservative mapping from EL rates to capital requirements is needed to ensure enough loss‑absorbing resources at each point in time. We estimate that – over our sample period – such a mapping would have resulted in capital requirements being (at least) twice as large on average as yearly AL for three-quarters of the banks (Graph 1.A). Actual capital requirements exhibit stronger conservatism, not least because real-life uncertainties are higher than those underpinning our stylised exercise.

By contrast, banks fare well when it comes to signalling the relative riskiness of their credit portfolios. The only exception is in 2021, in line with the extraordinary nature of the Covid-19 pandemic and related support measures. For all other years in the sample, the rank-ordering of EL rates across banks closely matches that of the corresponding AL rates (Graph 2).

We confirm and expand on these descriptive findings on the basis of regression analysis.

In a first step, we consider EL rates as the sole explanatory factor for AL rates. We find that EL rates explain less than 5% of the volatility in AL rates over time. By contrast, EL rates explain almost 70% of the dispersion in AL rates across banks.

In a second step, we study drivers of the portion of AL rates that is not explained by EL rates, ie drivers of “step-one errors”. In a sign that EL rates do not account fully for persistence in credit losses, we find that lagged AL rates help explain step-one errors. We also find a robust, statistically and economically significant relationship between step-one errors and country-level credit-to-GDP gaps. This suggests that banks abstract from macro indicators of overheating when forecasting credit losses. Likewise, we find that higher bank-level return-on-assets and price-to-book ratios tend to go hand in hand with higher and lower step-one errors, respectively. These findings are consistent with higher profitability introducing excessive optimism in risk measurement and with higher valuations enabling banks to afford greater conservatism in their EL rates.

Throughout, we are conscious of potential inconsistencies between the supervisory data behind EL rates and the vendor accounting data behind AL rates. We thus check – and confirm – the robustness of our results to the exclusion of observations for which measurable inconsistencies between the two data sets – ie a wedge between the credit risk exposures underpinning AL and EL rates – exceed a particular threshold. In addition, we control for potential differences between the defaults that affect AL rates and those underpinning the losses that EL rates are supposed to forecast. Ultimately, our exercise is an evaluation of regulatory credit risk estimates on the basis of the best available cross-jurisdiction data on large internationally active banks.

4. Policy takeaways

Our findings are closely related to the effects of post-GFC regulatory initiatives. For one, authorities sought to mitigate the tendency of risk assessments to be overly optimistic in tranquil times and spike in stress – ie to mitigate pro-cyclicality (BCBS (2021)). These efforts underpin banks’ use of “through the cycle” estimates of probabilities of default and “downturn” loss‑given‑default. Anchored in long-term historical default rates, such estimates tend to be stable and it is thus hardly surprising that they lead to EL rates that miss the evolution of AL rates. In addition, recent policy efforts have sought to ensure that differences in regulatory capital reflect genuine differences in underlying risks, rather than differences in risk-measurement practices across jurisdictions and entities (BCBS (2017)). The good performance of EL rates in explaining the dispersion of AL rates across banks is in line with such efforts.

The explanations we provide for shortcomings in EL rates as forecasts of AL rates are of relevance for prudential authorities. The possibility that systematically optimistic EL rates increase the vulnerability of high-RoA banks seems worthy of investigation. And the relative conservatism of banks with higher price-to-book ratios indicates a link between resilience and valuations. In addition, authorities may need to assess whether their use of the counter-cyclical capital buffer – whose activation is at national discretion (BCBS (2011)) – has been compensating sufficiently for the failure of banks’ EL rates to capture the evolution of AL rates.

We also complement previous assessments of banks’ regulatory credit risk estimates: most recently, BCBS (2022) concluded that the Basel III reforms increased the resilience of large internationally active banks and reduced systemic stress. Our analysis suggests that this resilience rests on accurate reporting of relative riskiness and conservative mapping from risk estimates to capital requirements.