Limits of stress-test based bank regulation

How should supervisory risk assessments, such as stress tests, inform bank regulation when such assessments provide imprecise signals to regulators about banks’ risk profile? What trade-offs do regulators face when redesigning assessments to improve accuracy? Does the disclosure of assessment results improve the effectiveness of capital requirements? We develop a theoretical framework showing that when supervisory risk assessments are noisy, capital requirements based on these assessments can induce banks to take ‘more’ risk. When the precision of assessment is below a certain threshold, a ‘one size fits all’ capital regulation works best. Above this threshold, as precision increases, bank-specific capital surcharges that become increasingly sensitive to assessment outcomes make sense. Tweaks of the supervisory assessment that result in a marginal increase in precision are also subject to trade-offs that may not necessarily justify higher surcharges.

The great financial crisis revealed that banks, especially large and complex banks, are opaque (Gorton, 2009). As a result, regulators in many countries are now increasingly relying on supervisory risk assessment tools to learn about bank-specific risk exposures and capital adequacy. Such tools complement financial reporting and disclosures in informing regulators about banks’ riskiness (Morgan et al., 2014) and allow them to better align the baseline capital requirements with individual banks’ risk profiles. In principle, capital regulation based on supervisory risk assessment can improve welfare by making requirements bank specific. However, as several studies have shown, these tools may end up providing noisy assessments of banks’ risk exposures (Plosser and Santos, 2018; Acharya et al., 2014). Given this imprecision, the natural question that arises is to what extent should capital regulation be shaped by supervisory risk assessments such as stress tests? We develop a theoretical framework to shed light on this question (Agarwal and Goel, 2021).1

Our main insight, contrary to conventional wisdom, is that higher capital requirements can create adverse incentives and lead to more risky banks when supervisory assessments are sufficiently noisy. This is because a bank may not have the ex-ante incentive to reduce the riskiness of its assets if there are high chances that even with a less risky portfolio it will be required to maintain a costly capital ratio. In this case, our model shows that in response to an increase in the capital requirement, a bank would choose to exert less effort towards improving the risk-adjusted return of its assets. As such, when the precision of assessment is below a threshold, the optimal strategy for the policymaker is to adopt a ‘one size fits all’ requirement – one that does not depend on the outcome of the assessment. When precision is above the threshold, higher capital requirements induce banks to exert more effort. In this case, capital requirements that are tied to banks’ assessed risk profiles are optimal (see Figure 1). Moreover, beyond this threshold, higher precision allows for higher capital requirements for a given level of assessed risk.

Figure 1: Risk assessment precision and capital requirements. Each point on the square characterises the precision of supervisory assessment. A lower value on the x-axis and/or a higher value on the y-axis indicate a more precise assessment. The optimal requirement is uniform across banks if precision is below the diagonal line i.e. in the shaded area. For precision above the diagonal line, the optimal requirement is higher for riskier banks. The main reason for these distinct phases is that as capital requirements tighten, a bank may increase or decrease the effort it exerts towards improving its risk-return profile depending on the precision of the supervisory assessment.

Our result suggests that improving the precision of supervisory risk assessments is desirable. This, however, is not trivial in practice. For one, there are costs involved, both for the supervisor and the bank. But more importantly, while tweaking the assessment may reduce, for instance, the rate of false negative signals (i.e. bad banks deemed to be good), the rate of false positive signals (i.e. good banks deemed to be bad) may rise. Indeed, simply making the assessment more stringent to better identify the ‘bad’ banks may lead some ‘good’ banks being incorrectly classified as ‘bad’ and worsen the overall precision of the assessment. Using an extended version of our baseline model, we discuss the implications of these trade-offs for the optimal policy choice of the regulator.

If the regulator could improve the precision of risk assessments (i.e. reduce both false positive and negative rates) without incurring a significant cost, we show that the optimal strategy is to do so and make capital regulation increasingly contingent on the outcome of risk assessment. However, if reducing both error rates simultaneously is prohibitively costly or technically infeasible, the optimal response is not unambiguously clear. We show that incurring a cost to improve one aspect of precision may not necessarily imply that capital requirements can be made more sensitive to assessment outcomes.

Our model also allows us to study how the noise in assessments affects the regulator’s decision to disclose assessment outcomes. In the absence of information frictions, disclosure of outcomes can improve market discipline by inducing banks to increase effort and improve their risk-return profile. However, if a bank knows that the assessment can classify it incorrectly too often, disclosure of outcomes to investors could amplify the adverse incentives that we described above and further reduce the effort the bank exerts to improve its risk-return profile.

A pertinent example of supervisory risk assessment is stress tests. They evaluate whether banks have sufficient capital to absorb losses resulting from adverse economic conditions. In the US, the Federal Reserve imposes a capital surcharge on banks based on their performance in the test. Banks that perform poorly – i.e. experience a larger projected decline in their Common Equity Tier 1 (CET1) capital ratios in the hypothetical stress scenario – face a higher capital requirement.

Figure 2: Banks’ performances in stress tests and in reality. A comparison of the decline in CET1 ratio in the 2020 stress test in the US and the actual decline observed between Q4:2019 and Q2:2020. The unit of both axes is in percentage points.

As an illustration of the noise inherent in risk-assessment exercises, we compare banks’ performances in the stress-tests conducted in the U.S. right before the Covid-19 crisis, and their actual performance during the pandemic. We document that the cross-sectional variance in test-driven changes in CET1 ratios is much higher than the changes observed during the pandemic, and that the two do not correlate (see Figure 2). In fact, while the ratios declined for almost all banks in the test, they rose for many in reality. In the case of Deutsche Bank USA, for instance, the CET1 ratio declined by 8 percentage points (pp) in the test, while during H1 2020, the same ratio rose by 5 pp. While there are limitations to directly comparing test and actual outcomes, as discussed in Agarwal and Goel (2021), broad concordance in the two would be desirable if stress tests are used in order to capture banks’ resilience to a generic crisis episode. The higher variance in test results and its lack of correlation with observed outcomes are therefore suggestive of the noise inherent in risk-assessments