Macroprudential stance assessment: problems of measurement and literature review

The ultimate objective of macroprudential policy (MP) is the stability of the financial system, by increasing its resilience, taming the build-up of vulnerabilities in the system and smoothing out the financial cycle, which should ultimately contribute to economic growth (ESRB, 2011). The question remains on how to measure the effects of macroprudential policy and what is its stance? In 2019, ESRB defined the macroprudential stance as “the balance between systemic risk and resilience relative to financial stability objectives, given implemented macroprudential policies; the stance metric represents residual systemic risk in the financial system, relative to a neutral level of risk considered sustainable in the long run”, and establishes a relationship between macroprudential policy actions and the financial stability objectives (ESRB, 2019, 2021). Thus, the board started to develop a framework in which the macroprudential policy measures are related to future GDP growth, based on the previous two definitions.

Compared to monetary and fiscal, macroprudential policy is still relatively new, and much more work must be done to identify and evaluate its transmission channels (ESRB, 2021). As the link between financial conditions, financial stability, and the real economy has gained more attention in the last decade, tools and frameworks have been developed to analyse it. The framework of interest in this paper, Growth-at-Risk (GaR henceforward), is one way the analysis could be done, as it links current macro-financial conditions in the economy with future GDP growth. Literature recognizes the importance of future growth forecasts due to the definition of financial stability. That is why empirical research on this topic has exploded in recent years.

The purpose of this paper was to identify results of related literature that utilizes GaR approach in measuring MP stance, and extract important messages. Besides this, the full paper on which this note is based provided an empirical analysis for the case of Croatia, to see what the challenges that occur in empirical studies are. The results are not near a complete picture on a stable measure of distance to tail, but provide initial basis for future work to improve on this to finally arrive at stable and usable measure that can help in macroprudential policy decision making. However, in this note, we focus on the insights from the related empirical literature.

First group consists of seminal papers related GaR with financial conditions and started the popularity of this stream of thought, which set the scene, as they related financial conditions to future GDP growth. Results of these seminal papers today are considered as stylized facts about macro-financial linkages. These papers were motivated to forecast the entire distribution of future GDP growth by using financial conditions as a new predictor in this area of research. Giglio et al. (2015) examine the predictive power of many systemic risk measures, and found that the observed indicators are more related to future downside risks, compared to the central tendency. Adrian et al. (2016) show that lower quantiles of future US GDP growth have greater volatility when compared to the upper ones, which are fairly stable over time. Extending the previous findings, Adrian et al. (2018) found a term structure of GaR: loose financial conditions are related to higher GaR in short-term, and lower in medium. Financial conditions indicators alone cannot be used, as it would be too late to act if decisions were based only on financial conditions. Nevertheless, papers in this section are the first ones that found the importance of the non-linear relationship between financial conditions and future GDP growth, alongside the term structure of this relationship.

Subsequently, the literature included the financial vulnerability indicators in the analysis, as MP tracks and obtains more helpful information about their medium-term predictive power of possible future risk materialization. Also, it takes some time before the effectiveness of MP instruments can be seen, so looking at the responses of future growth in the medium term would make more sense. One prominent thing in empirical literature is using (panel) quantile regression to estimate GaR, as this approach enables the researcher to obtain median and GaR growth estimates, which we seen are important in theoretical definition of MP stance. However, there are other approaches that have emerged in the last few years, as they also provide some insights into the subject matter. Authors want to see what is value added of one approach compared to other.

Some papers utilize the panel setting in the model estimation procedure. The question remains if controlling for fixed effects is enough to distinguish often different economies and financial systems that could exhibit very different structural characteristics. O’Brien & Wosser (2022) included structural variables in the analysis to see which of them best affect future downside growth risks. The main structural variables that authors consider are the size of the economy, degree of trade, financial openness, FDI (foreign direct investment) flows, and bank concentration variables. In the empirical results, smaller, more open economies with more concentrated banking systems were more vulnerable to financial shocks. Gächter et al. (2022) find the most relevant structural factors that affect the results are trade openness, financial sector size, public spending ratio, and government effectiveness.

A lot of work has been done either right before the Covid-19 shock hit, or if it was done afterwards, authors cut of the data before this period. Reason being that it is difficult to model this type of shock, even ex post, and GaR model’s purpose is not to capture such shocks. There are a couple of papers that try to explore what can be done and how the model performs in this setting. Alessandri and Di Cesare (2021) examine GaR performance during the Covid-19 pandemic crisis. As expected, the financial conditions variable, although used for short-term forecasting, was not good in predicting the plunge of real economic activity.

In order to talk about macroprudential policy stance, the policy itself needs to be defined as a variable and included in the model. Sánchez and Röhn (2016) was probably the earliest study important within the context of this survey. Authors evaluated various policies (besides MP, labour market, external policy and even quality of the institutions) and their effects on future GDP growth via panel quantile regression for the case of OECD countries (period 1970-2014). When focusing on macroprudential policy, the main results show that mean output growth is reduced, but the tail risk is also reduced. Duprey and Ueberfeldt (2018) showed that macroprudential tightening is more effective in reducing downside risks of future growth compared to monetary policy tightening. Results of the quantile regression (for period 1992 to 2020) indicate that macroprudential policy effectively reduces the GaR for the Canadian case. Galán and Rodríguez-Moreno (2020) focused on a panel of 27 EU countries, with quarterly data from 1970 to 2019. Interaction terms between MPI (MP indicator) and other variables were included in the study to account for different phases of the financial cycle and financial stress levels in economies. Heterogeneity of results in terms of the effects of MPI on different growth quantiles was found.

Macroprudential stance assessment includes the MPI as a crucial variable in the analysis. However, there are some things that need to be considered when doing empirical analysis. First and foremost, macroprudential policy is not measured via a policy rate as monetary policy is. Rather, it is measured through counting the number of measures over time, by constructing indices based on a binary variable, or a variable that takes a couple of values (e.g. -1, 0, or 1). This alone introduces a challenge of aggregation of heterogeneous measures, and on top of that, the intensity of different measures imposes an additional problem. First introduction of a measure, which is classified as a capital one, could have completely different effects to a borrower based measure that is, e.g. fine-tuned. Consequently, one has to have this in mind when using the MPI indicator in empirical research. These indicators reflect the frequency of macroprudential measures, not the magnitudes. A couple of papers emerged in the last couple of years that try to adjust the intensity of MPI indicators. Eller et al. (2020), Vandenbussche et al. (2015), and Richter et al. (2018, 2019) have been working on this.

Macroprudential policy is endogenous, and if GaR literature wants to talk about the effects of policy on future growth, this challenge needs to be considered. Even sourcing of MPI data also has some problems. If the aim of the analysis is to talk about causality, then non-systematic policy shocks should be used. They can be defined as random, i.e. portion of the policy that is not related to the state of the economy (McCallum, 1999). Future research on the effects of macroprudential policy should probably consider this approach. Non-systematic monetary policy shocks have been considered in empirical literature for a long time now, especially since the Lucas (1972) critique, who claimed that the non-systematic component of monetary policy is the part that is important for conducing the policy itself.

Finally, the source of the data collection also matters, as shown in Figure 1, where cumulative values of MPI for the case of Croatia show very different results, that affect the overall estimated results of a GaR model.

On the one hand, some countries only have a couple of years of MPI data, which disables a single-country analysis. On the other hand, measurement problems of MPI could discourage some authors from undergoing such analysis, as different results can be obtained concerning the definition and transformation of the MPI variable. Methodology to evaluate the MP stance has developed a lot, and could be useful once more MPI data comes along to evaluate the stance itself. However, one should have in mind that the GaR approach is just a “bird’s eye view”, i.e. channels of MP effectiveness, and its transmission cannot be seen in this approach. Thus, utilizing GaR approach to evaluate MP stance should be complemented with more granular data based analyses.

Panel analysis enables using more data and obtaining reliable results. However, differences in definitions of variables and their usefulness for specific countries warn that the “one size fits all” approach may not always be the best. A lot of reviewed papers introduce country-specific financial conditions or financial vulnerability indicators. Authors are motivated by some specific dynamics, characteristics, and/or problems of a single country, and to account for this, variables are modified to reflect this in the best possible way. This is probably the best approach for analysing a particular country, but disables a cross country comparisons. It is expected that the GaR framework will become more prevalent in climate change analysis. Bayoumi et al. (2021) and Kiley (2021) already provide an introduction. As climate disasters are becoming more frequent, it would not be surprising to see more and more applications to see the effects on financial stability.