Reevaluating Reinhart and Rogoff

This week one of the most cited papers and research findings in the past few years has been questioned. Reinhart and Rogoff's findings that GDP growth slows down after government debt levels reach 90% of GDP, which allegedly became the justification to use austerity as the best way to cut down the debt and reignite economic recovery, has been recalculated by three economists from University of Massachusetts  Thomas Herndon, Michael Ash and Robert Pollin. They have found coding errors in the excel spreadsheet, selective exclusion of some data, and unconventional weighting of summary statistics in the original paper. After fixing for these errors they found that GDP growth for countries above 90% public debt-to-GDP ratios is 2.1% on average, and not -0.1%, which was the initial Reinhart-Rogoff finding. This implies that it doesn't make a difference whether or not an economy is burdened with high public debt. You can read the details behind the mistakes on the Rotrybomb blog, or from Matt Yglesias, the FT, Tyler Cowen's comments, and some pretty harsh criticism from Paul Krugman

In comparison the findings from the two papers look something like this (data from 1945 to 2009): 

Reinhart & RogoffHerndon, Ash & Pollin
< 304.14.24.2NA
While a table with a longer time span, going back to 1800, looks like this (HAP paper didn't look into this data): 

Reinhart & RogoffHerndon, Ash & Pollin
< 303.73.9NANA
Source: FT 

Both tables still yield the same conclusion: GDP growth tends to diminish as debt-to-GDP increases. The difference is that the effect isn't as severe as the initial results of RR seemed to show. 

Source: The Economist
The database used in the original paper was from Reinhart and Rogoff's brilliant and several award-winning financial history book, This Time is Different. Their paper that followed afterward concluded the following:
"First, the relationship between government debt and real GDP growth is weak for debt/GDP ratios below a threshold of 90 percent of GDP. Above 90 percent, median growth rates fall by one percent, and average growth falls considerably more. We find that the threshold for public debt is similar in advanced and emerging economies. Second, emerging markets face lower thresholds for external debt (public and private)—which is usually denominated in a foreign currency. When external debt reaches 60 percent of GDP, annual growth declines by about two percent; for higher levels, growth rates are roughly cut in half. Third, there is no apparent contemporaneous link between inflation and public debt levels for the advanced countries as a group (some countries, such as the United States, have experienced higher inflation when debt/GDP is high). The story is entirely different for emerging markets, where inflation rises sharply as debt increases."
All these findings were highly receptive because they made a lot of logical sense. As was their financial contagion mechanism, and suggestions for policymakers. However, once it came under scrutiny, Reinhart and Rogoff admitted to the coding error, but dismissed other attacks and stood firmly by their conclusions: 
"…Herndon, Ash and Pollin accurately point out the coding error that omits several countries from the averages in figure 2. Full stop. HAP are on point. The authors show our accidental omission has a fairly marginal effect on the 0-90% buckets in figure 2. However, it leads to a notable change in the average growth rate for the over 90% debt group. 
HAP go on to note some other missing debt data points, which they describe as “selective omissions”. This charge, which permeates through their paper, is one we object to in the strongest terms. The “gaps” are explained by the fact there were still gaps in our public data debt set at the time of this paper. 
Our approach has been followed in many other settings where one does not want to overly weight a small number of countries that may have their own peculiarities. 
So do where does this leave matters on debt and growth? Do Herndon et al. get dramatically different results on the relatively short post war sample they focus on? Not really. They, too, find lower growth associated with periods when debt is over 90 per cent. Put differently, growth at high debt levels is a little more than half of the growth rate at the lowest levels of debt."
You can read their full response here and here. Cowen defends them.

What are the implications of all of this? The first one is the trustworthiness of non-peer reviewed papers (like NBER working papers of AER proceedings) to be used for policy implications. On the other hand, since journal articles became more restrictive to information access, it is no wonder working papers became much more attractive and widespread. As Ryan Avent concludes, this should "convince economists that more openness is better. Retreating back behind ivory walls will only protect their ideas by helping to make them irrelevant."

This brings us to the second implication; the blogosphere and the consequential immediate reaction of economists to the research has become more important than economic journals in enhancing and stimulating economic discussions and even at influencing policy, as Krugman has noted a few years back:
"First of all, policy-oriented research was never as centered on refereed journals as we liked to imagine. A lot of the discussion always took place via Federal Reserve and IMF working papers, and even reports from the research departments of investment banks. The rise and fall of Fed policy via targeting of aggregates, for example, was not a debate played out in the pages of the JPE and the QJE. 
Second, even for more academic research, the journals ceased being a means of communication a long time ago – more than 20 years ago for sure. New research would be unveiled in seminars, circulated as NBER Working Papers, long before anything showed up in a journal. Whole literatures could flourish, mature, and grow decadent before the first article got properly published – this happened to me with target zones back in the late 1980s, where my original 1988 working paper had spawned a large derivative literature by the time it actually got published. The journals have long served as tombstones, certifications for tenure committees, rather than a forum in which ideas get arguedWhat the blogs have done, in a way, is open up that process." 
Couldn't agree more, journal articles have a much higher influence on academic tenure than on spurring new ideas into discussion, especially if the discussion is on economic policy. 

As for the policymakers using new economic research as their justification for austerity, I only wish they are so receptive. It has been signaled by numerous research papers, discussions, op-eds, blogs, etc. that every European country is conducting a wrong tax-based approach to austerity, but this hasn't stopped the policymakers from doing so, hasn't it? The reasons are purely political, and have little to do with economics (as I've summarized back in September). 

Finally, how have these new findings affected our feelings on the issue of debt and the recovery? Well, not much. Debt is still bad. Even if there is a reversed causality in the debt and slow growth link, there is bound to be something wrong with an economy that doubles or even quadruples its public debt in a few years while using the classical Keynesian response to the crisis. In order to avoid several decades of stagnation that seems to be a necessary outcome of such an approach (combined of course with tax-based consolidations), the focus should still stand on broad-based pro-competition, pro-market reforms. Or to be more topical, revoke the Margaret Thatcher approach


  1. Today...again in the newspaper, same thing as with the Blanchard-IMF story: "economists were wrong, austerity is bad after all". Im going crazy over this even though I shouldn't. But its hard enough to explain certain things to people, especially in an environment where everyone in the country is a nobel laureate in economics (in addition to being the best football manager in the country)


    1. I know what you mean.. the toughest battle is against persistent ignorance

  2. I would like to go beyond this current debate and postulate something else. Are we, those who believe in free markets, wrong? Does it really matter how large the debt is? At what point would you be willing to concede that a nation can continually pump up the money supply as long as they can dance upon the top of the fence so to speak?

    My previous understanding of economics led me to believe that by now the entire edifice of both the USA and Europe economies would be in a total nosedive. But we still have only nominal inflation and investors are still willing to buy government bonds.

    Is there no point at which the whole thing becomes self defeating? And if so, does that mean that there are no effective limits upon whatever government wants to do?

    1. Kyle, thats a very interesting question. Im sure that at some point it will become self defeating. Looking from a point of a normal person in croatia buying a 100.000 eur place to live in, by taking on a 30yr loan, where total debt is much more then his yearly "output" i can see how all that can go very far in terms of debt/gdp. Guy at cyniconomics thinks its 150% gdp ( One thing that may be new for current politicians is environment of low inflation we have last 20 years. Monetary policy still has problems adopting to this phenomenon - its sort of a winners curse. Also, yields for biggest economies have been trending down since 80s, but you see that other countries that really have no significant productive/innovative base (together with horrible institutions) are being punished, maybe too harsh. If I would answer your question directly, I would say that this (ever shrinking) part of our economies left to market is doing a hell of a job keeping it all together.

    2. I would say that it is already self-defeating; growth is sluggish and countries with a huge debt burden like the UK are very likely to emulate the Japan scenario. The losers are the future generations which will at some point have to bear the pain of the necessary reforms.


Post a Comment

Popular posts from this blog

Short-selling explained (case study: movie "Trading Places")

Rent-seeking explained: Removing barriers to entry in the taxi market

Economic history: mercantilism and international trade

Graphs (images) of the week: Separated by a border