How should we empirically verify whether QE increased or decreased inflation?

Matt O Brien tweeted round an interesting piece by David Beckworth, which estimated a VAR to try to resolve a debate between Steve Williamson on the one hand, and Brad deLong, Paul Krugman, Noah Smith and others about whether QE was deflationary or inflationary.  I responded:  surely you need to identify QE shocks to recover this effect, and Matt asked 'how would you do it?'.  I'm going to teach stuff like this next term, so, if I can't explain this in plain English, I have a problem.

First, going over some stuff for the benefit of those not brainwashed into thinking the way empirical macro tend to think about these things.  We have to look for 'QE shocks', which are changes in QE not prompted by changes in the economy, in order to measure the effects of QE on the economy.  Why?  Because if we don't, we might conflate the effect on the economy of what policymakers are responding to (the terror at the great contraction turning into a depression) with the effects of QE itself.  This same problem crops up, naturally, when people try to figure out what the effect of changes in conventional policy rate and fiscal instruments are.  Hang on, you might say, why on earth should a sensible policymaker change an instrument that is supposed to be used for smoothing the business cycle in a way that is unrelated to the business cycle.  That would be mad, wouldn't it?  A well-functioning policymaker would never execute any policy 'shocks', and so we would never be able to estimate the effects of the instrument for someone doing their job well.  Well this is right, basically.   But actually history might provide us with many shocks nonetheless.  Changes in personnel at the top.  Revisions to data that the authorities use to decide how to move their instrument.  So policy typically has an unavoidably trembling hand, and researchers can use this to find out useful things.  Related to this there's an old debate about whether policy should deliberately experiment to generate the required noise to work out what should be done with the instrument.  Alan Blinder had stern words to say about this in his book 'Central Banking' and most senior central bankers will tutt-tutt at you if you mention this argument in the same way.

The basic problem with trying to recover QE shocks is that QE hasn't been going on for long enough to make a credible enough stab at disentangling QE shocks from QE prompted by the Fed following through on how it thinks it should be doing its job.  The sample time series is too small.

If we had enough data, then there would be a few ways of going about it.

One would be to make an assumptions about the signs of the effects a QE shock would have on some things, invoking knowledge we are confident of from theory.  For example, when people identify interest rate policy shocks, they often assume that a contraction reduces output and inflation and raises rates.  But we can't do that very well here.  The whole point is to try to investigate whether QE reduces or increases inflation.  And there isn't any theory of which we are confident to use to explore the rest of the theory that is more questionable.  It's all up for grabs.

Another would be to use what's known in the field as a 'recursive' identification method.  This is what David did.  Here you assume that inflation doesn't respond within the quarter to changes in QE.  By contrast, QE responds to inflation data as it comes in.  However, two problems.  Who is to say that inflation over a quarter [or month] won't respond to QE undertaken at the beginning of the quarter [or month]?  Perhaps if prices are sticky enough, this might be ok, but perhaps it won't.  Second, more worrisome, doing this with only 2 variables is very tricky indeed.  You hope that you have a policy shock because you have a movement in QE that wasn't caused by a movement in inflation.  However, because you don't have other stuff in the VAR you don't know that what you think is an unwarranted QE mistake isn't actually entirely warranted in response to something else you aren't measuring [like unemployment, or stock prices, or surveys, or whatever].  One of the set-piece debates the academic literature on interest rate shocks involves Chris Sims (recent Nobel laureate) explaining that if you try to measure the effects of interest rates without incorporating enough variables to capture what the Fed are responding to, you can find that a contraction in monetary policy raises inflation rather than reducing it.  This pathology became known as the 'price puzzle'.  It comes about because there is something outside your model the Fed thinks is going to push up inflation in the future [Sims conjectured commodity prices] and so the Fed raises rates to combat it;  the policy response is inevitably not entirely successful at choking off the inflationary threat, and inflation rises, and this looks, through the lens of the small model, like the Fed increasing inflation with a rise in rates.

To cut a long story short then, you can't hope to do this at all well with a two variable VAR.  People often found three or four variable VARs were inadequate when they tried to identify interest rate shocks.  The fancy way to say this is that with a small VAR, the shocks to your VAR equations can't hope to span the underlying economic shocks that you are trying to discover, so that you can compute their effects.

There are other ways to do it, using 'long run restrictions', but these would be completely incredible in these circumstances - with such a small time series - so I won't bother trying to explain them.  [Though if you want a great explanation, go to Karl Whelan's website and look at his lecture notes].

So, Matt asked 'how would you do it?' and my response is bleak:  you can't, not yet.

Are we stuck?

Not entirely.  For two reasons.  First, if you were willing to look just at the Treasuries buying part of QE, you can lengthen the time series, because then we are just talking about debt management really, and that's been going on for a very long time.  You have to go through the careful thought processes above, but this is doable.  Second, if the Williamson story were fleshed out completely, it would have implications for other data, probably, not just inflation and QE itself.  Perhaps there is some prediction for the comovement of a spread and the business cycle, and that could be verified, without confining oneself to the period during which QE was actually practiced.