« The Economics of the HDMI Cable Ripoff | Main | The FDA regulation of tobacco »
Russ Roberts has a question
It's about the influence of empirical economics:
I'd like one example, please. One example, from either micro or macro where people had to give up their prior beliefs about how the world works because of some regression analysis, ideally usually instrumental variables as that is the technique most used to clarify causation.
I will cite a few possible examples, although I won't stick with instrumental variables:
1. The interest-elasticity of investment is lower than people once thought.
2. We have a decent sense of the J Curve and why a devaluation or depreciation doesn't improve the trade balance for some while.
3. Dynamic revenue scoring tells us over what time horizon a tax cut is partially (or fully) self-financing.
4. Most resale price maintenance is not for goods and services involving significant ancillary services.
5. More policing can significantly lower the crime rate (that one does use instrumental variables).
6. The term structure of interest rates is whacky.
I see other examples but in general I agree with Russ's point that empirical work fails to settle a great number of important disputes, most disputes in fact. Many of the examples I would cite turn out to involve an elasticity being lower than we had thought. And many more involve macroeconomics (rather than micro) than you might expect.
What examples can you think of?
Posted by Tyler Cowen on June 12, 2009 at 07:26 AM in Economics | Permalink
Comments
As of the early 1970s there were approximately 100 studies finding that profits were positively correlated with industry concentration. Many people thought this finding was evidence of explicit or tacit collusion and that it justified strong antitrust policy. Harold Demsetz's clever empirical work (JLE 1973) supported a different interpretation: the leading firms in concentrated industries were competitively superior. After a few years profits-concentration studies became, to use Leonard Weiss's word, "unpublishable".
Posted by: Craig M. Newmark at Jun 12, 2009 8:03:58 AM
The equity premium puzzle?
Posted by: Steve Hamilton at Jun 12, 2009 8:19:42 AM
Corporate Finance (Mergers and Acquisitions) example:
Campa and Kedia, Journal of Finance 2002, I think, used IV to show that diversifying mergers do add value, whereas most studies previously had trumpeted the "Diversification Discount" but when CK used IV (and some other more sophisticated techniques) they showed that not only does the "discount" disappear, but it may be a premium.
Primarily, they had to control for self-selection bias in that most mergers happen during industry shake-ups, such that previous work had benchmarked too high for the comparison.
Posted by: Jesse Blocher at Jun 12, 2009 8:27:10 AM
The (lack of) social returns to job training programs.
Posted by: MW at Jun 12, 2009 8:29:02 AM
VARs killed naive monetarism: the finding that money shocks only explained maybe 25% of output volatility set off a modest monetarist literature trying to show that was a mistake and that money really did cause most output volatility.
Of course once the monetarists lost they didn't say 'we lost.'. Who's dumb enough to say that? The tearful mea culpa is common in movies because it is so rare in real life.
All the same VARs played a major role in killing the k% growth rule.
Posted by: Garett Jones at Jun 12, 2009 8:47:42 AM
The Phillips curve's appearance and disappearance.
Posted by: myself at Jun 12, 2009 9:00:58 AM
And reappearance?
Posted by: Tyler Cowen at Jun 12, 2009 9:13:12 AM
A recent paper from the New York Fed showing that subprime lending probably wasn't as racist or as predatory as some have accused:
http://www.newyorkfed.org/research/staff_reports/sr368.html
Posted by: Jim at Jun 12, 2009 9:17:32 AM
Tyler,
Can we get some citations for #4? I was under the impression this was still disputed.
Posted by: ao at Jun 12, 2009 9:20:38 AM
The findings that raising the minimum wage does not reduce employment.
Also the meta study techniques that are showing clear publication bias in most labor market studies.
Posted by: mickslam at Jun 12, 2009 9:23:44 AM
I was unaware that anyone serious thought that tax cuts were anything other than partly self-financing. The only issue was that some, mostly non-economists, thought they were fully self-financing. And good empirical work should have put an end to that nonsense.
Posted by: Mike at Jun 12, 2009 9:25:10 AM
Research from the NY Fed in late 2005 that showed that fundamentals could largely explain housing prices, and any potential fall in prices would not harm the larger economy.
http://www.newyorkfed.org/research/staff_reports/sr218.html
http://www.newyorkfed.org/research/economists/mccarthy/athens_bubble_paper.pdf
Posted by: tedm at Jun 12, 2009 9:34:30 AM
Oh and VARs are methodologically similar to IV.
Another example: Barro's 2 papers showing that money shocks influenced output for 2 years. Made it tough to believe in Lucas-style money surprise models; are people fooled for 2 years?. The lore of the profession is that this spurred RBC since the flexible price types couldn't embrace New Keynesian models.
The Lucas Revolution was cancelled by the empirical hump-shaped response of output to money shocks. Again, near-IV techniques were used.
Again, few recantations were issued: after all these people were busy building a new research agenda!
The visible implications of mind-changing are few.
Posted by: Garett Jones at Jun 12, 2009 9:40:09 AM
I believe that the jury is still out on the minimum wage affect on unemployment. The original Card study was full of holes.
There's some work from a while back that showed that the elasticity of demand for alcohol is much higher than people believe.
There are many people who have given up on the efficient market hypothesis. Even Fama had to add two variables to the "beta equation". That 1992 paper was clearly an admission that beta didn't work as theory said it should.
Posted by: Highgamma at Jun 12, 2009 10:16:19 AM
The series of articles over the past 3 decades that show low (and declining) intergenerational mobility in the U.S.
Posted by: GregN at Jun 12, 2009 10:40:15 AM
Can someone elaborate on how VARs are methodologically similar to IV?
Posted by: anony at Jun 12, 2009 10:47:08 AM
Joseph Doyle's papers in the AER and JPE from 2007 and 2008 showing foster care causes worse outcomes in crime, employment, and teen pregnancy. IV is probably one of the only ways I would've believed any result regarding foster care's effect on outcomes since studying foster care outcomes has selection problems.
Posted by: anony at Jun 12, 2009 10:52:38 AM
Almost all public financing of professional sports arenas is captured by the team value and almost none is captured by those taxed. (Though many economists may have had suspicions, the common wisdom of urban planners was the opposite.)
Posted by: MW at Jun 12, 2009 10:53:06 AM
Seatbelt laws kill -- as well as other applications of the, so called, "Peltzman Effect."
Posted by: MW at Jun 12, 2009 10:57:52 AM
This will be a tough battle to fight, as Russ Roberts as already proved himself unwilling to update his priors based on statistical evidence: http://www.overcomingbias.com/2007/10/if-not-data-wha.html
Less flippantly, I would say that one or two statistical studies are often insufficient to change peoples' minds, as there are usually studies that find the opposite and people are too lazy (?) to investigate the correctness of the various sides. The Donohue-Levitt abortion paper is a great example of this. There is substantial criticism, but a close examination, at least to my eyes, shows that the critiques are by and large very weak. Yet the mere existence of criticism is enough for people with strong priors to refuse to change their minds.
I think Prof. Roberts falls into this trap with the gun-crime relationship, as a close reading suggests that the statistical evidence is much stronger for one camp than the other, but Roberts refuses to address this and says that there is disagreement, and who is he to evaluate the evidence.
Posted by: Andy at Jun 12, 2009 11:24:14 AM
Andy makes good points. More generally, sifting through data is a humbling task, as is sifting through empirical results. The easy response is to take sides, but the hard part is to try and jump into the study and understand its strengths and weaknesses.
Causal effects literature seems to be progressing at a lightning fast pace, so even when one feels really sure about a result, it seems like new studies come out which make you do a double take. Take the weak IV literature. IV, to be valid, must be exogenous and relevant. In the 1990s, almost everyone focused on exogeneity, with little to no attention to relevance. Now we're up to our necks in weak IV tests. Or take all the hoo-hah over clustering.
Even when I feel convinced by a good causal effects study, I'm no longer 100% convinced. I'm either mostly convinced or mostly non convinced. Observational data makes it very difficult, for me, to be as sure about findings.
Posted by: anony at Jun 12, 2009 11:53:43 AM
Before the public choice revolution, both theory and evidence, people actually believed that regulation was in the public interest.
Posted by: MW at Jun 12, 2009 12:03:02 PM
I don't understand his question- any situation where the elasticity of an issue could be positive or negative would fall under this, and there are plenty of those issues. Or is he one of those fellows that thinks we can a priori know, regardless of the question, whether the income or substitution effect dominates?
Posted by: confused at Jun 12, 2009 12:23:01 PM
Andy, you seemed to have proven Russ's point.
You read very weak data, pro or con, as a strong argument.
Let me guess where your priors are.
Posted by: Tom at Jun 12, 2009 12:26:41 PM
Tom, how did you decide the data was weak? The data is weaker than it would be if everyone agreed, but that is obvious. My point is that not all data is created equal, and we shouldn't throw our hands up in the air merely because people disagree. We should evaluate people's statistical techniques, the plausibility of the exclusion restriction, and so forth. I'm not saying we shouldn't have a degree of skepticism and of course we should be willing to change our mind in the future, but it's obvious we can evaluate methodology when we evaluate researchers' results.
Personally, my prior on the abortion-crime link was that the theory made some sense, but that they were unlikely to find anything as the variation was too limiting and the effect was likely to be small. Reading the literature carefully changed my mind. Isn't that the point of this discussion and most research?
Posted by: Andy at Jun 12, 2009 12:40:26 PM
