« The best paragraph I read today | Main | This is either the worst or the best news I have ever heard »
Joshua Angrist writes to TNR on Levitt
He defends Levitt's work, his own work, and the use of instrumental variables in economics...
In "Freaks and Geeks" (April 2, 2007) Noam Scheiber praises my work with Alan Krueger on the economic effects of compulsory schooling but argues that economics Ph.D. students today are obsessed with headline-grabbing trivia of little substantive importance. The root cause is said to be excessive attention to a good or clever research design at the expense of the relevance of the underlying question. Scheiber's story is engaging and he lands a few punches, but his account is misleading in two important ways. For one thing, he exaggerates the problem of small-bore studies. America's half-dozen top Ph.D. programs produce scores of Ph.D. students every year. Most of this work is still on traditional topics. At MIT (where Levitt studied), we continue to supervise empirical theses on, among other things, health insurance, immigration, unions, and human capital. High-quality research on these traditional topics gets students high-quality jobs. I'll plead guilty, however, to being especially pleased when students manage to come up with clean identification--that is, they have a convincing strategy for uncovering causal effects. Clean identification is not a fetish; without it, little of value is learned. On this score, our students typically do better than the empiricists of H.G. Lewis's generation. In two of the most dynamic empirical microeconomics subfields, the economics of education and economic development, there has been a virtual credibility revolution, with the increase of randomized field trials as well as compelling natural-experiments research designs (see, e.g., the work done at MIT's Poverty Action Lab).
Second, in his rush to tar some up-and-comers with the "cute-o-nomics" brush, Scheiber misses a central feature of the clean-identification research agenda, best explained by example. One of the enduring scientific and policy questions in Labor economics is the sensitivity of hours worked to changes in pay (this matters for tax policy, for example). The best evidence labor economists have on the relation between wages and hours worked comes from a small experiment (by Ernst Fehr and Lorenz Goette) involving the wages of bicycle messengers in Switzerland. The second best comes from a study of stadium vendors by Gerald Oettinger. Who cares about the riders of Veloblitz or snack sellers at Camden Yards? We care because economics is predicated on the notion that a few simple principles explain behavior in many settings. These studies produce results that are convincing and may well be general, though, as always in science, it will take replication to know for sure. Some of the studies Scheiber dismisses can be understood in this spirit. Finally, as to the merits of Freakonomics the book: My 17-year old daughter picked it up on her own last year. She never knew economics could be so cool. Even better, she now asks me what I'm up to. So I tip my cap to Dubner and Levitt--I hope to see many of their readers in an economics classroom some day.
JOSHUA ANGRIST is a professor of economics at Massachusetts Institute
That is from TNR on-line. Here is Alex's earlier post on the Scheiber piece.
Posted by Tyler Cowen on April 25, 2007 at 06:39 PM in Economics | Permalink
Comments
The irony is that Steven D. Levitt's best known theory -- that legalizing abortion in 1970-1973 cut crime later -- contradicts Noam Scheiber's hypothesis that Levitt's work is brilliant but trivial. It's an extremely important topic, but one that Levitt notoriously botched up in a variety of ways, from miscoding his programming to failing to do simple reality checks on his results.
As a journalist, I have to admit that, overall, economists have acquitted themselves better in terms of critically evaluating Levitt's most famous Freakonomics theory than have journalists, such as, oh, Scheiber, who never mentions in his New Republic article critical of Levitt the flaws economists such as Joyce, Foote, Goetz, Dills, and Miron have identified in Levitt's theory. Though the mills of academic economics have ground slowly, they are grinding up Levitt's theory exceeding small.
In contrast, the book reviews of Freakonomics in prominent publications such as the New York Times were disgracefully credulous. Ten minutes of Googling would have revealed that there had been considerable empirical controversy over Levitt's abortion-crime theory for a number of years before Freakonomics' publication, but few reviewers did even this minimal due diligence.
Posted by: Steve Sailer at Apr 25, 2007 7:42:47 PM
Steve - A few questions. What exactly is a "reality check," and how did Donahue and Levitt fail to do that in that original 2001 article? One reason he may have failed to talk about Joyce's 2003 JHR paper is because of the 2004 JHR response by Donahue and Levitt that point out - interestingly enough, since this is essentially your argument - that because the Joyce only looked at the late 1980s, he fails to find anything because of an ommitted variable that is crack cocaine. Sound familiar? You harp on it enough that I thought you might find that curious. As for Foote and Getz - this is correct that they found the coding error. But the jury is still out as to whether it's a fatal problem or not. That is not a published article yet, and the response by Levitt and Donahue goes a long way in addressing the problems with the original dataset altogether. The Miron and Dills comment is just that - a comment which adds nothing to the debate except to describe our current knowledge of the effect of abortion legalization on criminal propensity. Even in that, Miron and Dills note that Levitt and Donahue appear to explain 25% of the decline in crime (as opposed to 50%).
There are other papers, too, that have been circulated on NBER that find evidence for the abortion legalization conjecture. There are also some that have been published - such asKerwin Charles and Melvin Stevens' 2006 JLE paper "Abortion Legalization and Adolescent Substance Use." But since you have become a harpy who repeats himself, rather than seriously enters into a very difficult scholarly debate about a serious question, I doubt you have had the time.
Posted by: jason voorhees at Apr 25, 2007 8:44:57 PM
I think you can argue that showing the relationship between abortion and crime is trivial because there really isn't much you can do with it after establishing the fact.
I think one of the main strengths of economics is the policy implications, where you analyze what the government or other institutions should do in light of the results of one's research. But what would such policy implications be of showing the causual relationship between abortion and crime rates? That abortion is good and should be legalized? Of course not, there are moral and political considerations to take in as well, so in the end the entire thing becomes a triviality. Obviously this is not true of all of his research, but one can do a similar thought experiment to the one above and realize that a few of the points he makes in Freakonomics are indeed quite trivial.
Posted by: Anonymous at Apr 25, 2007 10:32:28 PM
Yesterday, the Mexico City legislature
approved a bill to make abortion legal
during the first three months
of pregnancy. It passed 46-19 with one
absention.
It became the largest entity in Latin
America, to permit women to have
abortions on demand - no questions asked.
Some think that it will be challenged
in the courts.
Unwanted pregnancy leads to a million
abortions a year in Mexico today. So,
this law simply legalizes it. (Only
in Mexico City).
The Pope, as usual, butted in and
condemned the bill. Mind you, abortion
is legal around where he lives - Italy
(unless you quibble over the fact that
Vatican is legally a state - hence what
I said above is not true).
The archbishop of Acapulco went one
step further. He said that the lawmakers
who voted for it would be automatically
excommunicated.
I am looking forward to a lower crime
rate in Mexico City in two decades. If
not Levitt, demographic shift will help.
Tapen
Posted by: Tapen Sinha at Apr 25, 2007 10:42:31 PM
Anonymous - there's no value to positive economics?
Posted by: jason voorhees at Apr 25, 2007 11:27:37 PM
Steve - When you have time, here are other papers finding more evidence for the abortion legalization hypothesis.
"Abortion and Selection" by Elizabeth Oltmans Ananat, Jonathan Gruber, Phillip Levine and Douglas Staiger (2006) NBER working paper 12150 (http://www.nber.org/papers/w12150)
"The Impact of Legalized Abortion on Teen Childbearing, "by John J. Donahue III, Jeff Grogger and Steven D. Levitt (July 2002)
Charles and STevens, JLE 2006 (see above post)
"The Effect of Abortion Legalization on Teenage Out-of-Wedlock Childbearing in Future Cohorts," by Serkan Ozbeklik (June 2006).
The Charles and Stevens article is peer-reviewed published 2006 JLE. The Donahue, Grogger and Levitt is currently under major revisions. The Ananat, et al. (2006) NBER working paper is especially relevant because their "approach is to include all [interaction terms] in every specification and to test the sensitivity of the results to alternative identification strategies." Their results show "that Donahue and Levitt's results that increased abortion access was associated with a decline in crime are not sensitive to the alternative identification strategies that [they] describe. However, it is unclear how much of the decline in crime came through the selection effect."
This last paper you should read very closely, Steve. Since Levitt's not an author on it, but the paper confirms the main original results from Donahue and LEvitt (2001), and includes the worrisome state-year interactive fixed effects, I'm curious as to how you'll spin away these results. Although I suspect you'll just wave your hands in ignorance over the empirical work done, and start immediately into ad hominem and straw men accusations.
Posted by: jason voorhees at Apr 25, 2007 11:50:03 PM
Off the top of my head, the main critique of Levitt's book from the very beginning was that it was too trivial, to which the ultimate response was, and is, so what?
Even if it is, I know a number of people who have read Levitt's book who would otherwise not read a book on economics. Etc.; that argument has been made.
An interesting aside is the treatment that most economists receive from the media in general. I'm thinking of Richard Posner's "Public Intellectuals" wherein he lists many (though not all) of the most highly regarded experts who have been nothing but wrong in their declamations and predictions, and yet no one ever seems to notice, or bothers downgrading their credibility rating.
I’ve seen some arguments on Levitt’s work (see above) but nothing that would come close to how frighteningly wrong the media’s selected, anointed and never criticized experts have been.
Posted by: Ray G at Apr 26, 2007 12:03:08 AM
I personally found Scheiber's article a bit silly. There's plenty of room for all sorts of different work and approaches to problem solving, and actually achieving some causal identification is a good thing.
Scheiber also missed an important opportunity to discuss one problem that did happen in economics during the early part of the instrumental variables fad/revolution. Many economists didn't fully understand the statistical properties of instrumental variables regression and did some work that didn't hold up. Basically, it turns out that you need a very strong relationship between your instrument(s) and what you're instrumenting (even significantly better than 99% significance when you have one instrument)in the first-stage regression that generates your fitted values, and the F-statistic that you need rises exponentially in the number of instruments you use. If you don't have those things, your instruments are weak, you inference will be weak, you need to use slightly different estimation methods that are robust to your weak instruments, and even then you'll likely have wider confidence intervals.
Many economists during the late 80s and early 90s did not know this, so some mistakes were made.
But we know that now, so that's one instance where we learned from our mistakes.
Posted by: Keith at Apr 26, 2007 6:56:24 AM
I will say, however, that the examples on labor supply elasticity that Angrist cites do worry me. I would have to go back and read the papers, but unless the experiments took place over longer periods of time, they likely capture only short-term labor supply elasticities, which are inevitable lower than long-term labor supply elasticities. So you might not want to take those elasticities to the bank for policymaking purposes.
Posted by: Keith at Apr 26, 2007 7:00:14 AM
The really big thing that Scheiber missed is that the Levitt, Angrist, et al. revived interest in empirical work in general. The alternative for most top students was not DEEP research about human capital or development but navel-gazing mathematical exercises on trigger strategies, refinements on sub-game perfect equilibria, nth order econometric filters, and ex post mathematical rationalizations of mechanisms that had not been fully formalized but which were well understood. The new applied econometricians made it possible for non-mathematicians to get good jobs at top universities and get published in the top journals. For all their flaws, Levitt and Angrist (and Murphy and Shleifer) are better role models for the best grad students than Rubinstein or Nash. Unfortunately, grad schools STILL select students as if the ideal students were candidates for the Fields Medal. And the easiest way for someone at a lower ranked school to move up is STILL to play math games. Therefore, it is imperative that the Levitt tradition flourish if one is to push back against the trends that say, better a boring but technically sophisticated paper than an interesting -- if trivial -- empirical piece. What is certain is that the days of theorizing a la Friedman, Akerlof or Schelling -- where low tech models meet deep questions, is virtually ended in the top journals.
Posted by: AJ at Apr 26, 2007 10:15:23 AM
For Pete's sake, Sailer, let it go.
Posted by: josh at Apr 26, 2007 11:20:31 AM
balance transfer canadian
Posted by: uddogu at May 20, 2007 10:45:34 PM
大家好，我是臺灣人，從臺灣一個人搬家來到美國，環境很陌生，感覺很孤單。以前在臺灣幾家知名的徵信社工作過，我是一個優秀的徵信工作者，希望早點找到適合自己的工作。希望通過貴站，認識更多的朋友。
Posted by: 謝文豪 at Apr 2, 2008 2:27:32 AM
Please come to FFXI Gil, we will give you a great surprise.
Posted by: cheap Final Fantasy XI Gold at Dec 31, 2008 1:40:00 AM
Please come to FFXI Gil, we will give you a great surprise.
Posted by: cheap Final Fantasy XI Gold at Dec 31, 2008 1:42:47 AM
