Marginal Revolution: Explaining Regression

November
S	M	T	W	Th	F	St
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

« My food bleg | Main | Avian flu update »

Explaining Regression

During his Daily Show appearance Steve Levitt said that in estimating the effect of abortion on crime he controlled for other variables like police and prisons. Jon Stewart pressed Steve for an explanation of how someone could "control" for other variables - amazingly, Stewart seemed genuinely interested in an answer but, wisely, Steve demurred. The exchange got me to thinking, What is the shortest, non-technical, yet reasonably accurate explanation of how this is done?

I think the way to go is to use the Frisch-Waugh-Lovell Theorem. Here's my attempt:

Suppose you want to figure out the effect of weight on life expectancy. Heavy people tend to be tall so you have to control for height. You can do this with a two-step procedure. First, calculate how height correlates with weight. Let's say that you discover that every 1 inch increase in height above say 5'7 correlates with a 5 pound increase in weight; you now subtract from each person's weight that portion which can be explained by height. For example, you would subtract 5 pounds from everyone in the data of height 5'8 and 10 pounds from everyone who is 5'9. Since height doesn't explain weight perfectly, you are left with a new variable, weight2. In the second step, you calculate how life expectancy correlates with weight2, since weight2 is "weight after controlling for height" you have now calculated the effect on life expectancy of weight after controlling for height.

To be clear, I am not suggesting that this is what Steve should have said! (If asked I would have said, "Well, I could tell you that Jon, but then I would have to bore you.") I've opened the comments section if you have some other ideas.

By the way the ubiquitous Steve Levitt will be here on Wednesday, but I repeat myself.

Posted by Alex Tabarrok on May 3, 2005 at 07:12 AM in Economics | Permalink

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341c66b253ef00e5508348ee8833

Listed below are links to weblogs that reference Explaining Regression:

» h.g. wells, freakonomics and jon stewart from South(west)paw
Although I have some quibbles with Freakonomics, overall, I think it is great that Levitt and Dubner have increased the percentage of Americans who are interested in economics and the statistical approach to answering societal questions. I think it is... [Read More]

Tracked on Jul 29, 2005 2:26:46 PM

» Explaining regression and CEO compensation from 1-800-BLACK-HOLE
One of the most egregious violations of this principle occured during the dotcom boom / late 90's stock market bubble, namely the linking of CEO compensation to stock prices without controlling for the general effect of a rising market. [Read More]

Tracked on Sep 6, 2005 6:17:05 PM

Comments

Perhaps you could ask Mr. Levitt to address the substantive criticisms of his shallow analysis of The Oakland A's.

Thus far, he has succeeded only in making hay of straw men.

Posted by: Bob Dobalina at May 3, 2005 9:08:33 AM

I am not sure you would have to be so detailed, but the concept of 'overweight' is a good example to use. Just say something like, John, you understand what I mean by overweight, and that the actual weight that is overweight depends on how tall someone is? That would be controlling for height. We have formulas that help us determine that more rigorously, but that is basically what we are doing.

Posted by: theCoach at May 3, 2005 9:58:25 AM

In trying to simplify something, you've made it more confusing. I think the most important thing is to drop the statistics speak altogether--no need to mention variables, correlations and "controlling."
I've explained regression analysis to my math-phobic wife a couple of times and she now seems to understand. The way I do it is by saying something like this (using your weight example): It allows us to create two identical individuals who are alike in every way except for one characteristic, in this case weight. We then compare life expectancies and arrive at a reasonable estimation of the impact of individual weight. To be sure, I've glossed over some important facets, but the basic concept is still accessible to lay-people.

Bob: You should be more concerned about analyzing the A's wins/losses than some economist's take on their organizational structure. Just saying.

Posted by: Aidan at May 3, 2005 10:09:52 AM

Thank you for explaining that Alex, I have been wondering exactly how they do that since I read The Bell Curve. Now can someone explain how the process goes wrong? Would you have to mismeasure a correlation or is overlooking one single contributing factor enough to throw all the results off?

Posted by: Noumenon at May 3, 2005 10:13:57 AM

I watched the first few minutes of the Daily Show interview, but turned it off after realizing that Levitt and the Daily Show format was not a good fit. (And I like the Daily Show.) Now, I realize I should have stuck it out.

Oh, well. The extra minutes of sleep I got meant I could spend a little more time reading more substantive analysis of Levitt's work on the blogosphere.

Posted by: Nick at May 3, 2005 11:17:23 AM

I don't think Jon Stewart was really looking for an explanation of how regression analysis works; he was looking for an explanation of why he should believe in it. I think he would have been satisfied if Levitt said something like this:

"Economists who study traditional subjects like stocks or interest rates are always trying to figure out what real-world factors have an effect on whatever they're studying, and they've developed a lot of statistical analysis methods to help them out. These methods work well and are widely accepted in the financial world. In Freakonomics I'm applying those same methods to other areas of life."

I normally prefer to give people real explanations rather than hand-waving, but I think this answer does a good job of explaining why the methods yield believable results.

Posted by: Mark Draughn at May 3, 2005 11:17:39 AM

I took it as a measure of Stewart's interest that Levitt was on for two segments. Most guests are only on for one; it's usually only the big names that merit two.

Posted by: Matt Newman at May 3, 2005 11:52:00 AM

If Levitt had truly wanted to kill, he could have said,
"Jon, suppose you wanted to measure if CNN was hurting America, but didn't know if America was actually getting dumber because of CNN or because of Fox. You would try to control for Fox by seeing how the viewership of Fox news went up and down in America, and how the knowledge of some set of facts went up and down. Then you would do the same for CNN, controlling for Fox's pain, that is, see if CNN had some separate affect from Fox, and finally you could see whether or not Tucker Carlson was, in fact, a dick."

Pandering, not exactly cogent, but it would have killed.

Posted by: Joelw at May 3, 2005 12:12:51 PM

This is a problem we wrestle with often. We have to describe things for laypersons and legislators, but there simply isnt a synonym for 'linear regression analysis'.

I personally think the description included (for free!) in Freakonomics is the best I have read. If you've read the book, it's the one about the circuit boards. If you haven't, its on page 163.

Posted by: Aaron Morris at May 3, 2005 1:01:43 PM

Controlling for things is one of those examples of statistical bogosity that we should resist.

For example, suppose your friend needs a kidney transplant and has asked for your help in picking a hospital. You do a study and find that patients are twice as likely to die of kidney transplants at hospital A than at hospital B. You decide to "control for age", and produce a a hypothetical population patients2. You find that patients2 are actually more likely to die at hospital B then hospital A. What a relief, right?

But perhaps you might ask the question which is the better hospital for your friend to attend? A little more research shows you actually that hospital B is better for younger people and hospital A is better for older people. Controlling for age actually erases the data you need to make a good decision!

Applying this point to Levitt, controlling for police and prisons would not mean statistical adjustment. It would mean taking every combination of police and prisons levels and showing that in all cases abortion decreases crime. I don't know if he did that, but the adjustment approach Alex Tabbarock is describing is really not relevant to the problem.

Posted by: Alex Jacobson at May 3, 2005 4:30:12 PM

But Levitt isn't trying to prove that every single abortion reduced crime. He's only trying (successfully, I think) to prove that in the aggregate, legalizing abortion had the effect of reducing crime. It could be that some significant number of abortions were of fetuses that would have grown up to murder people, but some small number of abortions were of fetuses that would have grown up to stop crimes. Nevertheless in that scenario, the abortions in toto would have ended up reducing crime.

So regression is the right statistical tool for the job.

Posted by: Cardinal Fang at May 3, 2005 5:53:35 PM

Forget about it.

I teach physics and advanced math to high school students and explaining statistical methods to marginally interested people is difficult and boring (for them.)

Not that I don't try, but on national television. My cynicism says of the viewing public, the river doesn’t run that deep.

Posted by: Ray at May 4, 2005 1:56:04 AM

Sorry to intrude, but Levitt's abortion-cut-crime theory fails, catastrophically, most of the obvious tests of plausibility. For example, the first generation born after the legalization of abortion went on the worst teenage murder spree in American history. Similarly, the FBI's annual survey of crime victims shows that the worst cohort for juvenile crime was the one born in the late 1970s. Moreover, groups that had the highest legal abortion rates (beginning in 1970 in NY and CA) had the earliest and worst run-ups in youth murder rates in the late 1980s and early 1990s.

Besides, Levitt's simplistic model of how legalizing abortion is supposed to cut crime -- by reducing unwanted births -- is severely undermined by his own admission that following legalization, the birthrate declined only six percent but the number of pregnancies went up by almost 30%. In other words, legalizing abortion, according to Levitt, vastly increased the number of unwanted pregnancies. The actual effect on who got born therefore becomes much more uncertain.

You can read for yourself what the facts actually are at http://www.iSteve.com/abortion.htm

Posted by: Steve Sailer at May 4, 2005 4:58:08 PM

I think Alex' explanation is awful! Aidan is on the
right track but he is actually explaining matching
methods, not linear regression. For a discussion of
same, see many papers on my web page at:

www.glue.umd.edu/~jsmithz

Matching has its problems, but its key virtue is that
it is easy to explain to non-technical folks.

The other Alex is raising what the literature calls
the "support problem". The support problem says that
to measure the effect of something on a person with
characteristics X, you also need a person who did not
experience the something but also has characteristics
X. Regression, on the other hand, says that there is
some functional relationship between outcomes, the
something and the X that can be estimated on data on
individuals with many combinations of these, even if
no two individuals have the same X but different values
of the something.

Jeff Smith

Posted by: Jeff Smith at May 5, 2005 12:45:09 PM

Since I've been following medical research reports for decades what I've learned is that it is exceedingly difficult to control for variables in large population studies.

Look at the recent report claiming that mild to moderate obesity does not decrease life expectancy. Lots of studies have been done by generations of sharp medical researchers producing contrary results. Did it take decades to finally come up with a way to control for all the variables that could distort the results? Or will we find out a few years hence that the latest study is flawed?

Some economist confidently proclaims that he has picked the signal of abortion's effects on crime against the background of countless other changes in society. Some other economists think that since this economist is so bright he must be right. Well, sorry, I've seen too many very bright medical researchers get things wrong after repeated attempts and I've seen the conventional wisdom swing back and forth too many times on other questions that had many more people working on them than this question of abortion and crime. It is at very least premature to proclaim that Levitt got it right.

I happen to think that Levitt is wrong for the reasons Steve Sailer has outlined. But even if Sailer hadn't made his case I'd still be thinking:
But look at how complex the causes are for crime and how many other debates about crime and guns and crime and broken families have raged for decades without definitive answers. Social science is not that easy.

From what I'd call a scientific philosophical standpoint my problem with Levitt's conclusion is that he doesn't offer evidence that proves his vague mechanism. Attempts to control for variables have to do that because this isn't real experimental science. This is "lets statistically control for variables without the ability to repeat experiments" science.

His mechanism: "Wantedness"? What social science evidence is there that "wantedness" has increased among the children being born? Also, what social science evidence is there that "wantedness" reduces the odds that a kid will grow up not to be a criminal?

An obvious detail about "wantedness" needs to be answered: "wantedness" by mothers or by fathers? Or both? Illegitimacy has been rising since abortion was legalized. So hasn't paternal "wantedness" been declining substantially post Roe v Wade? His proposed mechanism seems highly dubious.

It is extremely difficult to control for variables when you don't have evidence for mechanisms on how those variables might influence some target variable under study. Taking Alex's example of height, under different circumstances with different populations height might be higher because of genetics, better nutrition, or even worse nutrition (high glycemic index diet causing higher average blood sugar causing higher insulin and higher IGF which might cause excess growth) or even use of growth hormones. A genetically heterogeneous population for height might have other genotypes on the same chromosome that also influence longevity independently even though they are linked to the height gene. Or greater height might be correlated with greater symmetry which might in turn be correlated to less toxin exposure and less stress during fetal development and hence tall people might be born with fewer internal flaws on average (though perhaps only in some environments). It is naive to attempt to control for variables without a lot of knowledge about how those variables influence each other and the phenomenon (in this case crime) under study.

Posted by: Randall Parker at May 8, 2005 11:42:59 PM

We could eliminate crime completely by legalizing everything.

Posted by: Ryan Israelsen at May 10, 2005 12:32:12 AM

google排名
 wow power leveling
wow power leveling
wow powerleveling
wow powerleveling
wow power level
wow power level
hongkong hotel
hongkong hotel
beijing tour
beijing tour
翻译公司
 翻译公司
 上海翻译公司
 上海翻译公司
 北京翻译公司
 北京翻译公司
 门禁
 wow gold
wow gold
powerleveling
powerleveling
wow powerleveling
wow powerleveling
wow power leveling
wow power leveling
power leveling