« "As a child, Peter Leeson was pirate-obsessed." | Main | The Post-American World »
Hail Emily Oster!
The paper is titled "Hepatitis B Does Not Explain Male-Biased Sex Ratios in China"; here is the abstract:
Earlier work (Oster, 2005) has argued, based on existing medical literature and analysis of cross country data and vaccination programs, that parents who are carriers of hepatitis B have a higher offspring sex ratio (more boys) than non-carrier parents. Further, since a number of Asian countries, China in particular, have high hepatitis B carrier rates, Oster (2005) suggested that hepatitis B could explain a large share (approximately 50%) of Asia's missing women". Subsequent work has questioned this conclusion. Most notably, Lin and Luoh (2008) use data from a large cohort of births in Taiwan and find only a very tiny effect of maternal hepatitis carrier status on offspring sex ratio. Although this work is quite conclusive for the case of mothers, it leaves open the possibility that paternal carrier status is driving higher sex offspring sex ratios. To test this, we collected data on the offspring gender for a cohort of 67,000 people in China who are being observed in a prospective cohort study of liver cancer; approximately 15% of these individuals are hepatitis B carriers. In this sample, we find no effect of either maternal or paternal hepatitis B carrier status on offspring sex. Carrier parents are no more likely to have male children than non-carrier parents. This finding leads us to conclude that hepatitis B cannot explain skewed sex ratios in China.
We should hold up Emily Oster as a role model of a truth-seeker. If the abstract does not make it clear, Emily Oster first won her fame by reporting the opposite result about sex ratios. Here are our previous posts on Emily Oster.
A more general lesson, of course, is simply how difficult it is to get at truth. This is a well-defined data set with a (more or less) well-defined answer. Most policy questions aren't so tractable.
Posted by Tyler Cowen on May 12, 2008 at 06:44 AM in Medicine | Permalink
Comments
Role model is right. I once heard Emily say that we (meaning economists) should be about trying to find the truth, and I now believe her.
Posted by: jason voorhees at May 12, 2008 7:53:18 AM
This is why when we as social scientist enter these areas for which we have little real theory that we need a model of the phenomena to generate the hypotheses that we take to the data. Yes, I mean a mathematical or computational model. It is one thing to build an empirical specification when we have a well trod theory to guide our choices. But in many of these "freakonomics" type areas (forgive the simplification but I think you know what I mean) we only have a story or a metaphor (usually to price theory). What we are observing is the curse of dimensionality eating the data set while a researcher over-fits the line. Bare-assed empiricism is ok as a first cut, but I am less and less convinced by this sort of work. Even when done by a really bright scholar who is very careful with their empirics.
Posted by: goodnessOfFit at May 12, 2008 9:06:18 AM
Its nice to think she was just investigating this topic with no direct purpose, but I think the 1 child per family law is the reason. China probably said there are so many more men than women because of hepatitis B, because thats how the government tried to control the situation, by placing blame on something it cannot directly control. The truth is, the Chinese have been murdering their female children in an effort to get the much more valuable Male offspring.
Posted by: brainwarped at May 12, 2008 9:12:10 AM
As a physicist I approve wholeheartedly of the capability to contradict your earlier work with new evidence. In fact, I would go beyond saying that you should not be wedded to your beliefs about the validity of certain theories or models; rather, I think you should be theory-agnostic. I do work in particle theory, but I have no real faith at all in the existence (or non-existence) of the Higgs boson. Even in a discipline such as mine where ideology is not really a concern, you see far too many scientists with pet theories for which there is no real evidence...
Posted by: Matt at May 12, 2008 9:23:36 AM
As a physicist I approve wholeheartedly of the capability to contradict your earlier work with new evidence. In fact, I would go beyond saying that you should not be wedded to your beliefs about the validity of certain theories or models; rather, I think you should be theory-agnostic. I do work in particle theory, but I have no real faith at all in the existence (or non-existence) of the Higgs boson. Even in a discipline such as mine where ideology is not really a concern, you see far too many scientists with pet theories for which there is no real evidence...
Posted by: Matt at May 12, 2008 9:25:37 AM
"Emily Oster first won her fame by reporting the opposite result about sex ratios. "
I thought it was witches.
Posted by: PLW at May 12, 2008 9:53:52 AM
Matt,
I agree that we should not be wedded to our models or theories in the face of contradictory empirical evidence. However, we cannot be theory-agnostic when using the tools of micro-econometrics. Virtually all the techniques assume that there is a theory and it has given you a prediction about a ceteris paribus relationship and that is what you are testing. When you run the empirical test you are relying on your underlying model to help you pick the proper specification. In most empirical quantitative social science that comes from the econ world our tools are built to test predictions from models.
Posted by: goodnessOfFit at May 12, 2008 11:47:00 AM
Gof - what's wrong with generating a hypothesis and testing it with data? Isn't that the scientific method? Emily's hypothesis, incidentally, did have some scientific basis. It's been a while, but I remember her going through and discussing the background on HepB and male birth probabilities. Her innovation was to suggest the imbalanced sex ratios in Asia were a consequence of the virus and not female maltreatment (or at least, that the latter couldn't explain all of it). This is good science as far as I can see. Observe, hypothesize, test hypothesis, repeat. Ultimately, this entire affair has been good for everyone, because I think we can rule out a hypothesis and have ultimately more confidence about other hypotheses than we did before it happened.
Posted by: jason voorhees at May 12, 2008 1:26:29 PM
PLW - witches was something she published from college, but she was a "star" because of HepB and the missing women.
Matt - I agree wholeheartedly. I think you see the same thing in economics, actually. Ideology matters much less in 2008 than it probably did in 1948, but pet theory biases abound. When I teach a research methods class, I plan to take my class through this sex ratio material that Emily's work fits into. I think students need to learn this as much as identification as soon as possible.
Posted by: jason voorhees at May 12, 2008 2:00:33 PM
Jason,
We are indeed testing a hypothesis, but the tools we use need more than a hypothesis. The tools of micro-econometrics are predicated on an implicit model of the data-generating process. They were and are designed to test the conditional marginal effects that are derived from a coherent model. That is how we build our empirical specifications. For example when you decide to include x_2 because it is correlated with y and x_1 you are basing that on a model that you have in your head. So the extent to which I believe the results of the empirical specification is in part based on how much I believe that model. So I want to see that model as much as I want to know all about the data set and the specification used. I want to know that your hypothesis, choices for control variables and specification form are all logically consistent and that is what a formal model does for you. Now it does not have to be your model. If you are using a well worn model then I have no problem with the jump right to empirics. But if the researcher is breaking new ground as we often see with modern applied micro work I want to see the model. An appeal to this literature to justify x_3 and an appeal to that literature to justify x_4 is really weak IMHO. That sort of thing reeks of the curse of dimensionality and finding one particular vector of specification choices that happens to fit the data (see de Marchi's 2005 Cambridge book is short and explains this well).
Don't get me wrong about Emily's work, I like it. As I like much of the interesting applied work being done now. Consider this a criticism of someone on the freak-train as it were. However, I do think there is a huge methodological blind spot in some of this work in that they are running empirical test as if they have a comparative static out of price theory as their hypothesis. At best they usually have a metaphor that appeals to price theory. At worst they have a patched together story that would fall apart if they ever tried to write it down mathematically or computationally.
Posted by: goodnessOfFit at May 12, 2008 2:14:01 PM
Okay, but how about giving some credit to all the non-economists who thought her initial hypothesis was silly? The evolutionary scientists I stay in touch with dismissed her theory quickly.
Posted by: Steve Sailer at May 12, 2008 5:21:40 PM
This is a good example of the pitfalls of the freakonomizing temptation.
The basic problem is that economists don't really know that much about the real world outside of economics. There aren't many incentives within the economics profession to read deeply in anthropology, evolutionary biology, psychology, history, sociology, criminology, psychometrics, political science, and other non-economic fields. That wouldn't be a problem except for the recent tendency of economists to attempt to imperialize other fields, but without doing the hard work of familiarizing yourself with the subjects.
My guess is that the freakonomics fad was an outgrowth of the prosperity of recent years. Back in the 1970s, economists weren't freakonomizing much because the economy was a mess and their prestige was low. So, they stuck to economics. Lately, the economy has been going fine, so, bored, they've used the prestige acquired from the prosperous economy to take fliers on other scholars turf. But, with the economics profession coming back into disrepute due to the obvious mismanagement of recent years, perhaps it's time for economists to get back to specializing on the economy?
Posted by: Steve Sailer at May 12, 2008 5:30:45 PM
Steve, there are plenty of people who "specialize in the economy." They're called macroeconomists, and it's always the hottest area. But there has been for decades now also economists who think all of human behavior, and not just "economic" behavior, can be fruitfully studied with the tools that were partly borne out of economic theory. Visit political science and sociology departments, and you'll increasingly see game theory and rational choice crowding out the older methods. Or at least, having a place therein.
Posted by: jason voorhees at May 12, 2008 6:15:41 PM
Sure, economists have useful tools. The problem is that economists tend to be infatuated with their tools and underestimate how much they need to know about other social scientists' specialties before parachuting in. Thus, the many embarrassments suffered by freakonomists.
A better model would be for economists to bring their tools and collaborate with people who actually know what they're talking about empirically.
Posted by: Steve Sailer at May 12, 2008 7:08:52 PM
Here's a current example of economists' negligence of their basic responsibilities.
The freakonomists claim that they're good at catching cheating, such as in sumo tournaments. Yet, massive mortgage fraud went on across America in 2005 and 2006 involving "liar loans," which now threaten to plunge America into hard times. How good a job did economists, with their cheater detection tools, do of noticing and publicizing and heading off this historic catastrophe?
Posted by: Steve Sailer at May 12, 2008 7:28:24 PM
Steve, how do you know when someone "knows what they're talking about empirically"? At what point can an economist say, "I've read the relevant literature and now have a firm grasp of what is currently the state of knowledge" before proceeding to develop their own original work? Is it that you read Emily's paper and found it sorely lacking in her knowledge of the missing women problem in Asia or the epidemiology of Hepatitis B?
Posted by: jason voorhees at May 12, 2008 7:57:04 PM
There are two somewhat different questions here:
First, if you want to get a serious understanding of another field, like Paul Krugman wanted to get to understand evolutionary biology in the 1990s, you have to do some heavy reading over a lengthy period of time. Krugman worked hard and found out that most of the stuff he thought he knew about Darwinism from reading Stephen Jay Gould was misleading, and that Gould was the John Kenneth Galbraith of biology.
Second, if you just want to see if your brainstorm idea is worth investing your time and other people's time in it, then do reality checks.
There are a lot of ways to do reality checks, such as asking experts.
Several years ago, I asked Greg Cochran, who knows far more about infectious diseases than any economist, about Emily Oster's theory and he didn't think her idea was worth pursuing.
Similarly, when a certain famous young Harvard economist was profiled by Stephen Dubner in the NYT Magazine as the next genius, Dubner devoted much space to the economist's theory that the reason African-Americans suffer so much from hypertension was that a slave with "a higher capacity for salt retention might also retain more water and thus increase his chance of surviving" the Middle Passage in slave ships.
So, I put this economist in touch via email with Cochran, who explained to him the arithmetic that shows that even the strongest natural selection acting on a single generation of ancestors can't account for the big genetic differences in salt retention between African-Americans and white Americans. Instead, it was the difference between the climate in Africa and in Europe, working over hundreds of generations, that selected for different gene variants.
But this economist didn't want to hear about it and cut off the dialogue.
The main thing is -- don't just talk to other economists. For example, Steven Levitt took his abortion-cut-crime theory comparing crime rates in 1985 and 1997 around to seminars in leading departments of economics in 1998-99. And, apparently nobody pointed out to Levitt that, in contradiction to his theory, in between 1985 and 1997, serious crime among young people (born right after Roe v. Wade) had soared, the opposite of what his theory predicted. All these economists had forgotten (or never noticed) the Crack Years.
Economists really aren't that good at remembering facts -- that's why they like theories. Conversely, historians are good at remembering facts, but they aren't very good at theories. Everybody has different strengths and weaknesses.
The bottom line is you have to get out of your comfort zone and go mix it up with people who don't think like you do.
Posted by: Steve Sailer at May 12, 2008 9:43:14 PM
My guess is that the freakonomics fad was an outgrowth of the prosperity of recent years. Back in the 1970s, economists weren't freakonomizing much because the economy was a mess and their prestige was low.
Steve, there's a simpler explanation: the growth of large-scale household surveys (especially panel data) and cheap computing power. Much of the low-hanging fruit involving publicly available economic data has already been picked. There's the temptation to start analyzing criminal justice statistics or surveys typically used by political scientists or sociologists.
As for mortage fraud, doing "cheater" analysis depends on actually having data. Banks are not going to give their lender databases to a bunch of economists just like that, especially if they know what it will be used for. That said, Asim Khwaja showed there was massive corruption in Pakistani state bank loans granted in the 1990s. He managed to show this by getting loan records from the central bank after Nawaz Sharif was booted out of office by Musharraf for, among other things, accusations of corruption. So it helps to be able to get data from someone with an agenda.
Posted by: Ricado at May 12, 2008 9:43:27 PM
Steve,
Glad to hear that you are so opposed to people talking about areas in which they have so specialty or training in. I hope that means you will stop criticizing econometric techniques and empirical specification choices that you have no knowledge whatsoever about.
Of course you have pet theories about race, marriage, crime and genetics all things which you have absolutely no training or expertise in. Every time you write an op-ed or blog post exposing your made-up explanation about whatever the topic of the day is you are committing the same sin that you always accuse economist of. The thing is that at least these economist have the skills to rigorously test the theories they come up with, you don't even have that.
Posted by: goodnessOfFit at May 13, 2008 10:27:39 AM
Steve,
Glad to hear that you are so opposed to people talking about areas in which they have so specialty or training in. I hope that means you will stop criticizing econometric techniques and empirical specification choices that you have no knowledge whatsoever about.
Of course you have pet theories about race, marriage, crime and genetics all things which you have absolutely no training or expertise in. Every time you write an op-ed or blog post exposing your made-up explanation about whatever the topic of the day is you are committing the same sin that you always accuse economist of. The thing is that at least these economist have the skills to rigorously test the theories they come up with, you don't even have that.
Posted by: goodnessOfFit at May 13, 2008 10:28:32 AM
