« Landsburg at GMU | Main | The economics of strikes, update »
Facts and True Facts: More on Divorce
My initial guest post noted that recent divorce statistics were misinterpreted widely in both the media, and by the academics interviewed by the press. The question is what went wrong with the latest data?
First, some necessary background. This table was published by the Census Bureau counting the proportion of those who had wed in each year who subsequently celebrated various anniversaries. Here’s a quick test: Look at the data, and decide for yourself what is happening to marital stability. Or if you are lazier, let me help with an example: the Census reported that 76.4% of men whose first wedding occurred in 1985-89 had celebrated a tenth anniversary; this declined rather dramatically to 70.0% among those who marrying in 1990-94. By jingo, it looks like recent marriages have become much less stable!
Not so fast. The marital history data were collected from July-September 2004, and hence those who had married in, say, October 1994, simply could not have reached their tenth anniversary by the survey date. Because this affected around one-in-ten of those wed from 1990-94, this statistical factor alone explains what looked like a decline in marital stability.
How do we interpret what happened?
- The Census Bureau reported true and useful facts: The data are interesting, and the table includes a small footnote, noting “Approximately 10 percent of the cohort has not reached the stated age by the end of the latest specified time period. Because of this, estimates for this group for the highest anniversary are low.” With this qualification, one should not conclude that divorce is rising. (But what should one conclude? No guidance is given.)
- The Census Bureau reported true, but useless facts: The tables measure exactly what it says it measures. The Census Bureau is like Fox news: We report, you decide. And we report, even if the number we report is meaningless.
- The Census Bureau reported misleading facts: It is obvious that a qualifying footnote will be ignored by most. Indeed, the New York Times printed the table but omitted the footnote. But let’s not be too harsh on the NY Times: I talked about these data with several excellent economists, and none even noticed the footnote. Headline numbers deserve headline qualifications.
- The survey was flawed: If the Census is interested in measuring the survival of a set of marriages to their tenth anniversary, then failing to wait ten years after a wedding to measure this is a surveying glitch.
So what is the mission of a statistical agency? If the Census’ job is to just report back what we (the surveyed population) tell them, then they performed that task adequately. If their job is to report parameters – useful facts – then they failed miserably, as the data they reported are hopeless biased indicators of marital stability. Alternatively, the question is: Does the Census provide facts, or interpretation? I’m happy if they present only facts and leave the interpretation to experts. But is there an obligation to report only interpretable facts?
Stephen Colbert’s term “truthiness”, the reigning word of the year, refers to what you want the facts to be as opposed to what the facts are. I’m wondering, what is the right word is for something that is a fact but isn’t true? Untruthiness, anyone?
Posted by Justin Wolfers on October 1, 2007 at 07:25 AM in Data Source, Economics | Permalink
Comments
How can anyone miss the footnote? There is a superscript, and with these kind of tables looking at what that refers to is the first thing you ought to be doing. The statistical agency is not responsible for people being lazy, not knowing economics, wishing to draw inferences without the data justifying doing so, etc.
Also, there is no such thing as a fact that isn’t true.
Posted by: datacharmer at Oct 1, 2007 8:18:28 AM
But every entry has the footnote except the first. This means that as long as they're consistent in their methodology, the trends excluding the first are valid.
For instance, both the cited 85-89 and the 90-94 1- yr data have the footnote. So if both are undercounted by 10%, then 85-89 is at most 86.4 and 90-90 is at most 80%.
As long as they're consistent in their methodology, and it appears from the table that they were, it seems perfectly correct to me to infer a 6% drop.
In fact, if we restrict ourselves to just the data with the footnote, 10yr marriage survival went from 85% from 60-64 to 70% in 90-94.
Posted by: Jody at Oct 1, 2007 8:32:05 AM
I should also add that either men or women are lying about being married in the stats or the survey population wasn't well constructed for gender.
Throughout, men are reporting that they've stayed married at a higher rate than women (about 2-4%), which seems impossible to me without either respondent error or poor survey design.
Note this shouldn't affect the downward trend as both genders exhibit it, but it is another interesting question.
Posted by: Jody at Oct 1, 2007 8:38:01 AM
"What is the right word is for something that is a fact but isn’t true"
In the journalism world, it's called "a story."
Posted by: M. Hodak at Oct 1, 2007 8:44:51 AM
Jody wrote:
"But every entry has the footnote except the first. This means that as long as they're consistent in their methodology, the trends excluding the first are valid."
This is actually an example of the problem. Actually, only the bottom number in each column was affected (the "latest specified time period"), and the 5-year anniversary data were unaffected. This means that the data do not represent the trends at all well.
As to the differences by gender, note that men in second marriages can be married to women in first marriages, so the numbers don't need to be exactly symmetric. But measurement error, recall bias, and differential mortality also may cause differences in the data for men and women.
Posted by: Justin Wolfers at Oct 1, 2007 8:53:03 AM
Good call on 1st vs 2nd marriages as the effect is indeed reversed.
Actually, only the bottom number in each column was affected (the "latest specified time period"), and the 5-year anniversary data were unaffected. This means that the data do not represent the trends at all well.?
Ahh. Then looking at the 5 yr, it's level (89.4 vs 89.3 staying within the same group) which is consistent with the numbers I had yesterday (steady 2:1 annual ratio of marriages to divorces).
So while marriage rates are going down, p(divorce) is level.
Posted by: Jody at Oct 1, 2007 9:01:56 AM
Close italics
Posted by: Jody at Oct 1, 2007 9:02:34 AM
Jody, you're confused. Only the LAST anniversary in each cohort is under counted. Comparing, e.g., the 10th anniversary for all cohorts, only the 1990-94 cohort is under counted - the others are counted correctly at their 10th anniversaries.
Also, men can stay married at a higher rate than women. For example, imagine two men and one woman. The woman and one man stay married for 1 year and then the woman remarries the second man and they stay married for 19 years. The first-marriage average for men is 10 years, while for women it's 1 year.
Posted by: Josh at Oct 1, 2007 9:14:07 AM
Wolfers, when is your popular economics book due?
I like the cut of your jib.
Good stuff.
Posted by: Jeff at Oct 1, 2007 9:20:23 AM
closing italics
Posted by: Steven Horwitz at Oct 1, 2007 9:23:26 AM
If you look at the 5, 10 & 15 year data you see what appears to be a strong downward trend until you get to the next to the last observation. The next to the last observations suggest that the downward trend may be ending, so the last observation is important in determining if there is a change in trend. But since the last observation is not comparable, one can not tell if the trend has changed.
The reported data implies that the next to the last observation was out of line and the downward trend is continuing. But we can not really determine that from the data. So looked at this way, publishing the data this way rather then waiting for the full data set raises serious questions about why they published incomplete data rather than just waiting to publish comparable data.
Given the record of this administration in spinning data one has to wonder if political consideration entered into the decision.
Posted by: spencer at Oct 1, 2007 9:53:46 AM
I've posted some additional thoughts over at Liberty and Power.
Posted by: Steven Horwitz at Oct 1, 2007 10:37:27 AM
People who don't marry legally, but have an equivalent arrangement may represent a specific demographic group. If this type of arrangement has become more prevalent then their influence in the divorce statics will change the overall picture.
When such a couple "marries" the event isn't recorded and neither when they "divorce".
So are we interested in those in traditional marriages or those in long-term social arrangements? By focusing only on marriage one could make an argument that the data is to provide fodder for those who see the decline of traditional "morals".
It is a sorry state of the way our country is run then one has to question the motivation (and independence) of those doing any sort of research.
In medicine many journals now require the authors to state their connections to commercial enterprises. Fifty years ago this was unheard of, one got one's funding from the institution where one worked and/or a grant from a government agency like the National Institute of Health.
How is society supposed to progress when objectivity is under question? We end up with things like "argue the controversy" instead of "argue the science". Those societies which shun this corruption of science will make progress and we will be left in the dust.
We can already see the effect in microbiology and genetics where the important work is being done in the UK and Asia, while we argue over whether a blastocyst has a soul. When the Chinese develop a cure for cancer, will they be willing to sell it to us?
If they have it and we don't they have an economic advantage. The cost of a healthy workforce becomes less than hours. Why should they want to give this up?
Posted by: robertdfeinman at Oct 1, 2007 11:39:10 AM
"Given the record of this administration in spinning data one has to wonder if political consideration entered into the decision."
Given that this post is about definitions, does spinning mean reporting something that spencer doesn't like?
Posted by: Tom at Oct 1, 2007 11:48:28 AM
The survey was probably used for many subjects, so timing it for the marriage data would not have made sense. The is a flaw in data presentation. If they had put the superscript next to the faulty data point, providing a visual clue, instead of in the first column it would probably not have been missed. Even better they could have made a statistical correction for the faulty number with a footnote indicating they had done so.
The census should and do provide raw data so experts can provide interpretation. However such data is difficult to work with so they need to provide summaries also. How and what to summarize is interpretation of data just as the news is always an interpretation of reality. Only Fox news claims they can report just the facts.
Posted by: joan at Oct 1, 2007 11:56:00 AM
Related to what Robert Feinman wrote, I wonder if the trend (declining divorce rates) can be explained by rising average age-of-marriage?
I.e., what if you control for age at which the people married?
Do 23-yr-old 1990 brides divorce at the same rate as 23-yr-old 1980 brides?
Speculation:
Seems like it would be possible for age-controlled divorce rates to be rising while overall divorce rates are falling due to rising age of marriage.
Which, perhaps, would mean that Joe Public has an accurate picture of things. I.e., the Divorce Myth is not a myth.
Probably doesn't hold up, but it's a thought.
Posted by: Bob Montgomery at Oct 1, 2007 4:02:00 PM
Oh, I get it. It is like showing, in rewinding mode, a video of an elephant walking; and calling his tail his trunk, and his trunk, tail. I suppose any fact presented intentionally to distort truth should be…simply, a lie.
Posted by: Yan Li at Oct 1, 2007 5:54:40 PM
I think it was mark twain who said that "there are three kinds of lies: lies damned lies and statistics" I have found personally that any "census" or "poll" or "chart" or "graph" presented to us by the media has some sort of spin on it. Its like the big news agencies have an agenda to feed us. They want us to think that there are more divorces or, just as an example, more murders or rapes. The Media trusts the American Populace to be to lazy to get up and find out information for themselves, so they dont feel bad when they pull the wool over our eyes. Nothing they tell us can be trusted
Posted by: John Scott at Oct 2, 2007 4:34:10 PM
癌症 肺癌 肝癌 胃癌 食道癌 宫颈癌 直肠癌 结肠癌 胰腺癌 乳腺癌 脑瘤
Posted by: gfhfgh at Oct 8, 2007 2:38:43 AM
I come from au, injoy 室內設計,now in a 搬家BIZ. i searching for 翻譯社 to trans a word- 離婚,who can help me
Thanks.
Posted by: law at Dec 1, 2007 2:23:00 AM
who know指甲彩繪？teach me some about it
Thank.
Posted by: cai at Dec 1, 2007 2:44:19 AM
I agree about that last observation on the trend issue. I have found some other useful divorce facts that helped me out during my divorce in California.
Posted by: Blake at Mar 27, 2008 6:01:01 PM
