Marginal Revolution

The Impact of No Child Left Behind on Student Achievement

Preliminary results are in, and they suggest it has helped with math skills but not with reading achievement, as measured in the 4th and 8th grades.  Via Thomas Dee and Brian Jacob:

The No Child Left Behind (NCLB) Act compelled states to design school-accountability systems based on annual student assessments. The effect of this Federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school-accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of 4th graders (effect size = 0.22 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in 8th grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased reading achievement in either 4th or 8th grade.

That is from an NBER paper, I do not yet see an ungated copy on-line.  To my skewed perspective, this is an intuitive result.  Math skills are more the result of drill, whereas you have to learn how to love to read and much of that happens within the family, not at school.  Math is therefore easier to "teach by central planning," so to speak.

The "paradox of choice" is not robust

I missed this one while traveling, so I am grateful to the loyal MR reader who pointed it out to me:

... the psychological effect may not actually exist at all. It is hard to find much evidence that retailers are ferociously simplifying their offerings in an effort to boost sales. Starbucks boasts about its “87,000 drink combinations”; supermarkets are packed with options. This suggests that “choice demotivates” is not a universal human truth, but an effect that emerges under special circumstances.

Benjamin Scheibehenne, a psychologist at the University of Basel, was thinking along these lines when he decided (with Peter Todd and, later, Rainer Greifeneder) to design a range of experiments to figure out when choice demotivates, and when it does not.

But a curious thing happened almost immediately. They began by trying to replicate some classic experiments – such as the jam study, and a similar one with luxury chocolates. They couldn’t find any sign of the “choice is bad” effect. Neither the original Lepper-Iyengar experiments nor the new study appears to be at fault: the results are just different and we don’t know why.

After designing 10 different experiments in which participants were asked to make a choice, and finding very little evidence that variety caused any problems, Scheibehenne and his colleagues tried to assemble all the studies, published and unpublished, of the effect.

The average of all these studies suggests that offering lots of extra choices seems to make no important difference either way. There seem to be circumstances where choice is counterproductive but, despite looking hard for them, we don’t yet know much about what they are. Overall, says Scheibehenne: “If you did one of these studies tomorrow, the most probable result would be no effect.”

That's by Tim Harford.  In my view, the so-called paradox of choice is one of the most overrated and incorrectly cited results in the social sciences.  The full account is here.

Sweet Trade

Here is a fun, easy and effective experiment that instructors can use to illustrate the gains from trade.  The instructor puts chocolate bars ("fun-size") or other candy in bags, one bag for each student. (Alternatively, you can use the type of small items that you can find at a dollar store.  Filling the bags is where the most work comes in especially if you have a large class). Students open the bag and are then asked to write down how much they would be willing to pay for the bag's contents.  But before snacking, students are allowed to trade.  After a few minutes of trade, ask the students to write down their valuation again.  Voila!  Gains from trade.  With a few numbers pulled at random from the students you can do a back of the envelope calculation for the total increase in value.  The experiment doesn't take long and the students will appreciate the candy!

A hat tip to Randy Simmons who first introduced this experiment to me.

The lessons of "Climategate"

I've had many readers emailing me, asking what I think of the "trove" of emails unearthed from climate change researchers.  I'll admit I haven't read through the rather embarrassing revelations, I've only read a few media summaries and excerpts.  I see a few lessons:

1. Do not criticize other people in emails or assume that your emails will remain confidential, especially if you are working on a politically controversial topic.  Ask a lawyer about this, if need be.  "Duh," they will say to you.

2. The Jacksonian mode of discourse, or mode of conduct for that matter, can do harm to your cause, especially if you are otherwise trying to claim the scientific high ground.

The substantive issues remains as they were.  In Bayesian terms, if it turns out that many leading scientists do not practice numbers one and two, I am surprised that you are surprised.  It's very often that the scientific consensus "sounds that way."

In other words, I don't think there's much here, although the episode should remind us of some common yet easily forgotten lessons.

I should add that this episode will seem very important to you, if you conceive of the matter in terms of the moral qualities of "us vs. them."

Addendum: Robin Hanson offers a similar opinion.  I wrote my post before reading his, yet we come to the same conclusions I think.

In praise of Robin Hanson

My fondest memory of Robin Hanson is when we interviewed him for a job and, during his on-campus visit, I gave him some papers I had been working on.  Later he emailed me back, before getting the offer I might add, and told me the papers weren't very good and what was wrong with them. 

Ten years later, as his colleague, I disagree with Robin on many topics, including futarchy, whether we will become computer uploads, and meta-ethics (oh, if Robin would only advocate the ethical theory he so consistently lives by!..instead of his contorted contractarian version of preference utilitarianism, which he sometimes calls "dealism.").  Despite our disagreements, Robin and I are oddly in frequent common agreement on practical "life topics."  Most of all, I view Robin as a reductionist thinker to a greater degree than I am comfortable with for myself; relative to Robin, I'm more attached to the mumbo-jumbo of the mess and the piling on of multiple perspectives to the point of squishiness.

Those of us who speak regularly with Robin know how brightly his star blazes.  He's a truly original and important thinker in a way that few are, plus on analytic back and forth he is blindingly fast and accurate.  But you can't expect him to be a "I'm going to agree with him all around" kind of guy; he isn't.  If you are one of his detractors, or even just a common sense skeptic, you can always find many of his beliefs to be outright absurd,  The real question, however, is how much you can learn from him and on that he is an A+.

Addendum: Robin responds.

MR vocabulary guide

1. "Self-recommending": the very nature of the authors and project suggest it will be good or very good.  This also often (but not always) means I haven't read it yet.  I am reluctant to recommend *anything* I haven't read, but I am signaling it is very likely recommendation-worthy and I wish to let you know about it sooner rather than later.

2. An "Assorted link" that ends with a question mark: Worth thinking about, but I wish to distance myself from the conclusion and the methods of the study, without being contrary per se.

3. Hansonian: of, or relating to Robin Hanson.  Yesterday I asked Garett Jones whether his date was as pretty as Robin is smart.

4. The Jacksonian mode of discourse.  I am opposed to this.  Political and economic pamphlets in the Jacksonian era were excessively polemical and sometimes the Jacksonian mode is still used today, in 2009, believe it or not.

5. Wunderkind: Take the average age of that person's relevant peers.  If said person is either under twenty or less than half that average, that person may qualify for "Wunderkind" status. 

6. Markets in everything: Some of these are celebratory but many of these are sad or tragic.  Usually I am trying to get you to think about -- as a philosophical question -- why the market exists at all and not whether it should be legal.  

7. Tyrone is my brother and alter-ego who believes the opposite of what Tyler believes.  Trudie offers personal advice.  Neither has good time management skills and thus they don't write very much these days.

8. "Shout it from the rooftops": What to do with wordy, obscure truths which the world badly needs to learn.

What have I left out?

How do you convince someone to stay away?

The feverish but resilient Megan McArdle refers us to a problem in signaling theory:

Slate ponders how to communicate the danger of radioactive waste to the far future.  The problem is, if they can't read English, or recognize the radiation trefoil, anything you do sounds more likely to intrigue future anthropologists than to warn them off...

Juliet Lapidos at Slate writes:

Even if future trespassers could understand what keep and out mean when placed side by side, there's no reason to assume they'd follow directions. In "Expert Judgment," the panelists observe that "[m]useums and private collections abound with [keep out signs] removed from burial sites."

...Likewise, a scavenger on the Carlsbad site in the year 12,000 C.E. may dismiss the menace of radiation poisoning as mere superstition. ("So I'm supposed to think that if I dig here, invisible energy beams will kill me?") Hence the crux of the problem: Not only must intruders understand the message that nuclear waste is near and dangerous; they must also believe it.

How can we solve this problem?  Similarly, if an attractive woman tells you "You don't want to go out with me" do you believe her and act on that?  What other problems have this structure?

Only in economics are floors above ceilings!

Only in economics are floors above ceilings!  It might be better to say "a minimum allowed price above the market price" and "a maximum allowed price below the market price," although that is a bit of a mouthful.  I find that the floors and ceilings language does work, however, if the instructor explicitly points out the oddity of floors above ceilings!  In that case, students find the distinction memorable.

Chindogu, making the simple complicated

Today's bizarrely fascinating cultural nugget from Japan: Chindogu. Literally translated as "weird tool," Chindogu is the Japanese art of creating deliberately complex devices that solve simple everyday problems.

Here is one example:

The Dumbbell Phone

People cite "lack of time" as the number one reason they don't work out more. With the dumbbell phone, that's no longer an excuse. Great for bulking up at your otherwise worthless telemarketing job, this phone will have you shaped and sculpted in no time.

This phone also makes a great gift, especially to that parent, friend, or girlfriend who's been known to talk your ear off on the phone. It's subtle, but effective, especially for those with weak arms.

You'll see a photo here, along with a discussion of other ideas, such as using "solar power" to light your cigarette or a fan to cool off your hot noodles.  The "grid-backed" shirt helps you tell your partner where to scratch your back.  It's a trend:

There's the International Chindogu Society, the Ten Tenets of Chindogu (Number Three: "Inherent in every Chindogu is the spirit of anarchy"), and scores of websites devoted to tracking the newest, and most ridiculous, Chindogu inventions.

Umberto Eco on lists

The list is the origin of culture. It's part of the history of art and literature. What does culture want? To make infinity comprehensible. It also wants to create order -- not always, but often. And how, as a human being, does one face infinity? How does one attempt to grasp the incomprehensible? Through lists, through catalogs, through collections in museums and through encyclopedias and dictionaries. There is an allure to enumerating how many women Don Giovanni slept with: It was 2,063, at least according to Mozart's librettist, Lorenzo da Ponte. We also have completely practical lists -- the shopping list, the will, the menu -- that are also cultural achievements in their own right.

...At first, we think that a list is primitive and typical of very early cultures, which had no exact concept of the universe and were therefore limited to listing the characteristics they could name. But, in cultural history, the list has prevailed over and over again. It is by no means merely an expression of primitive cultures. A very clear image of the universe existed in the Middle Ages, and there were lists. A new worldview based on astronomy predominated in the Renaissance and the Baroque era. And there were lists. And the list is certainly prevalent in the postmodern age. It has an irresistible magic.

...We like lists because we don't want to die.

Here is much more.  Make sure you read the quotation under the photo; I don't want to reproduce it on a family blog. 

I wonder if this interview was translated from some other language, given the difference between "lists" and "enumeration."  Here is an important MR post: Jeffrey Lonsdale writes.

I thank Cardiff Garcia for the pointer.