« But does Natasha like my books? | Main | The forever stamp »
Do football coaches maximize returns?
David Romer says no:
...the behavior of National Football League teams on fourth downs departs from the behavior that would maximize their chances of winning in a way that is highly systematic, clear-cut, and statistically significant. This is true even though the decisions are comparatively simple, the possibilities for learning and imitation are unusually large, the compensation for the coaches who make the decisions is extremely high, and the market for their services is intensely competitive...The departures from win maximization are toward "conservative" behavior...
In other words, too many punts and field goal attempts. But why?
...the natural possibility is that the actors care not just about winning and losing, but about the probability of winning during the game, and that they are risk-averse over this probability. That is, they may value decreases in the chances of winning from failed gambles and increases from successful gambles asymmetrically.
If you take a gamble and it fails, and you have lost for good, it hurts so so bad. Coaches value "being in the game until the end" for its own sake. They are unwilling to give up all sources of hope, even when the associated gamble would maximize their returns.
What does this say about how we run our lives?
That is from the just-arrived April 2006 Journal of Political Economy (I'm sorry Alex, but the JPE is better and more interesting than the QJE, all things considered). Here is an earlier version of the paper. Here is my earlier post on the NFL draft.
Posted by Tyler Cowen on May 4, 2006 at 07:28 AM in Sports | Permalink
Comments
Nice article.
Perhaps "being in the game until the end" is the true value? More exciting games, closer, more fans, better ratings, more talked about. If it's a clear win or loss early in the game, people tune out.
Then again, did anyone ever get fired for winning too many too easily?
Posted by: Lace Anderson at May 4, 2006 8:02:48 AM
In the particular example of fourth-down conversion, somebody--I think it was Bill James--pointed out a possibly rational reason for coaches to be conservative. The argument goes as follows: if a strategy succeeds and the game is won, everyone's happy. The real issue comes when it fails. If the coach did the standard strategy and fails, then hey, it's too bad, but everyone (well, everyone but George Steinbrenner) knows you can't win 'em all. But if the coach does something that is perceived to be "radical" and it fails, then he looks bad and is much more easily Monday-morning-quarterbacked. Even if the probability of winning is higher under the radical strategy, the medium-term expected payoff (i.e., probabability that the coach keeps his job at the end of the season) could be higher under the conservative strategy.
How does this differ from Romer's theory? Romer suggests a risk-aversion based on probability of winning. In my theory, the "default strategy" plays a key role. There is path dependence and an economic moitivation to follow the default strategy. (This is in addition to the much-observed psychological pheoomenon that people do the default, even at significant personal financial costs.) Romer doesn't mention the idea of defaults in his paper but I think that's the next step in studying the phenomenon of conservatism in decision making.
See here for my fuller discussion:
http://www.stat.columbia.edu/~cook/movabletype/archives/2006/05/conservative_de.html
Basically, I think it might help to follow some of the recent "judgment and decision manking" literature and think less about "gambles" abd more about "defaults."
Posted by: Andrew Gelman at May 4, 2006 8:11:41 AM
Gregg Easterbrook in his long-running Tuesday Morning Quarterback column has harped on this point for several years. His conjecture is that coaches are trying to minimize their margin of defeat for job security. It's a lot easier to rationalize a loss by 10 points than a blowout by 30, even if the latter occurs because of an all-out attempt at victory. He also points out the exception of Bill Belichick, the coach of the New England Patriots, who is not coincidentally also one of the most successful coaches.
Posted by: Carl at May 4, 2006 11:21:23 AM
The coaches don't just pay the price of losing a game. The cost of losing a game due to "unconventional" coaching is much higher than losing a game while "making the smart plays." I suppose it comes down to the magnitude of in-game advantage that unconventional (optimal) strategy would confer. If the increase in the probability of winning was high enough, then the possibility of being grilled in the local paper and possibliy losing the coaching job would be worth it (ie, you wouldn't get fired for wining 11 games, even if you lost a few by going for it on 4th and 4).
Posted by: Jason at May 4, 2006 11:43:40 AM
Yet another reason, as if anymore were needed, to hate the NFL.
Posted by: Robert Schwartz at May 4, 2006 11:56:57 AM
I'm generally in agreement with the others; nobody remembers a 4th and 2 that succeeds, but everyone remembers it if it fails, especially if it's considered to have cost the team the game.
If the coaches are trying to avoid putting the game out of reach, they should follow a more "rational" strategy early in the game. You can't generally put the game out of reach in the first quarter. So the question is, do coaches come closer to maximizing expected point differences in the first quarter than the fourth? (Also, if there are thirty seconds left in the game and you're down four, you shouldn't maximize your expected score, you should go for the touchdown -- maximization of probability of winning and maximization of expected point-difference line up better earlier in the game.)
Posted by: Steven Jens at May 4, 2006 12:03:17 PM
This was once called the "no one gets fired for choosing IBM" rule.
Posted by: DK at May 4, 2006 12:16:36 PM
Deadspin has a clip on the risks of going for it on 4th and short...
http://www.deadspin.com/sports/dallas-cowboys/the-unparalleled-genius-of-barry-switzer-168996.php
Posted by: Eric White at May 4, 2006 12:46:56 PM
Bill Belichik has a degree in economics, and is probably the best at managing salary cap issues in the NFL.
However, there was once a head coach who did make decisions based on probability. He was the Seattle Seahawks first head coach from 1977 to 1982. He took a woeful expansion team (2-12 in '77) to 9-7, and challenge for the playoffs, in his third year.
By going for it on 4th and one in his own territory in the first quarter, using numerous fake punts and field goals, and automatic onside kicks if his kicker saw the receiving teams front line cheat by retreating early to form a wedge for the return. He would have warmed the cockles of TMQ's heart.
Almost no one understood his tactics as far as I could tell--Howard Cosell once called him a 'fun coach'--except for former Redskins coach George Allen. Who, doing color commentary for one of the Seahawk games, corrected his play-by-play partner for castigating Patera after a fake FG resulted in an incomplete pass from holder Jim Zorn to PK Efren Herrera: 'Jack sees the strength of his team as his offense and he wants to have it on the field as much as possible.'
He eventually got fired during one of the NFL strikes and retired from coaching.
Posted by: Patrick R. Sullivan at May 4, 2006 12:54:06 PM
I don't think Romer's paper is just about coach preferences; the interesting question remains why the "default strategy" is default in the first place.
Posted by: Richard Phillips at May 4, 2006 2:16:17 PM
Interesting fact: There are "competitive" Madden videogame football players (that is, they play professionally). The number of onside kicks, two-point tries and fourth-down attempts is way higher. The reason, I would imagine, is the same given above: no one would "fire" them for unconventional play.
Posted by: cure at May 4, 2006 5:04:39 PM
Yeah, the answer is the 'default', or path dependence thing. It's all the conventional wisdom, and the backlash from losing while ignoring it.
It annoys the hell out of me, but what are you going to do. Occasionally you see coaches who break some of the extremely inefficient conventional wisdom; Belichek, Gibbs, Cowher, to name a few. But it never seems to stick and spread. You would expect some help from Darwinism, but it isn't enough.
Spurrier's brief NFL tenure was characterized by numerous economically smart 'go-for-its'. It was also characterized by abject failure.
Posted by: BillWallace at May 4, 2006 7:21:32 PM
This adresses a sore spot for me, I love football, but absolutely HATE conservative, lackluster football. I want to see teams go for it on fourth down when its only a few inches. Every time!
After all, its entertainment.
BTW Spurrier, is always interesting to watch, even when he dosent win.
Posted by: kyle n at May 4, 2006 7:45:04 PM
I'm sorry, but Romer's study is pretty close to worthless because he makes a catastrophicly bad assumption:
"Decisions to go for it on fourth down (that is, not to kick) are sufficiently rare, however, that they cannot be used to estimate the value of trying for a first down or touchdown. I therefore use the outcomes of third down
plays instead."
In other words, he didn't actually look at whether or not teams make it on fourth down. What he looked at was whether they make it on _third down_. But any football fan can realize the big difference between, say, third and five and fourth and five. It's easier to pick up, say, two yards on third down than on fourth down, just as it's easier to pick up two yards on first down than on third down.
Why? Because the lower the down, the more tries you have left to get a first down, so the safer it is to try a risky play for a long gain. Thus, the lower the down, the more the defense needs to play back to prevent giving up a touchdown or long gain.
Posted by: Steve Sailer at May 5, 2006 2:25:53 AM
For a more detailed explanation of why Romer's use of third down data to extrapolate to fourth downs gives such absurd recommendations as going for it on fourth and two on your _own_ ten yard line, see:
http://isteve.blogspot.com/2006/05/whats-worse-than-freakonomics-pseudo.html
Posted by: Steve Sailer at May 5, 2006 4:02:18 AM
Steve is right about the methodology of the study, of course.
However, near-goal fourth downs are much clearer a priori, and coaches consistently misplay them in the early game. The rules are such that even the failure to convert a 4th and goal from the 2 carries so much compensating benefit that one's chances for a touchdown would have to be absurdly low to make it not the correct decision. The downside is that you have what is (in other game contexts) dubbed "outstanding field position", with chances for a safety, turnover for a TD, or weak punt. A missed field goal (rare) is FAR worse, and a made field goal balances the 3 points against the far better field position the opponent is likely to have.
I must point out, though, that overly conservative play may have to do less with coaches' perception my the media and owners than with team psychology during a game. The streaking in basketball games is well known to be sufficiently statistically aberrant to be explained by psychology. So a football team might not be able to defend from the opposing 1 nearly as well after a failed 4th and goal as after a great downed punt.
Posted by: J. Goard at May 5, 2006 5:42:18 AM
"Decisions to go for it on fourth down (that is, not to kick) are sufficiently rare, however, that they cannot be used to estimate the value of trying for a first down or touchdown. I therefore use the outcomes of third down
plays instead."
I have to dispute the rarity of fourth-down attempts, too. They aren't common, but with `270 games per season, I'd guess there'd a sample of at least a 1000 4th-down attempts from just this decade (and probably more).
The harrummphing from some of the posters above is just pathetic, especially given the flaws in the study. Can we just in the future assume that famous head coaches who make millions of dollars a year in a glamorous profession are highly intelligent and have in-depth professional knowledge of their job?
Posted by: jult52 at May 5, 2006 7:43:25 AM
"I have to dispute the rarity of fourth-down attempts, too. They aren't common, but with `270 games per season, I'd guess there'd a sample of at least a 1000 4th-down attempts from just this decade "
But these are misleading because of the "cherry-picking" effect. You don't have to go for it on fourth down. Presumably, coaches are somewhat decent at choosing optimal occasions (are you playing the Texans or the Steelers?). This inlfates the conversion stats, because some misses are never attempted.
Posted by: butters at May 5, 2006 12:26:23 PM
I would also point out that Belichick majored in economics at Wesleyan and seems to have absorbed most of the lessons.
Posted by: Gabriel Kaplan at May 5, 2006 12:52:03 PM
Well, during the 2005 season, there were 465 4th-down attempts in the NFL and 223, or about 49%, of them were successful.
There are really two types of 4th-down attempts: Those in which there's not far to go for a first down, which are (theoretically) fairly easy to make and those desperation attempts late in a game that are almost impossible to convert. E.g., a team is losing by 7 points with 22 seconds to play, facing 4th and 10 at its own 25-yard line.
By comparison, 2611 of 6859 third-down attempts were successful. Thats 38.1 percent.
Posted by: Ralph Hickok at May 5, 2006 2:54:47 PM
"Can we just in the future assume that famous head coaches who make millions of dollars a year in a glamorous profession are highly intelligent and have in-depth professional knowledge of their job?"
Of course we COULD assume that. We could also assume that politicians always act in the best interests of their constituants becuase they have to answer to them in the next election. Of course we would be WRONG.
Football coaches are experts in their field for the vast majority of the tasks they have to perform. Choosing when to go for it is not one of them.
re: "conservative play may have to do... with team psychology during a game"
This is a common excuse given, but I don't believe it holds water. I remember, but don't have a link, that there was a study showing that the success rate of drives that start after the previous team failed on a 4th down conversion (removing desperation attempts) is actually slightly lower than normal, though not statistically significant.
As far as the psychology, it can be argued just as easily that a coach is showing faith and confidence in his team and defense by selecting the "riskier" play. And there are some common situations that also preclude the psychology explanation. For example at the end of a game. In the last NFL season there were a couple situations where a team could kick a near certain FG and go to overtime, or try for a touchdown to win. Overtime is close to 50-50 no matter how you slice it, and a Fourth and 1 is statistically around 70% on average, or at least definitely greater than 50%, making the touchdown attempt the obvious best solution. But when the coaches chose to go for it, they were hailed as geniuses, as if they had solved a great puzzle. Some coaches in the past have kicked field goals in these situations.
Posted by: BillWallace at May 5, 2006 5:28:43 PM
Doesn't it make sense for a team that regards itself as superior to its opponent to be more conservative in these situations than Romer's analysis suggests? If you think you're going to outscore the other guys the rest of the way it makes sense to avoid the risk of giving them an unexpected advantage by going for it instead of kicking.
In other words, especially early, it seems you want to reduce the variance of the outcome if you are a heavy favorite, even at the expense of using a non-score-maximizing strategy. The same would apply later on in the game if you are ahead.
Posted by: Bernard Yomtov at May 5, 2006 7:35:21 PM
I think the fairly favorable reception this uselessly flawed study, which absurdly claims you should be indifferent between punting and going for it on fourth and three from your _own_ 10 yard line, has gotten from other economists points out the dangers in the currently fashionable fad for "freakonomic" analyses of seemingly trivial topics like this.
Economists need to avoid being smug about their presumed analytical superiority when condescending to offer advice on topics outside the traditional bounds of their field. It may well turn out that Romer is right that NFL coaches are too risk averse -- that's my impression as well -- but J. Goard's short analysis in the Comments above is probably more useful than Romer's entire analysis.
Posted by: Steve Sailer at May 6, 2006 12:50:26 AM
So David Romer produces a paper in which he takes some simple assumptions, examines whether they seem reasonable and fit the data, works throught the implications of these assumptions using a mathematical model, and derives some surprising conclusions. He discusses the implications of the model and some possible caveats to interpreting it.
Then Steve Sailer looks at it, doesn't really understand the math, but decides he finds some of the conclusions implausible, and publicly calls David Romer an idiot on his blog.
And Sailer accuses *Romer* of being arrogant? Please.
(BTW, if you actually *read* the paper, you'll see that Sailer's criticisms are acknowledged and addressed on page 20.)
Posted by: ed at May 6, 2006 1:49:57 AM
What Romer should do is take his calculations and say, "All right, I admit it, I don't know anything about the chance of making a first down on 4th down. Obviously, my methodology of looking at 3rd down plays is useless for understanding the chances of making it on 4th down, except probably near the goal line. However, here are the expected return in all situations of making it versus not making it. Then, you football coaches should take _your_ estimates of the percentage chance of making it on different 4th down situations, which are obviously better than mine, and multiply it by my estimate of the returns of making it, which are pretty good, and the combined result of our efforts will be useful information."
If Romer really wants to continue to argue that teams should go for it on 4th and 2 on their own ten yard line, there is a source of data available. There are inner city high school football teams that seldom punt or try field goals because they can't persuade players to specialize in kicking. Romer could study them. My impression is that lacking a punter is a huge problem, but I could be wrong.
Posted by: Steve Sailer at May 6, 2006 4:57:07 PM
