« Wolfowitz at the World Bank | Main | The cultural foundations of capitalism »
The NBA study of referee bias
As a response to Justin Wolfers and Joseph Price, the NBA financed a study supposedly showing there is no racial bias in refereeing. Here is a WSJ analysis of that study. Here is part of what they found:
Columbia University statistician Andrew Gelman, who has blogged about the Wolfers-Price study and participated in a conference call with Segal and me, said, “What the statistics tell you is that there’s a pattern in the data that’s not explainable by chance.” University of California-Irvine statistician Hal Stern told me the NBA’s study “can’t be said to disprove the Price-Wolfers analysis.”
Meanwhile, the NBA’s study didn’t include players who weren’t called for any fouls, making Segal’s results “suspect,” according to Mr. Gelman. Mr. Fluhr responded, “I’m not sure if you’re looking at non-calls, it would affect the data.” He added that Segal had the data necessary to incorporate such players, but didn’t consider the data relevant, instead only focusing on foul calls. Messrs. Wolfers and Price included all players who appeared in the games they examined.
The NBA does promise to examine non-calls and redo some of the results. I do not think we have yet gotten to the bottom of this, but my "haven't read anything but the initial study" intuition (and Steve Levitt's comments; see also Voxbaby) is that the result of bias will hold up.
Thanks to Chris Masse for the pointer.
Addendum: Wolfers claims the NBA study supports his results.
Posted by Tyler Cowen on May 15, 2007 at 02:58 AM in Sports | Permalink
Comments
Quite apart from sample selection considerations etc,I am not entirely sure Wolfers and Price capture the effect they intent to.
Consider these three facts:
First of all, interpreting the rules of the game is to some extent subjective, a truth every sports fan is familiar with. How much of a push constitutes a foul? How aggressive must a player get before having to be disciplined? How important is intention to foul?
Secondly, black and white players have different playing styles. This is not only due to physical characteristics and body types but also due to the different environments where black and white players learn to play basketball.
Finally, this also goes for the referees and their perception of what constitutes foul play.
Combining the above, it's very likely black referees grew up learning to interpret the rules of the game amongst a majority of black players, and so came to tolerate some 'black' playing behaviour that a white referee would not view so favourably - and vice versa. If this is indeed the case, the results in the Wolfers and Price paper could be driven by this fairly innocuous, colour-blind factor - the authors' research design does not allow us to distinguish between these two hypotheses.
You can read more at
http://bluematter.blogspot.com/2007/05/racial-discrimination-in-nba-maybe-not.html
Posted by: bluematter.blogspot.com at May 15, 2007 3:35:51 AM
Bluematter, I very much agree with you. Although it would be highly subjective, it seems as if they could have put in some measure of how "black" or "white" a player plays. I would guess there are enough cross-overs in terms of playing style that you could detect an effect. Maybe not though - few come to mind.
Posted by: Josh at May 15, 2007 6:44:36 AM
In the Daily Pennsylvanian – which they describe as the independent student newspaper at the University of Pennsylvania – was a story by Emily Babay on the Price-Wolfers study. One might expect this story, entitled “Study indicates racial bias in NBA refereeing”, to simply re-hash the story that appeared in The New York Times. Babay, though, offered a bit more. Nestled inside Babay’s article are the following two quotes from Justin Wolfers:Wolfers, who has read the NBA report, said the league misinterpreted its own study. “Their data shows they actually agree with me,” he said. “Their statistics show there’s evidence of own-race bias.”
Posted by: JC at May 15, 2007 9:41:15 AM
Bluematter,
I think you are presenting a very interesting hypothesis, but I would interpret it as perfectly consistent with the Price-Wolfers interpretation of the results. I don't want to put words into their mouths, but it is likely that they would happily accept a cultural basis for the own-race bias they detect in the data. Much racial bias is likely driven by such differences in voice, speaking styles, and interpersonal interaction patterns across groups.
Posted by: Commenterlein at May 15, 2007 9:56:38 AM
I don't see why people seem to like the Wolfers-Price study so much. Here are my problems with it
(which others elsewhere have brought up):
1) "Calls against whites/blacks" isn't defined properly. If you call a foul against a black player, that
counts as an "anti-black" call, but what if his foul was against a black player? It helped one black and
hurt another. That should count as neutral for purposes to gauging bias, but the study recorded it as
"anti-black".
2) Often times, the best thing a ref can do is ignore a bona fide foul, which would not show up in the data.
If a ref lets blacks get away with fouls as much as he calls them, that would be bad ref-ing but it wouldn't
be anti-black, yet would still show up as anti-black in the method.
Those together create a big enough margin of error to eclipse the ~5% deviation they found.
To do a real study, you have to define "what is a foul". You'd then have to gather reams of clips of called
AND un-called fouls (the second is harder). Then have an evenly-mixed race panel view them. Then have each
give their judgment. Then have them discuss and see if they change their opinion. Have a good representation
of white-on-white, black-on-black, white-on-black, and black-on-white fouls, and see if their is any
variation. THEN compare to the call the refs made in the actual game.
But then ... that would be real work, wouldn't it?
Posted by: Person at May 15, 2007 10:33:28 AM
Why do you think the results will hold up? Check out Table 3 from the paper itself (http://bpp.wharton.upenn.edu/jwolfers/Papers/NBARace.pdf). Just in raw numbers, white refs are substantially more likely to call fouls on WHITE players -- a rate of 4.954 (compared to a rate of 4.330 for white refs and black players). So the only way that the scholars dug up racial bias was by adding in a bunch of controls for height, weight, position, all-star status, etc. Well, adding controls is all fine and well, but when the controls manage to flip the sign from minus to plus, shouldn't you worry about whether they're cherry-picking the controls and then rationalizing after the fact?
Posted by: John Doe at May 15, 2007 11:09:30 AM
Given that centers foul out much more often than guards, no, I don't see wh ywe should especially worry.
Posted by: Barbar at May 15, 2007 11:28:43 AM
How many actual fouls have nothing to do with the race of the referee/player? I'm talking about the end of game fouls when a team is deliberately fouling to stop the other team from running out the clock. And sometimes the foul is obvious to everyone watching (Richardson on Okur the other night). Neither of these types of fouls has anything to do with the race of the referee and the fouling player and would need to be removed to get a truer measure of "racial bias". Now, it may be that removing these fouls helps the Wolfers/Price paper, or it could be that it hurts it.
By the way, is anyone willing to suggest the Utah Jazz are the primary beneficiaries of foul calling? They have Kirilenko, Okur, Harpring, plus their starting "black" players aren't all that "black" - Boozer, Williams, and Fisher have a much lighter skin tone than say Baron Davis or Jason Richardson.
Posted by: AZ at May 15, 2007 12:44:27 PM
I wonder if the fact that many of the white players are Europeans may have something to do with it? The European game is somewhat different than the American game, and may involve less fouls.
Posted by: Foobarista at May 15, 2007 1:19:31 PM
Commenterlein
I am not sure that the paper would have attracted so much attention if it weren't for the moral implications of blatant racial discrimination. If 'discrimination' was seen to be due to an issue other than colour per se, the implications would be far less important. A similar paper demonstrating east coast white refs called more fouls on west coast white players relative to everyone else would never solicit the same reaction from the NBA, the NY times and the wider public.
Posted by: datacharmer at May 15, 2007 2:18:01 PM
Commenterlein
I am not sure that the paper would have attracted so much attention if it weren't for the moral implications of blatant racial discrimination. If 'discrimination' was seen to be due to an issue other than colour per se, the implications would be far less important. A similar paper demonstrating east coast white refs called more fouls on west coast white players relative to everyone else would never solicit the same reaction from the NBA, the NY times and the wider public.
Posted by: bluematter.blogspot.com at May 15, 2007 2:20:40 PM
The most interesting thing about this study is how the NYT chose to spin the results. You might expect them to frame it as implying that refs from the group representing the vastly dominant majority of NBA players were discriminating against the minority of players who only get 17% of the playing time.
But you can see why the Times discarded their normal majority discriminates against minority assumption here. because the majority of players are black (getting 83% of the playing time), and the Times' nurtures a deep hunger to find evidence of The White Man Keeping the Black Man Down (or Black Woman the case may be, as shown by the NYT's enormous enthusiasm for framing the Duke lacrosse team in 2006).
One fascinating finding of this vast study is that the team that plays more black players, as measured in minutes, wins 48.6% of the time. This suggests that there isn't much bias in the NBA, but what there is is in favor of blacks.
Posted by: Steve Sailer at May 15, 2007 3:39:54 PM
The most interesting thing about this study is how the NYT chose to spin the results. You might expect them to frame it as implying that refs from the group representing the vastly dominant majority of NBA players were discriminating against the minority of players who only get 17% of the playing time.
But you can see why the Times discarded their normal majority discriminates against minority assumption here. because the majority of players are black (getting 83% of the playing time), and the Times' nurtures a deep hunger to find evidence of The White Man Keeping the Black Man Down (or Black Woman the case may be, as shown by the NYT's enormous enthusiasm for framing the Duke lacrosse team in 2006).
One fascinating finding of this vast study is that the team that plays more black players, as measured in minutes, wins 48.6% of the time. This suggests that there isn't much bias in the NBA, but what there is is in favor of blacks. But that's of very little interest to the NYT!
Posted by: Steve Sailer at May 15, 2007 3:41:35 PM
You know, I read an interesting response to this study that cited the fact that only 1 of the most highly paid 25 players in the NBA is white. Now, there is surely more than 1 white player in the top 25 (Nowitzki, Nash, and maybe Kidd come to mind). Before I proceed, I want to point out that the response may have more been of the "well maybe refs discriminate against blacks but this is OK because they make more" nature, which I find to be very childish and immature. In any case, this person may have brought up an excellent point; that is, is the NBA a sport that is subject to customer-based taste discrimination against whites? In other words, are fans (the majority of whom are black) more willing to pay more to see an Allen Iverson play partially because of the way he may cater to the black audience in his mannerisms, style, etc. as opposed to a goofier Peja Stojakovic (obviously we would have to compare two players who we think to be extremely similar as indicated by statistics but who only differ in skin tone so this example here is not very valid). Now while this question is of an almost entirely different nature than the one that Wolfers and Price have investigated, discrimination is discrimination..
Posted by: bean14 at May 15, 2007 4:20:07 PM
CNN on the Wolfers NBA paper:
http://bpp.wharton.upenn.edu/jwolfers/media/NBARace(CNN-NBAwrong).avi.
Posted by: Chris Masse at May 15, 2007 5:41:14 PM
What a waste of time...
The NBA is like street ball...no blood, no harm...and if you're called for fouling, of course, it's not a foul - it's race - a black player fouls another black player and the black ref calls a foul and it's whiteys fault. Crap - there aren't enough white players or refs in the NBA to blame!
Why is race responsible for every injustice? And, why is whitey always responsible?
Why aren't there any 5' 10" Guamanians in the NBA? It's gotta be race. There's no other explanation. If there were any, they'd be fouled all the time because of white oppressive refs! I am sure of it!
Posted by: Mike at May 15, 2007 6:25:59 PM
This doesnt need to get into a racial battle. The NBA has the players to do the job. It is not because they are black or white or chinese or whatever. They want the players that will put the points on the score board. Yes, I do agree that some referees are biased toward different teams but not toward the race of the players. They dont have time to discriminate when refereeing. They have to think quickly to make the right call on the situation. I agree in some aspects of the NBA being like street ball because it is a contact sport. What do you think is going to happen in a basketball game? Your going to get hurt, your going to not like calls here and there, but you still play the game. I think its not right to blame the referee calls on race instead of what is really going on.
Posted by: Tyler Garrish at May 21, 2007 10:57:48 AM
Posted by: 鑽石 at Apr 2, 2008 8:21:20 PM
