« Strange laws | Main | China sentence of the day »
How important is overcoming bias?
Arnold Kling summarizes Robin's argument:
If you have a cause, then other people probably disagree with you (if nothing else, they don't think your cause is as important as you do). When other people disagree with you, they are usually more right than you think they are. So you could be wrong. Before you go and attach yourself to this cause, shouldn't you try to reduce the chances that you are wrong? Ergo, shouldn't you work on trying to overcome bias? Therefore, shouldn't overcoming bias be your number one cause?
Here is Robin's very similar statement. I believe these views are tautologically true and they simply boil down to saying that any complaint can be expressed as a concern about error of some kind or another. I cannot disagree with this view, for if I do, I am accusing Robin of being too biased toward eliminating bias, thus reaffirming that bias is in fact the real problem.
I find it more useful to draw an analogy with statistics. Biased estimators are one problem but not the only problem. There is also insufficient data, lazy researchers, inefficient estimators, and so on. Then I don't see why we should be justified in holding a strong preference for overcoming bias, relative to other ends.
When I think of a blog that tries to eliminate or reduce bias, say by considering a wide variety of views and methods, I think of Dan Drezner or Matt Yglesias. I view Robin's blog as exemplifying bias, and indeed showing that bias can be very useful, especially if embedded in a broader discovery process with checks and balances. (I would describe Robin's blog as one of the dozen "must reads" out there.) Robin's blog is one very select group of very smart people, pushing one unpopular, specialized, but very interesting and analytically powerful research method as far as it can go.
If I were allowed to retitle Robin's blog (and I am not), I would call it "Reaping the Fruits of Bias."
Posted by Tyler Cowen on August 25, 2007 at 05:44 AM in Philosophy | Permalink
Comments
I also think that "overcoming laziness" is important.
Posted by: Chris Masse at Aug 25, 2007 5:57:45 AM
The bais agains torture is not a utilitarian one of
calculating incrimental pain, but rather a qualitative
matter of drawing a line in the sand beyond which a
regime has descended from civilized practice to barbarism.
Officially, rather than accidentally (ie. prison rape).
Fortunately for us Jefferson spelled out these qualitative
landmarks before his involvement in Epicurianism tempered
his rights theory with eudamonism!
Posted by: Mark Sunwall at Aug 25, 2007 7:00:31 AM
I have to ask since there are about 30 blogs under your "blogs we like" what are the other 11 "must reads"
I've always thought of Megan McArdle as the blogger most concerned about dealing with her own bias.
Posted by: stuart at Aug 25, 2007 7:00:46 AM
Great post.
I agree completely that Robin's blog is interesting, there are smart people there, and they come up with some very valuable insights (as well as some extremely questionable conclusions). I also agree completely that the blog is highly biased and is in no way accomplishing its stated goal of overcoming bias. You are absolutely correct that in order to overcome bias, you have to consider a wide variety of views and methods, and this is where the O.B. blog falls badly short.
I would recommend a title change to "Bayesian Reductionists and Singularity Musings" which seems a far more accurate description of the direction and content of the blog.
I will continue to read it, keeping in mind the new title I have given it, appreciating it for what it is instead of being frustrated at it for what it is certainly not (a forum for the overcoming of bias where people welcome new and controversial ideas and question their deeply held assumptions and beliefs).
Posted by: Matthew C. at Aug 25, 2007 9:01:08 AM
An even cleaner analogy with statistics follows from the relation
mean square error of out-of-sample predictions = (bias of predictions)^2 + variance of predictions
Reducing bias is great, but needs to be traded off against the benefit of reducing variance. Another way of saying that is that you want your model to be correct on average (unbiased) but you also want to make sure that your predictions in a given situation don't depend too strongly on the particular, finite, random data set you use to fit your model.
Often this trade off leads one to prefer simpler, somewhat biased models over more complicated, less biased models.
Posted by: Jeffrey Miller at Aug 25, 2007 9:06:49 AM
They don't seem all that Singularity focused to me. Eliezer has a separate blog where he talks more about that.
Posted by: TGGP at Aug 25, 2007 11:47:58 AM
You once asserted that, while economists was a good topic for a blog, philosophy was especially unsuited. I remember mentally agreeing with you.
Overcoming Bias has caused me to reconsider that.
Posted by: luispedro at Aug 25, 2007 1:18:41 PM
Many bloggers, academics and politicians are not interested in finding out the "truth". They already know the truth. It is what they believe. They are interested in explaining this truth to everyone else and convincing the unconvinced.
Some distort things intentionally, some don't see their own biases (call it self deception) and some are paid to make a case.
In the hard sciences there are mechanisms designed to prevent errors from propagating for too long or too far. Sometimes they work well and sometimes it takes a long time for things to be resolved. The mechanisms are well-known. The procedures a person used to reach their conclusion (hypothesis) are supposed to be shown in refereed published articles. Their experiment or analysis should be repeatable by others and yield the same result. Any findings which contradict the theory mean that it must be discarded or revised.
In the social sciences it seems the mechanisms don't work as well. Data is often incomplete, or used selectively. Many theories can seemingly fit the same data. There are so many factors at work that it is hard to determine which are the controlling ones. Much "research" work is published without the same review processes used in the hard sciences. This means anonymous reviewers, no bias on the part of the journal and its editors and a credible appeals process.
The fact that academic departments in the social sciences and some of the humanities can be seen as belonging to a certain camp or another is a clear sign that the is no intention of following the rigid rules of science.
Posted by: robertdfeinman at Aug 25, 2007 1:22:59 PM
Another approach is to find an analytical market niche that is underutilized for reasons of bias. Mine, for example, is that I'm always on the lookout for the importance of relatives.
Posted by: Steve Sailer at Aug 25, 2007 2:50:32 PM
I like to read OB, but I find that much of the time when the posts regarding ethics or metaphysics claim to be attacking "bias" the authors seem quite disconnected from what I'd call "bias that actually exists" (this is epitomized by the recent post on torture). Even when I agree with the overall points of their posts (what they're affirming), what I wish they'd spend more time on is establishing that there are people who hold the views they are addressing (what they're negating) by, for instance, citing a point (or collection of points) that someone actually wrote and responding to that (those) point(s). But posts on economics are more likely to do that and, perhaps not unrelatedly, are usually top-notch.
Posted by: hrh at Aug 25, 2007 3:58:05 PM
robertdfeinman, you might be interested in this from Jared Diamond where he explains how the "hard sciencies" are really "easy sciences" and the "soft sciences" are "difficult sciences". I came to a different conclusion than (I infer) Diamond wanted me to: I respect the "difficult" sciences less because we get less out of them for the same effort. Perhaps if I believed in the labor theory of value I would respect them more, but I am a staunch subjectivist.
Posted by: TGGP at Aug 25, 2007 5:51:09 PM
There's an interesting example of bias showing up at Sharp As A Marble. Robb issued an invitation to victim disarmament advocates (now you know my bias) to take part in his comment thread for purposes of Rational Discourse. One took him up on it, but cannot conceive of any reason for people not to share his position other than that they are stupid and ignorant, and feels it his duty to point this out. Seek out the last few gun posts with nine or more comments.
Posted by: triticale at Aug 26, 2007 10:15:47 AM
say by considering a wide variety of views and methods
That is far more likely to reduce variance, than to reduce bias. If there is something systematically wrong, then a wide variety of views does not help, and can be a great hinderance. Considering a wide variety of views on economics or on traffic management is utterly useless.
Of course, we could only consider some views (those of the 'experts', say). But that means that we've already drawn firm conclusions as to who's biased and who's not. What is that conclusion based on in the first place?
The truth is not the midpoint between the two talking faces.
Posted by: Stuart Armstrong at Aug 27, 2007 10:10:51 AM
Tyler,
Could you respond to the broader point about allocating resources between cause selection and cause implementation? The former would include attempts to compensate for known cognitive biases, but would also involve study, performing tests, etc.
Given the wide range (orders of magnitude) in efficacy of work on different causes (Katrina relief vs oral rehydration therapy vs the Archon X-prize) if you're going to invest 2000 hours in doing good it seems reasonable to invest 1000 of those hours in evaluating causes through various means.
Posted by: Carl Shulman at Aug 27, 2007 3:45:25 PM
I agree with Chris Masse.
I think my point and (poorly structured) comment I made on Robins post bears repeating:
I think why it is not a highly held value is that it is very resourse intesive. The effort to overcome bias is great and would consume lot of time and energy. As a wealth society, we have the time to do this. But during the evolution of society we had to remain active to simply survive. Taking the time to think things through could cost you your life.Perhaps that is why depression is tied to rational thought. The prospect that there is no god or that life is meaningless isn't of itself good or bad, yet this often leads to suicidal thoughts (the idea being that being to contemplative and not social or productive enough could burden a social group).
In my IOE courses in undergrad, when evaluating projects, due to the time value of money, it is often optimal to have a good solution fast than to take the time to calculate the best solution.
Also, even knowing the truth, reality isn't very certain. Our biases push us to pursue diverse courses of action which improves the likelyhood that some of the courses of action will turn out to be net beneficial.
Posted by: aaron at Aug 28, 2007 8:41:26 AM
(if you spend 1000hrs evaluating causes, now you only have 1000hrs to invest in a cause)
Posted by: aaron at Aug 28, 2007 8:54:07 AM
Aaron, you are suspended from posting in this comment thread (I wish I had that power).
Posted by: Hopefully Anonymous at Aug 31, 2007 10:23:22 AM
Hi Best wishes。Allow me to offer my heartiest wishes.xicao loves-流水线娱乐博客常年提供高、中、低压锅炉钢管、流体钢管、结构钢管、化肥专用钢管、石油裂化钢管、地质钢管、液压支柱钢管及合金钢管-无缝管-无缝钢管等论文发表资讯/刊物信息,协助客户制定论文发表方案
Posted by: 无缝管 at Nov 13, 2007 7:58:02 PM
China Tours and Yangtze Cruises A professional China travel agency offering private & group package tours, tailor-made vacations for families, couples and individual travelers, Yangtze cruises, hotel & ticket booking services.
China Travel and Tours One-stop China travel service delivering China package tours, discount Yangtze River cruises and best rate hotels.
Tibet Train Travel A reliable tour operator specializing in Tibet tour packages, adventures, hotel and train ticket booking.
China Maps A free web-based source for China travel and tour map users
Voyage Chine, Tour de Chine, Circuit Chine - Un agence spécialiste voyage en Chine
Great Wall of China A website for the Great Wall of China.
Posted by: China tour at Nov 27, 2007 2:25:17 AM
Posted by: 翻译公司 at Feb 13, 2008 7:51:51 AM
Mabinogi online gold
Mabinogi money
Mabinogi Gold
cheap Mabinogi gold
buy Mabinogi gold
2moons dil
2moons gold
buy 2moons dil
2moon dil
cheap 2moons dil
Flyff gold
flyff Penya
flyff money
buy flyff penya
cheap flyff penya
cheap flyff gold
aion gold
buy aion gold
cheap aion money
aion money
cheap aion gold
Dofus kamas
buy dofus kamas
cheap kamas
dofus kama
dofus gold
dofus money
Knight Online Gold
Knight Gold
Knight Noah
Knight Online Noah
Posted by: aion at Jul 9, 2009 2:06:37 AM