« Not a small detail -- Department of Whoops | Main | What does the Dale and Krueger education paper really say? »
Assorted links
1. Blog of Italian economists, sometimes in English.
2. Do recommender systems produce on-line monoculture?
Posted by Tyler Cowen on March 23, 2009 at 11:07 AM in Web/Tech | Permalink
Comments
I certainly won't claim I fully understand the analysis of "online monoculture", but ISTM that the models described fail to differentiate between recommender systems for products where there really is a hierarchy of value (e.g., computers, consumer electronics) and those where there isn't one (e.g., books, music).
If online recommender systems say that Vendor X's newest plasma display is the best value you can get, and this causes a clumping of choices by consumers, I don't really see a problem. After all, that generally means that all the other plasma display vendors will have to work harder to earn my dollar the next go-round...and there will be another clump, different than the first one.
Posted by: David Hecht at Mar 23, 2009 11:53:20 AM
Re: Do recommender systems produce on-line monoculture?
The whole construct in the linked post rests on implicit assumption that moving from 48 customers and 48 products to millions of customers/products spread over multitude of social/cultural strata (not to mention emergence of smart/AI recommender systems) will not introduce factors rendering the entire thesis incongruous.
Yet another example of how economists keep getting Nobels and tenures while real economies keep veering and swerving clueless of their theories.
Posted by: Ozornik at Mar 23, 2009 12:54:06 PM
From #2: So I wrote a program that does basically what they do in their paper and tweaked it to highlight this result.
But a priori, we don't know how a particular recommendation algorithm will impact a particular media market.
Posted by: Cyrus at Mar 23, 2009 1:05:02 PM
The link to the english posts from italian economists is this:
http://www.noisefromamerika.org/index.php/articles/c18
You can find the rss feed there.
Posted by: Andrea Moro at Mar 23, 2009 1:20:09 PM
Here here. Finally some common sense from planners.
Posted by: Doug at Mar 23, 2009 2:51:57 PM
The Virginia regulations seem not aimed at cul-de-sacs per se, but rather at subdivisions with single entrances and exists. Out in sprawl land around here, many such subdivisions exist (with only one or two entrances), but the streets within the subdivisions often have at least a partial grid configuration (there's no real incentive to avoid the grid completely if the roads don't actually lead anywhere outside the subdivision).
Posted by: Slocum at Mar 23, 2009 3:09:08 PM
David:
The point of recommender systems is not so much to snoop out the power drill that works best for everybody but to match each customer with the product that best fills their individual needs. In addition recommender systems are employed in very large product spaces where each individual customer can only survey a small fraction of the space but (it is hoped) the population of users can illuminate the whole space. There are two potential problems with recommender systems: first, signaling a preferred choice leads to increasing returns for that choice. Items that are more visible might attract more attention simply because they're more visible, not because they're better than other (new or insufficiently visible) alternatives -- the product space never gets sufficiently illuminated and product-customer matching is suboptimal. And second, in some markets monoculture is detrimental even if the "mono" choice is in fact the best alternative, since lack of diversity has a negative effect on product development.
That said, these two little simulations don't come near establishing that monoculture is in fact a serious problem in online markets. They only shows how it could occur. So there is as of now no reason to distrust our gut instincts based on their findings.
Posted by: ogmb at Mar 23, 2009 3:15:20 PM
The recommender systems post is mine - thanks for the comments.
Ozomik - "The whole construct in the linked post rests on implicit assumption that moving from 48 customers and 48 products to millions of customers/products ... will not introduce factors rendering the entire thesis incongruous". I disagree: I'm not saying that this model captures everything about the real world. I am saying that even in a simple little case like this there can be unexpected results that cast doubt on the oft-heard mantra of "recommender systems help us deal with diversity and promote niche markets".
Cyrus - "we don't know how a particular recommendation algorithm will impact a particular media market" Agreed of course. But that doesn't stop people making general claims about the benefits of recommender systems, and I'm just trying to cast doubt on that.
ogmb - "So there is as of now no reason to distrust our gut instincts." I agree with much of what you say, but surely the burden of proof is on those who claim that recommender systems are a tool for promoting diversity?
Posted by: tom s. at Mar 23, 2009 4:02:13 PM
tom:
I'm not aware of a specific claim that recommender systems promote diversity, so let me try to formulate my own: Recommender systems do not promote diversity per se, online markets do (by increasing the accessible product space over brick-and-mortar stores). Recommender systems concentrate the product space to facilitate choice, so if the product space is invariant between online store and offline store, it is possible that concentration of sales goes up through recommender systems, but this fails to take into account the purpose of the recommender system and its effect if product space differ between online and offline store. I agree that it is up to the originator of such a claim to validate it empirically, but I believe your simulation misses out on the core driver of diversity of a recommender system: that it allows the customer to survey a much larger product space than an offline store.
Posted by: ogmb at Mar 23, 2009 4:27:35 PM
ogmb - The Long Tail is one widely-read source of claims that recommender systems promote diversity (they are listed as one of the "three forces of the long tail" or some such. Others from Clay Shirky to Yochai Benkler refer to collaborative filtering and recommender systems as, at least, positive forces for good and for a broader, more participative culture.
I hope to post something else soon that should address your second point. It's certainly one of the appeals of recommender systems, but I'm not convinced that they do it well - at least, comparing one big online store to the several smaller physical stores people typically frequent.
Posted by: tom s. at Mar 23, 2009 6:46:17 PM
There is a simple solution regarding the costs of maintaining cul-de-sacs. Privatize the roads. I live in a small cul-de-sac development - seven houses. We went to the county and changed our street to a private road; that allowed us to gate the entrance and we use homeowner association fees to maintain it. The cul-de-sac and gate meant that our children were able to safely play in the street, riding bikes, playing whiffle ball, street hockey, etc. while they were growing up. And our new neighbors - with young triplets - will enjoy the same safety. Developers could certainly offer this alternative to Virginia counties, couldn't they? However, it appears from the story that the cost argument may just be a cover for a collectivist desire to minimize privacy and force "community" on homeowners.
Posted by: allen dalton at Mar 23, 2009 9:26:20 PM
The cul-de-sac article contains the sentence The system forces drivers to enter these traffic-choked roads to go even 50 yards or so to the neighborhood coffeehouse or elementary school.
If they are too lazy to walk 50 yards, I say let them starve.
Posted by: doctorpat at Mar 23, 2009 9:57:25 PM
allen dalton:
did you read the article? VA is privatizing the roads. That's all the law does: VA won't maintain new cul de sacs.
Posted by: Douglas Knight at Mar 24, 2009 12:31:42 AM
I grew up on a cul-de-sac, and liked it up until my early teens. Actually, it wasn't the cul-de-sac itself that bothered me -- I didn't mind using the low-speed collector streets that routed to neighboring subdivisions, but there were some major gaps in the network. I once made an attempt to ride my bike to the local mall while avoiding all busy arterials, and found it to be impossible. Too many subdivisions did not connect to each other.
As a cyclist who currently lives in a urban area with a grid layout, I have no trouble navigating my way through the city on 25 mph (and sometimes 35 mph) streets. I would not be able to do this in many suburbs, especially those designed since the 1950s. I am not one of those cyclists who thinks that we require much specialized infrastructure, nor am I an evangelical new urbanist who thinks that the grid plan should be imposed upon the suburbs. However, I do think a minimum level of connectivity between subdivisions by way of at least one low-speed street in each direction should be required. It should be possible to walk and cycle throughout an urban area without taking the lane on a 65 mph arterial. To be honest, I mainly advocate this because it is in my own personal interest (which doesn't bother me, as politics is more about competing interests than abstract moralistic ideologies). I don't think that walkability or bikeability is very high on the priority list of most suburbanites, and I might not be able to convincingly press my interests were I to live in such an area. Unless the popularity of cycling ever overflows from the cities into the burbs, nobody is likely to push for it (although city planners are pushing home the congestion and emergency vehicle access arguments).
Posted by: chriswnw at Mar 24, 2009 6:52:50 PM
Also, by "subdivision", I specifically meant "superblock". I don't think that every private community or development should be a thoroughfare, but that it should be possible to get through every superblock without using the arterials.
Posted by: chriswnw at Mar 24, 2009 7:00:43 PM
