« The eleven best foods you aren't eating | Main | Why Chinese pollution is such a tough problem »
Has "The Long Tail" been refuted?
Prof. [Anita] Elberse looked at data for online video rentals and song purchases, and discovered that the patterns by which people shop online are essentially the same as the ones from offline. Not only do hits and blockbusters remain every bit as important online, but the evidence suggests that the Web is actually causing their role to grow, not shrink.
Here is the summary article. Here is the Elberse paper. Here is Chris Anderson's response. Overall I cannot call this one for Elberse. If you take a genre as given, the web looks less revolutionary but part of the long tail is the creation of new genres. We have blogs now, for instance, and we didn't fifteen years ago, even though blog readership is quite concentrated among the top sites. Or maybe the "Quickflix rental distribution" isn't so skewed to the left (the least-rented titles aren't so popular) but where were Quickflix, Netflix, and other such services fifteen years ago?
Static estimation by deciles and related measures is often misleading since in part the "long tail" effect is to make the top deciles thicker than before, not necessarily to raise the status of the bottom decile relative to the top. In his response, Chris Anderson nails this point:
The best example of this is in what she describes as a growing "concentration" of sales around a relatively small number of blockbuster titles. In the Rhapsody data, she finds, the top 10% of titles (out of more than a million in that data sample) accounted for 78% of all plays, and the top 1% account for 32% of all plays. That sounds pretty concentrated around the head, until you reflect, as she notes, that "one percent of a million is still 10,000--[...]equal to the entire music inventory of a typical Wal-Mart store."
Nor does showing that most of the sales are in the top of the distribution refute the claim. Arguably it is the middle tail which is suffering and the long tail, and the best sellers, are growing in import. That seems compatible with Anderson's core thesis. The long tail hypothesis may be oversold but the data in the Elberse piece don't really dent it.
Elberse wants to define the Long Tail hypothesis as claiming there is more money to be made in the niches than in the blockbusters; while I believe you might find a quotation to that effect from Chris Anderson the more general idea is simply how important the niches are becoming. Elberse concedes a lot at one point:
It is undeniable that online commerce has significantly broadened customers’ access to products of all varieties, including the most obscure. However, my findings suggest that it would be imprudent for companies to upend traditional practice and focus on the demand for obscure products.
You could have rewritten that as "The Long Tail hypothesis is basically true, just don't sell to the Long Tail alone." On that we should all be able to agree.
Posted by Tyler Cowen on July 3, 2008 at 07:18 AM in Web/Tech | Permalink
Comments
You could have rewritten that as "The Long Tail hypothesis is basically true, just don't sell to the Long Tail alone." On that we should all be able to agree.
Why? If you're the only company in a genre, it's foolish to sell only to the tail. But if there are established incumbents who're doing a good job of selling to the head, selling to the tail alone could be a perfectly sensible business model. It all depends on the structure of competition, not just on the structure of demand.
Posted by: James Grimmelmann at Jul 3, 2008 7:56:47 AM
I don't disagree with anything you say here. But on my view, the more interesting question is whether or over what domain power laws provide a good estimate of demand.
If demand is integrated up from a poisson distribution (i.e., from a random sequence of consumption events), then the long tail is an exponential rather than an algebraic decay. That in turn will have a very real impact on marginal production.
Posted by: Michael F. Martin at Jul 3, 2008 8:01:12 AM
I should add that there's no reason that I know of to believe that information is different from other goods in terms of patterns of consumption. Information obeys the same laws of physics. What spooky mechanism would give rise to infinite consumer surplus for certain types of information (i.e., the fat head of a power law)?
Posted by: Michael F. Martin at Jul 3, 2008 8:03:43 AM
Brynjolfsson Hu and Simester compare concentration online and offline for a retailer that sells the same product through both channels. See http://papers.ssrn.com/sol3/papers.cfm?abstract_id=953587
The study finds that there is significantly less concentration online (aka the long tail effect). Because the retailer has an identical product selection in both channels, this controls for the effect of increasing the number of products.
Posted by: a student of economics at Jul 3, 2008 9:12:24 AM
James,
The long tail is only applicable to situations like online, where you can drastically chop the labor, inventory, premises rental and overhead costs associated with brick and mortar stores.
In this situation, if you've spent the time and effort to put the software and systems and automation into place to be able to sell millions of items in the long tail, how hard would it be to add the 10,000 or so bestsellers? Why on Earth would you omit them and risk turning away dissatisfied customers? On the contrary, you'd probably make them loss leaders.
Merchants often specialize by genre (say, science fiction books), but why would they try to specialize by selling only the long tail? That would only make sense if you ran, say, a used bookstore where people went to browse lazily and serendipitously discover obscure books on random topics, rather than a store where people go to with a specific purchase in mind. But usually when we refer to the "long tail", we refer to stores that sell substantially the entire long tail (iTunes, Amazon.com, etc) rather than a limited assortment of random, unclassified, unsearchable obscure items.
Posted by: at Jul 3, 2008 9:15:48 AM
Agree 100% with James Grimmelman. It's not necessary for the long tail to form a significant percentage of the overall market for it to be a potentially good business strategy for a small niche producer to focus on a small niche market. And this can be done much more easily via the internet than it could previously.
I would bet, by the way, that proof of this could most easily be found in what is too politely termed the "adult entertainment" industry, where certain "niches" can be small indeed and yet profitable if catered to... (Hoping to avoid any spam filters here :-)
Posted by: Alex R at Jul 3, 2008 9:19:46 AM
My theory isn't to call it for one or the other. The internet is communication. It also reduces the cost of transaction and system design and automation.
This will increase the network effect as well as increasing the chance of finding that one great work which is popular because it is good. It also reduces the cost of finding that work that is good but obscure. The internet reduces the cost of getting those things everyone want as well as those things only you and a handful of others might want. Since the costs of the latter have been higher, I'd side with the long tail.
The internet provides more opportunity for both the popular and enigmatic, what does that mean? It is both too early to measure the effect of the internet and too late. Too early because once everyone has everything, then they will focus more on individualization. It's too late because you are right that you can't measure something with and without the internet. The first to benefit is that which is already popular.
In politics, the Ron Paul phenomena is evidence of the network effect of the internet. Barack Obama is not proof of the lack of a long tail. They both benefit from the internet, Ron Paul benefitted moreso on a percentage basis.
Posted by: Andrew at Jul 3, 2008 10:34:14 AM
Alex R,
Niche marketing of specialty items isn't the same thing as "long tail". Long tail is basically a sum of all niches, unlimited shelf space. You don't specialize, because you can offer everything.
Posted by: at Jul 3, 2008 1:07:41 PM
I can give you one concrete meatspace versus online example of the long tail. Try finding a booklet (aka "saddle") stapler at Staples or Office Depot or any other store in your city. Now, simply go to Amazon, and because they lengthen their tail with affiliates, you can find a few vendors who stock them and ship them out quickly. Furthermore, by aggregating the rather low demand for this product across the nation, the booklet stapler is even competitively affordable to the more common "long reach" stapler. And since it's available, I can recommend it to people and others can recommend it to other people, and its niche probably manages to maximize itself, although it will probably never break into mainstream.
Posted by: BoscoH at Jul 3, 2008 2:15:16 PM
In practice, the long tail model does predict certain kinds of consumption very well. I speak from experience analyzing view data at Veoh where, except for the very few top titles, a unit power law described number of views pretty well.
Economically speaking, though, you have to factor in a few kinds of friction. These are reasonably modeled as per unit sale costs and per unit of stock costs. The per unit sale costs don't affect this distribution, but the per stock unit costs do. If you draw the classic Zipf curve and assume that revenue is proportional to consumption for all items, then a per stock unit cost offsets the curve vertically. The resulting profit curve tells you pretty much immediately how things will fall out.
A good example of a per stock unit cost is given in p2p download schemes. The first few downloads of each item have to be paid for by the original source while additional downloads make use of sunk network resources that apply minimal marginal cost back to the source. For systems like bit-torrent, you need more than a hundred downloads before you get to high p2p efficiency which makes the system useful for mainstream content and less useful for body and tail content. The Veoh p2p system has a much lower threshold and is thus useful far down into the tail. Either system, though, leads to an economic return different from the theoretical zero-cost long-tail model.
Posted by: Ted Dunning at Jul 3, 2008 4:25:19 PM
You mean like this one? You can get those at a regular Staples by special ordering it. Like you can in a meatspace bookstore.
Posted by: Mo at Jul 3, 2008 4:34:19 PM
Nope. I mean like this.
Now, say you can special order it from a Staples store. See how that works for you compared to Amazon.
Posted by: BoscoH at Jul 3, 2008 11:41:27 PM
I used to research a closely related issue, and did in fact invest in the Long Tail, since it was appropriate for what I was doing.
Some explanations and thoughts here.
Posted by: Jon Kay at Jul 4, 2008 2:44:05 AM
