Oil prices and time

Allow me to quote the ever-intelligent Arnold Kling:

In forecasting oil prices, I tend to defer to the efficient markets hypothesis. In some sense, oil in the ground has to compete with bonds and other interest-bearing assets. So, a reasonable approximation is that oil prices should be expected to go up at the interest rate. So, if the interest rate is 5 percent, then oil prices should go up at 5 percent, plus something for storage cost. If people thought oil prices were going to rise faster than that, they would keep the oil in the ground. If they thought that oil prices were going to rise more slowly than 5 percent (if that is the interest rate), then they would sell oil and buy bonds.

So my tendency is to assume that markets are efficient and predict that oil prices will rise at the interest rate.

This is called the Hotelling rule. But hey, didn't Julian Simon tell me that most natural resource prices, in real terms, are falling over time? And surely the evidence is on his side. What gives?

Is oil exempt from Simon's prediction? (How do VCRs fit into the picture?, the gentle reader might ask...)

Do markets systematically underestimate the rate of technical change, and thus the rate of price decline?

Or are falling prices an equilibrium because arbitrage-ready inventories are already essentially zero (no more selling is possible and the Saudis won't pump more), and natural resource costs of production are falling rapidly as well? In other words, yes prices are falling but you can't sell tomorrow's oil today at a profit. Prices will be lower tomorrow but costs of production will be lower too. Good enough, but then the Hotelling rule doesn't hold at the relevant margin.

I've never had good answers for these puzzles. Here is one useful survey of some models and the literature, scroll down a bit for the most relevant parts. Try this article as well. This literature suggests that most of the standards "outs," such as changing costs of production, or new discoveries, don't square the circle for anything other than the short-run. Holding oil still ought to yield (risk-adjusted) market rates of return, unless again investors are fooled into underestimating how much prices will fall.

My take: In my gut I don't believe in the Hotelling rule. But I couldn't tell you why not; I do understand that the arbitrage conditions of the model are fairly simple. If you wish to create some disagreement over lunch, raise this question with some economists you know. In the meantime, if you're not confused, it means you don't know what is going on.