« Cap and trade vs. carbon tax | Main | Norman Borlaug on the Food Crisis »
Sentences to ponder
As far as I know, there has never been a rigorous ex post evaluation of CGE [computable general equilibrium] models in practice, one that compares predicted to actual outcomes.
Here is the broader argument, hat tip to Mark Thoma (there is more at the link), and comments are open so can you prove him wrong?
Posted by Tyler Cowen on June 4, 2008 at 04:13 AM in Economics | Permalink
Comments
1. The Australian Treasury was given three months to use CGE models to generate estimates of the economic impact of the government's proposed cap and trade carbon sytem. They have asked for another three months, it seems because the scenarios being modelled are so far outside previous experience that their models are blowing up. If the models have not had ex post evaluation, and as well are operating outside the data set, what are the results worth?
2. And does the same apply to climate models?
Posted by: John at Jun 4, 2008 5:11:40 AM
I'd like to know about climate models as well. Also, weren't these used to estimate the outcome of the US invasion of Iraq in 2003?
Posted by: john at Jun 4, 2008 6:02:21 AM
This may sound flip, but I thought that only people who were estimating CGE models actually believed in them. The reason why no one evaluates their predictions is that the models make horrible predictions.
Posted by: Tim at Jun 4, 2008 6:47:21 AM
I don't even think that people who use CGE believe in them. They are looking for *some* answer, put in a bunch of parameters and hope for the best.
What we are seeing in the carbon and climate change debates is a "battle of the CGE models" -- with the winner likely to be the person who ex-post justifies the ex-ante choice of the dominant political group.
You can never "confirm" a CGE model in the same way that you can never confirm a demand function -- it's a tool to make you think about things NOT a description of what's so. (and I could go on about econometrics, etc. :)
Posted by: David Zetland at Jun 4, 2008 8:28:44 AM
Since demand curves are incalculable, how could a general equilibrium be?
Posted by: ideogenetic at Jun 4, 2008 8:42:50 AM
If I am not mistaken, Norway's socialist economist Leif Johansen pioneered this field in the 50's (hoping to facilitate the plan economy) and altough we don't have many communists around here anymore, parts of his legacy still remains within the walls of the central statistical bureau (www.ssb.no).
The interesting part is that their CGE-model for policy impact studies and general forcasting seems to be consistently outperforming any of the large commerical banks best guesses.
Posted by: JBJ at Jun 4, 2008 9:46:17 AM
An evaluation of the performance of applied general equilibrium models of the impact of NAFTA
Timothy J. Kehoe http://minneapolisfed.org/research/sr/sr320.pdf
Abstract
This paper evaluates the performances of three of the most prominent multisectoral static applied general equilibrium models used to predict the impact of the North American Free Trade Agreement. These models drastically underestimated the impact of NAFTA on North American trade. Furthermore, the models failed to capture much of the relative impacts on different sectors. Ex-post performance evaluations of applied GE models are essential if policymakers are to have confidence in the results produced by these models. Such valuations also help make applied GE analysis a scientific discipline in which there are well-defined puzzles with clear successes and failures for competing theories. Analyzing sectoral trade data indicates the need for a new theoretical mechanism that generates large increases in trade in product categories with little or no previous trade. To capture changes in macroeconomic aggregates, the models need to be able to capture changes in productivity.
Posted by: R at Jun 4, 2008 10:22:38 AM
Here is an article in The Economist about the performance of CGE models.
Is the fact that these models have not delivered reliable predictions yet reason enough to ditch them? This is a young research program and people working on it are precisely addressing some of the issues that Dorman raises (for example heterogeneous agents models replacing representative agent models). The question is I think whether this is a promising research avenue or not.
Is this comparable to other research that is not commercially or even physically viable yet and thus has not been "approved" by the market (nuclear fusion comes to mind)?
Posted by: D. at Jun 4, 2008 10:54:47 AM
John asked:
2. And does the same apply to climate models?
The difference between climate models and CGE models is in the nature of uncertainty. In CGE models the largest form of uncertainty is in functional relationships between variables (see page 15 of the Kehoe paper mentioned earlier). Climate models follow the laws of thermodynamics, unless you are a flat earther of some kind there's no reason to doubt the functional relationships. The major uncertainty in climate models is in the input parameters, especially aerosol effects on albedo and rainfall, which is almost the opposite problem from economics where inputs are known very precisely, with the exception of black market activities, but relationships between terms are not.
UCSD hosts a group that maintains an ongoing evaluation of climate models versus predictions.
Posted by: Q-Tip at Jun 4, 2008 12:42:01 PM
I have limited experience with CGE models, but it would seem the the smaller the area being modeled the more reliable the predictions are. I think they are generally built for small regions, as the amount of data needed borders on ridiculous.
With regard to the NAFTA predictions, that is such a terribly difficult problem to model in a CGE model that no wonder the estimates were off.
CGE is simply another useful tool we have to help us think. Of course it's not exact, and sometimes it's wrong. Testing it is unrealistic because generally we change one aspect holding all else constant, while real life has many simultaneous changes. One helluva natural experimemt would be needed....
Posted by: Stan at Jun 4, 2008 1:56:22 PM
aion gold
aion money
cheap aion gold
cheap aion money
buy aion gold
Mabinogi online gold
Mabinogi gold
buy Mabinogi gold
cheap Mabinogi gold
Mabinogi money
2moons dil
2moons gold
buy 2moons dil
2moons dil
cheap 2moons dil
flyff gold
flyff penya
flyff money
buy flyff penya
cheap flyff penya
cheap flyff gold
Dofus kamas
buy Dofus kamas
cheap kamas
Dofus kama
Dofus gold
Dofus money
Knight online gold
Knight Gold
Knight Noah
Knight online Noah
Posted by: aion at Jul 10, 2009 3:50:18 AM
