« She only hinted at Rainier cherries | Main | The World We Have Lost »
Economic Inquiry has a new policy
R. Preston McAfee (a great choice) is the new editor, and he writes in a mass email today:
More insidious, in my view, is the gradual morphing of the referees from evaluators to anonymous co-authors. Referees request increasingly extensive revisions. Usually these represent improvements, but the process takes a lot of time and effort, and the end result is often worse owing to its committee-design. Authors, knowing referees will make them rewrite the paper, are sometimes sloppy with the submission. This feedback loop - submitting a sloppy paper since referees will require rewriting combined with a need to fix all the sloppiness - has led to our current misery. Moreover, the expectation that referees will rewrite papers, combined with sloppy submissions, makes refereeing extraordinarily unpleasant. We - the efficiency-obsessed academic discipline - have the least efficient publication process.The system is broken.Consequently, Economic Inquiry is starting an experiment. In this experiment, an author can submit under a 'no revisions' policy. This policy means exactly what it says: if you submit under no revisions, I (or the co-editor) will either accept or reject. What will not happen is a request for a revision.I will ask referees: 'is it better for Economic Inquiry to publish the paper as is, versus reject it, and why or why not?' This policy returns referees to their role of evaluator. There will still be anonymous reports.
Authors who receive an acceptance would have the option of publishing without changes. If a referee noticed a minor problem and put it in the report, self-respecting authors would fix the problem. But such fixes would not be a condition of publication.
You could try dating women on this basis as well; we'll see how it goes. Elsewhere in the world of journals, Science is ending its link to JSTOR, a sad moment for scholarship.
Posted by Tyler Cowen on July 26, 2007 at 06:17 AM in Science | Permalink
Comments
It's nice to see some experimentation aimed at cutting submission-to-publication times, which are ridiculous. It'll be interesting to see how many (and which) authors choose to take advantage of this option. The likelihood of what would have been a revise-and-resubmit decision becoming a rejection might affect author behavior only by scaring them away from this option.
Posted by: Pinkston at Jul 26, 2007 9:05:32 AM
For more on "as-is" reviews see the paper by Eric Tsang and Bruno Frey discussed here:
http://organizationsandmarkets.com/2007/04/17/the-as-is-journal-review-process/
Posted by: Peter G. Klein at Jul 26, 2007 10:17:56 AM
Great, now could somebody please explain to me, speaking slowly enough for my dim little brain to understand, why we can't submit the same paper to multiple journals in economics?
Posted by: Keith at Jul 26, 2007 10:21:54 AM
Physics has a well-organized system of publishing pre-prints online. This is what the author wrote. When the paper appears in the refereed journal it is possible to compare the two versions, if anyone wishes to do so.
Some disciplines (like medicine) control the flow of information so as to maximize the visibility (and ad revenue) of the top journals. This is, of course, a perversion of the role scholarly publishing - to advance the discipline, not the fortunes of the publisher.
Posted by: robertdfeinman at Jul 26, 2007 10:33:03 AM
But there are lots of papers that are good, with a couple fatal errors that are easily fixed. To give a 'yeah or neigh' ignores these cases, and would either publish them with their dumb, easily correctable errors, or not publish them. I think he's being too extreme, and should just say we are going to edit less and reject more, and leave it at that. Most journals have a very long backlog anyway.
Posted by: eric at Jul 26, 2007 12:15:18 PM
Uh, isn't this already how most people date? It may be possible to give couched-as-a-compliment feedback ("I really liked how you wore your hair yesterday"), but not many people try demanding serious revisions ("dye your hair, loose 20 pounds, and stop wearing Hawaiian shirts, then we can date").
Posted by: David Wright at Jul 26, 2007 12:22:33 PM
lose.
Posted by: shawn at Jul 26, 2007 1:28:02 PM
McAfee seems to have made the Editor's abdication from editing complete.
Posted by: Daniel Klein at Jul 26, 2007 1:39:58 PM
"I cannot judge this paper until it is rewritten in English". A few more referees saying that would be a Good Thing.
Posted by: dearieme at Jul 26, 2007 2:57:13 PM
Well, as an editor (JEBO), I have some views on this.
First, there is a problem, although I doubt that McAfee is offering a satisfactory solution.
There is a lot of egomaniacal refereeing that goes on, "cite me, cite me again, praise my work
to the skies!" etc. Editors need to be on top of this and not simply succumb to it. There is
indeed all too much of this, editors not taking control and kowtowing to referees, especially
on these endless rewrites that really go nowhere in circles.
However, the McAfee solution, and the more general approach advocated by Bruno Frey, who lurks
behind this, in fact requires more brilliant and knowledgeable editors, Renaissance People editors.
Now, that would be wonderful. Would that all of us editors were such brilliant and all-knowning
Renaissance People. But, the hard reality is that with the increasing specialization and fragmentation
of knowledge, we are less and less able to fulfill such functions. We must rely to some degree on
referees.
Now, I guess with this option, one can still do that, except that now one is taking up or down
recs from the refs. Maybe that is not too bad.
I think there is a problem, and some commentators have already noted it, that there may be some
major problem in the paper that could be easily fixed. There is also the issue of improper or
inadequate citations, something else that can be fixed. Of course, this is one of the bugaboos
of the egomaniacal refs. But then, sometimes the complaints are legit. There is a major problem
of proper people not getting cited. In the extreme this can even verge on plagiarism, people
claiming credit for ideas that have already been pubbed. This is a serious problem, a very
serious problem. Just having this up or down can make this problem a lot worse.
My own change when I took over editing JEBO was to impose on the board, without any big public
announcement, that we effectively make an up or down decision after the first revision. I think
that there is a strong case for having revision. Very few papers are accepted without any revision,
although some only require a little bit. I think McAfee is going to find himself drifting into
his loophole very quickly and expanding it. Just what constitutes a "minor editorial revision,"
is going to grow quickly. But, once a revision is in, I think there should not be any further
fiddling around. Further revisions beyond that should be effectively minor. It is the going on
and on with multiple revisions that is the real stinker here.
Posted by: Barkley Rosser at Jul 26, 2007 4:09:21 PM
If McAfee has abandoned his role as editor it seems only because the authors have abandoned their role as a writer.
The policy makes sense. Surely if there are significant, but easily fixed flaws that can be communicated in the pass-back. Same for small edits. What needs to be policed is the lazy first draft that is submitted.
All that said, isn't there an efficiency to having early drafts submitted? If an author is going to devote time to a paper, doesn't s/he want to ensure its publication? If all polishing is done pre-submission, and it is still rejected, then a lot of inefficient work has gone into writing.
Posted by: ah at Jul 26, 2007 5:45:11 PM
ah,
There is something else going on here. At the more well known journals, acceptance rates are generally
below 20% and often below 10% because of sheer space constraints. Given that the majority of papers
submitted are generally 1) not completely stupid or incorrect, and 2) usually have at least some
shred of an original idea or extension of existing ideas, this means that what is going on is that
the majority of papers are getting rejected, but not because they are obviously wrong or full of
screaming errors or just completely unoriginal. What is ultimately involved is a judgment call
regarding how interesting or new or innovative or important the paper is. That is not such an
easy matter.
So, most of the papers rejected are not all that bad. It is just that they are deemed not be
quite good enough.
Posted by: Barkley Rosser at Jul 26, 2007 6:11:43 PM
I am happy that Dr. McAfee is describing this approach as an experiment. I think that it is likely that Dr. Rosser is right and that he'll find it hard to stick with no revision. But I think that a serious effort to minimize the revisions cycle is not a bad idea. If nothing else, maybe he'll gain some new ideas as to how to improve the revisions cycle.
Having been caught in the endless loop of revisions (without any guarentee of eventual publication) and watched it with my colleagues, I agree that this is unpleasant -- especially when it requires large amounts of additional analysis or data collection between submissions.
What I would like to see is a quick up or down decision of "Should we work with this author to get this paper into press at this journal". I do not, and never will, mind doing major work or implementing constructive suggestions on a paper that is going to be published. But huge amounts of time and effort to not get published slow everything down and reward nobody!
Posted by: Joseph Delaney at Jul 26, 2007 8:39:30 PM
I'm surprised that nobody mentioned "The Slowdown of the Economics Publishing Process," Glenn Ellison, JEP, Oct. 2002. In a very real sense, McAfee is returning to how journals ran a few decades ago. One quote: "Most of the year-to-year changes are fairly small, but the magnitude of the increase over the thirty-year period is startling. At Econometrica and the Review of Economic Studies we see review times lengthening from 6-12 months in the early 1970's to 24-30 months in the late 1990's. My data on the AER and JPE do not go back nearly as far, but I can still see submit-accept times more than double (since 1979 at the JPE and since 1986 at the AER)."
Here's my favorite quote from the paper: "An anecdote I find revealing is that a senior economist told me it looks odd to him to see young economists' resumes trumpeting that papers have been returned for revision. When he was young he never would have listed a revise-and-resubmit on his resume because he would have been embarrassed that something was wrong with his initial submission."
Posted by: Bill Goffe at Jul 30, 2007 1:27:18 PM
I should have also mentioned that "Evolving Standards for Academic Publishing: A q-r Theory," Glenn Ellison, JPE, Oct, 2002, describes a theory of how changing social norms in publishing can explain the slowdown. Yep, two articles in the same JPE.
Posted by: Bill Goffe at Jul 30, 2007 1:31:01 PM
Posted by: _lika_18 at Oct 23, 2007 4:14:41 PM
I was influenced greatly by Glenn Ellison's study, and I have discussed cures for the refereeing slowdown with him. One of the things we discussed is that changing a journal, in his case, Econometrica, is very difficult because the referees have expectations that an editor can't change.
I want to make two points. I spent over 9 years as an AER co-editor and rejected over 2000 papers. I don't have a problem with commitment and have already rejected no_revisions EI papers with fatal defects that could have (likely) been fixed. Balancing the lost papers, EI has received submissions from Steve Levitt and Jim Heckman that it wouldn't have otherwise obtained.
I made the policy extreme so that a failure to follow it is demonstrable. A policy like "only minor editorial revisions" isn't verifiable, which makes it useless. It is also an advantage to use an extreme policy because it influences the way referees think about evaluation.
Second, much of the discussion here and elsewhere neglects that no_revisions is an OPTION. Young, inexperienced authors should use the old system. People who know what they want to say, how to say it, and who know the literature will benefit from no_revisions, as will 5th year assistant professors who need a decision.
Posted by: Preston McAfee at Dec 2, 2007 11:38:30 AM
