« Claims my Russian wife laughs at, a continuing series | Main | How to rebuild New Orleans: legalize shantytowns »
The Great Editors
After 18 months and repeated promises from the journal editors, I had no referee reports and reluctantly withdrew my article. I had never withdrawn an article before but this was the second time I had been ill-treated at this journal. I thought about venting my anger here but remembering Ayn Rand's dictum that justice is about rewarding virtue more than punishing vice I decided to take the high road and reflect on some truly great editors.
Top of my list is Sam Peltzman at the Journal of Law and Economics. In my experience, JLE referees are very good but none better than Peltzman himself. Peltzman reads submissions and usually returns his own comments. Every paper I have ever submitted to the JLE has been improved because of Peltzman's comments.
Aaron Edlin at the Berkeley Press Journals. I'm a big fan of the BePress journals; referee reports in 60 days, submit once and be evaluated for four journals simultaneously, electronic submission and referee reports and an opportunity to ask the referees questions anonymously. The technology wouldn't work without a great editor, however, and Edlin writes very thoughtful, intelligent comments, he also has a good sense for quality and what is important.
Robert Higgs at the Independent Review. No one helps an author improve not just the quality of argument but the quality of writing more than Bob Higgs. Working with him is always a pleasure (note that I am assistant editor at the Review as well as a contributing author). The Independent Review is not a top academic journal but it's an intelligent blend of philosophy, politics and economics that is accessible to laypeople as well as to academics.
Ed Glaeser, Robert Barro, and Lawrence Katz at the Quarterly Journal of Economics. I've published several papers at the JLE, the BePress journals and the Independent Review but I have only rejections to show for my submissions to the QJE. Nevertheless, I hold the editors in high esteem because they read submissions before they go to referees and they send back quick responses. It's one thing to be rejected with a stupid report after 12 months it's quite another to be told, interesting paper but not for us, try at journal X after just 3 days (I have heard that Barro used to reject papers in hours but this made people mad so now he holds them for at least a few days). The QJE is the most interesting journal in economics today.
The great editors are productive and generate quick turnarounds and they share some other virtues. The great editors have personality and verve and it shows in their journals - the journals I have mentioned are not just collections of articles they are a reflection of the editor's vision about what economics is and where it should be going. Finally, unlike many others, the great editors in my view don't model themselves as bouncers whose job is to prevent the riff-raff from penetrating the sanctum sanctorum. Instead, the great editors encourage bold ideas and work to raise the marginal product of their authors and of the profession.
Comments are open if you want to nominate some of the other great editors.
Posted by Alex Tabarrok on April 18, 2006 at 07:08 AM in Economics | Permalink
Comments
Renee Stulz from Ohio State used to be the editor of the Journal of Finance and did an amazing job. He seemingly read every submission from beginning to end and the letters from the editor were more detailed and useful than the referee reports.
Unfortunately he was forced out as editor, and the finance profession has been regretting it ever since.
Oh, and my best guess for the 18 month mistreatment would be the JPE. I honestly don't understand why anyone submits to that journal any more. There is nothing fun about receiving an incompeternt referee report written by a third year grad student from Chicago after 8 to 18 months. Not that I am bitter or anything :-).
Posted by: Commenterlein at Apr 18, 2006 7:36:48 AM
Could we get a copy of your paper? Thanks in advance.
Posted by: tibu at Apr 18, 2006 9:22:54 AM
Sorry to stray, but what is meant by referee reports? Thanks.
Posted by: brad at Apr 18, 2006 9:58:16 AM
When you submit a paper to an academic journal they send the paper out to peer reviewers or referees. The referees write a short report on the paper which goes first to the editor and then to the authors with a recommendation to the editor to accept, reject or "revise and resubmit."
Referees, however, are not paid and there is very little punishment for delay or for writing poor quality reports. Thus it is not uncommon to have to wait months, sometimes even years for a referee report only to find that the referee didn't try very hard. It's the editors job to find good referees and not to let this sort of nonsense go on. Often, however, editors do not do a good job which is what makes the great editors such a breath of fresh air.
Alex
Posted by: Alex Tabarrok at Apr 18, 2006 10:19:37 AM
My favorite editors are at arxiv.org. They've accepted and published every one of my papers in under 24 hours without giving me any grief, let me make revisions later (while preserving old versions online), and best of all, made my papers available for free to the whole world, instead of just rich universities.
Manny Knill also did a great job as an associate editor of IEEE transactions on information theory.
And Brad, a referee report is a review of a submitted paper by an anonymous peer academic. They're used to help editors decide which papers to accept.
Posted by: aram at Apr 18, 2006 10:27:21 AM
I haven't sent them anything recently, but I'd like to commend the journal Economic Inquiry for the quality of its referee reports during the past decade. Whether my article was accepted or rejected, I received more useful comments from EI than from many more prestigious journals.
I quite agree with Prof. Cowen about QJE and its good quick-reject policy. Like him, I've never had anything accepted but am grateful for the quick rejections. I also felt it as a compliment when my last paper there, on big empty parking lots (http://www.rasmusen.org/papers/parking-rasmusen.pdf)
was passed along to referees-- though then rejected by them for other reasons.
Posted by: Eric Rasmusen at Apr 18, 2006 11:00:51 AM
These problems are certainly not restricted to economics! My crypto papers seem to get a wide range of comments, about 1/3 completely missing the point of the paper, 1/3 giving moderately useful editing help, and about 1/3 giving some kind of actual information (like "I think your formula here is wrong, here's why" or "You should reference the paper by Whoosit in the Journal of Obscure Results in Urdu.").
Posted by: albatross at Apr 18, 2006 11:16:36 AM
Alex,
Great post. I agree 100% on Peltzman (though my JLE is your QJE). I would also mention Gordon Tullock, long time editor of Public Choice. While his tenure there ended in ~1990, he is still remembered for speed and deciciveness. On a related note, I recommend Bruno Frey's paper, "Publishing as Prostitution," which is a good introductory analysis of the malincentives between editors, referees, and contributors. Great post.
EJL
Posted by: Ed Lopez at Apr 18, 2006 11:51:30 AM
my question : is the referee system still adapted to the times..
Posted by: jck at Apr 18, 2006 12:35:50 PM
aram:
Heh. I love the arxiv too (as a Math PhD student at Cornell, one would expect so), and I highly recommend people submitting their papers there. However, there's hardly much "editing" in that case.
Posted by: John Thacker at Apr 18, 2006 12:42:31 PM
I have to disagree strongly about JLE. I (and many of my peers) have had the same, negative, "18 months and two stupid ref reports" experience. (All of these papers have subsequently been published in journals of comparable quality, with much shorter time lags, and better commentary from reviewers).
Posted by: mikev at Apr 18, 2006 1:08:37 PM
In philosophy, Ernest Sosa (editor of Philosophy and Phenomenological Research and Nous) deserves credit.
Posted by: J. at Apr 18, 2006 1:12:58 PM
I think if every journal reduced it acceptance rate so that it had only a 1 year backlog of accepted papers, it would be a good thing. There are too many papers, which while often true enough, are simply uninteresting (eg, like the hard sciences, where every bit of trivia is published). The bottom line is, good gatekeepers are valuable and thus rare, so most journals are blather, especially as the cost of distribution declines.
Posted by: efalken at Apr 18, 2006 1:50:25 PM
Hmmm, well as a journal editor who has fortunately not sat on a paper
by Alex (he has not submitted to my journal), although Tyler might have
some grounds for complaints, it might be worth throwing in a few cents
from the other side of the fence.
It is a very difficult problem, this one of getting back referee reports.
I am completely sympathetic with the complaints of authors regarding
delays. As noted, referees are not paid, and unless one is an author
with a paper under review, or a very close friend who owes one something
(and as an editor, one uses these owings up soon enough, I fear), there
is simply no incentive/punishment one can offer to get these darned
referees to get papers back, especially if it is not someone likely to
want to submit to one's journal.
This leads to all kinds of judgment calls, sending papers to new
referees after some do nothing. I appreciate would be referees who
say they are not willing to referee a paper upfront much more than
someone who takes a paper, says they will referee it, and then just
sits on it forever. There is all too much of this out there, I fear.
While I do my best to get papers back quickly, there are an embarrassing
number that have had unconscionable delays that authors certainly have
every reason to be pissed off about.
This is an equivalent, I suppose, of at least some authors preferring
the swift upfront rejection than the long, slow death. As the submission
rate to the journal I edit has risen (more than doubled in the last
four years), I have found myself sharply increasing that rate of upfront
rejections. Most seem to appreciate it, but some people really get pissed
off. I just had somebody send me one of the nastiest emails I have ever
gotten, calling me a rather long list of thoroughly non-repeatable names
because of doing so.
There is also a lot of learning involved, although some people are just
naturally wonderful and wise editors, I suppose. One learns about the
attitudes and preferences and proclivities of both referees one uses
regularly and also the members of one's editorial board. Some people
are tougher, some people are easier, just like professors and their
grading habits. Some are slower, some are quicker.
There is a rather curious counterpoint to all this. I have a paper
forthcoming in my journal (JEBO) that I actually disagree with quite
strongly (the author added a section to respond to some of my complaints
directly). The paper argues that there is an optimal "first response
time" (f.r.t.) for papers to authors. This paper also argues that this
has been increasing in economics. What is involved is essentially a
signaling argument: longer f.r.t.s will hold people back from sending
papers to journals that they really have no chance of getting into.
There may be something to this, but I responded that I think it is more
of a social norm thing, especially as I am aware that the social norm
is quite different in some other disciplines, such as physics, where the
average f.r.t. is much shorter than in economics. This author's reply
to that was that physics papers tend to be shorter than ones in economics,
which I do not find an entirely satisfactory argument. (BTW., the paper
is by Ofer Azar.)
I do note that the social norm aspect cuts several ways. Thus, if one
knows that there is a norm of say, six months, before authors really
start to get testy, then referees themselves will sit on papers for
that long often. Indeed, there is a strong disincentive for a referee
to get a paper back too soon. S/he might find themselves being viewed
as a "good referee" and will start getting bombarded with more papers
to referee. There is an unfortunate phenomenon that the better known
someone is, the more papers get sent to them to referee, and so they
can start to accumulate in a pile on one's desk or wherever. One does
not want to get back too quickly, or the pile will get worse. I think
there is an unfortunate drift element here that can lead to longer and
longer refereeing times as a creeping (and creepy) social norm.
For the record, I do not like this and do my best to get papers back
as soon as possible. However, I am not surprised at the contradictory
reports regarding the JLE above. Some papers get to responsible referees,
others get to irresponsible ones, or have trouble even finding a referee
who will deal with the paper. I will in such cases eventually rely on
a few special associate editors I use for such cases, or even myself, but
some papers I receive I am not really competent to judge. Some papers
just get stuck in unfortunate situations.
That reminds me of something that I saw in one of the papers by
Frey (not the prostitution one, an earlier one). He made an argument
I disagreed with that editors should make decisions and referee reports
should only be used to advise on revisions. My objection to that is
that this only works if one is either a Renaissance genius or is the
editor of a very narrowly specialized journal for which the editor really
is a big expert. Otherwise, one must rely on somebody, even if just other
members of the editorial board (who can also sometimes be slow) to at
least provide some advice. Frey's argument involved the (generally false)
claim that editors have "property rights" while referees do not. I agree
that there are plenty of game playing referees out there, but that does
not mean the editors are necessarily much better.
Which brings me to another point. Strong editors are praised in this post,
and that is certainly an implication of Frey's argument. However, while
I agree that some of the more interesting journals around have or do
reflect such editors, I have also observed some cases I shall not name
of strong editors who were too narrow and personalistic, where the way
to get into the journal was to kiss the editor's ass and cite a bunch
of their work very approvingly, and so forth. Maybe this is an inevitable
outcome of a strong and creative editor who is putting a strong imprint
on a journal. There is no getting around them imposing their views and
their ideas, although the better ones are open minded, and I have seen
these two phenomena coincide more than once.
Posted by: Barkley Rosser at Apr 18, 2006 5:21:44 PM
Regarding European-based journals, I would think that Backhaus at the European Journal of Law & Economics is doing a very good job. It seldom takes more than three months to get competent referee reports.
Also, Geoffrey Hodgson at the newly founded and (I think) quite promising Journal of Institutional Economics appears to be a very efficient editor. Three reports are standard, and they come quickly (about two months).
Posted by: statler at Apr 18, 2006 5:29:35 PM
One useful advance is on-line submission and refereeing. Not least because the websites automatically email the refereee to nag him. (A personal nag from an editor is even more effective, but just getting reminded by a robot that you owe a review has a behavioral impact in my experience.)
On the strong editor question, I would rather have a portfolio of strong editors at different journals, each of whom may be biased in a different way (or require some obeisance), than a norm of weak editors who delegate decision-making to the referees. (BTW, if you think there are problems in econ, you should see what we put up with in management journals.)
Posted by: srp at Apr 18, 2006 6:06:36 PM
"referees are not paid"
Well, maybe they should be. Doesn't AER pay 75$ to it's refrees?
Posted by: Teller at Apr 18, 2006 7:06:54 PM
I wonder if book editors have incentives to work more efficiently than journal editors since their names are usually included in the citations.
Posted by: Andres at Apr 18, 2006 9:49:11 PM
Electronic submitting and tracking is speeding things up a bit,
for whatever reason.
Last time I reffed for AER, they did not pay. I hear some finance
journals do. I have never been paid for refereeing an article
anywhere. Some journals do have some neat incentives, such as,
reduced submission fees if papers are gotten back on time (JEDC).
I am not going to diss anyone in particular, but some of what I
am reading here sounds like claims that are made by journals,
not necessarily their actual performance.
I have run into some pretty godawful slow book editors in my time.
Posted by: Barkley Rosser at Apr 18, 2006 9:55:10 PM
On the question of whether to pay referees or not:
"Engers and Gans (AER 1998) examine incentives to provide effort when concern for the output produced is a primary motivation. That paper provides an efficiency rationale for why referees may not be paid. Specifically, upon receipt of a paper to referee, if the accompanying payment for the task is large enough, the referee correctly anticipates that, were he to decline the refereeing assignment, the next person asked would be likely to accept it. If the accompanying payment is low, however, then chances are high that the next referee would not accept the task. In that case, the referee motivated by professional concern accepts the assignment."
(copy/paste from P. Francois, "Making a Difference")
Posted by: OxEcon at Apr 19, 2006 5:46:35 AM
Having made some vague, negative remarks, let me praise
somebody. It is not always the editor who is responsible
for a journal's success, however authors are treated. The
Journal of Economic Perspectives must be viewed as one of
the most successful of recently started journals, now in the
top ten in citation rates and widely read by many economists.
It has had many editors since its inception.
However, the main person responsible for what the JEP is today
is its Managing Editor, who has been with it since its start,
Timothy Taylor. I have some disagreements with Tim, especially
his insistence on paginating by issue rather than volume, which
seriously screws up referencing, but he deserves the credit more
than all the top editors put together for the indisputable
success of the JEP.
BTW, while I know that Aaron Edlin is getting papers back
quickly, or reasonably quickly, I am still waiting to see
people citing papers published in those journals to any
noticeable degree. It has not been happening so far. He
is not a Tim Taylor, not yet anyway.
Posted by: Barkley Rosser at Apr 19, 2006 10:10:35 AM
Hendrik Houthakker ran a tight ship at Review of Economics and Statistics. Papers got matched up with referees knowledgeable about the subject, deadlines on both sides of the review were enforced, and accepted papers got into print quickly. Oh, and one could learn a lot about tight writing when a referee report came back with "consider this and this and take out 2,000 words."
Posted by: Stephen Karlson at Apr 19, 2006 11:01:36 AM
While I've only published 1 piece at the Independent Review (I hope to publish more there in the future), I can confirm your commendation of that journal's editor, Robert Higgs.
Posted by: George Bragues at Apr 19, 2006 12:36:45 PM
The Journal of Financial Economics lists the turnaround times/workload for their editorial board and referees:
http://jfe.rochester.edu/
Posted by: Brian at Apr 19, 2006 1:26:47 PM
The Journal of Financial Economics also has a $550 submission
fee for nonsubscribers and a $500 submission fee for subscribers.
Excuse me while I vomit in the corner rather than applaud the
practices of that particular journal.
Posted by: Barkley Rosser at Apr 19, 2006 1:59:43 PM
