As editor of an economics journal, I regularly hear authors complain about long lags for referee reports. I am very sympathetic with these complaints and frustrated myself. What is especially frustrating is that I know that these lags are much shorter in many other disciplines, especially the hard sciences. Physicists and others are simply shocked at the lags in economics refereeing and publishing, and this has led to much of the recently emerged econophysics literature being published in physics journals like Physica A and European Physical Journal B, which are not indexed in the Journal of Economic Literature, and hence not read and not even known about by most economists.
Ofer Azar has argued in a paper in Economic Inquiry this year ("The slowdown in first response times of economics journals: can it be beneficial?" 45(1), 179-187) that indeed the increasing lag of authors hearing back from journals could be a good thing, maybe even optimal. This is because people need to pay a cost for sending papers to journals that are too far above them and in which they have no chance of publishing. Hence, they will get the signal if they have to wait a long time and just get rejected and send their papers to appropriate outlets. Azar argues (in a paper forthcoming in JEBO, which I edit) that the shorter response times in physics reflect that their papers are shorter on average than those in economics. This latter point is correct, but I think that what has happened is that a bad social norm has evolved in economics where referees simply assume that they can sit on papers from economics journals for a long time and just put them aside, fearing that if they get their reports back quickly, they will simply be punished by having more papers sent to them to referee, given that the average response times are so long. I dislike this social norm, but it is very hard to overcome (and the Berkeley Electronic Press's effort to shorten refereeing times has so far resulted in published articles that barely get cited in other papers).
it also sounds like some peculiarly economist type thinking is taking place:
ReplyDelete"pay a cost" "get a signal" "be punished.."
i'd like to think normal people don't think in these terms, but i remember the skinnerians.
and of course Republicans in general think there is a moral equivalence between "sending a message" and horribly punishing real human beings.
I have lots of experience in physics publishing and the way things work there is that the speed of publishing is stratified. The fastest journal is Physical Review Letters. The time lag is weeks to months. They impose very strict rules on what gets accepted. There are length restrictions, there are novelty restrictions, and the editorial staff weeds out many papers before they get refereed.
ReplyDeleteThe second way delay is combated is via a formalized system of issuing preprints. Those who need to know the results can get them quickly online and/or through informal distribution lists. Many papers are published mostly to improve an authors vita and when they appear doesn't matter as long as they can put down that it has been accepted when seeking promotion or a new job.
There are also conditions imposed on referees as to how quickly they turn papers around. Those who take too long are dropped. The one constraint on the major journals (like Physical Review) is that they have page budgets and can only publish so many articles per issue. There have been attempts to deal with this as well by getting authors to shorten their papers and put supplementary material online. There are formal procedures for this as well.
Lastly there are an ever increasing number of journals (especially in biology) which cater to ever narrower fields. Sometimes these have enough prestige to serve the career goals, but sometimes this prestige is only known to those in the field and not to university review committees.
Economics seems to allow much non-refereed material to be "published". I don't know how this is treated when promotions and the like are being considered.
Robert,
ReplyDeleteThere is also a proliferation of ever more specialized journals in economics. Not clear where that is going to end.
Regarding your last remark, I do not know where you got that from. With a very few odd exceptions, almost all (like 998-99%) of academic economics journal artcles are refereed. Indeed, that is why there are so many delays and lags. It is mostly the referees who have adopted these unpleasant social norms of sitting on papers for many months that is the main source of the problem.
You mention that in physics referees who do not get papers done in time are threatened with "being dropped." For referees who are not listed on an editorial board, being threatened with being dropped is not much of a threat to most referees. Refereesing is mostly grunt work that is tedious and gets no rewards. The only real pull an editor has is if a referee would like to publish in the journal. That is about it.
Rosserjb:
ReplyDeleteI know this discussion is supposed to be limited to "academic" research, but much of the material that gets the widest notice is put out by think tanks like Hoover, Cato, and their liberal counterparts.
Those not in the field can't distinguish one from the other especially since the think tanks are good at issuing press releases to go with their white papers. I don't know if this work carries any weight when making appointments or promotions, but there is no counterpart in the hard sciences that I'm aware of.
Some of the biological journals, like "Nature" are good at the PR game, but at least the articles are refereed. I'm also upset about the trend of late to embargo mention of such articles in the press until the journal is "released". I don't know why the press plays along.
Someone in the field will have to explain to me what the position of NBER is, it doesn't seem to adhere to scholarly standards.
As for getting referees to "behave" it might be useful for the editors of some of the most problematic economics journals to consult with the editors at some of the physical science journals and get some feedback on how they deal with the referee problem.
Robert,
ReplyDeleteSome of the outlets from the think tanks are refereed. I am thinking specifically of the Cato Journal and the Brookings Papers on Economic Activity. The papers in the latter tend to be of especially high quality, although they are invited, even if still refereed. Inviting is one of the exceptions, but the journals that do that tend to be highly rated, like the Journal of Economic Perspectives and the Journal of Economic Literature. The BPEA is highly cited and very influential. The Cato Journal is less so, but better than a lot of what comes out of there.
BTW, I am about to leave the country for over a week tomorrow. How many of you spotted the latest posting on Dani Rodrik? We are now 84 th in the rankings, up from our former 96th. Also look pretty respectable academically.
ReplyDeleteThis is rather off topic. But why is there a "Read More" button when there is no more to read? As a blog reader, this seriously bothers me. I click through expecting further commentary, and instead just end up wasting a minute. (If I'd wanted to read the comments, I would have clicked on the "comments" link).
ReplyDeleteIt's a really basic and simple thing, but it makes the blog very unpleasant to read. I either click through on every post and take twice as long to read the page, or I miss half of the posts.
I note that ProGrowthLiberal's 2:40 post doesn't have a Read More link, so it seems like there's no technical barrier to getting this right.
Anon.,
ReplyDeleteThe Read More button just got added. We're still working on this, new blog and all that. Sorry about any confusion.
Thanks for the reply. I don't know how the tone of my comment sounded, but this was meant as constructive criticism and I'm glad that you're aware of the issue and working on it. I complained only because I want to read the blog more efficiently!
ReplyDelete--Same Anon
as a physicist turned economist, I think people don't really understandhow the preprints work.
ReplyDeleteThere is a rather large preprints database in which just about everyone can submit their papers. Many people read these preprints, thus by the time ot gets to the referee stage, it has already been thoroughly reviewed by others.
Arxiv.org is the site, it's one thing i really felt the economics discipline was lacking.
(Am in an internet cafe in Ancona,
ReplyDeleteItaly at the moment. Got 26.38
euros for US 50 this morning at
the Munich airport, dollar dive
indeed).
There is not such a centralized
mechanism in economics, but most
papers get up onto websites as
working papers. I have even been
told by associate editors of my
journal that we should abandon
double blind refereeing because
"everybody can figure out who the
author of a paper is,° and if the
paper is not available on the
internet, °it is probably not any
good anyway,° although I have ignored this last piece of insider arrogance and retained double blind refereeing, at least for now.
Ciao.
Barkley
(giving a plenary on econophysics here tomorrow)
Barkley:
ReplyDeleteI think it would make following your discussions (on multiple sites) easier if you would adopt a consistent screen name.
Since you don't seem to want to remain anonymous your name would seem to be the best option.
Cool post you got here. I'd like to read more about that theme. Thanx for giving this material.
ReplyDeleteJoan Stepsen
Cyprus escort