Over on Overcoming Bias, Robin Hanson has proposed that journals report the papers that they reject, including names of authors and dates of rejections, possibly even with the referee letters. He argues that this might improve the efficiency of the economics journal publishing process by "raising the bar" so that people will not send papers to journals that they are unlikely to get their papers accepted in.
As a journal editor I disagreed, noting that this would be very humiliating for many would-be authors, with some I know having a hard enough time submitting papers given their fears and unhappiness about rejections and nasty comments by refererees. I also noted that there are other proposals out there along similar "efficiency" lines, but that they go against practices and trends in the hard sciences. Thus one says that lengthening the times to first responses from journals (which has been a trend) would achieve this result also, and there are journals that charge very high submission fees, but then return them if papers are accepted (last time I checked, $650 at the Journal of Financial Economics, with the Journal of Monetary Economics not far behind). As it is, in the hard sciences, very rapid turnaround and publishing times are emphasized, and rather than punishing submitters who get rejected and rewarding those who are accepted, many hard science journals have no submission fees, but make authors pay for pages of papers that are being published, something I am unaware of any economics journal doing, whatever one thinks of that. But it is certainly the opposite of the practice of the J. Fin. Econ. and the J. Mon. Econ.
20 comments:
Am I to understand that the economics journals cited and in general do not provide some detailed outline of what is expected of a submission for possible publication? In addition to which I'd be perplexed to know that within the field there is not some reasonably clear idea of what constitutes a quality submission, accepting that the criteria for judging the quality of a paper may differ from one journal to another.
Furthermore, if an economist has gone through the trouble of preparing a presentation of some subject matter, does that person have no peer group from which to obtain an objective opinion regarding the quality of the presentation? Otherwise economics would seem to be a very peculiar branch of the social sciences.
If the rejection rate at a specific journal is relatively high, it would imply that the editors of that journal have not made it clear what is expected of submissions for publication. That is the fault of the editors and publisher of the journal.
I have published probably 25 papers in science/engineering journals and the only time I might have had to pay was if there were color prints that were done outside the regular print process but needed to be included.
Further, referees are encouraged to give other journals to which an unacceptable paper could be sent if the subject matter fits better at the second journal.
Jack,
The answers to your first questions are yes, no, yes.
Rejection rates at the higher ranked economics journals tend to be quite high. It is 85% at the journal I edit. At the Quarterly Journal of Economics, one of the top four, and published out of Harvard, about 90% of papers submitted are "desk rejected" up front without even being sent out to referees. I only desk reject about 50%. Things are rough.
Bill,
I do not have exact figures, but I know that there are journals in the hard sciences that charge per page. However, I am also aware of some that do what you have described, only charging for figure or colors on pages, or other extra stuff. In any case, that sort of thing just about never happens in economics.
Barkley,
Those rejection levels imply either that, as I implied previously, the editors are not making clear what their criteria for publication are. Or, the individuals submitting those rejected papers are out of touch with what they themselves are producing. In effect they're subjective opinions of the quality of their work is biased. That may be due to the elite nature of the publisher, Harvard as an example, causing submissions to be too optimistic regarding the quality of their own work.
Why not require that submissions from previously unpublished individuals, by the specific journal, be accompanied by third party reviews from within their own departments? As I noted above, it seems reasonable that if you think you've invented the wheel maybe you should ask your friends and neighbors their opinion of the product. I've not been academically involved with the field of research psychology for a long time. When I was it was clear to most researchers what a good paper read like, and how a quality project was constructed. Even so it would have seemed the hight of hubris to submit a paper for publication that had not been vetted by one's colleagues. I'm not referring to popular journals, but to the more serious ones like Journal of Experimental Psych.
It would be great if the journals published the rejected articles on line with the reviewers' comments.
It would be great if the journals published the rejected articles on line with the reviewers' comments.
It would be easy enough to set up a "journal rejects blog." But who would read it?
Reading rejected work and knowing which journal rejected it would tell as much about the editors of the journal as it does about the author of the rejected work. Knowing that rejected work will be published as such in an alternate manner would likely give pause to those submitting for publication. That might discourage innovation, but it might also encourage those who produce innovative work to have it peer reviewed close to home before being submitted.
Sandwichman, Jack explained the reasoning for publishing rejected articles & comments.
If you submitted an article about the lump of labor fallacy & the comments just said everybody knows ...., wouldn't that tell you something?
Heh. Jack,
I have had a paper 'desk rejected' by the QJE. No hard feelings about it and when I submitted it (it was a first paper I ever submitted to a serious journal) I pretty much expected it. But hey, I don't buy scratch tickets, go to Vegas or even play poker with friends. So I got to get my gambling thrills somehow - I thought my paper was pretty good, but it was the QJE.
As far as knowing the standards or getting an objective assessment of your work - I do think that this is actually pretty difficult, particularly for younger folks like myself who haven't been around that much. And especially if your field is something a bit out of ordinary or you're doing something sort of weird (on the other hand, if what you're doing is plain vanilla then it will probably be rejected too - part of the reason I tried QJE is that in my opinion they do tend to publish quirky papers). One telling piece of evidence is all papers that became 'classics' that initially got rejected all over the place.
(Akerlof's Lemons paper is one that springs to mind, and I think Robert Hall's "consumption follows a random walk" (though that one is still considered weird - one referee wrote back that he must've been on drugs when he wrote it) but there's also many others. In fact there's a whole book on this whose title completely escapes me at this moment).
notsneaky
If you submitted an article about the lump of labor fallacy & the comments just said everybody knows ...., wouldn't that tell you something?
Is that a rhetorical question?
This is perhaps off topic, but funny:
http://www.cartoonistgroup.com/store/add.php?iid=31210
Signe Wilkinson's cartoon on economics departments. I'm surprised (or am I?) that Greg Mankiw did not post it as he has a thing for cartoons if they favor his positions, whatever they may be.
Jack,
Part of the problem is a classic scarcity one: the overwhelming majority of journals (including the one I edit) have a given number of pages they can publish per year, so there is very severe page constraint. Thus, if people are sending a lot more papers than can be published, given that, many of those will end up getting rejected, even if the majority of them are quite good papers.
It is not a matter of "here are our requirement, if you meet them you get in." No such exist anywhere for any journal, although there are certainly some things that can get you automatically rejected, in some cases banned from ever submitting again, such as plagiarism or submitting the same paper to more than one journal at a time.
Some journals have reputations for favoring people who are part of an in-crowd with special links to the journal. This has long been alleged to be the case with the QJE, where supposedly having a Harvard connection helps, and a similar rep holds for the Journal of Political Economy (JPE), another of the top four, which is based at the University of Chicago, and where a Chicago connection is reputed to help. I think for the AER (another one), it helps to have a link with somebody on their editorial board, and this is often the case at many journals, although not necessarily so everywhere.
Given journals tend to have certain themes and attitudes, and the more one can figure out what those are and fit in with them, usually the better one's chances. But those things are rarely set in concrete or stated officially, and can and will change over time as the editor and editorial board change. Ultimately it is they who determine what goes in and what does not, and it is a matter of judgment.
Let me be more personal. So, I reported desk rejecting 50% myself of papers submitted. Probably only about 5% of the submitted papers are really bad, unworthy of publication anywhere. Beyond that it is really a matter of judgment, deciding that somehow this paper looks more innovative or more interesting to one's readers than that other one, which may be a well done paper with original ideas and nothing terribly wrong with it. The basis of such judgments simply cannot be put down as criteria by a journal in any serious way, although most journals do have stated goals that give some indication of the sorts of topics or themes they are most interested in, and of course one should look and see what is published in a journal before one attemptst submitting to it.
michael p.,
Well, of course there is difference between actually "publishing rejected papers" themselves versus simply listing titles and authors and dates of such rejections, which is quite a different matter. On the former, one runs into these page limit matters again, although I suppose one could put up an electronic file somewhere. But then, does one publish the desk rejected ones? What if a paper was revised, possibly more than once, does one publish the original submission, the last version submitted before rejection? I see a lot of problems with this.
(One of those problems may involve copyrights, especially for potential future publishers of the paper: do you want to have people unable to publishe a paper that was rejected because the rejecting journal "published" it already for their "rejected papers" pile?)
There is also the problem of who is going to read all this? Heck, I have enough problems just keeping the remotest track of what is published. I suppose it might be of interest for someone thinking of submitting to a journal, but who elss?
[S-man makes this point also.]
And there remains the matter that I think few potential authors are going to like it, but then we are back to Robin Hanson's argument about efficiency and how this will keep people from submitting papers inappropriately to journals they have little chance of getting published in.
yns,
Yes, it is hard to get "objective" advice, with some of the hardest being exactly this sort of advice about where one should submit one's papers. This is the sort of thing that "mentoring" senior faculty should do for junior faculty, not to mention one's professors, especially major prof. But, when it gets to the top journals those people are in the same boat anyway, trying to get into those journals and not really sure what will fly and what will not when one is looking at rejection rates in excess of 95%.
Barkley, your objections make sense. My hope would be that there could be a way to shine light upon the editorial process.
Barkley,
My comments were only intended as the first step in a process. In effect, let them know what you expect
in a publication. Constraints of space always drives the rest, unfortunately. The high rate of rejection might imply the need for additional editorial space. Certainly the last analysis is a subjective one. Good journals that attract many readers by definition have editorial staff with good inclinations, people who know what they want said and recognize something worth sharing.
Isn't this conversation about how to improve the design of the buggy whip? The issue, it seems to me, is how can there be some kind of mechanism for objectively or scientifically establishing reputations of quality scholarship.
Economic journals arose within a historically specific institutional and technological context, which is in the process of being surpassed. Old media don't have any inherent moral claim to adjudicate what transpires in the new media. There might indeed be a transition period during which the weight of tradition and established reputation holds sway.
I used to pay attention to Letters to the Editor in newspapers I respected. But with the convenience of Email, the quality of such letters seems to have diminished as the speed of submitting them has made them more timely responses (perhaps not well thought out in the haste to submit them) to articles the prior day in the newspapers. Perhaps newspapers should let their readers know about the letters they rejected so that readers can get a better idea of the selectivity process of the Editor. I am not serious about this primarily because of personal time limitations on reading a tad too much. But don't my time limitations apply to economists who would be interested in rejected submissions? How about SSRN? There are many articles available there that probably were rejected by economic publications. Some may be good, some bad. But who has the time to wade through the many rejections in life?
Over on Overcoming Bias, where I have been debating this matter with Robin Hanson, he has just confirmed that what he has in mind is simply listing titles and authors and dates of rejection, not printing the full papers, although he is also open to listing names of referees, I think, and perhaps reasons for rejection.
"how can there be some kind of mechanism for objectively or scientifically establishing reputations of quality scholarship."
I think that by definition a reputation is subjective. Quality scholarship itself should be objective, and in place of scientific I would say rigorous. Is first rate science like pornography? It's hard to define, but you know it when you read it, or when you try to replicate its findings. This is a separate though related issue to that of the rejection of submitted product.
It does seem strange to me, however, that with the advent of the internet, mega storage capacity and file sharing it should be possible to publish all good work. Then watch the fun begin when the real research scientists, if they do exist, come out swinging with their critiques of that which is inadequate. If colleagues are unwilling to critique in a public forum, I don't see how economics or any science can advance to a better understanding and better ideas.
Don't academics gossip about which editors take what? Science fiction writers do, and while they may not want to talk in public about their rejections, they will talk at parties and in the hotel bar. Over time, if you attend enough conventions, you get a fairish idea of editorial preferences; and you learn who is doing good work and having trouble getting published.
That's a process that may be fine for fiction writers. The publication of worth while intellectual product(is that what economics is all about?) needs a process that's a bit less social and a bit more incorporating.
Post a Comment