An odd spinoff of the ongoing debate over at http://rjlipton.wordpress.com is a proposal that the genius logician Harvey Friedman (first taught as a prof at Stanford in philosophy at age 18, a world record for youthful professoring) has put forward in the middle of it for refereeing papers at journals, obviously inspired by the ongoing, now many hundreds of entries long, debate over the proposed proof by Vinay Deolalikar that P does not equal NP in computational complexity theory (current consensus: current proof flawed, but argument might still be right, or more likely, proof strategy may be very productive for lesser results).
So, the proposal is that an author is offered the option of public refereeing rather than the standard secretive double-blind type usually done. This public refereeing involves the journal putting the paper up on a website where anybody can publicly critique it. The author can respond and put up new revised versions (and can criticize the editorial board of the journal as well). When the author is satisfied with what has transpired out of the process, s/he can propose that the ed board consider it for publication. They then decide either to publish or not to publish. If it is not published, the last version can either remain up "hanging" on the website or be taken down.
"This public refereeing involves the journal putting the paper up on a website where anybody can publicly critique it."
ReplyDeleteIf too many cooks may spoil the broth, what might "anybody" do to the broth?
It has been brought to my attention that there is a journal that does this in economics, Economics e-journal, based in Kiel, Germany. One difference is that this public refereeing is the only way they do things, not offering the standard alternative. Looking at it closely, I would say that this journal has not done all that well, another hyped hopeful that has not really lived up to its hype.
ReplyDeletei note a well known crank, matteo marsili publishes there. unless he's not. (at abdus salam centre--not even a real muslim, eh.) i allus wondered why 'anybody' like darwin or einstein could ruin the broth-ren. i note m davis on fom mentioned galois. not to mention the model theory---course, its a good ide4a. 'jr'. (vs g kreisel)
ReplyDeleteMatteo Marsili is a one of the better econophysicists around, and I know him personally. He is a nice and reasonable guy and has written and published quite a few good papers. He is most definitely not a crank.
ReplyDeleteAbdus Salam won a Nobel Prize in physics. He was also not a crank.
my comment was a joke (facetious or something). i looked at that journal---it actually looks to be 'state-of-the-art' (or possibly generic in that alot of this stuff is now standard----M aoki has a paper in a theme issue, and he was a pioneer of stat mech/econophysics whose stuff was what i used to like---emphasizing combinatorics more than most). marsilis has also been around a while with tons on the arxiv. as for salaam, his is a a branch of islam which in pakistan is viewed as outside the mainstream ('cranky')---they get hassled there or outlawed.
ReplyDeletethe open source review process seems interesting (one paper on comparing exogenous shocks with endogneous 'news' propogation in a 'post-keynsian' model generated some interesting exchanges; though i dont think the idea is that new---comparing noise versus determinsim (eg Ornstein and Weiss BAMS 91).
on the 'math under the microsope' blog there is a discussion of open-source reviewing of free e-textbooks, coming from sun microsystems apparnelty. i think plos has some of this too. or, wikipedia.
i guess i'm not sure alot of this stuff is any worse than the mainstream stuff, and same with the econ e-journal (though it does appear somewhat generic if state of the art). i guess the old clr james idea of every cook a governor can be threatening---who could sell a special sauce that disintguishes memes such as french fries from hamburgers, if the task could be left to typing monkees? would such lost cultural diversity prove suboptimal? sounds like a grant.
Public refereeing sounds like a great idea if it could be applied to journalism, and what passes for journalism on our mass media. I'm thinking of a web site along the lines of CJR which would post "interesting" examples of faux news and analysis for all to comment on. hopefully journalists would themselves be attracted to such a site, and possibly report on its activities and commentary. Bull shit economics "research" would be far less harmful if the bull shit "news" journals that report on such bull shit work were identified as such.
ReplyDeletePete Boettke @ Coordination Problem has a post today on the same basic idea. It seems like the future to me. The Wisdom of Crowds and all that. But I wonder if we know the right design to get the Wisdom of Crowds and avoid the Madness of Crowds. Do we need to filter the reviewers by restricting them to, say, tenured profs or something? Otherwise won't we get things like a rich scholarly literature on how we are being visited by aliens who do creepy experiments on us at night?
ReplyDelete