Mob rules for scientific peer review?
science Tuesday, February 2, 2010 . This is a SciScoop post by David Bradley
According to the BBC, leading stem cell researchers (Robin Lovell-Badge, NIMR and Austin Smith, Wellcome/U Cambridge) have apparently complained in an open letter to the editors of scientific journals* that their work is being stifled by a small group of elite scientists vetoing high-quality research using the anonymous peer review system. (*If anyone can find a link for this letter, I’d be grateful.)
They claim that it is being done to deliberately to stifle the research of competitors. Missing a placement of one’s research paper in prestigious journals such as Cell, Nature, and Science, can mean the difference between getting hundreds of thousands of pounds in public funding for a research project…and not.
Billions of pounds of public money is spent on funding stem cell research, says the BBC report.
Once again, the whole peer-review process is under the spotlight, with disgruntled scientists asserting that the whole system, in which other scientists get to anonymously review one’s work, is broken. Its defenders would disagree and say that it’s the best mechanism we have for validating science.
The BBC reports that Philip Campbell, Editor of Nature, said: “Last year we used about 400 reviewers in stem cell and developmental biology, and we constantly recruit new referees. The idea that there’s some privileged clique is utterly false.”
Of course, there are some problems with the system. After all, the number of scientific frauds that have been exposed over the years seems to be rising. There is an inherent delay to the publication process because of the refereeing step. There is also the issue of the constant clamoring for publication in “prestigious” journals and the disappointment and failed grants that ensue when one fails to make the grade. An open system with crowd-sourced review might avoid such issues.
In such a system, there would be no distinction between the journals, papers would simply be deposited in a vast open database. No one would worry about impact factors, and any Etonian type playing fields that were not leveled would be relaid horizontal.
That said, one can imagine that niches and cliques would ultimately emerge even in such a system. This quickly became apparent on social media sites like Digg and delicious where crowd review is the norm. I cannot see how would science avoid that in an open system.
- peer review research science
Previously: « Alcohol addiction and nutrition
- Next:Valentine for a scientist »
Comment Form
About
SciScoop Science News is a forum for news, views and controversial conjectures. Please contact us if would like to submit a guest post.
SciScoop Top Authors
- Total posts: 3177
- David Bradley (416)
- Bunyip (35)
- Chad (85)
- Drog (407)
- DV8 2XL (25)
- Wayne Goode (69)
- Gpmap (27)
- Ricky James (1367)
- Mertero (22)
- Mtigges (34)
- AP Smith (171)
- Sweetwind (64)
5 Responses to Mob rules for scientific peer review?
Hector
February 2nd, 2010 at 1:43 pm
This is a concern that is constantly being brought up in scientific circles. I have seen an example of it once, but that is why there are 3 or more reviewers of an article. In this case, the other two reviewers position on the article was so strong it over rode the one reviewer.
Unfortunately, you see this complaint mostly from individuals who did not get their papers accepted by a panel of their peers. I think that any time you do work that goes against the established body of evidence, you need to make sure that there is no question regarding experiments, i.e. controls. It took me 3 years to get the proper controls finished for an observation on an enzymatic activity that questioned the established model for a key system in DNA precursor metabolism and 2 years later, the papers is still under review. That is how science works.
Hector
David Bradley
February 2nd, 2010 at 4:40 pm
Many thanks for your input on this debate Hector. When I was in the business of selecting referees for chemistry journal papers a long time ago, the process got tougher the more specialist the area. Moreover, our standard was to pick two initial referees and if they disagreed we went to a third, if there was still an issue, then an editorial board member was enlisted for a final decision.
I presume that in defense of the current open letter, that the niche of experts is perhaps so small in some fields that two referees, and even three referees all know each other well (something is awry if the experts in a field don’t know each other) then collusion is easier if their aim is to exclude work from specific individuals, or just competitors in general.
I don’t personally buy into this conspiracy theory, but it is a possibility. I don’t think it’s necessarily sour grapes on the part of anyone who simply didn’t get into everyone’s favourite journals…
Maximilian Koskull
February 7th, 2010 at 12:33 pm
An interesting topic and thanks for posting. I have not heard from it before.
Besides, here is the link to the open letter:
http://www.eurostemcell.org/commentanalysis/peer-review
It has already been published in July 2009, so I am wondering why it took so long until several media discussed and spread the letter´s content… (Or perhaps I have missed any former news concerning this matter?)
Best regards,
Maximilian.
Manu Field
February 8th, 2010 at 9:46 pm
“our standard was to pick two initial referees and if they disagreed we went to a third, if there was still an issue, then an editorial board member was enlisted for a final decision.”
David, you have high standards and from bitter experience I wish some of the journals I have had the misfortune to deal with were as enlightened. Often, I’ve had papers rejected when only one of two or three reviewers had an issue (i.e. a minority verdict), with no option to address any issues raised or to appeal to a member of the editorial team.
Perhaps journal Editors should take on the role of deciding what submitted papers would or would not be suitable for publication in their journal. Thus, reviewers would essentially have to have to justify why the Editor was wrong. Obviously, this process would still have problems, but then at least the decision to reject would be taken by someone not hiding behind the screen of anonymity!
David Bradley
February 9th, 2010 at 7:40 am
Yes, I realise that the publisher for which I worked had high standards. There is another model akin to what you cite used by Proc Natl Acad Sci, in which papers have to be submitted on one’s behalf by an NAS member, that’s akin to an extended editorial board first choosing what’s worth publishing before it gets refereed. I’m not 100% sure, but someone can correct me, but don’t the editors at Nature, Cell, and Science filter out the papers before assigning referees. We had the prerogative to do that, but only in extreme cases where the paper was obviously not even readable or was so obviously out of the field covered by the journal in question.