Speaking of the retraction of scientific papers, Neuroskeptic is pondering the question of peer review:
Is it the job of peer reviewers to detect scientific fraud?
I’ve been pondering this question for a while but lately my interest was sparked by the case of a retracted cancer biology paper in the high-profile journal Nature Cell Biology. Written by Taiwanese researchers Shih-Ting Cha et al., the article was published on the 15th August and retracted just three months later, after anonymous posters on PubPeer noticed several anomalies in the results.
For instance, there was image duplication: the paper contained identical images that were meant to be of different mice [image omitted].
It seems to me that a publisher should make every effort to validate the papers it publishes, not as a matter of honor or good taste, but as a matter of survival. Like other institutions, publishers are subject to evolutionary pressures, and in this case we’re talking about putting a premium on truth and reality. A publisher that gains a reputation for shoddy, fallacious papers within the community of scientists will lose both readers and quality content – a vicious vortex.
[I’ll now pause and consider the evolutionary pressures on religious publishers.]
I think scientific publishers should be taking a systematic approach to the problem, and that should include the use of our quasi-artificial intelligence systems to investigate possible image duplication, not only within papers, but stolen from other papers as well, as well as attempting to do the tedious validation of statistical analysis, if only in the mathematics – actually judging the validity of any particular approach may be beyond an AI system. (Or maybe it’s easy. I do not keep up with AI advances.)
Certainly the role of a peer reviewer remains important in judging the importance and quality of a paper overall. But we do need to remember that, after all this effort, a paper can still be wrong or irrelevant. Something both Neuroskeptic and his correspondents either ignored or forgot about is the role that study replication plays in the process of producing good science. A single experiment is rarely adequate; it’s more like a single torch on the path. Replication is as important, if not more so, as peer review.
Or, as engineers working on far more critical systems than I do, think about it, it’s all about redundancy.
