Professors emeritus Les Hatton and Gregory Warr think peer review has run its course, as noted in the blog on Times Higher Education:
First, peer review is self-evidently useful in protecting established paradigms and disadvantaging challenges to entrenched scientific authority. Second, peer review, by controlling access to publication in the most prestigious journals helps to maintain the clearly recognised hierarchies of journals, of researchers, and of universities and research institutes. Peer reviewers should be experts in their field and will therefore have allegiances to leaders in their field and to their shared scientific consensus; conversely, there will be a natural hostility to challenges to the consensus, and peer reviewers have substantial power of influence (extending virtually to censorship) over publication in elite (and even not-so-elite) journals.
Publication in the highest-profile journals reinforces the hierarchies of status in the scientific community and promotes very effectively the prestige-, career- and profit-driven motives of authors, journal editors, publishers and (less directly) universities. This state of affairs exerts a particularly baleful influence on interdisciplinary research.
I suppose that, as someone who’s never published a paper and doesn’t hardly read beyond abstracts, I should be properly bashful and keep my thoughts to myself – but what fun is that?
I can’t help but notice these condemnatory paragraphs are all about implementation and not about design. That is to say, some constructs, tangible or intangible, are designed to fail, and no matter the quality of the components to build the construct, it will fail when implemented. Once a design is proven to be inadequate to its needs, it should be discarded, and in a better world, it will be instantly.
But that’s not what I’m reading here. Read that first paragraph again. These are not complaints about how peer review is designed, only in how it’s being implemented. The design of peer review is to have personnel competent in the relevant field review the methods described in a given paper to ensure they are appropriate for the collection and analysis of the data, and to ensure those methods have been properly employed (i.e., did they do 2+2 and get 4?). This must encompass papers that argue new methodologies for collecting and analyzing data.
None of this can be construed for necessarily requiring “… protecting established paradigms and disadvantaging challenges to entrenched scientific authority.” In poker, you can bluff all you like, but in the end, the cards talk. Same in the best papers and research – your ideology may call for water to have a memory, but when the evidence says no, it says no.
Source: Futurism.com
But as any good design engineer knows, you can always design something that is worthless because the proper materials are not available or are too expensive. Think of the proposed Space Elevator, basically a cable anchored at some point on the equator, extending straight upwards well beyond geostationary orbit (35,786 km altitude) with a counterweight on the end. Once available, it’d make it far cheaper than today’s lift rates to get payloads into orbit – but no known material has the characteristics necessary to implement the Space Elevator (some might argue that, so put one up and falsify the general argument, eh?).
If we extend this idea to peer review, then the question is whether the parts – the human beings – involved are incapable of fulfilling the requirements of honestly evaluating the methodologies, calculations, and claims of a research paper. And I suspect the professors have actually answered that question with a resounding No!, because they note elsewhere that the papers themselves are not reaching the rigorous standards appropriate for scientific research:
However, for any innovations in scientific publication to succeed two conditions would need to be met. The first, as noted above, is the provision with a publication of all the information necessary for independent reproduction and repeatability of the research, and the second is the improvement in the culture of science such that less than rigorous work and deceptive publication practices are no longer tolerated.
Amen. This applies to the peer review, in spirit, as well, and I believe is attainable. In that first paragraph, they suggest that peer review simply functions to shore up old paradigms and reputations. But this is not self-evident, because, ideally, peer reviewers are anonymous and honest. The first is easy enough to ensure, while the second is somewhat problematic. But the point is that because they are anonymous, no reputations are at risk or enhanced – at least, not profitably to the honest peer reviewer.
I suspect in today’s environment the wrong motivations / goals are employed. The goals of cash and jealous prestige are always the flies in the ointment that is science, and occasionally these must be met and put down, like rabid dogs, to remind the individual members of the scientific community that cutting corners in the pursuit of momentary wealth, or jealously guarding your reputation by shattering incipient new paradigms, only does harm to the entire scientific enterprise. At their very core, scientists are striving for the clearest possible vision of reality; letting the cataracts of wealth, fame, jealousy, and other such human foibles form over your eyes simply wrecks the view – for everyone.
