The problem is how to replace peer review and with what.

The time has come for us to take a serious look at the institution of peer review. It is not a good way to catch misconduct or outright fraud in science, because the reviewer quite naturally accepts the account of how and why the research was done. But detecting fraud is not the purpose of peer review.

Peer review is instead a good way to identify valid science. Of course, a referee may occasionally fail to appreciate a truly visionary or revolutionary idea, but, by and large, peer review works pretty well as long as scientific validity is the issue at stake. However, it is not at all suited to arbitrating an intense competition for research funds or for editorial space in prestigious journals. There are many reasons for this, not the least being the fact that the referees have an obvious conflict of interest: they are themselves competitors for the same resources.

This point seems to be another of those relativistic anomalies, obvious to any outside observer but invisible to those of us who are falling into the black hole. It would take impossibly high ethical standards for referees to avoid taking advantage of their privileged anonymity to advance their own interests. As time goes on, more and more referees risk having their standards eroded because they themselves, as authors, have been victimized by unfair reviews.

Peer review is thus one example among many of practices that were well suited to the period, lasting until the early 1970s, when science was undergoing exponential expansion. But that time is gone forever, and the system of peer review will become increasingly dysfunctional in the difficult future we face.

The problem is how to replace peer review and with what. As long as journal editors and contract monitors find it a convenient way of making hard choices, I fear that peer review will continue to hold its exalted place in our system of science.