How many elite athletes take performance enhancing drugs? Sporting bodies say that it is a very small minority. But a documentary broadcast in Germany last month suggested a much higher figure. Several Russian athletes claimed that nearly all of their colleagues dope, and with the knowledge of officials. The World Anti-Doping Agency (WADA) immediately launched an investigation, which is expected to report this year.

Science helps to keep sport clean by developing tests to screen athletes for banned substances. Bodies such as WADA and the US Anti-Doping Agency (USADA) say that they are doing all they can to deter doping. But they have so far neglected to carry out a simple scientific analysis of how widespread the problem is. Or if they have, they have not published the results. This makes it impossible for the rest of us to assess whether anti-doping policies are working.

Drug testing in sport, as currently implemented, might catch the occasional cheat and could deter others, but these results do little to help design an anti-doping strategy, and to independently assess whether it works. For that, we need to know whether the number of athletes doping is going up or down. And to do that, we need a reliable measure of what proportion of athletes dope. The problem — and the best way to manage it — is very different if 1% of athletes dope than if 50% of them do.

Although the stated goal of anti-doping agencies is to prevent prohibited drug use, they simply do not gather the data to enable evaluation of how effective their policies are. This is despite sporting bodies across the world spending an estimated US$350 million on drug testing each year.

Estimating the number of elite athletes who dope is straightforward, and perfectly suited to the tools of science. Determining this number is much easier than other efforts by scientists to quantify unknowns, such as estimates of the number of planets in the Galaxy or whales in the sea. In probability-speak, it is a ball and urn problem: how do we determine how many black balls there are in an urn that contains 1,000 white and black balls if we can sample only a small number?

To assess the prevalence of sports doping, such an analysis needs two things: a reliable estimate of the total population of elite athletes and a proper randomized testing protocol. The first is readily available. For instance, at the London 2012 Summer Olympics, nearly 11,000 athletes participated from more than 200 countries. Each country conducted Olympic trials with its own pool of registered, domestic competitors seeking to qualify for the games. For the second requirement, because screening every athlete over a year is impractical, anti-doping agencies could carry out randomized tests designed to support estimates of the prevalence of doping alongside existing testing programmes at a marginal cost.

Anti-doping agencies suffer from a sort of institutionalized blindness.

Current doping tests are anything but random, at least in a statistical sense. Some athletes are tested several times, others not at all. In 2013, USADA says that it conducted 9,197 tests on 4,640 athletes. Decisions on which athletes were subjected to these tests were determined ‘strategically’, it says. The number of positive tests, then, cannot be used to say anything about a broader population. The same is true for existing global statistics. WADA says that it tested 176,502 samples (not individual athletes) in 2013, and that 1% gave ‘adverse analytical findings’ (AAFs).

But such a red flag does not necessarily mean that doping has occurred, because some athletes have exemptions for prohibited substances, for instance. Nor do the data allow for the matching of AAFs to sanctions against athletes. So of the 176,502 samples, what does a 1% AAF actually mean? It is impossible to say, and that is the problem.

Why has there been no effort to quantify the problem of doping in sport (or if it has been done, why is it not published)? Evidence suggests that the leaderships of these organizations do not want to know the true extent of doping or their effectiveness in regulating it. In 2012, Richard Pound, the first president of WADA, oversaw an agency committee called Lack of Effectiveness of Testing Programs. The committee’s report concluded that within the sports community, “there is no general appetite to undertake the effort and expense of a successful effort to deliver doping-free sport”.

WADA, created after a major drug scandal in cycling in the late 1990s, is unique in that it is overseen by governments in partnership with non-governmental sports organizations, and operates under the provisions of a United Nations treaty. In principle, this signifies a public responsibility and expectations of accountability to stated goals. The UN treaty gives these agencies legitimacy, and thus no excuse not to be transparent.

In my opinion, anti-doping agencies suffer from a sort of institutionalized blindness that has been characterized by Steve Rayner, who studies science and civilization at the University of Oxford, UK, as the “social construction of ignorance”. This is a strategy that organizations use necessarily to make their way in a complicated world. Organizations also create zones of ignorance to ‘manage uncomfortable knowledge’, and this can sometimes lead to dysfunction.

In the case of doping in sport, uncomfortable knowledge includes the possibility that doping among athletes is much more prevalent than is recognized and that anti-doping programmes are not very effective. But without a proper effort to gather the data, we just don’t know.

figure a

Univ. Colorado