Chinese swimmer Ye Shiwen broke the world record for the women's 400-metre individual medley event at the Olympic Games on 28 July.  Credit: L. Neal /AFP / Getty Images

See also Editors’ note | Letter from Lai Jiang | Editors’ note (continued)

At the Olympics, how fast is too fast? That question has dogged Chinese swimmer Ye Shiwen after the 16-year-old shattered the world record in the women's 400-metre individual medley (400 IM) on Saturday. In the wake of that race, some swimming experts wondered whether Ye’s win was aided by performance-enhancing drugs. She has never tested positive for a banned substance and the International Olympic Committee on Tuesday declared that her post-race test was clean. The resulting debate has been tinged with racial and political undertones, but little science. Nature examines whether and how an athlete's performance history and the limits of human physiology could be used to catch dopers.

Was Ye’s performance anomalous?

Yes. Her time in the 400 IM was more than 7 seconds faster than her time in the same event at a major meet in July 2011. But what really raised eyebrows was her showing in the last 50 metres, which she swam faster than US swimmer Ryan Lochte did when he won gold in the men’s 400 IM on Saturday, with the second-fastest time ever for that event.

Doesn't a clean drug test during competition rule out the possibility of doping?

No, says Ross Tucker, an exercise physiologist at the University of Cape Town in South Africa. Athletes are much more likely to dope while in training, when drug testing tends to be less rigorous. “Everyone will pass at the Olympic games. Hardly anyone fails in competition testing,” Tucker says.

Out-of-competition tests are more likely to catch dopers, he says, but it is not feasible to test every elite athlete regularly year-round. Tracking an athlete over time and flagging anomalous performances would help anti-doping authorities to make better use of resources, says Yorck Olaf Schumacher, an exercise physiologist at the Medical University of Freiburg in Germany, who co-authored a 2009 paper proposing that performance profiling be used as an anti-doping tool1. “I think it’s a good way and a cheap way to narrow down a large group of athletes to suspicious ones, because after all, the result of any doping is higher performance,” Schumacher says.

The ‘biological passport’, which measures characteristics of an athlete’s blood to look for physiological evidence of doping, works in a similar way to performance profiling (see 'Racing just to keep up'). After it was introduced in 2008, cycling authorities flagged irregularities in the blood characteristics of Antonio Colom, a Spanish cyclist, and targeted drug tests turned up evidence of the banned blood-boosting hormone erythropoietin (EPO) in 2009.

How would performance be used to nab dopers?

Anti-doping authorities need a better way of flagging anomalous performances or patterns of results, says Schumacher. To do this, sports scientists need to create databases that — sport by sport and event by event — record how athletes improve with age and experience. Longitudinal records of athletes’ performances would then be fed into statistical models to determine the likelihood that they ran or swam too fast, given their past results and the limits of human physiology.

The Olympic biathlon, a winter sport that combines cross-country skiing and target shooting, has dabbled in performance profiling. In a pilot project, scientists at the International Biathlon Union in Salzburg, Austria, and the University of Ferrara in Italy, developed a software program that retroactively analysed blood and performance data from 180 biathletes over six years to identify those most likely to have doped2. The biathlon federation now uses the software to target its athletes for drug testing.

Could an athlete then be disciplined simply for performing too well?

“That would be unfair,” says Tucker. “The final verdict is only ever going to be reached by testing. It has to be.” In recent years, cycling authorities have successfully prosecuted athletes for having anomalous blood profiles, even when banned substances such as EPO could not be found. But performance is too far removed from taking a banned substance and influenced by too many outside factors to convict someone of doping, Tucker says. “When we look at this young swimmer from China who breaks a world record, that’s not proof of anything. It asks a question or two.”

EDITORS’ NOTE (updated 6 August 2012)

This article has drawn an extraordinary level of outraged response. The volume of comments has been so great that our online commenting system is unable to cope: it deletes earlier posts as new ones arrive. We much regret this ongoing problem. The disappearance of some cogent responses to the story has fuelled suspicions that Nature is deliberately censoring the strongest criticisms. This is absolutely not the case: Nature welcomes critically minded discussion of our content. (We intentionally removed only a few comments that violated our Community Guidelines by being abusive or defamatory, including several that offensively stereotyped the many Chinese readers who commented on the story.)

[UPDATE 8 August 2012: The technical problem has now been resolved and all of the posts that were inadvertently hidden have now been restored. In order to keep this problem from recurring, we have closed the story to further comments.]

Many of the commenters have questioned why we changed the original subtitle of the story from “‘Performance profiling’ could help catch sports cheats” to “‘Performance profiling’ could help dispel doubts”. The original version of the title was unfair to the swimmer Ye Shiwen and did not reflect the substance of the story. We regret that the original appeared in the first place. We also regret that the original story included an error about the improvement in Ye’s time for the 400-metre individual medley: she improved by 7 seconds since July 2011, not July 2012. We have corrected the error.

We apologize to our readers for these errors, and for the unintended removal of comments because of technical issues with our commenting system. Below we reproduce one of the most thorough and thoughtful of the hundreds of responses we received. Beneath it, we continue with our response.

FROM LAI JIANG, Department of Chemistry, University of Pennsylvania

It is a shame to see Nature — which nearly all scientists, including myself, regard as one of the most prestigious and influential physical-science magazines — publish a thinly veiled biased article like this. Granted, this is not a peer-reviewed scientific article and did not go through the scrutiny of picking referees. But to serve as a channel for the general populace to be in touch with and appreciate science, the authors and editors should at least present the readers with facts within the proper context, which they blatantly failed to do.

First, to identify Ye’s performance increase, Ewen Callaway compared her Olympic 400-metre IM time with her performance at the World Championships in 2011 (4:28.43 and 4:35.15, respectively) and concluded that she had an “anomalous” improvement of around 7 seconds (6.72 s). In fact, her previous personal best was 4:33.79 at the Asian Games in 2010. This leads to an improvement of 5.38 seconds. In a sporting event in which 0.1 s can be the difference between the gold and silver medal, I see no reason for 5.38 s to be treated as 7 s.

Second, as previously pointed out, Ye is only 16 years old and her body is still developing. Bettering oneself by 5 seconds over two years may seem impossible for an adult swimmer, but it certainly happens among youngsters. An interview with Australian gold medallist Ian Thorpe revealed that his 400-metre freestyle time improved by 5 seconds between the ages of 15 and 16. For regular people, including Callaway, it may be hard to imagine what an elite swimmer can achieve as he or she matures and undergoes scientific and persistent training. But jumping to the conclusion that it is “anomalous” based on ‘Oh that’s so tough I cannot imagine it is real’ is hardly sound.

Third, to compare Ryan Lochte’s last 50 metres to Ye’s is a textbook example of ‘cherry-picking’ your data. Yes, Lochte was slower than Ye in the last 50 metres, but Lochte had a huge lead in the first 300 metres, so he chose not to push himself too hard and to conserve his energy for later events (whether this conforms to the Olympic spirit and the ‘use one’s best efforts to win a match’ requirement that the Badminton World Federation recently invoked to disqualify four badminton pairs is another topic worth discussing, though probably not in Nature). Ye, on the other hand, was trailing behind after the first 300 metres and relied on freestyle, in which she has an edge, to win the race. Failing to mention this strategic difference, as well as the fact that Lochte is 23.25 seconds faster (4:05.18) than Ye overall, creates the illusion that a woman swam faster than the best man in the same sport, which sounds impossible. Putting aside the gender argument, I believe this is still a leading question that implies to the reader that there is something fishy going on.

Fourth is another example of cherry-picking. In the same event, there are four male swimmers who swam faster than both Lochter (29.10 s) and Ye (28.93 s) in the final 50 metres: Kosuke Hagino (28.52 s), Michael Phelps (28.44 s), Yuya Horihata (27.87 s) and Thomas Fraser-Holmes (28.35 s). As it turns out, if we are just talking about the last 50 metres in a 400-metre IM, Lochter is not the example I would have used if I were the author. What kind of scientific rigorousness is Callaway trying to demonstrate here? Is it logical that if Lochter is the champion, we should assume that he leads in every split? That would be a terrible way to teach the public how science works.

Fifth is the issue I oppose the most. Callaway quotes Ross Tucker and implies that a drug test cannot rule out the possibility of doping. Is this kind of agnosticism what Nature really wants to teach its readers? By that standard, I estimate that at least half of the peer-reviewed scientific papers in Nature should be retracted. How can one convince the editors and reviewers that their proposed theory works for every possible case? One cannot. One chooses to apply the theory to typical examples and to demonstrate that in (hopefully) all scenarios considered, the theory works to a degree, and that that should warrant publication until a counterexample is found. I could imagine that Callaway has a sceptical mind, which is crucial to scientific thinking, but that would be put to better use if he wrote a peer-reviewed paper that discussed the odds of Ye doping on a highly advanced, non-detectable drug that the Chinese have come up with in the past 4 years (they obviously did not have it in Beijing, otherwise why not use it and woo the audience at home?), based on data and rational derivation. This article, however, can be interpreted as saying that all athletes are doping and the authorities are just not good enough to catch them. That may be true, logically, but definitely will not make the case if there is ever a hearing by the governing body for water sports, FINA, to determine if Ye has doped. To ask whether it is possible to obtain a false negative in a drug test looks like a rigged question to me. Of course it is possible: other than the athlete taking a drug that the test is not designed to detect, anyone who has taken quantum 101 will tell you that everything is probabilistic in nature, and so there is a probability that the drug in an athlete’s system could tunnel out right at the moment of the test. A slight chance it may be, but should we disregard all test results because of it? Let’s be practical and reasonable, and accept that the World Anti-Doping agency (WADA) is competent at its job. Ye’s urine sample will be stored for eight years after the contest for future testing as technology advances. Innocent until proven guilty, shouldn’t it be?

Sixth, and the last point I would like to make, is that the out-of-competition drug test is already in effect, which Callaway failed to mention. As noted in the president of WADA’s press release, drug testing for Olympians began at least six months before the opening of the London Olympics. Furthermore, 107 athletes have been banned from this Olympics for doping. That may be the reason that “everyone will pass at the Olympic games. Hardly anyone fails in competition testing” — those who did dope have already been caught and sanctioned. Callaway is free to suggest that a player could have doped beforehand and fooled the test at the game, but this possibility is certainly ruled out for Ye.

Over all, even though Callaway did not falsify any data, he did (intentionally or not) cherry-pick data that, in my view, are far too suggestive to be fair and unbiased. If you want to cover a story of a suspected doping from a scientific point of view, be impartial and provide all the facts for the reader to judge. You are entitled to your interpretation of the facts, and the expression thereof in your piece, explicitly or otherwise, but showing only evidence that favours your argument is hardly good science or journalism. Such an article in a journal such as Nature is not an appropriate example of how scientific research or reporting should be done.

EDITORS’ NOTE (continued)

The news story was triggered by a debate that was already active, concerning the scale of Ye Shiwen’s victory. Such debates have arisen over many outstanding feats in the past, by athletes from many countries, and it is wrong to suggest, as many of the critics do, that we singled her out because of her nationality.

The story’s intention as an Explainer was to examine how science can help resolve debates over extraordinary performances, not to examine those performance statistics in detail. Several analyses done by others convinced us that it was fair to characterize Ye’s performance as ‘anomalous’ — in the sense that it was statistically unusual. But we acknowledge that the combination of errors discussed above and the absence of a more detailed discussion of the statistics (which with hindsight we regret) gave the impression that we were supporting accusations against her, even though this was emphatically not our intention. For that, we apologize to our readers and to Ye Shiwen. Tim Appenzeller Chief Magazine Editor, Nature Philip Campbell Editor-in-Chief, Nature