Sir

In his Commentary 'The science of doping' (Nature 454, 692–693; 2008), Donald Berry uses data from the Floyd Landis case as grounds for his claim that drug-testing practices are often based on flawed statistics. His concerns stand in contrast to the US Anti-Doping Agency's view of the outcome of the case (quoting from R. Mukhopadhyay and J. Griffiths Anal. Chem. 79, 8823–8825; 2007): “It's really easy to play Monday-morning quarterback and see an i that's not dotted or a t that's not crossed, but that in no way undermined the validity or the reliability of the work that was done by the French lab.”

This quote seems frivolous in the context of a class of drug-testing practices. These are based on separation of a sample's constituents by chromatography and detection of target compounds by mass spectrometry, one of the work-horses in anti-doping research. Identification is reported as positive when the test and reference sample signals agree within a particular tolerance window.

The size of this window is not constructed with an acceptable risk of false positives in mind. Rather, fixed decision criteria hold, regardless of the quality of the laboratory or the signal properties of the target compound. However, a laboratory that produces relatively precise results should deploy stricter criteria. Likewise, target compounds should be differentiated so that information in their signals can be respected.

Always deploying the same rigid criteria leads to a probability of false positives that depends on the particular laboratory and target compound in an undefined way. This situation is frustrating because the statistical solution — flexible criteria that account for various complications — was already published and thoroughly tested five years before these rigid criteria were introduced.

So, is it ignorance of the literature or failure to understand the analytical problem at hand that underlies the ongoing usage of these arbitrary decision rules? Laboratories, as well as their clients and (re-)accrediting organizations, should start reflecting on their accountability with regard to this avoidable malpractice.

See also Doping: world agency sets standards to promote fair play