Most claims that men and women are affected differently by disease-associated gene variations are poorly founded. A team of researchers has found that the data supporting such claims are often poorly analysed statistically or come from studies that were not adequately designed to show these links.

“The abysmal standard of statistical analysis in much of genetic epidemiology is little short of scandalous,” says David Balding, professor of statistical genetics at Imperial College London, UK, who was not involved in the study. “This paper reveals an entire industry of prominently reported results that are largely unjustified and probably mostly false.”

John Ioannidis and his colleagues at the University of Ioannina School of Medicine in Greece evaluated 432 claims in 77 research papers (N. Patsopoulos et al. J. Am. Med. Assoc. 298, 880–893; 2007). The team applied a set of criteria to determine whether the papers' authors had performed the correct analysis, such as comparing like with like, and had taken steps to show that the association was not due to chance. Worryingly, only 12.7% of claims satisfied these criteria. “There is quite a gap between what should have been done and what the journals and reviewers should have asked for, compared with what the authors did,” says Ioannidis.

Many studies were not designed to test for a link between sex and gene variants, with researchers trying to extract associations from their data after the fact. Sample sizes were at least ten times smaller than they needed to be to yield statistically robust results, Ioannidis adds.

“This paper tells us that we don't have a clue whether gender is a real biomarker for any of the clinical areas assessed,” says Howard McLeod, director of the UNC Institute for Pharmacogenomics and Individualized Therapy in Chapel Hill, North Carolina. “Gender, as well as age and race, are crude ways of understanding the complex factors regulating clinical effect,” he adds.