Meta-analysis is common in clinical research, less so in basic biology, but it is also proving useful in some basic research contexts. It should help improve research reproducibility.
Research is by nature collaborative. Whether working in individual labs on specific problems or in larger groups analyzing 'omics' data sets, most researchers seek to synthesize and compare their output with that of other related work. And yet such comparisons are often relatively qualitative and anecdotal. Meta-analysis, a statistical approach that combines independent studies testing the same hypothesis and that can determine whether a result holds across that larger sample, is not widespread across basic research.
In the clinical world, meta-analysis is common. Such analysis may combine studies on whether a drug affects a particular condition, for example, or studies investigating the influence of dietary habits on health. Organizations like Cochrane and the Medical Letter are dedicated to conducting meta-analyses or critical, unbiased reviews of medical evidence to help doctors, patients and others make informed decisions about health guidelines and care.
This prevalence in clinical research is in many ways not surprising: clinical studies typically have a few, relatively simple endpoints or measures and, unlike exploratory basic research, are more likely to adhere to a standard study design. As such they are much more conducive to meta-analytic synthesis.
To be sure, there are areas in basic biology where meta-analysis is far from a foreign concept. Genome-wide association studies (GWAS) determine statistical correlations between genomic markers and traits or diseases; combining data from many studies increases the power to detect weak effects and permits the robustness of associations to be assessed across multiple cohorts. Meta-GWA studies, for instance, have identified many more markers for Crohn's disease than individual studies have. Researchers in metagenomics are also making increasing use of the approach, either to compare samples across studies or to test the robustness of a single set of conclusions. These examples illustrate a feature of meta-analysis: it is valuable when additional statistical power is needed to overcome small effect sizes or sample sizes.
In most basic research, scientists may well be able to collect enough data within their own labs to detect the effect they are studying, reducing the need for this type of synthetic, cross-study effort. But in the face of contradictory conclusions or when results cannot be reproduced, neither of which is a rare scenario, meta-analysis could help resolve the conflict, identify and correct for confounders, or point to the fact that the biology is more complex than initially thought.
Meta-analysis of mouse behavioral phenotyping, for instance, has been proposed as the basis for modeling genotype–laboratory interactions and for setting thresholds to identify phenotypes that should be more readily replicated across laboratories. In separate work, meta-analysis of studies spanning more than a decade was used to resolve contradictory reports on brain regions involved in short term memory in the fruitfly. Similarly, in metagenomics, meta-analysis of the composition of the human fecal microbiome in lean and obese people found no evidence for an 'obesity-associated' microbiome, as had been reported in some (but not all) previous studies; instead it found that interstudy variability was larger than the difference between lean and obese hosts. In yet another example, an analysis of the electrophysiological properties of neurons measured in multiple studies identified experimental factors (animal age and electrode type, among others) as a source of variability; correcting for these rendered the data more reproducible across labs.
Reproducibility also stands only to gain from the increased attention to transparent reporting of methods and experimental metadata that is needed for this approach. Effective meta-analysis requires that data be accessible in a suitable format, well annotated, and comparable in a way that is biologically meaningful. To determine whether the last criterion is met, researchers must be able to assess the details of the underlying experiments: the studies to be combined must use methods that are measuring comparable parameters; the samples and how they were processed should be compatible, given the biological question being asked. Any filtering or bioinformatic data processing must be assessed for its potential to bias the results, in which case appropriate normalization is needed. And finally, any criteria or bias in data selection must be stated.
Although the notion of combining similar experiments may seem simple, meta-analysis is far from trivial; to do it properly one needs both the statistical and the biological chops. It does not fit as naturally with all basic research as it does with the clinical. But it would be no mistake to more frequently consider this approach to help make better sense of the vast body of complex and sometimes contradictory evidence supporting our understanding of biological systems.
Rights and permissions
About this article
Cite this article
Meta-analysis in basic biology. Nat Methods 13, 959 (2016). https://doi.org/10.1038/nmeth.4102
Published:
Issue Date:
DOI: https://doi.org/10.1038/nmeth.4102