Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Quantifying research waste in ecology

Abstract

Research inefficiencies can generate huge waste: evidence from biomedical research has shown that most research is avoidably wasted and steps have been taken to tackle this costly problem. Although other scientific fields could also benefit from identifying and quantifying waste and acting to reduce it, no other estimates of research waste are available. Given that ecological issues interweave most of the United Nations Sustainable Development Goals, we argue that tackling research waste in ecology should be prioritized. Our study leads the way. We estimate components of waste in ecological research based on a literature review and a meta-analysis. Shockingly, our results suggest only 11–18% of conducted ecological research reaches its full informative value. All actors within the research system—including academic institutions, policymakers, funders and publishers—have a duty towards science, the environment, study organisms and the public, to urgently act and reduce this considerable yet preventable loss. We discuss potential ways forward and call for two major actions: (1) further research into waste in ecology (and beyond); (2) focused development and implementation of solutions to reduce unused potential of ecological research.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Stages of the classical research life cycle.
Fig. 2: Overall estimate of research waste of ecological research based on a meta-analysis of waste at each stage (with examples of causes).
Fig. 3: Estimates of the main components of research waste.

Data availability

The data needed to reproduce the analyses and create the main text and supplementary figures have been deposited at Zenodo35 https://doi.org/10.5281/zenodo.6566100. These include the original effect sizes as extracted from studies and the final set of the effect sizes used in the meta-analysis. Source data are provided with this paper.

Code availability

The codes/scripts needed to reproduce the analyses and create the main text and supplementary figures are deposited at Zenodo35 https://doi.org/10.5281/zenodo.6566100.

References

  1. Ioannidis, J. P. A., Fanelli, D., Dunne, D. D. & Goodman, S. N. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 13, e1002264 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  2. Hampton, S. E. et al. The Tao of open science for ecology. Ecosphere 6, art120 (2015).

    Article  Google Scholar 

  3. Rothstein, H. R., Sutton, A. J. & Borenstein, M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (John Wiley & Sons, 2005).

  4. Sutton, A. J. in The Handbook of Research Synthesis and Meta-Analysis (eds Cooper, H. et al.) 435–452 (Russell Sage Foundation, 2009).

  5. Nakagawa, S., Koricheva, J., Macleod, M. & Viechtbauer, W. Introducing our series: research synthesis and meta-research in biology. BMC Biol. 18, 20 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Nakagawa, S. et al. A new ecosystem for evidence synthesis. Nat. Ecol. Evol. 4, 498–501 (2020).

    Article  PubMed  Google Scholar 

  7. Coolidge, H. J. & Lord, R. H. in Archibald Cary Coolidge: Life and Letters 308 (Houghton Mifflin Harcourt, 1932).

  8. Nickerson, R. S. Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220 (1998).

    Article  Google Scholar 

  9. Touchon, J. C. & McCoy, M. W. The mismatch between current statistical practice and doctoral training in ecology. Ecosphere 7, e01394 (2016).

    Article  Google Scholar 

  10. Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483, 531–533 (2012).

    Article  CAS  PubMed  Google Scholar 

  11. Aarts, A. A. et al. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).

    Article  CAS  Google Scholar 

  12. Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).

    Article  PubMed  Google Scholar 

  13. Fraser, H., Parker, T., Nakagawa, S., Barnett, A. & Fidler, F. Questionable research practices in ecology and evolution. PLoS ONE 13, e0200303 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  14. Culina, A., van den Berg, I., Evans, S. & Sánchez-Tójar, A. Low availability of code in ecology: a call for urgent action. PLoS Biol. 18, e3000763 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Jennions, M. D. & Møller, A. P. Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method. Biol. Rev. Camb. Philos. Soc. 77, 211–222 (2002).

    Article  PubMed  Google Scholar 

  16. Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behavior. Behav. Ecol. 14, 438–445 (2003).

    Article  Google Scholar 

  17. Cassey, P., Ewen, J. G., Blackburn, T. M. & Møller, A. P. A survey of publication bias within evolutionary ecology. Proc. Biol. Sci. 271, S451–S454 (2004).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kardish, M. R. et al. Blind trust in unblinded observation in ecology, evolution, and behavior. Front. Ecol. Evol. 3, 51 (2015).

    Article  Google Scholar 

  19. Jennions, M. D. & Møller, A. P. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proc. Biol. Sci. 269, 43–48 (2002).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016).

    Article  CAS  PubMed  Google Scholar 

  21. Chalmers, I. & Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet 374, 86–89 (2009).

    Article  PubMed  Google Scholar 

  22. Altman, D. G. The scandal of poor medical research. BMJ 308, 283–284 (1994).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Glasziou, P. & Chalmers, I. Is 85% of health research really ‘wasted’? BMJ Opinion (14 January 2016).

  24. Glasziou, P. & Chalmers, I. Research waste is still a scandal. BMJ 363, k4645 (2018).

    Article  Google Scholar 

  25. Chalmers, I. et al. How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165 (2014).

    Article  PubMed  Google Scholar 

  26. Kunin, W. E. Robust evidence of declines in insect abundance and biodiversity. Nature 574, 641–642 (2019).

    Article  CAS  PubMed  Google Scholar 

  27. Christie, A. P. et al. Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nat. Commun. 11, 6377 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Campbell, H. A. et al. Finding our way: on the sharing and reuse of animal telemetry data in Australasia. Sci. Total Environ. 534, 79–84 (2015).

    Article  CAS  PubMed  Google Scholar 

  29. Koricheva, J. Non-significant results in ecology: a burden or a blessing in disguise? Oikos 102, 397–401 (2003).

    Article  Google Scholar 

  30. Bennett, L. T. & Adams, M. A. Assessment of ecological effects due to forest harvesting: approaches and statistical issues. J. Appl. Ecol. 41, 585–598 (2004).

    Article  Google Scholar 

  31. Duval, S. & Tweedie, R. A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98 (2012).

    Google Scholar 

  32. Brlík, V. et al. Weak effects of geolocators on small birds: a meta-analysis controlled for phylogeny and publication bias. J. Anim. Ecol. 89, 207–220 (2020).

    Article  PubMed  Google Scholar 

  33. Hurlbert, S. H. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187–211 (1984).

    Article  Google Scholar 

  34. Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behaviour. Behav. Ecol. 14, 438–445 (2003).

    Article  Google Scholar 

  35. Culina, A., Purgar, M. & Klanjscek, T. Datasets and codes for Purgar et al. 2022: quantifying research waste in ecology. Zenodo https://zenodo.org/record/6566100#.YrLWB-zMIqs (2022).

  36. Ferguson, C. et al. Europe PMC in 2020. Nucleic Acids Res. 49, D1507–D1514 (2021).

    Article  CAS  PubMed  Google Scholar 

  37. Huang, C.-K. et al. Meta-Research: Evaluating the impact of open access policies on research institutions. eLife 9, e57067 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Ross-Hellauer, T. Open science, done wrong, will compound inequities. Nature 603, 363 (2022).

    Article  CAS  PubMed  Google Scholar 

  39. Smith, A. C. et al. Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s ‘Mirror journal’ system. Quant. Sci. Stud. 2, 1123–1143 (2021).

    Article  Google Scholar 

  40. Christie, A. P. et al. Reducing publication delay to improve the efficiency and impact of conservation science. PeerJ 9, e12245 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Desjardins-Proulx, P. et al. The case for open preprints in biology. PLoS Biol. 11, e1001563 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  43. O’Dea, R. E. et al. Towards open, reliable, and transparent ecology and evolutionary biology. BMC Biol. 19, 68 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Culina, A. et al. Navigating the unfolding open data landscape in ecology and evolution. Nat. Ecol. Evol. 2, 420–426 (2018).

    Article  PubMed  Google Scholar 

  45. Culina, A., Crowther, T. W., Ramakers, J. J. C., Gienapp, P. & Visser, M. E. How to do meta-analysis of open datasets. Nat. Ecol. Evol. 2, 1053–1056 (2018).

    Article  PubMed  Google Scholar 

  46. Roche, D. G., Kruuk, L. E. B., Lanfear, R. & Binning, S. A. Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  47. Grainger, M. J., Bolam, F. C., Stewart, G. B. & Nilsen, E. B. Evidence synthesis for tackling research waste. Nat. Ecol. Evol. 4, 495–497 (2020).

    Article  PubMed  Google Scholar 

  48. Nørgaard, B. et al. Systematic reviews are rarely used to inform study design—a systematic review and meta-analysis. J. Clin. Epidemiol. 145, 1–13 (2022).

    Article  PubMed  Google Scholar 

  49. Webb, J. A. et al. Weaving common threads in environmental causal assessment methods: toward an ideal method for rapid evidence synthesis. Freshw. Sci. 36, 250–256 (2017).

    Article  Google Scholar 

  50. Collins, A., Coughlin, D., Miller, J. & Kirk, S. The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide (Joint Water Evidence Group, 2015).

  51. Carrick, J. et al. Is planting trees the solution to reducing flood risks? J. Flood Risk Manag. 12, e12484 (2019).

    Article  Google Scholar 

  52. Nuñez, M. A. & Amano, T. Monolingual searches can limit and bias results in global literature reviews. Nat. Ecol. Evol. 5, 264 (2021).

    Article  PubMed  Google Scholar 

  53. Morrison, A. et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28, 138–144 (2012).

    Article  PubMed  Google Scholar 

  54. Wu, T., Li, Y., Bian, Z., Liu, G. & Moher, D. Randomized trials published in some Chinese journals: how many are randomized? Trials 10, 46 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Vorobeichik, E. L. & Kozlov, M. V. Impact of point polluters on terrestrial ecosystems: methodology of research, experimental design, and typical errors. Russ. J. Ecol. 43, 89–96 (2012).

    Article  Google Scholar 

  56. Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366 (2011).

    Article  PubMed  Google Scholar 

  57. Transforming Our World: the 2030 Agenda for Sustainable Development (United Nations, 2015).

  58. MacCoun, R. & Perlmutter, S. Blind analysis: hide results to seek the truth. Nature 526, 187–189 (2015).

    Article  CAS  PubMed  Google Scholar 

  59. Parker, T. H. et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 (2016).

    Article  PubMed  Google Scholar 

  60. Announcement: reducing our irreproducibility. Nature 496, 398 (2013).

  61. Moher, D. et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet 387, 1573–1586 (2016).

    Article  PubMed  Google Scholar 

  62. Smaldino, P. E. & McElreath, R. The natural selection of bad science. R. Soc. Open Sci. 3, 160384 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Vrieze, J. Landmark research integrity survey finds questionable practices are surprisingly common. ScienceInsider https://www.sciencemag.org/news/2021/07/landmark-research-integrity-survey-finds-questionable-practices-are-surprisingly-common (2021).

  64. Woolston, C. Impact factor abandoned by Dutch university in hiring and promotion decisions. Nature 595, 462 (2021).

    Article  CAS  PubMed  Google Scholar 

  65. Directorate-General for Research and Innovation (European Commission). Towards a Reform of the Research Assessment System. Scoping Report (Publications Office, 2021).

  66. Athena Research & Innovation Center, Directorate-General for Research and Innovation (European Commission), PPMI, UNU-MERIT. Monitoring the Open Access Policy of Horizon 2020. Final report (European Commission, 2021).

  67. Kwon, D. University of California and Elsevier forge open-access deal. TheScientist https://www.the-scientist.com/news-opinion/university-of-california-and-elsevier-forge-open-access-deal–68557 (2021).

  68. Vines, T. H. et al. Mandated data archiving greatly improves access to research data. FASEB J. 27, 1304–1308 (2013).

    Article  CAS  PubMed  Google Scholar 

  69. NPQIP Collaborative Group. Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci. 3, e000035 (2019).

    Article  Google Scholar 

  70. Glasziou, P. et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 383, 267–276 (2014).

    Article  PubMed  Google Scholar 

  71. Fecher, B. & Friesike, S. in Opening Science: the Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing (eds Bartling, S. & Friesike, S.) 17–47 (Springer International Publishing, 2014).

  72. Hardwicke, T. E. et al. Calibrating the scientific ecosystem through meta-research. Annu. Rev. Stat. Appl. 7, 11–37 (2020).

    Article  Google Scholar 

  73. McKiernan, E. C. et al. How open science helps researchers succeed. eLife 5, e16800 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Fidler, F. et al. Metaresearch for evaluating reproducibility in ecology and evolution. Bioscience 67, 282–289 (2017).

    PubMed  PubMed Central  Google Scholar 

  75. Cornwall, C. E. & Hurd, C. L. Experimental design in ocean acidification research: problems and solutions. ICES J. Mar. Sci. 73, 572–581 (2016).

    Article  Google Scholar 

  76. Fidler, F., Burgman, M. A., Cumming, G., Buttrose, R. & Thomason, N. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol. 20, 1539–1544 (2006).

    Article  PubMed  Google Scholar 

  77. Forstmeier, W. & Schielzeth, H. Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav. Ecol. Sociobiol. 65, 47–55 (2011).

    Article  PubMed  Google Scholar 

  78. Gillespie, B. R., Desmet, S., Kay, P., Tillotson, M. R. & Brown, L. E. A critical analysis of regulated river ecosystem responses to managed environmental flows from reservoirs. Freshw. Biol. 60, 410–425 (2015).

    Article  Google Scholar 

  79. Haddaway, N. R., Styles, D. & Pullin, A. S. Evidence on the environmental impacts of farm land abandonment in high altitude/mountain regions: a systematic map. Environ. Evid. 3, 17 (2014).

    Article  Google Scholar 

  80. Heffner, R. A., Butler, M. J. & Reilly, C. K. Pseudoreplication revisited. Ecology 77, 2558–2562 (1996).

    Article  Google Scholar 

  81. Holman, L., Head, M. L., Lanfear, R. & Jennions, M. D. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol. 13, e1002190 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  82. Hurlbert, S. H. & White, M. D. Experiments with freshwater invertebrate zooplanktivores: quality of statistical analyses. Bull. Mar. Sci. 53, 128–153 (1993).

    Google Scholar 

  83. Johnson, W. T.3rd & Freeberg, T. M. Pseudoreplication in use of predator stimuli in experiments on antipredator responses. Anim. Behav. 119, 161–164 (2016).

    Article  Google Scholar 

  84. Kozlov, M. V. Pseudoreplication in ecological research: the problem overlooked by Russian scientists. Zh. Obshch. Biol. 64, 292–307 (2003).

    CAS  PubMed  Google Scholar 

  85. Kozlov, M. V. Plant studies on fluctuating asymmetry in Russia: mythology and methodology. Russ. J. Ecol. 48, 1–9 (2017).

    Article  Google Scholar 

  86. McDonald, S., Cresswell, T., Hassell, K. & Keough, M. Experimental design and statistical analysis in aquatic live animal radiotracing studies: a systematic review. Crit. Rev. Environ. Sci. Technol. 52, 2772–2801 (2021).

    Article  Google Scholar 

  87. Møller, A. P., Thornhill, R. & Gangestad, S. W. Direct and indirect tests for publication bias: asymmetry and sexual selection. Anim. Behav. 70, 497–506 (2005).

    Article  Google Scholar 

  88. Mrosovsky, N. & Godfrey, M. H. The path from grey literature to Red Lists. Endang. Species Res. 6, 185–191 (2008).

    Article  Google Scholar 

  89. O’Brien, C., van Riper, C.3rd & Myers, D. E. Making reliable decisions in the study of wildlife diseases: using hypothesis tests, statistical power, and observed effects. J. Wildl. Dis. 45, 700–712 (2009).

    Article  PubMed  Google Scholar 

  90. Parker, T. H. What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biol. Rev. Camb. Philos. Soc. 88, 511–536 (2013).

    Article  PubMed  Google Scholar 

  91. Ramage, B. S. et al. Pseudoreplication in tropical forests and the resulting effects on biodiversity conservation. Conserv. Biol. 27, 364–372 (2013).

    Article  PubMed  Google Scholar 

  92. Sallabanks, R., Arnett, E. B. & Marzluff, J. M. An evaluation of research on the effects of timber harvest on bird populations. Wildl. Soc. Bull. 28, 1144–1155 (2000).

    Google Scholar 

  93. Sánchez-Tójar, A. et al. Meta-analysis challenges a textbook example of status signalling and demonstrates publication bias. eLife 7, e37385 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  94. Waller, B., Warmelink, L., Liebal, K., Micheletta, J. & Slocombe, K. Pseudoreplication: a widespread problem in primate communication research. Anim. Behav. 86, 483–488 (2013).

    Article  Google Scholar 

  95. Van Wilgenburg, E. & Elgar, M. A. Confirmation bias in studies of nestmate recognition: a cautionary note for research into the behaviour of animals. PLoS ONE 8, e53548 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  96. Yoccoz, N. G. Use, overuse, and misuse of significance tests in evolutionary biology and ecology. Bull. Ecol. Soc. Am. 72, 106–111 (1991).

    Article  Google Scholar 

  97. Zaitsev, A. S., Gongalsky, K. B., Malmström, A., Persson, T. & Bengtsson, J. Why are forest fires generally neglected in soil fauna research? A mini-review. Appl. Soil Ecol. 98, 261–271 (2016).

    Article  Google Scholar 

  98. Zvereva, E. L. & Kozlov, M. V. Biases in studies of spatial patterns in insect herbivory. Ecol. Monogr. 89, e01361 (2019).

    Article  Google Scholar 

  99. RStudio Team. RStudio: Integrated Development for R (RStudio, 2020).

  100. Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge funding from the Croatian Science Foundation project IP-2018-01-3150-AqADAPT that supported T.K. and M.P.

Author information

Authors and Affiliations

Authors

Contributions

A.C. conceived the study and wrote the manuscript draft. A.C. and M.P. analysed the data. M.P., T.K. and A.C. designed the analysis, contributed to data collection, interpretation of the data and the manuscript revisions.

Corresponding author

Correspondence to Antica Culina.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Ecology & Evolution thanks Matthew Grainger and Alec Christie for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

A. Supplementary Methods, 1. Literature review and meta-analysis, 1.1 Search procedure, 1.2 Inclusion screening, 1.3 Data extraction, 1.4 Data analysis and synthesis. 2. Estimating the percentage of published ecological literature that is not open access. B. Supplementary results – Supplementary Fig. 4, C. References.

Reporting Summary

Peer Review File

Source data

Source Data Fig. 3

Data needed to recreate Fig. 3 (using Fig3_plotting.R script, available at 10.5281/zenodo.6566100). This .xlsx file contains three tabs; each have to be saved as a separate .csv file (with the name corresponding to the Excel sheet name: Dataset_MA_final.csv; Meta_analytic_means.csv; Study_planning_ma_means.csv) to recreate Fig. 3.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Purgar, M., Klanjscek, T. & Culina, A. Quantifying research waste in ecology. Nat Ecol Evol 6, 1390–1397 (2022). https://doi.org/10.1038/s41559-022-01820-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41559-022-01820-0

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing