Abstract
Research inefficiencies can generate huge waste: evidence from biomedical research has shown that most research is avoidably wasted and steps have been taken to tackle this costly problem. Although other scientific fields could also benefit from identifying and quantifying waste and acting to reduce it, no other estimates of research waste are available. Given that ecological issues interweave most of the United Nations Sustainable Development Goals, we argue that tackling research waste in ecology should be prioritized. Our study leads the way. We estimate components of waste in ecological research based on a literature review and a meta-analysis. Shockingly, our results suggest only 11–18% of conducted ecological research reaches its full informative value. All actors within the research system—including academic institutions, policymakers, funders and publishers—have a duty towards science, the environment, study organisms and the public, to urgently act and reduce this considerable yet preventable loss. We discuss potential ways forward and call for two major actions: (1) further research into waste in ecology (and beyond); (2) focused development and implementation of solutions to reduce unused potential of ecological research.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data needed to reproduce the analyses and create the main text and supplementary figures have been deposited at Zenodo35 https://doi.org/10.5281/zenodo.6566100. These include the original effect sizes as extracted from studies and the final set of the effect sizes used in the meta-analysis. Source data are provided with this paper.
Code availability
The codes/scripts needed to reproduce the analyses and create the main text and supplementary figures are deposited at Zenodo35 https://doi.org/10.5281/zenodo.6566100.
References
Ioannidis, J. P. A., Fanelli, D., Dunne, D. D. & Goodman, S. N. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 13, e1002264 (2015).
Hampton, S. E. et al. The Tao of open science for ecology. Ecosphere 6, art120 (2015).
Rothstein, H. R., Sutton, A. J. & Borenstein, M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (John Wiley & Sons, 2005).
Sutton, A. J. in The Handbook of Research Synthesis and Meta-Analysis (eds Cooper, H. et al.) 435–452 (Russell Sage Foundation, 2009).
Nakagawa, S., Koricheva, J., Macleod, M. & Viechtbauer, W. Introducing our series: research synthesis and meta-research in biology. BMC Biol. 18, 20 (2020).
Nakagawa, S. et al. A new ecosystem for evidence synthesis. Nat. Ecol. Evol. 4, 498–501 (2020).
Coolidge, H. J. & Lord, R. H. in Archibald Cary Coolidge: Life and Letters 308 (Houghton Mifflin Harcourt, 1932).
Nickerson, R. S. Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220 (1998).
Touchon, J. C. & McCoy, M. W. The mismatch between current statistical practice and doctoral training in ecology. Ecosphere 7, e01394 (2016).
Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483, 531–533 (2012).
Aarts, A. A. et al. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).
Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).
Fraser, H., Parker, T., Nakagawa, S., Barnett, A. & Fidler, F. Questionable research practices in ecology and evolution. PLoS ONE 13, e0200303 (2018).
Culina, A., van den Berg, I., Evans, S. & Sánchez-Tójar, A. Low availability of code in ecology: a call for urgent action. PLoS Biol. 18, e3000763 (2020).
Jennions, M. D. & Møller, A. P. Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method. Biol. Rev. Camb. Philos. Soc. 77, 211–222 (2002).
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behavior. Behav. Ecol. 14, 438–445 (2003).
Cassey, P., Ewen, J. G., Blackburn, T. M. & Møller, A. P. A survey of publication bias within evolutionary ecology. Proc. Biol. Sci. 271, S451–S454 (2004).
Kardish, M. R. et al. Blind trust in unblinded observation in ecology, evolution, and behavior. Front. Ecol. Evol. 3, 51 (2015).
Jennions, M. D. & Møller, A. P. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proc. Biol. Sci. 269, 43–48 (2002).
Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016).
Chalmers, I. & Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet 374, 86–89 (2009).
Altman, D. G. The scandal of poor medical research. BMJ 308, 283–284 (1994).
Glasziou, P. & Chalmers, I. Is 85% of health research really ‘wasted’? BMJ Opinion (14 January 2016).
Glasziou, P. & Chalmers, I. Research waste is still a scandal. BMJ 363, k4645 (2018).
Chalmers, I. et al. How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165 (2014).
Kunin, W. E. Robust evidence of declines in insect abundance and biodiversity. Nature 574, 641–642 (2019).
Christie, A. P. et al. Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nat. Commun. 11, 6377 (2020).
Campbell, H. A. et al. Finding our way: on the sharing and reuse of animal telemetry data in Australasia. Sci. Total Environ. 534, 79–84 (2015).
Koricheva, J. Non-significant results in ecology: a burden or a blessing in disguise? Oikos 102, 397–401 (2003).
Bennett, L. T. & Adams, M. A. Assessment of ecological effects due to forest harvesting: approaches and statistical issues. J. Appl. Ecol. 41, 585–598 (2004).
Duval, S. & Tweedie, R. A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98 (2012).
Brlík, V. et al. Weak effects of geolocators on small birds: a meta-analysis controlled for phylogeny and publication bias. J. Anim. Ecol. 89, 207–220 (2020).
Hurlbert, S. H. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187–211 (1984).
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behaviour. Behav. Ecol. 14, 438–445 (2003).
Culina, A., Purgar, M. & Klanjscek, T. Datasets and codes for Purgar et al. 2022: quantifying research waste in ecology. Zenodo https://zenodo.org/record/6566100#.YrLWB-zMIqs (2022).
Ferguson, C. et al. Europe PMC in 2020. Nucleic Acids Res. 49, D1507–D1514 (2021).
Huang, C.-K. et al. Meta-Research: Evaluating the impact of open access policies on research institutions. eLife 9, e57067 (2020).
Ross-Hellauer, T. Open science, done wrong, will compound inequities. Nature 603, 363 (2022).
Smith, A. C. et al. Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s ‘Mirror journal’ system. Quant. Sci. Stud. 2, 1123–1143 (2021).
Christie, A. P. et al. Reducing publication delay to improve the efficiency and impact of conservation science. PeerJ 9, e12245 (2021).
Desjardins-Proulx, P. et al. The case for open preprints in biology. PLoS Biol. 11, e1001563 (2013).
Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).
O’Dea, R. E. et al. Towards open, reliable, and transparent ecology and evolutionary biology. BMC Biol. 19, 68 (2021).
Culina, A. et al. Navigating the unfolding open data landscape in ecology and evolution. Nat. Ecol. Evol. 2, 420–426 (2018).
Culina, A., Crowther, T. W., Ramakers, J. J. C., Gienapp, P. & Visser, M. E. How to do meta-analysis of open datasets. Nat. Ecol. Evol. 2, 1053–1056 (2018).
Roche, D. G., Kruuk, L. E. B., Lanfear, R. & Binning, S. A. Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295 (2015).
Grainger, M. J., Bolam, F. C., Stewart, G. B. & Nilsen, E. B. Evidence synthesis for tackling research waste. Nat. Ecol. Evol. 4, 495–497 (2020).
Nørgaard, B. et al. Systematic reviews are rarely used to inform study design—a systematic review and meta-analysis. J. Clin. Epidemiol. 145, 1–13 (2022).
Webb, J. A. et al. Weaving common threads in environmental causal assessment methods: toward an ideal method for rapid evidence synthesis. Freshw. Sci. 36, 250–256 (2017).
Collins, A., Coughlin, D., Miller, J. & Kirk, S. The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide (Joint Water Evidence Group, 2015).
Carrick, J. et al. Is planting trees the solution to reducing flood risks? J. Flood Risk Manag. 12, e12484 (2019).
Nuñez, M. A. & Amano, T. Monolingual searches can limit and bias results in global literature reviews. Nat. Ecol. Evol. 5, 264 (2021).
Morrison, A. et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28, 138–144 (2012).
Wu, T., Li, Y., Bian, Z., Liu, G. & Moher, D. Randomized trials published in some Chinese journals: how many are randomized? Trials 10, 46 (2009).
Vorobeichik, E. L. & Kozlov, M. V. Impact of point polluters on terrestrial ecosystems: methodology of research, experimental design, and typical errors. Russ. J. Ecol. 43, 89–96 (2012).
Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366 (2011).
Transforming Our World: the 2030 Agenda for Sustainable Development (United Nations, 2015).
MacCoun, R. & Perlmutter, S. Blind analysis: hide results to seek the truth. Nature 526, 187–189 (2015).
Parker, T. H. et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 (2016).
Announcement: reducing our irreproducibility. Nature 496, 398 (2013).
Moher, D. et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet 387, 1573–1586 (2016).
Smaldino, P. E. & McElreath, R. The natural selection of bad science. R. Soc. Open Sci. 3, 160384 (2016).
Vrieze, J. Landmark research integrity survey finds questionable practices are surprisingly common. ScienceInsider https://www.sciencemag.org/news/2021/07/landmark-research-integrity-survey-finds-questionable-practices-are-surprisingly-common (2021).
Woolston, C. Impact factor abandoned by Dutch university in hiring and promotion decisions. Nature 595, 462 (2021).
Directorate-General for Research and Innovation (European Commission). Towards a Reform of the Research Assessment System. Scoping Report (Publications Office, 2021).
Athena Research & Innovation Center, Directorate-General for Research and Innovation (European Commission), PPMI, UNU-MERIT. Monitoring the Open Access Policy of Horizon 2020. Final report (European Commission, 2021).
Kwon, D. University of California and Elsevier forge open-access deal. TheScientist https://www.the-scientist.com/news-opinion/university-of-california-and-elsevier-forge-open-access-deal–68557 (2021).
Vines, T. H. et al. Mandated data archiving greatly improves access to research data. FASEB J. 27, 1304–1308 (2013).
NPQIP Collaborative Group. Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci. 3, e000035 (2019).
Glasziou, P. et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 383, 267–276 (2014).
Fecher, B. & Friesike, S. in Opening Science: the Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing (eds Bartling, S. & Friesike, S.) 17–47 (Springer International Publishing, 2014).
Hardwicke, T. E. et al. Calibrating the scientific ecosystem through meta-research. Annu. Rev. Stat. Appl. 7, 11–37 (2020).
McKiernan, E. C. et al. How open science helps researchers succeed. eLife 5, e16800 (2016).
Fidler, F. et al. Metaresearch for evaluating reproducibility in ecology and evolution. Bioscience 67, 282–289 (2017).
Cornwall, C. E. & Hurd, C. L. Experimental design in ocean acidification research: problems and solutions. ICES J. Mar. Sci. 73, 572–581 (2016).
Fidler, F., Burgman, M. A., Cumming, G., Buttrose, R. & Thomason, N. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol. 20, 1539–1544 (2006).
Forstmeier, W. & Schielzeth, H. Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav. Ecol. Sociobiol. 65, 47–55 (2011).
Gillespie, B. R., Desmet, S., Kay, P., Tillotson, M. R. & Brown, L. E. A critical analysis of regulated river ecosystem responses to managed environmental flows from reservoirs. Freshw. Biol. 60, 410–425 (2015).
Haddaway, N. R., Styles, D. & Pullin, A. S. Evidence on the environmental impacts of farm land abandonment in high altitude/mountain regions: a systematic map. Environ. Evid. 3, 17 (2014).
Heffner, R. A., Butler, M. J. & Reilly, C. K. Pseudoreplication revisited. Ecology 77, 2558–2562 (1996).
Holman, L., Head, M. L., Lanfear, R. & Jennions, M. D. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol. 13, e1002190 (2015).
Hurlbert, S. H. & White, M. D. Experiments with freshwater invertebrate zooplanktivores: quality of statistical analyses. Bull. Mar. Sci. 53, 128–153 (1993).
Johnson, W. T.3rd & Freeberg, T. M. Pseudoreplication in use of predator stimuli in experiments on antipredator responses. Anim. Behav. 119, 161–164 (2016).
Kozlov, M. V. Pseudoreplication in ecological research: the problem overlooked by Russian scientists. Zh. Obshch. Biol. 64, 292–307 (2003).
Kozlov, M. V. Plant studies on fluctuating asymmetry in Russia: mythology and methodology. Russ. J. Ecol. 48, 1–9 (2017).
McDonald, S., Cresswell, T., Hassell, K. & Keough, M. Experimental design and statistical analysis in aquatic live animal radiotracing studies: a systematic review. Crit. Rev. Environ. Sci. Technol. 52, 2772–2801 (2021).
Møller, A. P., Thornhill, R. & Gangestad, S. W. Direct and indirect tests for publication bias: asymmetry and sexual selection. Anim. Behav. 70, 497–506 (2005).
Mrosovsky, N. & Godfrey, M. H. The path from grey literature to Red Lists. Endang. Species Res. 6, 185–191 (2008).
O’Brien, C., van Riper, C.3rd & Myers, D. E. Making reliable decisions in the study of wildlife diseases: using hypothesis tests, statistical power, and observed effects. J. Wildl. Dis. 45, 700–712 (2009).
Parker, T. H. What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biol. Rev. Camb. Philos. Soc. 88, 511–536 (2013).
Ramage, B. S. et al. Pseudoreplication in tropical forests and the resulting effects on biodiversity conservation. Conserv. Biol. 27, 364–372 (2013).
Sallabanks, R., Arnett, E. B. & Marzluff, J. M. An evaluation of research on the effects of timber harvest on bird populations. Wildl. Soc. Bull. 28, 1144–1155 (2000).
Sánchez-Tójar, A. et al. Meta-analysis challenges a textbook example of status signalling and demonstrates publication bias. eLife 7, e37385 (2018).
Waller, B., Warmelink, L., Liebal, K., Micheletta, J. & Slocombe, K. Pseudoreplication: a widespread problem in primate communication research. Anim. Behav. 86, 483–488 (2013).
Van Wilgenburg, E. & Elgar, M. A. Confirmation bias in studies of nestmate recognition: a cautionary note for research into the behaviour of animals. PLoS ONE 8, e53548 (2013).
Yoccoz, N. G. Use, overuse, and misuse of significance tests in evolutionary biology and ecology. Bull. Ecol. Soc. Am. 72, 106–111 (1991).
Zaitsev, A. S., Gongalsky, K. B., Malmström, A., Persson, T. & Bengtsson, J. Why are forest fires generally neglected in soil fauna research? A mini-review. Appl. Soil Ecol. 98, 261–271 (2016).
Zvereva, E. L. & Kozlov, M. V. Biases in studies of spatial patterns in insect herbivory. Ecol. Monogr. 89, e01361 (2019).
RStudio Team. RStudio: Integrated Development for R (RStudio, 2020).
Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).
Acknowledgements
We acknowledge funding from the Croatian Science Foundation project IP-2018-01-3150-AqADAPT that supported T.K. and M.P.
Author information
Authors and Affiliations
Contributions
A.C. conceived the study and wrote the manuscript draft. A.C. and M.P. analysed the data. M.P., T.K. and A.C. designed the analysis, contributed to data collection, interpretation of the data and the manuscript revisions.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Ecology & Evolution thanks Matthew Grainger and Alec Christie for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
A. Supplementary Methods, 1. Literature review and meta-analysis, 1.1 Search procedure, 1.2 Inclusion screening, 1.3 Data extraction, 1.4 Data analysis and synthesis. 2. Estimating the percentage of published ecological literature that is not open access. B. Supplementary results – Supplementary Fig. 4, C. References.
Source data
Source Data Fig. 3
Data needed to recreate Fig. 3 (using Fig3_plotting.R script, available at 10.5281/zenodo.6566100). This .xlsx file contains three tabs; each have to be saved as a separate .csv file (with the name corresponding to the Excel sheet name: Dataset_MA_final.csv; Meta_analytic_means.csv; Study_planning_ma_means.csv) to recreate Fig. 3.
Rights and permissions
About this article
Cite this article
Purgar, M., Klanjscek, T. & Culina, A. Quantifying research waste in ecology. Nat Ecol Evol 6, 1390–1397 (2022). https://doi.org/10.1038/s41559-022-01820-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41559-022-01820-0
This article is cited by
-
Enhancing biodiversity conservation and monitoring in protected areas through efficient data management
Environmental Monitoring and Assessment (2024)
-
Opportunities and challenges for Registered Reports in ecology and evolution
Nature Communications (2022)
-
Quantifying research waste in ecology
Nature Ecology & Evolution (2022)