Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Questionable survey methods generate a questionable list of recommended articles

Nature Ecology & Evolution volume 2, pages 1336–1337 (2018)Cite this article

Subjects

To the Editor — Courchamp and Bradshaw1 provided a list of 100 papers that “every ecologist” should read; a broad and sobering statement. The list was constructed by soliciting opinions from 665 “experts” as defined by editorial positions at ecology journals. The study is described as a survey and a correlation analysis was performed on the data, implying methodological rigour and robust results. However, we find that this study (and its companion2 using the same data) suffered from severe methodological omissions, errors and flaws, calling into question any utility of the results.

When soliciting opinions and views from human subjects, social scientists adhere to rigorous methods that aim to prevent unusable results corrupted by bias and oversight. Such papers provide ample methodological details that permit future replication. Publishable peer-reviewed studies must meet these general expectations for accuracy, adequate sample size and representativeness, high response rate, and lack of (or controlled for) non-response bias. Below we describe how refs 1,2 demonstrate shortcomings on each point.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Courchamp, F. & Bradshaw, C. J. A. Nat. Ecol. Evol. 2, 395–401 (2018).

    Article  Google Scholar 

  2. Bradshaw, C. J. A. & Courchamp, F. Preprint at https://www.biorxiv.org/content/early/2018/01/03/219824 (2017).

  3. Carley-Baxter, L. R. et al. Survey Practice https://doi.org/10.29115/SP-2009-0033 (2009).

  4. Bruna, E. M. Nat. Ecol. Evol. 2, 202 (2018).

    Article  Google Scholar 

  5. Ramirez, K. S. et al. Nat. Ecol. Evol. 2, 200 (2018).

    Article  Google Scholar 

  6. Peytchev, A. Ann. Am. Acad. Polit. Soc. Sci. 645, 88–111 (2013).

  7. Baruch, Y. & Holtom, B. C. Hum. Rel. 61, 1139–1160 (2008).

    Article  Google Scholar 

  8. Fincham, J. E. Am. J. Pharm. Educ. 72, 43 (2008).

  9. Dillman, D. A., Smyth, J. D. & Christian, L. M. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method (John Wiley & Sons, Hoboken, 2014).

  10. Davern, M. Health Serv. Res. 48, 905–912 (2013).

    Article  Google Scholar 

  11. National Research CouncilNonresponse in Social Science Surveys: A Research Agenda (The National Academies Press, Washington DC, 2013).

    Google Scholar 

  12. Baum, J. K. & Martin, T. G. Nat. Ecol. Evol. 2, 201 (2018).

    Article  Google Scholar 

  13. Gilbert, G. S. Nat. Ecol. Evol. 2, 203 (2018).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Social Sciences, Michigan Technological University, Houghton, MI, USA

    Audrey L. Mayer & Adam M. Wellstead

  2. School of Forest Resources and Environmental Science, Michigan Technological University, Houghton, MI, USA

    Audrey L. Mayer

Authors
  1. Audrey L. Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam M. Wellstead
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Audrey L. Mayer.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mayer, A.L., Wellstead, A.M. Questionable survey methods generate a questionable list of recommended articles. Nat Ecol Evol 2, 1336–1337 (2018). https://doi.org/10.1038/s41559-018-0637-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41559-018-0637-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing Anthropocene

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: Anthropocene