Nature | Research Highlights: Social Selection

Could students solve the irreproducibility crisis?

Scientists on social media debate a call to require PhD students to replicate research before they can graduate.

Article tools

A growing backlog of psychology findings that have never been reproduced has shaken confidence in the field. One possible remedy is to require PhD students to replicate at least one study from their own specialism as part of their education, write UK psychologists Brian Earp and Jim Everett in an opinion piece in Frontiers in Psychology1. Jonathan LaTourelle, a PhD student in philosophy and cognitive science at Arizona State University in Tempe tweeted:

But ethicist Owen Schaefer at the University of Oxford, UK, suggested in a comment on a blog post that the proposal could end up “disproportionately burdening” graduate students.

Claims of irreproducible research have plagued the field of psychology (see Nature 485, 298–300; 2012). Earp and Everett, both researchers at the University of Oxford, write that too few psychologists are attempting to replicate earlier studies because of the lack of incentives in a competitive field. Researchers might feel that their careers would benefit more from doing original work, and it can be difficult to get a replication study published — as Earp, Everett and their colleagues found out first-hand with their 2014 study (B. D. Earp et al. Basic Appl. Soc. Psych. 36, 9198; 2014) that failed to reproduce findings from a highly cited paper.

If such studies were a requirement of graduate education, they argue, not only would there be a clear incentive for students, but also vast brain power could be unleashed on the problem. Earp said in an interview that there are “thousands of important studies that could be done relatively inexpensively and in a manageable window of time”. Revisiting those studies could go a long way towards “cleaning up” the literature, he says.

But Schaefer, in a comment on the University of Oxford’s Practical Ethics blog, says that the plan risks exploiting students because it “takes advantage of a cheap, easily-pressured labor pool”. He suggests instead that senior scientists have an important role in tackling the irreproducibility crisis. Reviewers should be more willing to recommend replication studies for publication, he says, and hiring committees should look more favourably on such work when evaluating CVs.

Reached for comment, Schaefer says that forcing graduate students to conduct replication studies could actually deepen the stigma against such work. “Replication studies could come to be seen as grunt work you have to get through to get your degree, not the sort of thing a serious, established researcher will be involved in,” he says.

LaTourelle disagrees that such tasks would be too onerous — he thinks that graduate students would benefit from doing the work by learning to critically evaluate high-quality research. As a graduate student, he says that having the chance to repeat an interesting and important experiment “sounds like a dream”.

Earp adds that trying to recreate someone else’s work drives home the importance of sticking to transparent, easy-to-follow methods. If more psychologists learned this lesson early in their career, he says, future research would be easier to reproduce, and some of the confidence in psychology results could be restored.

For more, see www.nature.com/socialselection.

Journal name:
Nature
Volume:
524,
Pages:
9
Date published:
()
DOI:
doi:10.1038/524009f

References

  1. Everett, J. A. & Earp, B. Front. Psychol. http://dx.doi.org/10.3389/fpsyg.2015.01152 (2015)

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments

Commenting is currently unavailable.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.