WORLD VIEW

A journal club to fix science

ReproducibiliTea can build up open science without top-down initiatives, says Amy Orben.
Amy Orben is a research fellow at the University of Cambridge, UK, where she studies how technology affects teen well-being.
Contact

Search for this author in:

If science had generations, mine would not be defined by war or Woodstock, but by reproducibility and open science. Early-career researchers such as myself have been trained in an era when scientists are acutely aware of problems in the scientific process. We are taught about replication issues and failures, and are encouraged to question results that have been left unchallenged for decades. Our ability to connect through social media lets us sidestep conventional hierarchies and scrutinize current research practices. Of course, we want to adapt how we do research to improve the scientific landscape that we will be navigating for decades.

But we are often overlooked. I gave a talk at a workshop on open and reproducible science earlier this year; I was a PhD student at the time, and the only invited speaker who was not a tenured faculty member. Another speaker recounted recent changes brought about by lobbying the highest echelons of the scientific community, such as mandates to share data openly. He argued that early-career researchers have little agency to push for such improvements.

I could not disagree more. We need all those who care about better research to stay invested, and this will not happen by telling the next generation of scientists to just sit back and hope. Early-career researchers do not need to wait passively for coveted improvements. We can create communities and push for bottom-up change.

ReproducibiliTea is one way to do this. Sam Parsons, Sophia Crüwell and I (all trainees) started this grass-roots journal club in early 2018, at the experimental-psychology department at the University of Oxford, UK. We hoped to promote a stronger open-science community and more prominent conversations about reproducibility. The initiative soon spread, and is now active at more than 27 universities in 8 countries.

During each meeting, a scientific paper lays the groundwork for a conversation. Concerns vary from field to field and institution to institution, so each club focuses on aspects of scientific methods and systems that concern them most. Topics for my group ranged from discussions on replicability (Open Science Collaboration Science 349, aac4716; 2015), to debates about open-access publishing (J. P. Tennant et al. F1000 Res. 5, 632; 2016), the problems of analytical flexibility (J. P. Simmons et al. Psychol. Sci. 22, 1359–1366; 2011) and the potential of Registered Reports, a publication format in which papers are reviewed primarily on the research question and protocol, before results are known (C. D. Chambers Cortex 49, 609–610; 2013).

These conversations can become a crucial line of support for young scientists. They give researchers a space to explore ideas such as publishing in open-access journals, preregistering studies and sharing data even if their supervisors are wary and warn that such practices will undermine their careers. ReproducibiliTea discussions have emboldened trainees to go back to their lab groups and advocate for change, often backed by real-world examples. Sometimes it works, and sometimes it doesn’t. Either way, ReproducibiliTea members tell me how valuable it is to know they are not alone in how they want to see science practised.

ReproducibiliTea works because it is easy and visible. Setting up a journal club does not require jumping over administrative hurdles, and it does not need senior support or funding (although we did very much appreciate a small grant to cover popcorn, strawberries and the like). The group at Oxford consists of about a dozen psychology researchers, but people show up from a range of departments, such as zoology and anthropology, and even other institutions. Beyond attendees, ReproducibiliTea puts open science on the radar of other academics and senior staff. We have posters, e-mails and a weekly slot in our departmental newsletter.

To launch their own ReproducibiliTea group, motivated researchers need only to select some articles and set a time and a place. No minimum group size or meeting frequency is required. They will then join a community of ReproducibiliTea journal clubs that continually discuss improvements and support each other. (For more information, see https://reproducibilitea.org/.)

Some of my favourite sessions had representatives from all career stages present — from undergraduate students critical of what they were being taught to very senior professors. Those meetings had a unique feeling of camaraderie, even during intensely personal discussions, including how much blame for subpar practices should fall on individuals versus a broken system.

Similar to other initiatives, such as the Open Science Community organizations in the Netherlands, or the UK Reproducibility Network, ReproducibiliTea groups tend to bring together those already interested in reforms. But we conscientiously strive to make the discussion open to diverse and critical voices.

In practice, I have found our meetings underscore the idea that open science is a process, not a one-time achievement or a claim to virtue. Our groups have had impassioned arguments about the validity of measures used in psychology, and on whether preregistration is even possible for cognitive-modelling research, which is more dependent on continually tweaking analyses and models. Discussions exploring current limitations of open science often draw the largest and most diverse crowds. One attendee told me, “Before, I thought everything was black and white in open science, and now I see there are caveats and difficulties and things to overcome.” ReproducibiliTea’s low-key grass-roots meetings will encourage a new generation of scientists to feel motivated to master these challenges.

Nature 573, 465 (2019)

doi: 10.1038/d41586-019-02842-8

Nature Briefing

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.