This four-part Working Scientist podcast is supported by the University of Sydney. Explore our research at sydney.edu.au.
Adam Levy
Hello, I’m Adam Levy and this is Working Scientist, a Nature Careers podcast. In this series, we’re looking at the challenging chore of publishing a paper.
So far, we’ve looked at getting your paper published and making sure that other people actually read it, but this has all been with the conventional approach to scientific publishing in mind, and the reality is that in the 21st century, things are changing, and so this week in the final episode of this four-part series, we’re taking a look at how papers are evolving. We’re discussing the future of the academic paper. There are many ways that manuscripts are mutating, from different approaches to peer review to reformatting the structure of the paper itself. But just adding some extra details to publications can also have profound impacts. Here’s Antarctica scientist Pippa Whitehouse of Durham University in the UK.
Pippa Whitehouse
I think something which is very positive for the publishing process is the summaries that are often included. I think it started with papers that sort of forced you to include maybe three bullet points that summarise the important results, and often now, there’s a short layman’s review of the work, and I find those really useful in subjects slightly outside my field. I see a title that looks useful and don’t quite understand the language in the technical abstract, but sometimes the layman’s abstract can just give me enough insight into that study, and I think that’s a really, really useful thing that’s happened in the last four or five years.
Adam Levy
But this is far from the only change that researchers will have noticed over the last half-decade or so. Another, of course, is the rise of preprint servers. These provide researchers with the opportunity to present their work to wider audiences before their papers have been peer reviewed and published in a journal. For environmental scientist Jen Burney of the University of California, San Diego, this can have big benefits.
Jen Burney
You can spend two years trying to get it into the right place, and nobody really knows that you’ve gotten anywhere with this, and so I think there’s a benefit both to the individuals who have worked on a project and to the community as a whole, who’s trying to think about where the next pushes need to be so what people are doing and where I might make a contribution, to have preliminary versions of things out there for people to see.
Adam Levy
Preprint servers have, in fact, been around for decades, the best known being Rxiv, which hosts papers in mathematics and the physical sciences. But they’re still evolving and growing and researchers in different fields are still discussing their benefits. One such researcher is systems biologist Sarvenaz Sarabipour of John Hopkins University in Maryland, USA. Sarvenaz recently published a paper in PLOS Biology titled ‘On the value of preprints: An early career researcher perspective.’ She’s become a big fan of preprint servers, not least because of the easy access they provide to research.
Sarvenaz Sarabipour
I’m originally from Iran, and I have seen for many years how researchers are struggling with access to scientific data in countries where it’s hard to even gain funds to perform research, let alone have access to research or publish research, and we realised that many early career researchers are unaware of the benefits, so we decided to write this piece.
Adam Levy
I called Sarvenaz up to discuss why researchers are turning to preprints and why some might still feel a little bit cautious. I started by checking the fundamentals: what preprints servers actually serve to do.
Sarvenaz Sarabipour
They are online servers where you can deposit your manuscript or nearly complete scientific work or even your abstract free of charge, and everyone in the world who has access to the internet would be able to access your work.
Adam Levy
And why would a researcher want to publish on a preprint rather than holding out until their paper hopefully passes peer review.
Sarvenaz Sarabipour
Well, it’s very beneficial for researchers to deposit their work immediately because journals are not able to do that, and what preprinting is doing is that it’s decoupling dissemination form the peer-review process. It’s wonderful to have it published earlier when journals would spend six months to years, sometimes, sending it through cycles of review. So, the peer-review process is inhibitory to dissemination, but the peer-review process, of course, has added value, so the preprint server clearly states that the work is not peer reviewed, but the work is accessible to all immediately.
Adam Levy
Are there times when researchers might want to choose not to publish their work on preprint severs and what kind of times would those be?
Sarvenaz Sarabipour
The only incidents that I could think of would be if your work is patentable, and in that case, I think researchers wait and patent the work and then preprint it, if they are hoping to send it to a journal later for peer review. And it’s key that researchers first check a journal that they’re interested in publishing in to find out about their preprint policies.
Adam Levy
Now, you’ve written a paper specifically on preprints for early career researchers. How does the role of preprints change at different stages of an academic career?
Sarvenaz Sarabipour
As an early career and very early career researcher, of course, you don’t have many papers, so it’s wonderful to have something out quicker and be able to discuss that with your colleagues and more senior researchers. Researchers can notice each other’s work quicker, they contact each other, if they have something similar they may start collaborating and producing a common paper, and it accelerates the early career researcher’s training and the quality of training.
Adam Levy
Now, how do preprint servers change the process of writing a paper, dealing with the publication process, and also promoting a paper once it finally comes out?
Sarvenaz Sarabipour
After your work is disseminated on a preprint server, you have a chance to receive feedback from the community. So, this is different from the classic process where researchers would send maybe their manuscript to a limited number of colleagues confidentially and then proceed to journal submission. The authors then can respond to the comment and subsequently, if needed, revise their preprint before submitting to a journal for peer review.
Adam Levy
And for you, what do you think the future of preprint servers is?
Sarvenaz Sarabipour
Oh, the future is very bright. There are already many servers in specific fields available to researchers. So, there is EarthArXiv, engrXiv, there is PsyArXiv, there is one for palaeontology and there are also servers for different regions of the world and countries. Millions of people read and review preprints.
Adam Levy
Do you have any other thoughts on how the process of paper publishing in journals will change in the future?
Sarvenaz Sarabipour
I also do think that the open access aspect of preprints is very valuable for researchers in other countries, not just the ones that are privileged to have subscriptions through their libraries to journals. I think that it will accelerate science as a global enterprise and it will really benefit the public globally.
Adam Levy
That was Sarvenaz Sarabipour, and many researchers, like Sarvenaz, swear by preprints, and it’s not unusual for journalists to cover preprint papers, albeit hopefully with more caution than they would a peer-reviewed manuscript. But not every researcher in every field is happy to show their work in progress in this way. Here’s catalyst researcher Ben List of the Max Planck Institute for Coal Research in Germany.
Ben List
With these preprint servers, this is something that’s a bit risky, I would say, in my field of chemical synthesis because it’s a different thing in physics or biology, where experiments take a long time and typically you know who are working on these experiments. In chemistry, you see something and then within a few days you can actually reproduce this work, so I’m not 100% sure, even though I’ve just submitted a paper on ChemRxiv, if this is the future of publishing in chemistry, at least.
Adam Levy
But Ben isn’t a stickler for tradition. In fact, he’s embarked on an experiment to tinker with one of the central pillars in all of science – the peer-review process. He’s doing this because while he is a chemist, he has another important role.
Ben List
I’m editor in chief of the journal SYNLETT, and that’s a journal that specialises in chemical synthesis. Last year, we celebrated our 30th birthday, so it’s a relatively young – in comparison to Nature, for example – journal.
Adam Levy
Well, even though it is 30 years old now, it is still trying to do things in a youthful and new way. What is SYNLETT doing perhaps a bit differently to some other journals?
Ben List
So, in the past, we would do everything pretty much the same as other journals, but what we really do differently now is our peer reviewing. Pretty much everybody in scientific publishing thinks about peer review, right? The principle is exactly the same as it used to be several hundred years ago, in fact, and to my knowledge other attempts have been made to advance this, but our idea was really how can we make use of all these powerful tools of the internet to come up with a better and faster way to do peer review because not only can the quality sometimes suffer in peer reviewing, but also the pace can be a bit slow and then in the end of it you get like, I don’t know, a couple of sentences that are not very useful.
Adam Levy
So, what did you actually set out to do differently at SYNLETT?
Ben List
Our idea was let’s use this energy of the internet where people like to connect with each other and think about WhatsApp and Facebook and these social media, and also crowd reviewing, which I think originally is a trend that came from product reviews. And we wanted to use these elements in a new format for peer review, and the way we did it was we increased the referee pool, so we have typically, let’s say, 70 people at the same time but these are selected experts on chemical synthesis, and let them interact. Basically, what happens is they get a link in an email, they click on it, a PDF appears in a browser, and then they can start commenting on that PDF. It sounds very simple and it is very simple, but it’s also very effective, and so now the referee will see the PDF and he might put his opinion there in a little text box, and then all the 69 other referees see this opinion and can, in principle, also comment on it. And that’s it. So, you have a larger referee size and you have the ability of these referees to interact with each other, and that’s what makes it so powerful because if you think about classical peer review, one of the problems is you have only two people typically, sometimes it’s three, sometimes it’s more, that comment on your work and imagine one of them is biased, for example. It could be your good friend, which is maybe good for you if you’re the author, but not so good for science, and then 50% of your material is then basically useless. So, if you have 70 referees, you will always get enough information and because there are so many and not all of them need to write even, within two days, typically, we have more than enough substantive feedback. And the biasing problem, if he’s like either your good friend or your enemy, will be immediately corrected because there are 69 other referees that see your bias and will say, ‘No, this is not right, actually.’
Adam Levy
You often hear stories about it being hard to find reviewers for papers even when only two or three reviewers are required, so expecting a pool of some-70 reviewers to be there as soon as you need them, isn’t that making that even more challenging?
Ben List
Yeah, you’re right, it sounds like it. In practice, actually, it does work. With SYNLETT, we now completely switch to only this format because it works so well, but I tell you what the secret is. The secret is while there are 70 referees at the same time, not all of them need to write. In fact, if you’re busy and you don’t have time, you don’t have to do anything because there are 69 others that will do the job. In a sense, this is sort of crowd intelligence based refereeing, and so we get our results quickly and we get enough substantive information to make a really good decision. That’s how it makes my life easier as the editor.
Adam Levy
Now, of course, you’re singing its praises, you’re very happy with how it’s going. Have there been any hiccups or challenges that you’ve encountered along the way of setting it up?
Ben List
Just in case you have somebody misbehaving or people don’t even comment, they just go there and watch and never comment, then we can just replace them. So, this occasionally happens but as I said, it’s actually kind of rare. But if it happens, it’s like an organism that some cells die and new cells come in, and so we just hire new referees.
Adam Levy
Are any other journals looking to start this?
Ben List
I travel the world as so many scientists do and I give talks about my science, but now I find myself often in the situation where I get more questions to crowd reviewing than to my science, which is a little bit annoying, right? I want to talk about my science. But suffice to say there are several other journals and several other publishing houses that have developed interest in this method but also funding agencies because they have the same problem, right?
Adam Levy
And how have the paper authors themselves been responding? Have they managed to adapt to this new style of peer review?
Ben List
Well, they are also happy if they get the results quickly, right? Sometimes you submit a paper, then you wait four weeks and then you get a triage from the editor because he was so busy and then, of course, that’s super disappointing for authors. We all have papers rejected, manuscripts rejected, but if we get this information quickly, I think it’s kind of nice, especially if it’s substantive with substantive comments. And as I said, it’s going really, really well. All our manuscripts are reviewed after three days. Do you know any other journal who is that fast and still doing a very detailed job?
Adam Levy
Beyond just this kind of pool of peer reviewers, what’s your vision or hope for the future of academic paper publication?
Ben List
One thing I see how peer reviewing could evolve is imagine if the authors are also involved in this crowd reviewing situation. The editor can be there and the authors can be there and they can argue with the referees, right? Then there’s a little discussion going on. I would like that.
Adam Levy
Ben List there. SYNLETT’s approach to peer review is just one example of how journals are beginning to tinker with the new tools that online social media provide. Another is to keep the practice more or less the same, but to make it transparent. In fact, Nature recently announced that it would give authors the option of having referee reports published, along with their own responses and rebuttals once a manuscript is better. Pippa, who you heard from earlier, thinks this kind of approach has some important benefits.
Pippa Whitehouse
I’ve published in some of the EGU journals, where the whole process is open and so the reviews and the response to the reviews are open to the public, and I like that process. I think it forces people to think carefully about the comments that they write, and it’s actually useful to sometimes read the reviewer comments as well as the final paper. It gives some insight into how science is really done and how it’s discussed to reach the conclusions that are published.
Adam Levy
And neuroscientist Agustin Ibanez of the Institute of Cognitive and Translational Neuroscience in Argentina and the University of California, San Francisco agrees.
Agustin Ibanez
I think that this is so important because science is not a static object. It is really a dynamic one. All the results or even the argumentations is a dynamic process that doesn’t finish with your first draft of your paper, so having a more transparent process for this is, I think, of extreme importance in the process of publication.
Adam Levy
No matter how the process evolves and whatever the paper of the future looks like, one thing is for certain – papers are the backbone of science, building careers and spreading research around the world to other academics and to the general public. So, whether you’re setting out to publish your very first paper or you’re just beginning to put together your 100th manuscript, what are you waiting for? Get to work and start sharing your research. Thanks for listening to this miniseries of Working Scientist, a Nature Careers podcast. Find all the episodes and much, much more over at nature.com/careers. Until next time, I’m Adam Levy.