A clear scientific protocol — a set of detailed instructions for a specific experimental method — is a valuable resource. Good protocols facilitate the transfer of knowledge, equip new laboratory members with essential skills and ensure that experiments can be replicated after researchers move on. Scientists and journal editors offer their tips on creating and sharing effective protocols.
NICOLAS ROHNER: Reach out to your community and involve early-career researchers
Assistant professor at Stowers Institute, Kansas City, Missouri.
While doing my postdoctoral training at Harvard Medical School, from 2010 to 2015, I started working on blind cavefish, specifically Astyanax mexicanus. They are not a typical animal model, but they are an excellent species for studying processes such as loss of vision (because their surface-dwelling relatives can see) and many metabolic adaptations to extreme environments, such as nutrient-poor caves. Although the cavefish research community is steadily growing, it is still small compared with those studying more conventional model organisms such as mice or zebrafish (Danio rerio). Our cavefish protocols are less established and require more time and resources to optimize before they can be used in each laboratory.
For example, a major challenge I had initially was breeding the cavefish. I knew that lowering the temperature and salinity of the water, which is what happens to the cavefish’s environment when it rains, can trigger breeding in some fish, but this did not work well for this species. When I reached out to and visited other labs working on cavefish, they shared a protocol that triggered breeding by raising the water temperature from 25 to 28 °C. That worked in part: I also had to make sure the water temperature was uniform, and modify other parameters, such as light cycles and food supply. Further challenges in working with cavefish include the lack of genetic tools and even appropriate commercial lab supplies. The community has to develop new protocols from scratch, even for simple experiments.
To encourage more researchers to join and expand this field to a critical mass, protocols need to work well, but they also should be collaboratively created and shared. The community has created a website called CAVEFIN to share cavefish-specific protocols. The website also lists labs working on cavefish, enabling researchers to easily reach out to one another for technical support. I also find it useful to involve early-career researchers and technicians in creating, reviewing and sharing protocols. For instance, my lab members led a team to publish a protocol1 on in vitro fertilization in cavefish (see below). Labs should try to involve junior researchers as early as possible in protocol development because, compared with senior researchers, they have more recent hands-on experiences and make use of protocols more regularly, almost on a daily basis.
MELANIE CLYNE: Keep protocols timely and updated
Chief Editor of Nature Protocols, London.
Research papers often lack detailed information about methodology. When describing a technique, authors will often cite a previous paper where it has been used. Often, this paper cites a previous paper, which might cite an even earlier paper, and the wild goose chase goes on. Methodologies evolve over time, so even if one can track down the original source of the technique, it might well have changed a great deal over the years. This can cause serious problems because even the smallest variations can lead to major differences in outcomes.
Protocols help to minimize the amount of resources wasted in trying to reproduce the work of others, while ensuring authors get the credit they deserve for methodological developments. When we at Nature Protocols decide to publish a protocol, the method is typically a gold-standard or cutting-edge technique that has been used to generate data for a well-respected, peer-reviewed journal.
Protocols are also a valuable tool in the Reproducibility Initiative, which aims to validate research projects, methods and reagents through independent replication by expert labs. Researchers can submit a request to replicate a paper, and the experiments are matched with an appropriate, expert lab. Once the lab has recreated the experiment, the results are published in one of the reproducibility collections of the publisher PLOS. If they are reproduced successfully, a Reproducibility Initiative ‘Independently Validated’ badge is awarded to the original paper.
To be effective, protocols should contain background information on the technique, such as how it was developed, how it compares with other techniques, its advantages and limitations, the diversity of applications and details on how to adapt or modify it. It is also helpful to include anticipated results in protocols so that users will know if their experiments have worked (see ‘How to structure a great protocol’).
Protocols should be kept updated to reflect progress in science, and this is why Nature Protocols, as well as other protocol journals such as Current Protocols, give authors the option to publish updates or extensions to their protocols.
LENNY TEYTELMAN: Develop protocols using digital tools and share them
Co-founder and chief executive of protocols.io, Berkeley, California.
Researchers devote a lot of resources to reproducing and correcting methods, but they are not always rewarded. As a postdoctoral researcher in 2009, I spent more than a year correcting a microscopy technique for imaging RNA, because the reagent volume and incubation time had been incorrectly reported. Many other members of my lab had to redo experiments because the data we collected were misleading. However, my updated protocol was not considered a new method and it was only for one model organism, yeast. So, eventually, we just posted the correction on the lab website. At that time, there was also no good platform out there to share the updated protocol with my scientific community. This was frustrating: I could only imagine many other researchers wasting their time trying to troubleshoot.
I think that a good protocol is a dynamic piece of work to be shared publicly, with the authors also getting credit for sharing. For instance, at protocols.io, we have more than 9,000 public protocols, which are continually being iterated and improved through feedback from authors and users. Researchers are able to compare different versions of a protocol to see what has changed and why, and they can cite the protocols, too.
Digitalization is a powerful way to strengthen protocols. A digital protocol can contain a greater number of fine details such as the choice of reagent, the equipment to use and safety precautions that might be too wordy and would typically be left out of a paper protocol. Important information such as safety warnings for chemical and biological materials, which would otherwise make a paper protocol too lengthy, can be included as individual sections of a digital protocol and accessed as separate tabs.
A digital protocol can also facilitate documentation and optimization. Our software interface at protocols.io enables users to digitally record how they might have executed a step differently in real time as they run their experiments. This helps in developing methods and tracing the root causes of unexpected experimental outcomes. Because digital protocols can be accessed remotely by colleagues at different locations, there would also be a livelier exchange of ideas to make them better for all stakeholders.
Today, if my RNA imaging technique was already published on protocols.io, I could share corrections and optimizations with a few clicks. The original protocol is preserved and the scholarly record is intact, but sharing updates is easy. Anyone using the older version will be alerted to an update.
NATHALIE GAUDREAULT: Be creative in protocol development and share only robust ones
Director of Microscopy Pipeline at the Allen Institute for Cell Science, Seattle, Washington.
At the Allen Institute for Cell Science, we are developing open-access cell lines, gene-editing and image-analysis tools, and protocols for the scientific community. The methods that make up these tools are not straightforward and can take up to a year to be optimized before they can be executed reproducibly across different labs. For instance, when we were developing our protocols to culture and image genetically engineered stem cells, members of different groups at our institute had to come over and watch one another before they could reproduce the steps to thaw and culture the cells.
This experience motivated us to share video tutorials with our users. There are more than 500 labs in 26 countries using our cell lines, and it is our responsibility to make sure that the protocols can be robustly replicated internally before we share them with external collaborators.
The Allen Institute has a dedicated communications team that helps us make video tutorials. But smaller labs can also do this kind of work: we’ve created multiple videos with an inexpensive GoPro camera, iPhone and tripod, such as our video on our automated tissue culture platform (see below).
I’d suggest that those developing new video protocols be creative and think of the audience: what details do they need to see and learn? We once went to the lengths of using an endoscope to capture a bird’s-eye view of a pipette tip in our automatic cell-culture-and-imaging robot, so that researchers interested in building their own robot can see exactly what’s happening. Those who are developing video protocols should also consider creating a storyboard and script to structure the video. Also, take multiple shots of the same step at different angles and edit the best shots together later.
Many papers from our institute have long methods sections, but because of page limits, these are relegated to the supplementary materials. In the future, protocol papers will be shared and published alongside original papers from our institute that have extensive methods. In doing so, I think we are one step closer to greater reproducibility, fairness and accountability in science.
RONALD MYERS: Take advantage of visual teaching and learning
Director of Editorial at JoVE (the Journal of Visualized Experiments), Cambridge, Massachusetts.
A good protocol should contain sufficient details, however small, for inexperienced researchers: for instance, how much time a process takes and whether there is a preferred brand for reagents. A good protocol should also try to provide a troubleshooting guide that lists commonly encountered issues, together with a list of plausible reasons and solutions. This can make troubleshooting more efficient, leaving more time for data collection.
Most scientific protocols are based on text. To the best of my knowledge, JoVE is the first journal dedicated to publishing video protocols and methods. Admittedly, some scientific fields benefit more from video protocols than others. Protocols in the mathematical and computational fields, for example, can be too abstract, whereas surgery is best taught using video.
In cases where there are too many fine details and a researcher’s choices can affect experiments, video protocols can strengthen reproducibility better than written ones. Even when the most experienced researchers have left, their knowledge could still be transferred to many subsequent generations of lab members through videos.
When JoVE started in 2006, we were publishing about ten visual protocols per month. The number has since increased to 100 per month and our website gets 30 million views annually. I expect the numbers to increase further. Society prefers to consume information in video form now, and our video-production team, which helps researchers create protocols, has expanded our operations worldwide.
These interviews have been edited for length and clarity.
Peuß, R. et al. J. Vis. Exp. 147, e59334 (2019).