To the Editor:
Improving reproducibility and transparency in science requires increased attention to the details of research. We have previously reported that studies citing the inability to replicate findings in preclinical research reveal economic estimates in the billions of dollars wasted1. Preclinical research relies on the use of high-quality preserved tissue and fluid specimens to infer disease mechanisms and develop biomarkers for disease tracking. Patients generously donate samples with the hope of advancing the treatment of disease; improving reproducibility with clinical specimens affirms a commitment to those patients. Among the factors that contribute to irreproducibility is whether tissue samples are handled equivalently in different studies2.
Research labs obtain samples from biorepositories, which process and distribute biological specimens. The processing steps can have profound effects on the outcome of subsequent experiments in which the processed samples are used. In a survey of recent literature, we identified more than 300 scientific papers demonstrating how variables in sample preparation (e.g., ischemia time, temperature, buffers, and freezing steps) affect the outcomes of assays. When researchers compare samples prepared via different standard operating procedures (SOPs), a type of apples-to-oranges problem arises that makes it difficult to reproduce results. For example, detection of the protein fragment amyloid-β (1–42) is important for the diagnosis of Alzheimer's disease, but measurements of the peptide will be incorrect when samples are not processed at the proper temperature3. Large variances between sample results continue to delay the validation of amyloid-β (1–42) as a diagnostic tool4.
Recent increases in the sensitivity of experimental assays have revealed the fragility and complexity of biological samples, which can be easily altered by processing methods. Cold ischemia times under one hour affect the phosphorylation state of up to 24% of phosphorylation sites on proteins detected by liquid chromatography–tandem mass spectrometry in ovarian and breast cancer tumor samples5. Anticoagulants, processing times, and the collection tubes used affect protein profiles in serum and plasma samples analyzed by surface-enhanced laser desorption/ionization mass spectrometry6. Subjecting a urine sample to a single freeze–thaw cycle greatly affects its peptide profile as detected by matrix-assisted laser desorption/ionization mass spectrometry7. Researchers can identify and account for the artifacts of sample preparation only when they know how collection and storage sites handled the samples.
The sample-preparation steps carried out prior to an experiment are rarely reported in publications. Often the researchers themselves do not have this information. Indeed, descriptions of how samples were prepared prior to their distribution can be difficult to obtain. Sixty-seven percent of biorepositories do not currently share their sample-preparation SOPs publicly, based on a sampling of 300 biorepository websites, and some even refuse to share their SOPs when they are requested, treating their SOPs as trade secrets despite receiving public or philanthropic funds. Some biorepositories discount the need for researchers to have access to their specific SOPs. During our own attempts to obtain SOPs, one biorepository provided the simple response, “We process...samples the same way everyone else does.”
None of this indicates that biorepositories are handling samples incorrectly, but it emphasizes the need to know processing details in order to compare results obtained with samples from different repositories. Neither journals nor funding agencies specifically request information on sample preparation, and a bias toward brevity in methods sections makes it easy to omit this information.
We propose an alternative to individual biorepository websites, SOP documents e-mailed upon request, and the need to piece together information from incomplete methods sections. We suggest that all biorepositories deposit their SOPs, in fully searchable text form, in a centralized database that links SOPs to the sample collections that used them. Thus a single SOP may get linked to several sample collections, and researchers can easily locate details on how their samples were processed. Publications that use samples could list the ID number of the sample-preparation SOP to save space in the methods section. SOPs could be derived with minimal effort from one another with lineage history to make the evolution and relatedness of processing methods traceable. This approach would enable researchers to identify how particular sample collections were processed and to find collections appropriate for their experiments. We have launched an open project, BiospecimenCommons.org, that accomplishes this by listing and linking biorepositories, collections they store, and the SOPs used to process those samples.
This approach expands and improves upon efforts already being undertaken by the biorepository community. College of American Pathologists (CAP) accreditation (http://www.cap.org) requires the establishment of SOPs that are verified by CAP inspectors, and the International Society for Biological and Environmental Repositories (ISBER) (http://www.isber.org) has published best practices for writing SOPs. However, neither requires (or even strongly encourages) repositories to share those SOPs with the researchers who use the samples. The Biorepositories and Biospecimen Research Branch (BBRB) of the US National Institutes of Health has initiated the Biospecimen Research Database (BRD) (https://brd.nci.nih.gov/brd/), a database of SOPs. The BRD is not focused on sample collection and it stores flat files with metadata, which limits the ability to search protocols electronically, for example, to identify variations in multiple protocols. The Biobanking and BioMolecular Resources Research Infrastructure–European Research Infrastructure Consortium has produced a database of European biorepositories (https://directory.bbmri-eric.eu/) that is based on the Minimum Information About Biobank Data Sharing (MIABIS) guidelines8. MIABIS details the information a biorepository should share regarding the patient demographics of samples and which tissue(s) the samples were collected from, but does not currently address sample processing. The Biospecimen Reporting for Improved Study Quality (BRISQ) guidelines9 suggest data to share about samples and their processing, but from our review of publications and biorepository websites, we infer that samples rarely have BRISQ data attached when they are sent to researchers. None of these efforts enable scientists to find and evaluate the best samples for their experiments. Most important, there is no current system that links clinical samples to the SOPs used to prepare them.
Biospecimen Commons was developed using the MIABIS guidelines for both biorepository and collection listings, and SOPs support the attachment of BRISQ codes. Biospecimen Commons already has 300 biorepositories listed and 154 sample-preparation SOPs published.
Biorepositories must adopt the use of SOPs (for those that do not practice this), and they should share their SOPs and link them to the appropriate sample sets. One way to encourage transparency might be to reward those biorepositories that do share SOPs with badges on their page entries and associated published articles. Journals could also encourage, or perhaps require, the use of links to protocol databases with accession numbers, similar to genomic and proteomic submissions. Funding agencies should insist that protocols used to collect samples using award money be made public. Researchers who obtain samples from biorepositories should request the SOPs used to prepare samples and cite the digital location of those SOPs in publications. By developing an environment of transparent processing of tissue samples, we can enable researchers to choose the best samples available for their experiments, leading to both more reproducible research and better allocation of these limited and valued resources.
Freedman, L.P., Cockburn, I.M. & Simcoe, T.S. PLoS Biol. 13, e1002165 (2015).
LaBaer, J. J. Proteome Res. 11, 5592–5601 (2012).
Sancesario, G.M. et al. Exp. Neurol. 223, 371–376 (2010).
Mattsson, N. et al. Alzheimers Dement. 9, 251–261 (2013).
Mertins, P. et al. Mol. Cell. Proteomics 13, 1690–1704 (2014).
Banks, R.E. et al. Clin. Chem. 51, 1637–1649 (2005).
Bruegel, M. et al. J. Proteomics 72, 608–615 (2009).
Merino-Martinez, R. et al. Biopreserv. Biobank. 14, 298–306 (2016).
Moore, H.M. et al. J. Proteome Res. 10, 3429–3438 (2011).
The authors declare no competing financial interests.
About this article
Cite this article
LaBaer, J., Miceli, J. & Freedman, L. What's in a sample? Increasing transparency in biospecimen procurement methods. Nat Methods 15, 303–304 (2018). https://doi.org/10.1038/nmeth.4684
Enhancing Cooperation Between Academic Biobanks and Biomedical Industry: Better Mutual Understanding and New Collaborative Models Are Needed
Biopreservation and Biobanking (2020)
Preanalytics and Precision Pathology: Pathology Practices to Ensure Molecular Integrity of Cancer Patient Biospecimens for Precision Medicine
Archives of Pathology & Laboratory Medicine (2019)