Reliability and accuracy of single-molecule FRET studies for characterization of structural dynamics and distances in proteins

Single-molecule Förster-resonance energy transfer (smFRET) experiments allow the study of biomolecular structure and dynamics in vitro and in vivo. We performed an international blind study involving 19 laboratories to assess the uncertainty of FRET experiments for proteins with respect to the measured FRET efficiency histograms, determination of distances, and the detection and quantification of structural dynamics. Using two protein systems with distinct conformational changes and dynamics, we obtained an uncertainty of the FRET efficiency ≤0.06, corresponding to an interdye distance precision of ≤2 Å and accuracy of ≤5 Å. We further discuss the limits for detecting fluctuations in this distance range and how to identify dye perturbations. Our work demonstrates the ability of smFRET experiments to simultaneously measure distances and avoid the averaging of conformational dynamics for realistic protein systems, highlighting its importance in the expanding toolbox of integrative structural biology.

Please include a "Data availability" subsection in the Online Methods.This section should inform readers about the availability of the data used to support the conclusions of your study, including accession codes to public repositories, references to source data that may be published alongside the paper, unique identifiers such as URLs to data repository entries, or data set DOIs, and any other statement about data availability.At a minimum, you should include the following statement: "The data that support the findings of this study are available from the corresponding author upon request", describing which data is available upon request and mentioning any restrictions on availability.If DOIs are provided, please include these in the Reference list (authors, title, publisher (repository name), identifier, year).For more guidance on how to write this section please see: http://www.nature.com/authors/policies/data/data-availability-statements-data-citations.pdfCODE AVAILABILITY Please include a "Code Availability" subsection in the Online Methods which details how your custom code is made available.Only in rare cases (where code is not central to the main conclusions of the paper) is the statement "available upon request" allowed (and reasons should be specified).
We request that you deposit code in a DOI-minting repository such as Zenodo, Gigantum or Code Ocean and cite the DOI in the Reference list.We also request that you use code versioning and provide a license.
For more information on our code sharing policy and requirements, please see: https://www.nature.com/nature-research/editorial-policies/reporting-standards#availability-ofcomputer-codeMATERIALS AVAILABILITY As a condition of publication in Nature Methods, authors are required to make unique materials promptly available to others without undue qualifications.
Authors reporting new chemical compounds must provide chemical structure, synthesis and characterization details.Authors reporting mutant strains and cell lines are strongly encouraged to use established public repositories.
More details about our materials availability policy can be found at https://www.nature.com/natureportfolio/editorial-policies/reporting-standards#availability-of-materialsORCID Nature Methods is committed to improving transparency in authorship.As part of our efforts in this direction, we are now requesting that all authors identified as 'corresponding author' on published papers create and link their Open Researcher and Contributor Identifier (ORCID) with their account on the Manuscript Tracking System (MTS), prior to acceptance.This applies to primary research papers only.ORCID helps the scientific community achieve unambiguous attribution of all scholarly contributions.You can create and link your ORCID from the home page of the MTS by clicking on 'Modify my Springer Nature account'.For more information please visit please visit <a href="http://www.springernature.com/orcid">www.springernature.com/orcid</a>.Please do not hesitate to contact me if you have any questions or would like to discuss these revisions further.We look forward to seeing the revised manuscript and thank you for the opportunity to consider your work.

Sincerely, Rita
Rita Strack, Ph.D. Senior Editor Nature Methods

Reviewers' Comments:
Reviewer #1: Remarks to the Author: This manuscript describes a very important new multilab study concerning the reliability of smFRET measurements for analyzing distances and dynamics in biomolecules.It is actually a study that probably many FRET researchers have waited for (at least I have) since the authors' previous Nature Methods publication, in which DNA conformation and dynamics were analyzed by multiple labs using smFRET.This new work extends the successful smFRET realiability, precision, and accuracy investigation to proteins and it is truly impressive how well smFRET can analyze also these significantly more complicated biomolecules.The manuscript clearly summarizes the main results and the online methods as well as an extensive Supporting Information file contain all necessary information to follow and possibly reproduce the work from the different labs.I also appreciated that the authors discuss difficulties (for example for the determination of the  correction factor) in detail and propose possible solutions to overcome the challenges.Overall, this study, which has been performed by internationally renowned labs, presents an important contribution to science and knowledge.It clearly shows that smFRET is an important method for biophysical analysis and is well suited for Nature Methods.I recommend the publication of this manuscript after minor revision (see comments below, which are mainly related to clarity of description).
Abstract: I would change "worldwide" to "international".Asia, Africa, Australia, and South America are not present in the study.P4: Introduction: Instead of directly describing the sub-field smFRET, I would suggest to shortly introduce FRET in general.I proposed to cite a recent FRET review paper from this very journal (https://doi.org/10.1038/s41592-019-0530-8)and a book chapter that describes the general theory of FRET (https://doi.org/10.1002/9783527656028.ch05).
P4: Was the study of Hellenkamp et al. not performed by 20 labs?P4: When discussing the differences between DNA and protein analysis, is the controlled bioconjugation (position and orientation within the protein) not also a point that is significantly more challenging when analyzing proteins?P5: Why was U2AF2 investigated in only 7 labs?There must be a specific reason why 12 labs within a network of collaborating labs did not perform those measurements.Are they too challenging?Is specific equipment and/or analysis required?P7: Was the dye-labeling also performed by each different lab or were the samples prepared by one lab?Dye labeling is probably an important factor of experimental uncertainly.How was this taken into account.Also, it seems there are 5 labeling positions (all cysteines) and the two dyes (D and A) were stochastically labeled.Does that mean that each dye can take any position?That would also mean that there is a very large distribution of possible D and A fractions and different positions for each system with a defined fraction of D and A. I assume that this dye distribution will significantly influence the FRET results and it would be interesting if those aspects can be discussed in more detail.P7: Why only values from 16 labs.What happened to the other 3 labs?I see in the caption of Figure 1 and TableS1 that there were measurement problems.I think that deserves a little more discussion.In the current form, it seems that 3 out of 19 labs face problems, which is more than 15%.Is it likely that the same problems occur for other labs?Are there suggestions to overcome those problems?P10: If measurements for MalE were performed by 16 labs (or 19), why are the setup dependent parameters and correction factors analyzed for only 8 labs?P12: Is there any possibility to improve the accuracy and precision of the  correction factor?For example by using a specific setup, pair of dyes etc.?
Equations: I recommend that the authors double check all their equations.I did not check them all but I found that, for example, Equation S7.1 contains an error: The refractive index should be to the power of 4. I assume it is a typo and the authors used the correct equation for calculating R0.

Reviewer #2: None
Reviewer #3: Remarks to the Author: In their work, Gam et al. present a comparative study to establish error margins for smFRET experiments of proteins.Whereas a previous study (Hellenkamp et al. 2018 Nat Methods, 15, 669) by some of the authors benchmarked smFRET experiments with DNA samples, the current study focuses on proteins that are more heterogeneous both chemically and structurally.The study includes 19 labs and two proteins, MalE (maltose binding protein) and U2AF2 (a spliceosome component).Notably, besides determining lab-to-lab variability of FRET efficiencies and distances, the authors also investigated the reliability of smFRET experiments to identify dynamic properties of the investigated proteins, a task that is clearly new and interesting.For transfer efficiencies, the authors find an uncertainty of ±0.06 across different labs, which is similar to the error found in the previous benchmark study.The authors spotted the sources of this variation between labs in the correction factor determination of the optical system, particularly the gamma-factor, which accounts for the brightness differences and detection differences between donor and acceptor channel.The uncertainty is highest at intermediate FRET efficiencies and using error propagation, the authors present a rigorous analysis of these effects in the Supplementary Information.An analysis of 7 data sets for U2AF2 gave an error of ±0.03 that was reduced to ±0.008 in an analysis of all data sets (and correction factor determination) by a single person, which demonstrates that instrument variations are not detrimental if correction factors are determined correctly -an important result.In the second part of the work, the authors discuss two methods to identify dynamics in biomolecules: BVA (burst variance analysis) and the combined analysis of donor fluorescence lifetimes and FRET efficiencies.Most labs identified dynamics (or no dynamics) correctly using these methods.
The effect of such dynamics on the accuracy of FRET-distance conversions is important.A combined analysis of FRET, anisotropy, and lifetime-FRET plots (Fig. 5) and the use of the positional distribution of the dyes (as given by AV or ACV models) shows that rotational and translational dynamics of the dyes due to their long linkers and dye sticking to protein surfaces can be a source of additional variability.Limits of average limiting anisotropies (<0.2-0.25) to estimate the impact of such processes where confirmed using a diffusion-with-traps (DWT) model and the optimal choice of dyes and labeling sites were stressed.The final part of the paper, a quantitative analysis of U2AF2 using smFRET, lifetime MEM analysis, filtered FCS, NMR and SAXS seems to be less informative, except maybe for researcher working on this particular protein.
Overall, this joint effort provides an excellent overview over the potential and limits of smFRET experiments together with a rich set of analysis tools that allow researcher to improve their analysis (if they are not already doing it anyway).The study will particularly be helpful for future integrative approaches in which different methods (smFRET, SAXS, SANS, NMR, EPR) are combined to generate structural ensembles of biomolecules.I am therefore very positive, but I would like to ask the authors to address a few comments: 1. Stressing the number of 19 labs in the abstract and discussion seems a bit exaggerated as only FRET of the protein MalE has been studied by 19 labs, whereas U2AF2 was studied by 7 labs, and also dynamics where studied by 7-8 labs.This is still a large number and I don't think this study requires any exaggeration.
2. The determination of gamma values and other correction factors has been done in a global manner, i.e., including experiments of different variants or conditions.This was necessary as FRET distributions of MalE or U2AF2 are unimodal.Yet, a significant difference in E (less in S) between two populations is required to determine the gamma factor precisely.It would have been interesting to determine the uncertainty in gamma as function of the separation of FRET peaks.For two FRET-peaks with coordinates (E, S) and (E+ΔE, S+ΔS) in the E-S plot that are corrected for background, leakage, and acceptor direct excitation, and assuming identical localization error dS and dE (which should be proportional to 1/√N with N being the average photon number), the error in determining gamma should scale as 1/ΔE, which seems significant to me.It would be helpful to see a more rigorous analysis in the paper.
3. Related to point 2, gamma determinations with dye solutions are an alternative used by some labs.Clearly, as a downside, dyes are not in a native environment, i.e., close to the protein surface, yet, shot noise doesn't play a role in correction factor determinations.The authors might want to comment on the pros and cons of the two strategies.
4. I didn't find the numbers used for the factors l1 and l2 for anisotropy measurements in a confocal setup (Supplement p. 21).I am asking since determining these factors is often associated with a significant error (personal experience).Did the authors determine the values for their systems or did they take literature values? 5.It was not clear to me how the authors chose the correct averaging regime for FRET-calculations based on structural models given that the dyes do not only rotate but also translationally diffuse within the AV (AVC) volume.When analyzing the effect of donor-quenching (Supplementary Note 10), the authors used Brownian diffusion of the dye in the AV-volume.For distance calculations in absence of quenching, the authors computed FRET efficiency averaged model distances between the two dyes (p.15, 2nd paragraph) and ref. 4 indicates that a static average across positions in the AV volume was chosen.Since translational motion across significant distances happens at a timescale of ns as suggested by the dye-diffusion coefficient provided by the authors (10 Å2/ns for one dye), I was wondering whether the authors take the partial averaging in the AV-volume during the dye fluorescence lifetime into account, e.g., via Brownian dynamics simulations alike those used to analyse donor quenching?6. Related to point 5, recent results (Klose et al. 2021 Biophys J, 120, 4842) suggest that rotamer libraries seem superior to coarse-grained AV calculations for computing FRET values from structures.Please comment.

Author Rebuttal to Initial comments Decision Letter, first revision:
Dear Don, Thank you for submitting your revised manuscript "Reliability and accuracy of single-molecule FRET studies for characterization of structural dynamics and distances in proteins" (NMETH-AS50048A).It has now been seen by the original referees and their comments are below.The reviewers find that the paper has improved in revision, and therefore we'll be happy in principle to publish it in Nature Methods, pending minor revisions to comply with our editorial and formatting guidelines.
We are now performing detailed checks on your paper and will send you a checklist detailing our editorial and formatting requirements in about a week.Please do not upload the final materials and make any revisions until you receive this additional information from us.
TRANSPARENT PEER REVIEW Nature Methods offers a transparent peer review option for new original research manuscripts submitted from 17th February 2021.We encourage increased transparency in peer review by publishing the reviewer comments, author rebuttal letters and editorial decision letters if the authors agree.Such peer review material is made available as a supplementary peer review file.Please state in the cover letter 'I wish to participate in transparent peer review' if you want to opt in, or 'I do not wish to participate in transparent peer review' if you don't.Failure to state your preference will result in delays in accepting your manuscript for publication.
Please note: we allow redactions to authors' rebuttal and reviewer comments in the interest of confidentiality.If you are concerned about the release of confidential data, please let us know specifically what information you would like to have removed.Please note that we cannot incorporate redactions for any other reasons.Reviewer names will be published in the peer review files if the reviewer signed the comments to authors, or if reviewers explicitly agree to release their name.For more information, please refer to our <a href="https://www.nature.com/documents/nr-transparentpeer-review.pdf"target="new">FAQ page</a>.ORCID IMPORTANT: Non-corresponding authors do not have to link their ORCIDs but are encouraged to do so.Please note that it will not be possible to add/modify ORCIDs at proof.Thus, please let your co-authors know that if they wish to have their ORCID added to the paper they must follow the procedure described in the following link prior to acceptance: https://www.springernature.com/gp/researchers/orcid/orcid-for-nature-researchThank you again for your interest in Nature Methods.Please do not hesitate to contact me if you have any questions.We will be in touch again soon.

Sincerely, Rita
Rita Strack, Ph.D. Senior Editor Nature Methods Reviewer #1 (Remarks to the Author): The authors have carefully revised their manuscript and appropriately answered to the comments of both reviewers.The additional explanations and data have further improved the clarity and quality of the manuscript.I recommend the publication of this revised version in Nature Methods.No more modifications necessary.
Reviewer #3 (Remarks to the Author): I thank the authors for providing detailed answers to my points.

Final Decision Letter:
Dear Don, I am pleased to inform you that your Analysis, "Reliability and accuracy of single-molecule FRET studies for characterization of structural dynamics and distances in proteins", has now been accepted for publication in Nature Methods.Your paper is tentatively scheduled for publication in our May print issue, and will be published online prior to that.The received and accepted dates will be August 2, 2022 and Jan 31, 2023.This note is intended to let you know what to expect from us over the next month or so, and to let you know where to address any further questions.
Acceptance is conditional on the data in the manuscript not being published elsewhere, or announced in the print or electronic media, until the embargo/publication date.These restrictions are not intended to deter you from presenting your data at academic meetings and conferences, but any enquiries from the media about papers not yet scheduled for publication should be referred to us.
Over the next few weeks, your paper will be copyedited to ensure that it conforms to Nature Methods style.Once your paper is typeset, you will receive an email with a link to choose the appropriate publishing options for your paper and our Author Services team will be in touch regarding any additional information that may be required.
After the grant of rights is completed, you will receive a link to your electronic proof via email with a request to make any corrections within 48 hours.If, when you receive your proof, you cannot meet this deadline, please inform us at rjsproduction@springernature.com immediately.
Please note that <i>Nature Methods</i> is a Transformative Journal (TJ).Authors may publish their research with us through the traditional subscription access route or make their paper immediately open access through payment of an article-processing charge (APC).Authors will not be required to make a final decision about access to their article until it has been accepted.<a href="https://www.springernature.com/gp/open-research/transformative-journals">Find out more about Transformative Journals</a> phone and fax) of someone who will be able to check the proofs and deal with any last-minute problems.
If, when you receive your proof, you cannot meet the deadline, please inform us at rjsproduction@springernature.com immediately.
Once your manuscript is typeset and you have completed the appropriate grant of rights, you will receive a link to your electronic proof via email with a request to make any corrections within 48 hours.If, when you receive your proof, you cannot meet this deadline, please inform us at rjsproduction@springernature.com immediately.
Once your paper has been scheduled for online publication, the Nature press office will be in touch to confirm the details.
If you have posted a preprint on any preprint server, please ensure that the preprint details are updated with a publication reference, including the DOI and a URL to the published version of the article on the journal website.
Once your paper has been scheduled for online publication, the Nature press office will be in touch to confirm the details.
Content is published online weekly on Mondays and Thursdays, and the embargo is set at 16:00 London time (GMT)/11:00 am US Eastern time (EST) on the day of publication.If you need to know the exact publication date or when the news embargo will be lifted, please contact our press office after you have submitted your proof corrections.Now is the time to inform your Public Relations or Press Office about your paper, as they might be interested in promoting its publication.This will allow them time to prepare an accurate and satisfactory press release.Include your manuscript tracking number NMETH-AS50048B and the name of the journal, which they will need when they contact our office.About one week before your paper is published online, we shall be distributing a press release to news organizations worldwide, which may include details of your work.We are happy for your institution or funding agency to prepare its own press release, but it must mention the embargo date and Nature Methods.Our Press Office will contact you closer to the time of publication, but if you or your Press Office have any inquiries in the meantime, please contact press@nature.com.
To assist our authors in disseminating their research to the broader community, our SharedIt initiative provides you with a unique shareable link that will allow anyone (with or without a subscription) to read the published article.Recipients of the link with a subscription will also be able to download and print the PDF.
directly from the figure legend.Spreadsheets can be submitted in .xls,.xlsxor .csvformats.Only one (1) file per figure is permitted: thus if there is a multi-paneled figure the source data for each panel should be clearly labeled in the csv/Excel file; alternately the data for a figure can be included in multiple, clearly labeled sheets in an Excel file.File sizes of up to 30 MB are permitted.When submitting source data files with your manuscript please select the Source Data file type and use the Title field in the File Description tab to indicate which figure the source data pertains to.