Uncovering developmental time and tempo using deep learning

During animal development, embryos undergo complex morphological changes over time. Differences in developmental tempo between species are emerging as principal drivers of evolutionary novelty, but accurate description of these processes is very challenging. To address this challenge, we present here an automated and unbiased deep learning approach to analyze the similarity between embryos of different timepoints. Calculation of similarities across stages resulted in complex phenotypic fingerprints, which carry characteristic information about developmental time and tempo. Using this approach, we were able to accurately stage embryos, quantitatively determine temperature-dependent developmental tempo, detect naturally occurring and induced changes in the developmental progression of individual embryos, and derive staging atlases for several species de novo in an unsupervised manner. Our approach allows us to quantify developmental time and tempo objectively and provides a standardized way to analyze early embryogenesis.

We request that you deposit code in a DOI-minting repository such as Zenodo, Gigantum or Code Ocean and cite the DOI in the Reference list.We also request that you use code versioning and provide a license.
For more information on our code sharing policy and requirements, please see: https://www.nature.com/nature-research/editorial-policies/reporting-standards#availability-ofcomputer-codeMATERIALS AVAILABILITY As a condition of publication in Nature Methods, authors are required to make unique materials promptly available to others without undue qualifications.
Authors reporting new chemical compounds must provide chemical structure, synthesis and characterization details.Authors reporting mutant strains and cell lines are strongly encouraged to use established public repositories.
More details about our materials availability policy can be found at https://www.nature.com/natureportfolio/editorial-policies/reporting-standards#availability-of-materialsSUPPLEMENTARY PROTOCOL To help facilitate reproducibility and uptake of your method, we ask you to prepare a step-by-step Supplementary Protocol for the method described in this paper.We <a href="https://www.nature.com/nature-research/editorial-policies/reporting-standards#protocols"target="new">encourage authors to share their step-by-step experimental protocols</a> on a protocol sharing platform of their choice and report the protocol DOI in the reference list.Nature Portfolio 's Protocol Exchange is a free-to-use and open resource for protocols; protocols deposited in Protocol Exchange are citable and can be linked from the published article.More details can found at <a href="https://www.nature.com/protocolexchange/about"target="new">www.nature.com/protocolexchange/about</a>.ORCID Nature Methods is committed to improving transparency in authorship.As part of our efforts in this direction, we are now requesting that all authors identified as 'corresponding author' on published papers create and link their Open Researcher and Contributor Identifier (ORCID) with their account on the Manuscript Tracking System (MTS), prior to acceptance.This applies to primary research papers only.ORCID helps the scientific community achieve unambiguous attribution of all scholarly contributions.You can create and link your ORCID from the home page of the MTS by clicking on 'Modify my Springer Nature account'.For more information please visit please visit <a href="http://www.springernature.com/orcid">www.springernature.com/orcid</a>.
Please do not hesitate to contact me if you have any questions or would like to discuss these revisions further.We look forward to seeing the revised manuscript and thank you for the opportunity to consider your work.

Sincerely, Madhura
Madhura Mukhopadhyay, PhD Senior Editor Nature Methods

Reviewers' Comments:
Reviewer #1: Remarks to the Author: The authors present essentially a robust method to compute and quantify phenotypical differences between images of larvae.Application of this method to a huge number of images from developing zebrafish embryos, by comparing different stages, results in the definition of a development trajectory for normal embryos.This is then used to analyze variability between "normal" individuals during the first 24 hrs of development, to identify individuals that deviate from the normal trajectory, and to compare trajectories of normal embryos from that of embryos treated with inhibitors for BMP or Nodal signaling.
Finally, the authors show that comparison of images from a single embryo, but at different stages, allows for definition of developmental epochs and stages, which they also apply to less well characterized species such as medaka, stickleback, and even C. elegans.This approach is certainly novel, it is convincing in the results it provides, and impressive in the scale of work presented.Only minor points need to be addressed: Minor points: 1) In line 62, the authors state that their dataset comprises 3 million images, in Fig. 1a they state 15 million: how comes?Could it be that the 15 million were required to obtain the 3 million "high quality" images used for model training?What happened to the remaining one's, were they ever used?2) Lines 152-175: the authors clearly illustrate that defects due to depletion of specific signaling pathways will be detected by their developmental staging method, however without giving any information about the specific defect that causes the divergence from the normal staging trajectory.Possibly (presumably), treatment with another compound affecting BMP signaling would result in a trajectory indistinguishable from the one shown here.However, I also presume that very different phenotypes may occur at similar stages, thus yielding a similar deviation from the normal trajectory, but also from the BMP-inhibition one.Would it be useful to have a "developmental trajectory database" that could be used for new studies, on other compounds even in a different lab.Could another lab use the images from this work and run the comparison with its own images, or would all the images need to be recaptured (different setting, different conditions)?3) Supplementary Fig. 6: I am not sure this representation is adding to the comprehension, given that Fig. 4b is already quite clear.4) Supplementary Fig. 8c: there seems to be a mismatch between the stage timing in minutes indicated along the images and the stage timing in hours in the "autostage strip".5) How many images were used for the medaka, stickleback and C. elegans autostage determination?I finally found it in "Image acquisition", but it should be indicated maybe in the figure legends.A lower number of images may explain the plethora of autostages seen for stickleback, given the presumably also high variability that was also observed in zebrafish.

Marc Muller
Reviewer #2: Remarks to the Author: noneThis manuscript is a follow-up paper of a previously published study employing neural networks to phenotype embryogenesis in fish.The current work makes substantial advances and is conceptually different as it employs a twin-network architecture.This interesting methodology allows the authors to quantify morphological distances in an unbiased manner.It is applied to normal development and signaling perturbation and even extended to other species of fish and even C. elegans.In general, the paper is informative, presents a novel approach and appears scientifically sound and well documented.The biological findings confirm long known concepts and are rather unsurprising, thus this is rather a methodological paper that does not provide much novel biological insight.However, the potential to build upon this methodological framework is large and thus, I think it is of exceptionally high scientific interest to a larger audience.
I have two main questions: Fig. 3 depicts how phenotypic differences after signaling perturbation can be detected in an unbiased manner.That is cool and has a multitude of potential applications.However, I didn't understand how robust in terms of sensitivity or specificity such a phenotyping method would be.Given that the perturbation is known, can the parameter space in which this is feasible be defined more clearly?I would love to get a feeling for how many embryos need to be treated and how penetrant the phenotype would have to be to be distinguishable from random noise, and how and where the detection would reach its limits.I know this depends on multiple, investigator specific parameters, threshold levels, … So I don't expect an easy answer, but could the dataset the authors gathered serve as an example case to derive a clearer picture of how usable the twin-network phenotyping would be in an applied setting?Is there a way to quantify how robust this method would distinguish a phenotype without prior knowledge?What parameters are crucial in which rages?
The distance matrices across developmental stages are impressive.It seems that Fig. 4 And those shown in Suppl Fig. 7 were each derived from a single embryo.Interestingly and unsurprisingly, they do show a certain degree of variability.Given the enormous dataset the authors have at hand.Could a metaanalysis of such distance maps be constructed that would depict this inter-individual variability more clearly?Or is the variability due to random fluctuations of the model?How can biological and technical variability be distinguished?
The authors could discuss how further methodological improvement may advance this technology to the next level.Also, the limitations of using twin-networks could be elaborated upon.
Reviewer #3: Remarks to the Author: Toulany&al propose a deep-learning based method of zebrafish embryo staging.The authors recorded a very large number (~10 000) of movies of developing zebrafish embryos.Using a deep-learning neural network, they show that they can then automatically « stage » an embryo by calculating a similarity measure between an embryo image and their dataset.They show that this prediction becomes more uncertain as developmental time progresses, and that abnormal embryos diverge from the general trend earlier than what the human eye would recognise.The authors show perturbed trajectories of similarity measures between unperturbed and perturbed embryos, following drug treatment.They also show that comparison of one embryo images to an earlier time points define broad « epochs » where embryo shapes are more comparable to each other than in between epochs.
Overall, this work is nicely done, clearly explained and relies on an interesting dataset of many developing embryos.It is nice to see a solid quantification of the so-far qualitative notion of « staging »; it is clear that this quantification is made possibly only by the use of deep learning and a very large dataset.
Beyond these positive points, I also find that the conclusions of the study are somewhat lacking in depth: we can see from the author analysis that perturbed embryos are different from unperturbed embryos; that some embryos do not develop normally; that there are recognisable broad stages of development based on embryo morphology; however it seems to me that none of these observations seems to go very far what can already be seen by eye.It is nice to see that this rigorous method recapitulates known observations, but how does it allow to go beyond them?For instance from the title referring to « tempo » I was expecting a more in-depth analysis of variability in developmental timing than what the manuscript provides.
Therefore, although the study introduces a new rigorous staging method, it is not clear to me at this point that the paper clearly shows this method can or will bring a significant change in the study of developmental dynamics.
Other comments: -I think that the caption of Fig. 4a is not sufficiently descriptive, it is difficult to understand what the schematics mean.

Author Rebuttal to Initial comments Decision Letter, first revision:
Dear Patrick, Thank you for submitting your revised manuscript "Uncovering developmental time and tempo using deep learning" (NMETH-A52047A).It has now been seen by the original referees and their comments are below.The reviewers find that the paper has improved in revision, and therefore we'll be happy in principle to publish it in Nature Methods, pending minor revisions to satisfy the referees' final requests and to comply with our editorial and formatting guidelines.
We are now performing detailed checks on your paper and will send you a checklist detailing our editorial and formatting requirements within two weeks or so.Please do not upload the final materials and make any revisions until you receive this additional information from us.
TRANSPARENT PEER REVIEW Nature Methods offers a transparent peer review option for new original research manuscripts submitted from 17th February 2021.We encourage increased transparency in peer review by publishing the reviewer comments, author rebuttal letters and editorial decision letters if the authors agree.Such peer review material is made available as a supplementary peer review file.Please state in the cover letter 'I wish to participate in transparent peer review' if you want to opt in, or 'I do not wish to participate in transparent peer review' if you don't.Failure to state your preference will result in delays in accepting your manuscript for publication.
Please note: we allow redactions to authors' rebuttal and reviewer comments in the interest of confidentiality.If you are concerned about the release of confidential data, please let us know specifically what information you would like to have removed.Please note that we cannot incorporate redactions for any other reasons.Reviewer names will be published in the peer review files if the reviewer signed the comments to authors, or if reviewers explicitly agree to release their name.For more information, please refer to our <a href="https://www.nature.com/documents/nr-transparentpeer-review.pdf"target="new">FAQ page</a>.ORCID IMPORTANT: Non-corresponding authors do not have to link their ORCIDs but are encouraged to do so.Please note that it will not be possible to add/modify ORCIDs at proof.Thus, please let your co-authors know that if they wish to have their ORCID added to the paper they must follow the procedure described in the following link prior to acceptance: https://www.springernature.com/gp/researchers/orcid/orcid-for-nature-researchThank you again for your interest in Nature Methods.Please do not hesitate to contact me if you have any questions.We will be in touch again soon.

Sincerely, Madhura
Madhura Mukhopadhyay, PhD Senior Editor Nature Methods Reviewer #1 (Remarks to the Author): As previously sated, the authors present a method to define developmental trajectories using images from zebrafish embryos.The authors extended their original submission by adding the analysis of zebrafish and medakas at different temperatures, as well as in the presence of different signalling inhibitors or mutations.These new data add to the validitynand applicability of the method, a more broader perspective is also included.I consider that my comments, and those of the other reviewers were accurately addressed.
Reviewer #2 (Remarks to the Author): I find the additional data included in the revised manuscript impressive and insghtful.The authors all of my concerns in full.I suggest to publish this paper in Nat Methods.
Reviewer #3 (Remarks to the Author): In their revision, Tulany&al have answered my comments and notably have added a study of temperature-dependence of development in Fig. 2 which strongly strengthens the manuscript.Therefore I recommend publication of this work.

Final Decision Letter:
Dear Patrick, I am pleased to inform you that your Article, "Uncovering developmental time and tempo using deep learning", has now been accepted for publication in Nature Methods.Your paper is tentatively scheduled for publication in our December print issue, and will be published online prior to that.The received and accepted dates will be 19 Mar 2023 and 15 Oct 2023.This note is intended to let you know what to expect from us over the next month or so, and to let you know where to address any further questions.
Acceptance is conditional on the data in the manuscript not being published elsewhere, or announced in the print or electronic media, until the embargo/publication date.These restrictions are not intended to deter you from presenting your data at academic meetings and conferences, but any enquiries from the media about papers not yet scheduled for publication should be referred to us.
Over the next few weeks, your paper will be copyedited to ensure that it conforms to Nature Methods style.Once your paper is typeset, you will receive an email with a link to choose the appropriate publishing options for your paper and our Author Services team will be in touch regarding any additional information that may be required.
You will receive a link to your electronic proof via email with a request to make any corrections within 48 hours.If, when you receive your proof, you cannot meet this deadline, please inform us at rjsproduction@springernature.com immediately.
Please note that <i>Nature Methods</i> is a Transformative Journal (TJ).Authors may publish their research with us through the traditional subscription access route or make their paper immediately open access through payment of an article-processing charge (APC).Authors will not be required to make a final decision about access to their article until it has been accepted.<a href="https://www.springernature.com/gp/open-research/transformative-journals">Find out more about Transformative Journals</a> to be accepted, including <a href="https://www.springernature.com/gp/open-research/policies/journalpolicies">self-archivingpolicies</a>.Those licensing terms will supersede any other terms that the author or any third party may assert apply to any version of the manuscript.
If you have any questions about our publishing options, costs, Open Access requirements, or our legal forms, please contact ASJournals@springernature.com Your paper will now be copyedited to ensure that it conforms to Nature Methods style.Once proofs are generated, they will be sent to you electronically and you will be asked to send a corrected version within 24 hours.It is extremely important that you let us know now whether you will be difficult to contact over the next month.If this is the case, we ask that you send us the contact information (email, phone and fax) of someone who will be able to check the proofs and deal with any last-minute problems.
If, when you receive your proof, you cannot meet the deadline, please inform us at rjsproduction@springernature.com immediately.
Once your manuscript is typeset and you have completed the appropriate grant of rights, you will receive a link to your electronic proof via email with a request to make any corrections within 48 hours.If, when you receive your proof, you cannot meet this deadline, please inform us at rjsproduction@springernature.com immediately.
Once your paper has been scheduled for online publication, the Nature press office will be in touch to confirm the details.
If you have posted a preprint on any preprint server, please ensure that the preprint details are updated with a publication reference, including the DOI and a URL to the published version of the article on the journal website.
Once your paper has been scheduled for online publication, the Nature press office will be in touch to confirm the details.
Content is published online weekly on Mondays and Thursdays, and the embargo is set at 16:00 London time (GMT)/11:00 am US Eastern time (EST) on the day of publication.If you need to know the exact publication date or when the news embargo will be lifted, please contact our press office after you have submitted your proof corrections.Now is the time to inform your Public Relations or Press Office about your paper, as they might be interested in promoting its publication.This will allow them time to prepare an accurate and satisfactory press release.Include your manuscript tracking number NMETH-A52047B and the name of the journal, which they will need when they contact our office.About one week before your paper is published online, we shall be distributing a press release to news organizations worldwide, which may include details of your work.We are happy for your institution or funding agency to prepare its own press release, but it must mention the embargo date and Nature Methods.Our Press Office will contact you closer to the time of publication, but if you or your Press Office have any inquiries in the meantime, please contact press@nature.com.
To assist our authors in disseminating their research to the broader community, our SharedIt initiative provides you with a unique shareable link that will allow anyone (with or without a subscription) to read the published article.Recipients of the link with a subscription will also be able to download and print the PDF.
As soon as your article is published, you will receive an automated email with your shareable link.
You can now use a single sign-on for all your accounts, view the status of all your manuscript submissions and reviews, access usage statistics for your published articles and download a record of your refereeing activity for the Nature journals.Nature Portfolio journals <a href="https://www.nature.com/nature-research/editorialpolicies/reporting-standards#protocols"target="new">encourage authors to share their step-by-step experimental protocols</a> on a protocol sharing platform of their choice.Nature Portfolio 's Protocol Exchange is a free-to-use and open resource for protocols; protocols deposited in Protocol Exchange are citable and can be linked from the published article.More details can found at <a href="https://www.nature.com/protocolexchange/about"target="new">www.nature.com/protocolexchange/about</a>.Please note that you and any of your coauthors will be able to order reprints and single copies of the issue containing your article through Nature Portfolio's reprint website, which is located at http://www.nature.com/reprints/author-reprints.html.If there are any questions about reprints please send an email to author-reprints@nature.com and someone will assist you.
Please feel free to contact me if you have questions about any of these points.

Best regards, Madhura
Madhura Mukhopadhyay, PhD Senior Editor Nature Methods