Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Reply to: "Proper reporting of predictor performance"

Nature Methods volume 11, pages 781–782 (2014)Cite this article

Subjects

Kumar et al. reply:

We disagree with Vihinen's1 suggestion that the performance of the EvoD2 method based on evolutionary stratification of prediction models was not evaluated correctly, and we affirm the importance of the method. The need for evolutionary stratification arose because we discovered that existing methods exhibited a very high rate of false positive diagnoses for variants occurring at the most highly conserved positions (ref. 2 Table 1). We had observed a high rate of false negatives for variants found in positions that have evolved the fastest2. These discoveries established the biological pitfalls of existing approaches, all of which fit a single prediction model that is agnostic to differences in evolutionary conservation among positions.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Vihinen, M. Proper reporting of predictor performance. Nat. Methods 11, 781 (2014).

    Article  CAS  Google Scholar 

  2. Kumar, S. et al. Nat. Methods, 9, 855–856 (2012).

    Article  CAS  Google Scholar 

  3. Liu, L. & Kumar, S. Mol. Biol. Evol. 30, 1252–1257 (2013).

    Article  CAS  Google Scholar 

  4. Wei, Q. & Dunbrack, R.L. PLoS ONE 8, e67863 (2013).

    Article  CAS  Google Scholar 

  5. Eiland, E.E. & Liebrock, L.M. Adv. Artificial Intelligence 2013, 427958 (2013).

    Article  Google Scholar 

  6. Obayashi, T. & Kinoshita, K. DNA Res. 16, 249–260 (2009).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This work is supported by research grants from the US National Institutes of Health (HG002096-12 and LM010834-04).

Author information

Authors and Affiliations

  1. Institute for Genomics and Evolutionary Medicine, Temple University, Philadelphia, Pennsylvania, USA

    Sudhir Kumar & Li Liu

  2. Center for Evolutionary Medicine and Informatics, Arizona State University, Tempe, Arizona, USA

    Sudhir Kumar, Jieping Ye & Li Liu

  3. School of Computing, Informatics and Decision Systems Engineering, Arizona State University, Tempe, Arizona, USA

    Jieping Ye

Authors
  1. Sudhir Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jieping Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudhir Kumar.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, S., Ye, J. & Liu, L. Reply to: "Proper reporting of predictor performance". Nat Methods 11, 781–782 (2014). https://doi.org/10.1038/nmeth.3039

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.3039

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing