Flaws in evaluation schemes for pair-input computational predictions

Article metrics

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Figure 1: Toy example (nine proteins) demonstrating prediction of protein-protein interactions and showing the shortcomings of a typical cross-validation scheme.


  1. 1

    Shen, J. et al. Proc. Natl. Acad. Sci. USA 104, 4337–4341 (2007).

  2. 2

    Wikberg, J.E. & Mutulis, F. Nat. Rev. Drug Discov. 7, 307–323 (2008).

  3. 3

    Yabuuchi, H. et al. Mol. Syst. Biol. 7, 472 (2011).

  4. 4

    Bellucci, M., Agostini, F., Masin, M. & Tartaglia, G.G. Nat. Methods 8, 444–445 (2011).

  5. 5

    Gottlieb, A., Stein, G.Y., Ruppin, E. & Sharan, R. Mol. Syst. Biol. 7, 496 (2011).

  6. 6

    Vert, J.-P. & Yamanishi, Y. Advances in Neural Information Processing Systems vol. 17 (eds. Saul, L., Weiss. Y. & Bottou, L.) 1433–1440 (MIT Press, Cambridge, Massachusetts, USA, 2005).

  7. 7

    Flicek, P. et al. Nucleic Acids Res. 39, D800–D806 (2011).

  8. 8

    Schaefer, M.H. et al. PLoS ONE 7, e31826 (2012).

  9. 9

    Tropsha, A. & Golbraikh, A. Curr. Pharm. Des. 13, 3494–3504 (2007).

  10. 10

    Olah, M., Bologa, C. & Oprea, T.I. J. Comput. Aided Mol. Des. 18, 437–449 (2004).

Download references


We thank W.S. Noble, A. Ben-Hur, J.-P. Vert and V. Helms for stimulating discussions and critical comments on the manuscript. This work was supported by grants to E.M.M. from the US National Institutes of Health, the US Army (58343-MA), Cancer Prevention and Research in Texas and the Welch (F1515) Foundation. Y.P. acknowledges financial support from the Deutsche Forschungsgemeinschaft (DFG-Forschungsstipendium).

Author information

Correspondence to Yungki Park or Edward M Marcotte.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Tables 1–4, Supplementary Methods and Supplementary Discussion (PDF 534 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Park, Y., Marcotte, E. Flaws in evaluation schemes for pair-input computational predictions. Nat Methods 9, 1134–1136 (2012) doi:10.1038/nmeth.2259

Download citation

Further reading