Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Reply to ‘Inflated performance measures in enhancer–promoter interaction-prediction methods’

Nature Genetics volume 51, pages 1201–1202 (2019)Cite this article

Subjects

Cao and Yip reply — Cao and Fullwood1 argue that the reported performance of JEME2 was affected by the presence of positive pairs of enhancers and transcription start sites (TSSs) in the training and testing sets that overlap in their window regions. They also show that our random targets approach to generating negative pairs led to very different distance distributions for the positive and negative pairs (Fig. 2g in ref. 1), thus making it trivial to separate the positives and negatives on the basis of genomic distance features alone.

Regarding the issue of overlapping positive examples in the training and testing sets, because ultimately the main use of JEME is to predict enhancer–target interactions in unseen samples, the across-sample tests are most informative. In these tests, all information from the training sample can be used to predict the enhancer–target pairs in the testing sample, and therefore there is no concern regarding overlapping positive examples. Cao and Fullwood report that in the across-sample test involving K562 as the training sample and GM12878 as the testing sample, JEME was not as accurate as a simplified version of JEME involving only genomic distance features. They achieved this result by tuning the maximum tree depth parameter of the random-forest model, whereas we used the default setting without any parameter tuning in our original paper, thus explaining the difference. For GM12878, only expression quantitative trait locus (eQTL) data were available for validating the predicted enhancer–target pairs, a scenario that is not ideal, because eQTLs could include indirect interactions and could miss true enhancer–target interactions with no eQTLs at the enhancers. We included eQTLs as validation data mainly because of a lack of direct chromatin-interaction data at the time of our study. We therefore checked MCF-7 cells, from which chromatin interaction analysis with paired-end tags (ChIA-PET) data targeting RNA polymerase II were available for validating the predictions. These data provide more direct and comprehensive information about enhancer–target interactions. When we trained the models by using K562 data and tested them on MCF-7 cells, JEME consistently performed better than the simplified version of JEME with only distance features at all values of the maximum tree depth parameter that we considered (Fig. 1c), including the values used by Cao and Fullwood. The AUPR improvement at these tree depths ranged from 0.021 to 0.065, which overlapped with the values that we reported in the original paper (0.040–0.050 for the three across-sample cases in Fig. 3c in ref. 2).

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Relationship between enhancer–TSS genomic distance and JEME performance.

Data availability

The data that support the findings of this study are available at http://yiplab.cse.cuhk.edu.hk/jeme/.

References

  1. Cao, F. & Fullwood, M. J. Nat. Genet. https://doi.org/10.1038/s41588-019-0434-7 (2019).

  2. Cao, Q. et al. Nat. Genet. 49, 1428–1436 (2017).

    Article  CAS  Google Scholar 

  3. Roy, S. et al. Nucleic Acids Res. 43, 8694–8712 (2015).

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Biomedical Sciences, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong

    Qin Cao

  2. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong

    Kevin Y. Yip

  3. Hong Kong Bioinformatics Centre, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong

    Kevin Y. Yip

  4. CUHK-BGI Innovation Institute of Trans-omics, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong

    Kevin Y. Yip

  5. Hong Kong Institute of Diabetes and Obesity, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong

    Kevin Y. Yip

Authors
  1. Qin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Y. Yip
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Q.C. and K.Y.Y. designed the analyses. Q.C. carried out the analyses. Q.C. and K.Y.Y. interpreted the data and prepared the manuscript.

Corresponding author

Correspondence to Kevin Y. Yip.

Ethics declarations

Competing interests

The authors declare no competing interests.

Supplementary Information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, Q., Yip, K.Y. Reply to ‘Inflated performance measures in enhancer–promoter interaction-prediction methods’. Nat Genet 51, 1201–1202 (2019). https://doi.org/10.1038/s41588-019-0452-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41588-019-0452-5

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing