Knowing the sequence specificities of DNA- and RNA-binding proteins is essential for developing models of the regulatory processes in biological systems and for identifying causal disease variants. Here we show that sequence specificities can be ascertained from experimental data with 'deep learning' techniques, which offer a scalable, flexible and unified computational approach for pattern discovery. Using a diverse array of experimental data and evaluation metrics, we find that deep learning outperforms other state-of-the-art methods, even when training on in vitro data and testing on in vivo data. We call this approach DeepBind and have built a stand-alone software tool that is fully automatic and handles millions of sequences per experiment. Specificities determined by DeepBind are readily visualized as a weighted ensemble of position weight matrices or as a 'mutation map' that indicates how variations affect binding within a specific sequence.
At a glance
- DNA binding sites: representation and discovery. Bioinformatics 16, 16–23 (2000).
- Origins of specificity in protein-DNA recognition. Annu. Rev. Biochem. 79, 233–269 (2010). et al.
- RNAcontext: a new method for learning the sequence and structure binding preferences of RNA-binding proteins. PLoS Comput. Biol. 6, e1000832 (2010). , , , &
- Direct measurement of DNA affinity landscapes on a high-throughput sequencing instrument. Nat. Biotechnol. 29, 659–664 (2011). et al.
- Protein-DNA binding: complexities and multi-protein codes. Nucleic Acids Res. 42, 2099–2111 (2014). &
- Advances in Neural Information Processing Systems (eds. Pereira, F., Burges, C.J.C., Bottou, L. & Weinberger, K.Q.) 1097–1105 (Curran Associates, 2012). , & in
- Speech recognition with deep recurrent neural networks. ICASSP 6645–6649 (2013). , &
- Rapid analysis of the DNA-binding specificities of transcription factors with DNA microarrays. Nat. Genet. 36, 1331–1339 (2004). et al.
- Rapid and systematic analysis of the RNA recognition specificities of RNA-binding proteins. Nat. Biotechnol. 27, 667–670 (2009). et al.
- Design and analysis of ChIP-seq experiments for DNA-binding proteins. Nat. Biotechnol. 26, 1351–1359 (2008). , &
- Multiplexed massively parallel SELEX for characterization of human transcription factor binding specificities. Genome Res. 20, 861–873 (2010). et al.
- Highly expressed loci are vulnerable to misleading ChIP localization of multiple unrelated proteins. Proc. Natl. Acad. Sci. USA 110, 18602–18607 (2013). , , &
- Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989). et al.
- Advances in Neural Information Processing Systems (Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F. & Weinberger, K.Q.) 1647–1655 (Curran Associates, 2011). , , & in
- Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014). , , , &
- Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012). &
- Evaluation of methods for modeling transcription factor sequence specificity. Nat. Biotechnol. 31, 126–134 (2013). et al.
- Quantitative analysis demonstrates most transcription factors require only simple models of specificity. Nat. Biotechnol. 29, 480–483 (2011). , , &
- Statistical mechanical modeling of genome-wide transcription factor occupancy data by MatrixREDUCE. Bioinformatics 22, e141–e149 (2006). , &
- RankMotif.: a motif-search algorithm that accounts for relative ranks of K-mers in binding transcription factors. Bioinformatics 23, i72–i79 (2007). , &
- Compact, universal DNA microarrays to comprehensively determine transcription-factor binding site specificities. Nat. Biotechnol. 24, 1429–1435 (2006). et al.
- A compendium of RNA-binding motifs for decoding gene regulation. Nature 499, 172–177 (2013). et al.
- Shape-specific recognition in the structure of the Vts1p SAM domain with RNA. Nat. Struct. Mol. Biol. 13, 160–167 (2006). et al.
- RRM-RNA recognition: NMR or crystallography...and new findings. Curr. Opin. Struct. Biol. 23, 100–108 (2013). , &
- The role of RNA sequence and structure in RNA–protein interactions. J. Mol. Biol. 409, 574–587 (2011). &
- ChIP-seq guidelines and practices of the ENCODE and modENCODE consortia. Genome Res. 22, 1813–1831 (2012). et al.
- Sequence features and chromatin structure around the genomic regions bound by 119 human transcription factors. Genome Res. 22, 1798–1812 (2012). et al.
- MEME-ChIP: Motif analysis of large DNA datasets. Bioinformatics 27, 1696–1697 (2011). &
- Application of experimentally verified transcription factor binding sites models for computational analysis of ChIP-Seq data. BMC Genomics 15, 80 (2014). et al.
- JASPAR 2014: an extensively expanded and updated open-access database of transcription factor binding profiles. Nucleic Acids Res. 42, D142–D147 (2014). et al.
- TRANSFAC and its module TRANSCompel: transcriptional gene regulation in eukaryotes. Nucleic Acids Res. 34, D108–D110 (2006). et al.
- DNA-binding specificities of human transcription factors. Cell 152, 327–339 (2013). et al.
- Transcriptional regulation and its misregulation in disease. Cell 152, 1237–1251 (2013). &
- The Human Gene Mutation Database: building a comprehensive mutation repository for clinical and molecular genetics, diagnostic testing and personalized genomic medicine. Hum. Genet. 133, 1–9 (2014). et al.
- Functional analysis of LDLR promoter and 5′ UTR mutations in subjects with clinical diagnosis of familial hypercholesterolemia. Hum. Mutat. 32, 868–872 (2011). et al.
- The 8q24 cancer risk variant rs6983267 shows long-range interaction with MYC in colorectal cancer. Nat. Genet. 41, 882–884 (2009). et al.
- A regulatory SNP causes a human genetic disease by creating a new transcriptional promoter. Science 312, 1215–1217 (2006). et al.
- GATA-4 regulates Bcl-2 expression in ovarian granulosa cell tumors. Endocrinology 149, 5635–5642 (2008). et al.
- COSMIC: mining complete cancer genomes in the Catalogue of Somatic Mutations in Cancer. Nucleic Acids Res. 39, D945–D950 (2011). et al.
- Evolutionarily dynamic alternative splicing of GPR56 regulates regional cerebral cortical patterning. Science 343, 764–768 (2014). et al.
- The transcription factor GABP selectively binds and activates the mutant TERT promoter in cancer. Science 348, 1036–1039 (2015). et al.
- TERT promoter mutations in familial and sporadic melanoma. Science 339, 959–961 (2013). et al.
- Highly recurrent TERT promoter mutations in human melanoma. Science 339, 957–959 (2013). et al.
- A general framework for estimating the relative pathogenicity of human genetic variants. Nat. Genet. 46, 310–315 (2014). et al.
- Deep surveying of alternative splicing complexity in the human transcriptome by high-throughput sequencing. Nat. Genet. 40, 1413–1415 (2008). , , , &
- MBNL proteins repress ES-cell-specific alternative splicing and reprogramming. Nature 498, 241–245 (2013). et al.
- RBFOX1 regulates both splicing and transcriptional networks in human neuronal development. Hum. Mol. Genet. 21, 4171–4186 (2012). et al.
- An RNA map predicting Nova-dependent splicing regulation. Nature 444, 580–586 (2006). et al.
- The RNA-binding protein TIA-1 is a novel mammalian splicing regulator acting through intron sequences adjacent to a 5′ splice site. Mol. Cell. Biol. 20, 6287–6299 (2000). et al.
- Genome-wide analysis of PTB-RNA interactions reveals a strategy used by the general splicing repressor to modulate exon inclusion or skipping. Mol. Cell 36, 996–1006 (2009). et al.
- Diversity and complexity in DNA recognition by transcription factors. Science 324, 1720–1723 (2009). et al.
- Supplementary Figure 1: An extended version of Figure 2a, depicting multi-model training and reverse-complement mode (90 KB)
To use the GPU’s full computational power, we train several independent models in parallel on the same data, each with different calibration parameters. The calibration parameters with validation performance are used to train the final model. Shown is an example with batch_size=5, motif_len=6, num_motifs=4, num_models=3. Sequences are padded with ‘N’s so that the motif scan operation can find detections at both extremities. Yellow cells represent the reverse complement of the input located above; both strands are fed to the model, and the strand with the maximum score is used for the output prediction (the max strand stage). The output dimension of the pool stage, depicted as num_motifs (*), depends on whether “max” or “max and avg” pooling was used.
- Supplementary Figure 2: Performance of in vitro trained TF models on in vivo data (DREAM5 ChIP-seq) (129 KB)
(a) All DREAM5 ChIP-seq AUCs used to compute mean performance shown in Figure 3b. The models were trained on the DREAM5 PBM training data only, and evaluated against three different backgrounds22. (b) Cross-validation performance of methods trained directly on ChIP-seq data (sequence length 100), evaluated against a dinucleotide shuffled background (Supplementary Table 1).
- Supplementary Figure 3: Performance on in vitro RBP data using several evaluation metrics (65 KB)
Box plots showing distribution of RNAcompete in vitro RBP performance over 244 different microarray experiments using 6 evaluation metrics (columns) and two types of correlation (rows). Models were trained on RNAcompete PBM probes labeled “Set A”, and tested on “Set B” probes.
- Supplementary Figure 4: Performance of in vitro trained RBP models on in vivo data (90 KB)
Performance of all RBP models for which RNAcompete in vivo data was available (c.f. Ray et al.19, Fig. 1C). Figure 3d shows only the subset of RBPs for which the in vivo test sequences has average length <1000. All AUCs are calculated with 100 bootstrap samples, and the standard deviation is shown as vertical lines. “Base counts” show the best performance achievable from ranking test sequences by the proportion of a single nucleotide or by sequence length; for example, ranking the QKI test sequences by 1/(fraction of Gs) gives AUC of 0.95. There are 9 RBPs for which at least one method can perform better than base counts on this test data. RNAcompete PFMs beat base counts for PUM2, SRSF1, FMR1, and Vts1p. DeepBind beats base counts for 8 RBPs (no significant improvement for FMR1). See Supplementary Table 3 (“In vivo AUCs”) for raw data for this plot.
- Supplementary Figure 5: ROC curves for the AUCs shown in Figure 3d (199 KB)
ROC curves for the AUCs shown in Figure 3d, where the RNAcompete-trained (in vitro) RBP models were applied to in vivo (CLIP, RIP) sequences. Importantly, several DeepBind models have higher recall at low false positive rates.
- Supplementary Figure 6: Detailed explanation of how ChIP-seq peaks were divided into training and testing data for each experiment. (66 KB)
The ChIP-seq performance from Figure 3e are reproduced at left with extra annotations for clarity. At right is the breakdown of ChIPseq peaks used to train a model on each ChIP experiment. We train each method on peaks labeled A (“top 500 odd”), then test each method on peaks labeled B (“top 500 even”). DeepBind* is a special case where we show that including the lower -ranked peaks labeled C (“all remaining peaks”) in the training set can significantly improve the accuracy when scoring the top-ranked peaks labeled B.
- Supplementary Figure 7: Evaluation of FoXA2 models learned from ChIP-seq data on EMSA-measured affinities (68 KB)
FoxA2 ChIP model predictions validated by EMSA-measured affinities of FoxA2 binding to 64 probe sequences32. The column marked “DeepBind†” is an extra model that we trained on the same ENCODE ChIP data as “DeepBind”, but where we used motif_len=16 instead of the usual motif_len=24. The shorter motif length was tried due to the post-hoc observation that our FoxA2 model learns patterns of length 10, and we heuristically found that motif_len of ~1.5x the true motif length often works well. The fact that DeepBind† performed best suggests that there is still room for refinement in the DeepBind training procedure we use.
- Supplementary Figure 8: ROC curves for the AUCs shown in Figure 3f (649 KB)
ROC curves for the AUCs shown in Figure 3f, where the HT-SELEX-trained (in vitro) TF models were applied to in vivo (ChIP) sequences. For the semi-automatic method of Jolma et al. we show the curve for whichever PWM performed best on the test data; summing the scores of their choices of PFMs resulted in worse performance overall, so it is not shown.
- Supplementary Figure 9: Schematic diagram of the DeepFind model (81 KB)
Schematic diagram of the DeepFind model, using 2n TF scores (n wild type, n mutant) as features to a deep neural network.
- Supplementary Figure 10: DeepFind score distributions for the observed and simulated SNVs. (72 KB)
DeepFind score distributions for the observed and simulated SNVs.
- Supplementary Table 1 (39 KB)
Performance of in vitro trained models on DREAM5 in vitro and in vivo test data
- Supplementary Table 2 (97 KB)
In vitro performance metrics for models trained on RNAcompete RBP data
- Supplementary Table 3 (18 KB)
In vivo performance metrics for models trained on RNAcompete RBP data
- Supplementary Table 4 (35 KB)
The list of all ENCODE ChIP-seq data sets analyzed
- Supplementary Table 5 (50 KB)
Performance of models trained on ENCODE ChIP-seq data on held out data
- Supplementary Table 6 (67 KB)
The list of all HT-SELEX data sets analyzed
- Supplementary Table 7 (34 KB)
Performance of models trained on HT-SELEX data on held out data
- Supplementary Table 8 (26 KB)
Performance of models trained on HT-SELEX data on ENCODE ChIP-seq data
- Supplementary Table 9 (13 KB)
P-values for differential binding scores of RBPs regulating alternatively-spliced exons
- Supplementary Table 10 (36 KB)
All calibration parameters for DeepBind models and the SGD learning algorithm. Each parameter is either fixed for all calibration trials, or is independently sampled for each trial from the given search space.
- Supplementary Software (134 MB)
This code download is distributed as part of the Nature Biotechnology supplementary software release for DeepBind. Users of DeepBind are encouraged to instead use the latest source code and binaries for scoring sequences at http://tools.genes.toronto.edu/deepbind/
Your access to and use of the downloadable code (the “Code”) contained in this Supplementary Software is subject to a non-exclusive, revocable, non-transferable, and limited right to use the Code for the exclusive purpose of undertaking academic, governmental, or not-for-profit research. Use of the Code or any part thereof for commercial or clinical purposes is strictly prohibited in the absence of a Commercial License Agreement from Deep Genomics. (firstname.lastname@example.org)