Benchmarking of the Oxford Nanopore MinION sequencing for quantitative and qualitative assessment of cDNA populations

Abstract

To assess the performance of the Oxford Nanopore Technologies MinION sequencing platform, cDNAs from the External RNA Controls Consortium (ERCC) RNA Spike-In mix were sequenced. This mix mimics mammalian mRNA species and consists of 92 polyadenylated transcripts with known concentration. cDNA libraries were generated using a template switching protocol to facilitate the direct comparison between different sequencing platforms. The MinION performance was assessed for its ability to sequence the cDNAs directly with good accuracy in terms of abundance and full length. The abundance of the ERCC cDNA molecules sequenced by MinION agreed with their expected concentration. No length or GC content bias was observed. The majority of cDNAs were sequenced as full length. Additionally, a complex cDNA population derived from a human HEK-293 cell line was sequenced on an Illumina HiSeq 2500, PacBio RS II and ONT MinION platforms. We observed that there was a good agreement in the measured cDNA abundance between PacBio RS II and ONT MinION (r_pearson = 0.82, isoforms with length more than 700bp) and between Illumina HiSeq 2500 and ONT MinION (r_pearson = 0.75). This indicates that the ONT MinION can sequence quantitatively both long and short full length cDNA molecules.

Introduction

Transcriptome sequencing using short read technologies (Illumina HiSeq 2500¹, Ion Proton²) provides valuable information on transcript abundance, rare transcripts and variable transcription start or end sites. Nevertheless, inferring alternatively spliced isoforms of genes from short read data through statistical assignment of the most probable combination of exons is still computationally challenging and not very accurate³. Uneven read coverage⁴, complex splicing⁵ and potential sequencing bias⁶ complicates even more the task. The PacBio RS II⁷ zero–mode waveguide (ZMW) long-read sequencing technology has proven capable of characterizing the transcriptome in its native, full-length form unravelling novel gene isoforms not previously observed in RNA-seq experiments⁸. Recently, another long-read DNA sequencing technology based on nanopore sequencing (MinION) was introduced from Oxford Nanopore Technologies Ltd (ONT)⁹. Similar to PacBio RS II, the ONT MinION has been shown that can resolve the exon structure of mRNA molecules transcribed from genes with a large variety of isoforms¹⁰.

Here, we assessed the ONT MinION performance for its ability to sequence the cDNAs with good accuracy in terms of cDNA abundance, sequence identity and in full length. To evaluate the performance of the ONT MinION platform, the cDNA of a commercially available defined set of 92 polyadenylated transcripts, that mimic mRNA species (ERCC RNA Spike-In mix), was sequenced with the Illumina HiSeq 2500 or MiSeq instruments, the PacBio RS II platform and the ONT MinION. Additionally, a complex cDNA population from a HEK-293 cell line was sequenced in the same sequencing platforms and the agreement in the cDNA abundance of the different transcript isoforms across the three platforms was assessed. The results indicate that the ONT MinION platform can sequence quantitatively cDNA molecules similar with the Illumina and PacBio RS II platforms paving the way for full length sequencing of cDNA molecules with nanopores.

Results

Sequencing the ERCC cDNA molecules on the ONT MinION platform

We sequenced between 9,525 and 197,014 ERCC cDNA molecules in four different ONT MinION flow cells as presented in Table 1. We used two different versions of the ONT MinION flowcells an old version r7 and a newer version r7.3. The different types of reads produced from the ONT MinION platform has been described in detail in other studies¹¹. Briefly, the ONT MinION platform can sequence both strands of the same DNA molecule at once due to the presence of a hairpin adaptor. The strand that is sequenced first is called the “template read”. The strand that is sequenced second is called the “complement read”. If both strands of the same molecule are sequenced, a consensus sequence (“2D read”) is produced from the “template read” and the “complement read”. The template reads group, the complement reads group and the 2D reads group are referred to as “read types” in this manuscript. After the sequencing, the ONT analysis pipeline separates the sequenced reads in two groups the “pass” and the “failed” group. The reads in the “pass” and “fail” groups are referred to as “high” and “low” quality read categories, respectively, in this manuscript. The “pass” group contains high quality 2D reads (average base quality score of the “2D read” >=9) along with their “template” and “complement” read sequences. Based on the data from this manuscript the median identity of the high quality template, the high quality complement and the high quality 2D reads is 67.9–70.7%, 68.7–69.7%, 83.5–87.5% respectively (Supplementary Fig. S1). The “failed” group contains the rest of the reads. The median identity of the low quality template, the low quality complement and the low quality 2D reads, that we observed, is 64.8–68.1%, 64.3–67.7%, 70.9–77% respectively (Supplementary Fig. S1). In Table 1 we see that all of the cDNA molecules were sequenced as template strand. A fraction of the cDNA molecules (7–49% of the total reads) were sequenced as both template and complement strand and eventually some of them (4–39% of the total reads) were assigned as 2D reads. The percentage of complement and 2D reads, produced from each one of the experiments, is variable. A variability in the performance of the ONT MinION flowcell runs has also been noticed in another study¹¹ and in our case both the ONT MinION flowcell run performance and the different complexity of the cDNA material used (low complexity for the ERCC and high complexity for the HEK-293) can contribute to this variability (Supplementary text section I, Supplementary Fig. S2).

Table 1 Number of reads sequenced in each MinION experiment.

The MinION community is currently mainly focused on the 2D read group from the high quality category. As has already been observed in other studies¹¹ the accumulation of high quality 2D reads gradually drops over the sequencing time and it is high only in the beginning of the sequencing run (Supplementary Fig. S3A,B). For accurate gene expression analysis, a large number of sequenced reads is preferable and to overcome the high quality 2D read limitation we examined whether some other read types (template, complement, 2D reads) from the low quality category can be used for quantitative expression analysis. We assessed which read type is closer to the expected number of ERCC cDNA molecules from both high and low quality categories and how many sequenced reads can be aligned to the reference ERCC database in each category and read type. We eventually selected the read type that had the highest number of reads and that was closer to the expected distribution of ERCC cDNA molecules.

The abundance of the different ERCC cDNA molecules sequenced from either the Illumina HiSeq 2500/MiSeq platforms, the PacBio RS II platform or the ONT MinION is comparable

To test whether the ONT MinION performs equally well as the Illumina platforms in estimating the ERCC cDNA abundance, the same full length ERCC cDNA population used in the ONT MinION experiments, was sequenced on an Illumina HiSeq 2500 or MiSeq instruments. To estimate the cDNA abundance from the Illumina sequencing, we examined both the FPKM (or the TPM) method and direct molecular counting of the ERCC cDNA molecules through the incorporation of 5′ and 3′ molecular tags during RT¹² (Supplementary text section II, Supplementary Fig. S4). The 5′ end molecular counting was selected as more accurate (Supplementary text section II, Supplementary Fig. S4C,D) and the number of cDNA molecules calculated from both the ONT MinION and Illumina HiSeq 2500/MiSeq was compared.

Additionally, the same ERCC cDNA population was sequenced on a PacBio RS II platform. We compared the agreement of the PacBio RS II estimated cDNA abundance with the one from the ONT MinION platform. The PacBio ZMW loading procedure that was used to sequence the ERCC cDNA transcripts, enriches for molecules longer than 700 bp. For this reason, only ERCC transcripts with length more than 700bp were used to compare these two platforms.

Initially we compared the agreement between the number of sequenced cDNA molecules from the ONT MinION platform with the number of RNA molecules as provided from the manufacturer (Ambion), for each one of the 92 ERCC species. In Fig. 1A we see that the ONT MinION sequenced reads are as good as the Illumina 5′ end molecular counts (Fig. 1B) or the PacBio RS II platform (Supplementary Fig. S5B) in estimating the cDNA abundance (r_p = 0.98 for the ONT MinION, r_p = 0.97 for Illumina, r_{p for ERCC longer than 700bp} = 0.94 for the PacBio RS II). We then assessed which ONT MinION read type (template, complement, 2D for both low and high quality categories) is in better agreement with the Illumina molecular counts. The Illumina molecular counts represent our best estimation of the true cDNA abundance for the ERCC transcripts that can be detected. We saw that independently from the read type of the ONT MinION reads used, the agreement for the low quality category is better than the one for the high quality category. The correlation with the Illumina molecular counts for the low quality category is r_{p raw values} = 0.98–0.99, r_{p log10 values} = 0.98 (Supplementary Figs S6A–C and S7A–C respectively). The correlation for the high quality category is r_{p raw values} = 0.93–0.94, r_{p log10 values} = 0.96–0.97 (Supplementary Figs S6D–F and S7D–F respectively). When we pooled together the reads from the low and high quality categories, the agreement for each one of the read types is again high (r_{p raw values}  = 0.96–0.99, r_{p log10 values} = 0.97–0.98, Supplementary Figs S6G–I and S7G–I). We then assessed which read type, from the pooled quality category, is in better agreement with the Illumina cDNA molecular counts. In both the flow cell v7.3 (Supplementary Fig. S6G–I) and flow cell v7 (Supplementary Fig. S8A–C) we see that the template reads gave a higher and more consistent correlation with the expected Illumina cDNA molecular counts than the complement or 2D reads. The correlation values are r_{p raw values v7.3} = 0.99 and r_{p raw values v7}  = 0.99 for the template reads, r_{p raw values v7.3} = 0.96 and r_{p raw values v7}  = 0.93 for the complement reads, r_{p raw values v7.3} = 0.99 and r_{p raw values v7}  = 0.72 for the 2D reads. As different number of reads are present in each read type group and read quality categories, in order to confirm the significance of all the presented differences, we subsampled from the low quality template reads equal number as either the high quality template reads, the low quality complement reads or the low quality 2D reads from each ERCC experiment (Supplementary Figs S9 and S10). The subsampling confirmed that the low quality template read group is in closer agreement with the expected cDNA abundance than either the high quality template read group, the low quality complement read group or the low quality 2D read group (Supplementary Table S1, Supplementary methods section VIII).

Figure 1

Estimation of the ERCC cDNA abundance with the ONT MinION platform.

We compared the ERCC cDNA abundance estimated from the ONT MinION (A) and the Illumina HiSeq 2500 or MiSeq platforms (B) against the expected number of RNA molecules as provided from the manufacturer (Ambion). The template reads from the ERCC MinION experiments number 1, 2, 3, 4 were pooled together and used for (A). Similarly, the corresponding Illumina data were pooled together for (B). The Illumina molecular counts data were derived using 5′ molecular tags at the RT step as described in material and methods. The total number of molecules presented on the x-axis corresponds to 3.5 pgs of ERCC RNA. In both axes the log10 transformation of the original count number is used.

In parallel, we used the sequenced ERCC template reads to examine the performance of different aligners (Margin-Align, BWA-mem, LAST, BLASr, BLAST, Smith-Waterman) and alignment parameters (Supplementary Figs S11,S12 and S13 and Supplementary Table S2). We tried to find an aligner that gives the maximum agreement with the expected cDNA abundance, an adequate number of aligned reads, an adequate alignment length and a high alignment accuracy. The LAST aligner (parameters of the lastal module: -a 1 –b 1 –q 1 –s 2 –T 0 –Q 0 –e 45) was eventually selected (Supplementary methods section VII) and was used throughout the manuscript for alignment of the ONT MinION reads.

The ONT MinION sequenced reads usually correspond to the full length of the ERCC cDNA molecules but some sequencing runs can produce partially sequenced molecules

The cDNA molecules produced during reverse transcription do not always correspond to the full length of the reference transcript either because the RNA molecules are degraded or because the reverse transcriptase fails to process the RNA molecules in their entirety. The template switching approach, that is used in this manuscript, can minimize the production of cDNA molecules from partially processed RNA molecules because the reverse transcriptase is more efficient in adding the necessary cytosines for the template switching step only when it reaches the end of the RNA molecule¹³. Our aim was to assess whether the ONT MinION can sequence the full length of the cDNA molecules by examining how many of the sequenced molecules corresponded to the full length of the reference ERCC transcript database. For this we initially examined how many of the cDNA molecules, produced after reverse transcription, corresponded to the full length of the reference ERCC transcript database. We used only the Illumina fragments where the 5′ adaptor sequence was clearly identified. These fragments correspond to the 5′ ends of the cDNA molecules. We then analyzed the position on the reference ERCC database where the 5′ end fragments align (Supplementary Fig. S14). We saw that 90.8% of them were full length (aligned position of the 5′ Illumina fragments maximum 5 bases from the 5′ position of the reference transcript). This indicated to us that the majority of the cDNA molecules are synthesized as full length which permitted us to examine the effect of the ONT MinION sequencing on the full length processing of the ERCC cDNA molecules.

Initially we separated the ONT MinION reads in two groups. The first group includes reads where the sequencing started from the 5′ end of the transcripts in the ERCC reference database (reads aligning as sense on the database, we call them “sense reads”). The second group includes reads where the sequencing started from the 3′ end (reads aligning as anti-sense on the database, we call them “antisense reads”). We examined what percentage of the sense or antisense reads reached until the 3′ or the 5′ end of the ERCC transcripts respectively. In Supplementary Fig. S15 we see that in all the ERCC runs except run number 4 the reads were sequenced as full length. In the case of run number 4, 80% of the “sense” template reads (that do not belong in either the low or the high quality 2D read type) and 77% of the “antisense” template reads are partially processed (the sequencing stops more than 5 bp away from the TES and TSS respectively). Since run number 4 gave the maximum number of reads we used it to assess whether any potential systematic bias during the sequencing run affected the complete processing of reads. For this we examined whether the reads were sequenced as full length over time.

The ERCC library has the advantage that it consists of two abundant groups of reads that can be separated based on their length, a group with short transcripts (average length ~600bp) and a group with long transcripts (average length ~1200 bp). This permits us to assess whether the ONT MinION platform preferentially sequences short or long DNA molecules as is the case for the PacBio RS II platform⁸. Additionally, it permits to assess whether the performance of the ONT MinION platform is constant throughout the sequencing run. In Supplementary Fig. S16 we see that in the beginning of every restart of the sequencing run (“re-mux” time11; time point zero and time points at the purple vertical lines) the median sequenced length (~650 bp) was approximately the same (Supplementary Fig. S16B) and the group with the short transcripts was sequenced equally well as the group with the long transcripts. Nevertheless every time there was an increase in the voltage, which drives the DNA translocation across the pore (“bias voltage”¹¹), by 5mV (time point at green vertical lines which corresponded to every 4 hours of a continuous sequencing run¹¹) we observed a decrease of the median sequenced length by 50–250 bp (Supplementary Fig. S16B). The drop of the median sequenced length followed the drop of the median alignment length on the reference database (from 95% down to 55–85% depending on the time point). Because the sequenced length drop affected both DNA molecules in the short and long transcript group, the overall decrease did not affect the relative quantification of the transcripts. As can be seen in Supplementary Fig. S16C the group with the short transcripts was always more abundant than the group with the long transcripts, independently of the time point during the flow cell run when the reads were produced.

This profile of the ERCC experiment 4 cannot be attributed to the presence of fragmented molecules before the sequencing. To assess the quality of the ERCC cDNA libraries sequenced we used the sequenced length distribution of the 2D reads for the r7 version and the sequenced length distribution of the high quality 2D reads for the r7.3 version of the ONT MinION platform (Supplementary Fig. S17C,F respectively). The presence of distinct peaks for the short and long ERCC transcript groups in the length distribution plots permits us to assess the quality of the ERCC cDNA libraries sequenced. All the different ERCC experiments show the characteristic two peaks which indicate that the ONT MinION library preparation procedure did not created fragmented molecules (Supplementary Fig. S17, Supplementary text section III).

The ONT MinION platform can sequence equally well short and long ERCC cDNA molecules and shows no specific preference for high or low GC content ERCC cDNAs

We examined whether the ONT MinION platform shows a preference for sequencing better either high or low GC content cDNA molecules and either short or long cDNA molecules. Initially we examined whether any deviation (log fold difference) between the observed ERCC cDNA abundance from the ONT MinION platform and the expected ERCC RNA/cDNA abundance, from either the Ambion molecular counts (RNA concentration) or the Illumina 5′ end molecular counts, is linearly correlated (positively or negatively) with the GC content. The comparison between the ONT MinION cDNA counts and the Ambion RNA counts showed no statistically significant trend (Fig. 2A). On the contrary when we compared the ONT MinION cDNA counts and the Illumina cDNA molecular counts we saw a positive trend (r² = 0.293) (Fig. 2B). This indicates that 29% of the variation in the deviation between the ONT MinION cDNA counts and the Illumina cDNA counts can be attributed to the difference in the GC content of the ERCC cDNA molecules. This is in agreement with previous observations that showed that, for the GC content range of 30–52%, the Illumina platform sequences more frequently the higher GC content molecules than the lower GC content molecules^6,13. On the contrary, the ONT MinION platform does not show this high GC content bias for the GC content range of 30–52%. The lower GC content ERCC molecules are sequenced equally well in both platforms. A similar comparison between the PacBio RS II and the ONT MinION platform showed that, for the GC content range of 30–52%, the high GC content molecules are also preferentially sequenced by the PacBio RS II platform (Supplementary Fig. S18D, Supplementary text section IV). This PacBio RS II platform GC bias is not as strong as the one of the Illumina platform (Supplementary Fig. S18E). A weak GC bias for the PacBio RS II platform has been reported in some studies^14,15. Contrary to other studies⁴ our experiments did not analyse sequences with extreme GC content, very high (>52%) or very low (<30%) GC content.

Figure 2

Effect of the GC content and ERCC length on the estimation of the ERCC cDNA abundance with the ONT MinION platform.

The figures present deviations of the ERCC expression level estimates with the ONT MinION platform from the Ambion RNA molecular counts (A,C) or from the Illumina HiSeq 2500/MiSeq estimated cDNA abundance (B,D) as a function of the GC content (A,B) and the ERCC length (C,D). We plot the log2 ratio of observed (ONT MinION) to expected (Ambion, Illumina) read counts for the ERCC spike-ins (y-axis, log) for each of the samples relative to their length or GC content (x-axis). Due to the variable sequencing depth from each ERCC MinION experiment, each point is the average value from different MinION flow cell runs if at least 5 reads have been detected for this point in the corresponding MinION runs. The points are colored differently based on the number of flow cell runs in which they were detected (red, green, cyan, purple correspond to values derived from one, two, three or four flow cell runs respectively). The standard deviation is also presented for points with values from two or more MinION runs.

We additionally examined whether the ERCC cDNA abundance deviation is linearly correlated (positively or negatively) with the ERCC transcript length. The comparison between the ONT MinION cDNA counts and Ambion RNA counts showed no statistically significant trend (Fig. 2C). Comparison of the ERCC cDNA abundance derived from the ONT MinION platform with the abundance derived from the Illumina 5′ end fragments showed a statistically significant negative trend (r² = 0.1, p = 0.04) (Fig. 2D). The same is observed when we compared the Illumina 5′ end molecular counts with the Ambion RNA counts (r² = 0.22, p = 0) (Supplementary Fig. S19D). This indicates that in our hands, the combination of Nextera library preparation followed with Illumina sequencing results in a length bias. This is probably caused from the fact that the 5′ end fragments from the short ERCC transcripts (less than 800 bp) can be smaller than the ones from the long ERCC transcripts (more than 800 bp) and consequently they are more efficiently sequenced on the Illumina platform (Supplementary Fig. S20).

Including both the GC content and the length of the ERCC transcripts in a component regression model explained 53% of the variation in the deviation (log fold difference) between the observed ERCC cDNA abundance from the ONT MinION platform and the expected ERCC cDNA abundance from the Illumina 5′ end fragments (r² = 0.53, BIC_{score GC+length}  = 74 versus BIC_{score GC} = 82, BIC_{score length}  = 96).

The GC content of the transcripts also affects the accumulation of high quality reads. In Supplementary Fig. S21A we see that the low GC content ERCC transcripts are more probable to give high quality MinION reads rather than the high GC content ERCC transcripts. This is in accordance with other observations in 16S rRNA amplicons that showed a lower number of mismatches, against the known reference sequence, with reduced GC content¹⁶. The length of the ERCC transcripts did not have any effect on the accumulation of high quality reads (Supplementary Fig. S21B).

We also used the ERCC experiment number 4 to calculate the limit of detection for transcripts at a sequencing depth of 113402 total aligned ONT MinION reads (Supplementary Fig. S22, Supplementary text section V). These total number of reads can detect, with around 2 ONT MinION reads or more, transcripts with approximate abundance of at least 1 in 51164 molecules.

The detection of isoforms in a complex cDNA population by the ONT MinION platform complements the Illumina HiSeq 2500 and PacBio RS II platforms

To assess the qualitative and quantitative representation of a complex cDNA population we used cDNA produced from a HEK-293 cell line. We used material from 200 HEK-293 cells in order to reduce the number of different isoforms that can be detected for the same gene, which permits us to detect more genes¹⁷. We sequenced the same cDNA on an Illumina HiSeq 2500, PacBio RS II and ONT MinION platforms. Initially, we compared the agreement between the long-read sequencing technology platforms PacBio RS II and ONT MinION. Secondly, we compared the agreement of each one of them with the short-read sequencing technology platform the Illumina HiSeq 2500. The comparison assessed both the identity and the abundance of the detected isoforms or genes from the ones available in the “NCBI Homo sapiens Annotation Release version 104” transcript database. The ONT MinION run produced 16540 sequenced reads (Table 1). The low number of sequenced reads was due to the limited amount of cDNA available to load on the ONT MinION flow cell at the end of the ONT MinION library preparation protocol (~30 ngs). This limited amount of the ONT MinION ready cDNA library is a consequence of both the available amount of HEK-293 cDNA at the start of the ONT MinION library preparation and of the amount of material lost at each step of the ONT MinION library preparation (Supplementary Table S3). The HEK-293 cDNA ONT MinION library preparation started with ~500 ngs of cDNA because the total amount of cDNA synthesized from the 200 HEK-293 cells was limited (~750 ng).

To assess whether in the case of the HEK-293 cDNA library run on the ONT MinION platform, a flow cell specific behavior affected the full length sequencing of at least the template strand of the cDNA molecules, we introduced into the HEK-293 total RNA, three selected RNA transcripts (“Spike-1”, “Spike-4”, “Spike-7”) at different concentrations. The full length sequencing of the cDNA molecules, from the ONT MinION platform, for these RNA Spikes was then examined. Due to the limited sequencing depth we were able to detect only “Spike-1”. We saw that the template strand of the ONT MinION reads assigned to “Spike-1” was processed overall as full length (Supplementary Fig. S23A,B) and the sequenced length did not change throughout the flow cell run (Supplementary Fig. S23C).

Initially, we compared the length size distribution of the ONT MinION template reads with the length size distribution of the raw cDNA electrophoresis signal using the Caliper Labchip GX instrument. In Supplementary Fig. S24A we see that both the length size distribution of the ONT MinION template reads and the raw cDNA electrophoresis signal are in good agreement, indicating that the ONT MinION processed all the cDNA molecules of the complex mRNA population without any specific preference for either the short or the long ones.

After alignment we found 634 isoforms with at least 2 ONT MinION reads assigned to them. In the case of the PacBio RS II platform we detected 6871 isoforms with at least 2 sequenced reads whereas in the case of the Illumina HiSeq 2500 platform we detected 12969 isoforms with an FPKM value of more than 1.

As was already presented in the case of the ERCC cDNA transcripts, the PacBio RS II ZMW loading procedure that was used to sequence the HEK-293 cDNA, enriches for molecules longer than 700 bp. When we compared the overall length size distribution of the PacBio RS II reads, either the longest subread or the high quality Circular Consensus Sequencing reads (CCS reads), with the length size distribution of the raw cDNA electrophoresis signal, there was underrepresentation of molecules lower than 700 bp (Supplementary Fig. S24A).

The long HEK-293 expressed isoforms detected with the ONT MinION platform (>700 bp in length) are more frequently found as highly expressed among the 10641 isoforms detected with the PacBio RS II platform (Supplementary Fig. S25A). A similar picture emerges when we compare all the HEK-293 expressed isoforms detected with the ONT MinION platform against the top expressed 10641 isoforms from the Illumina HiSeq 2500. Additionally, the top expressed 400 isoforms of the ONT MinION platform are more frequently found among the highly expressed Illumina HiSeq 2500 isoforms rather than among the highly expressed PacBio RS II isoforms (Supplementary Fig. S25B,D). As the Illumina HiSeq 2500 contains a mixture of short and long isoforms and the short isoforms have usually higher expression than the long ones, more common short isoforms are detected (the distribution of the expression values from the Illumina HiSeq 2500 dataset for isoforms less than 700 bp in length is higher than the one for isoforms more than 700 bp in length).

The example of the RPL41 gene shows the complementary information that the ONT MinION and PacBio RS II platforms can provide (Fig. 3). The ONT MinION provides information for the diversity of both short and long cDNA species whereas the PacBio RS II loading procedure followed in this manuscript can provide information only for the diversity of long cDNA species.

Figure 3

Detection of different cDNA species for the RPL41 gene from either the Illumina HiSeq 2500 platform, the ONT MinION platform or the PacBio RS II platform.

ONT MinION reads that were sequenced as full length, as defined by the presence of both the 5′ and 3′ RT adaptors, are presented. For the PacBio RS II example the corresponding “Circular Consensus Sequencing” reads are presented. These reads correspond to fully sequenced molecules. mRNA molecules from GenBank for the RPL41 gene are also shown. For the Illumina data a pileup of the sequenced paired-end fragments is presented.

The expression level estimation of transcripts in a complex mRNA population from the ONT MinION platform is comparable to the Illumina HiSeq 2500 and PacBio RS II platforms

We then assessed the expression level concordance of the isoforms detected from the ONT MinION platform with the expression levels estimated from the Illumina HiSeq 2500 and PacBio RS II platforms. We used the PacBio RS II reads to estimate the abundance of isoforms more than 700 bp in length and the Illumina HiSeq 2500 reads to estimate the abundance of isoforms irrespective of their length. We tested multiple transcript expression estimation methods (TopHat/Cufflinks¹⁸, TopHat/StringTie¹⁹, Kallisto²⁰, Sailfish²¹) and we selected the ones that gave the higher concordance in the isoform abundance or gene abundance comparisons across the three different platforms (Supplementary Figs S26 and S27, Supplementary text section VI). The Sailfish software was used for the isoform abundance comparisons and the TopHat/Cufflinks pipeline was used for the gene abundance comparisons.

Initially we compared the concordance of the cDNA abundance for the isoforms with length more than 700 bp between the PacBio RS II and Illumina HiSeq 2500 platforms. In Fig. 4A we see a good agreement (r_p = 0.86, r_s = 0.75) between the two platforms. We then compared the ONT MinION with the PacBio RS II and Illumina HiSeq 2500 platforms. In Fig. 4B,C we see that the estimation of the isoform expresson levels from the ONT MinION is equally good as the PacBio RS II (r_p = 0.82, r_s = 0.57) and the Illumina HiSeq 2500 platforms (r_p = 0.75, r_s = 0.62) platforms. We obtained similar results when we compared the expresson level concordance at the gene level (Supplementary Fig. S28).

Figure 4

Estimation of the HEK-293 cDNA isoform abundance with three sequencing platforms.

The comparison between the cDNA isoform abundance estimated from the Illumina HiSeq 2500 platform and from the PacBio RS II platform is presented in (A). The expression level of the HEK-293 isoforms estimated with the ONT MinION platform is compared with the one calculated from either the PacBio RS II (B) or the Illumina HiSeq 2500 platform (C). For the Illumina HiSeq 2500 platform the expression level, presented as TPM, was estimated with the Sailfish²¹ software. For the PacBio RS II or the ONT MinION platform the counts of sequenced molecules per isoform are presented.

The TSS and TES of the genes identified with the ONT MinION platform closely agree with the ones identified with the PacBio RS II platform

We compared the position of the transcriptions start site (TSS) and the position of the transcription end site (TES) for the genes detected in both the ONT MinION and PacBio RS II platforms. We selected only genes where we were able to assign at least 3 ONT MinION reads for the TSS comparison or at least 2 MinION reads for the TES comparison. Because the type of isoforms for the same gene detected from the two platforms can be different (Fig. 3), in each gene we assigned as representative of the TSS position agreement the pair of the PacBio RS II and ONT MinION reads with the closest TSS distance between them. Similar approach we used for the TES comparison. In Fig. 5A we see that the TSS of the ONT MinION reads closely follows the TSS of the PacBio RS II reads. Only in seven out of seventy-four genes is the distance difference between the TSS more than 5 base pairs. Similarly, in Fig. 5B we see again that the TES of the ONT MinION reads follows the TES of the PacBio RS II reads. In this case there is a greater variation in the distance difference between the PacBio RS II and ONT MinION reads. In contrast to the PacBio RS II signal the nanopore signal for homopolymer regions (poly-A/poly-T) cannot be adequately resolved in its constituent number of bases resulting in bigger differences.

Figure 5

Agreement between the position of the TSS (A) and TES (B) for the genes detected by both the PacBio RS II and the ONT MinION platform. For each gene we assigned as representative of the TSS position agreement, the pair of the PacBio RS II and ONT MinION reads with the closest TSS distance between them. The nucleotide distance difference between the position of the TSS in the two platforms is presented on the top row. In the bottom row the distance from the annotated TSS of the gene isoform, where the pair was assigned to, is presented for each one of the two platforms individually. Similarly, for the TES position agreement. The position of the points on the top row correspond to the genes that are present on the same position on the bottom row.

Discussion

The ONT MinION platform is a third generation/single-molecule sequencing platform that can be used to sequence full length cDNA molecules. Although the reads currently have a higher error rate than other widely used long-read sequencing platforms, in our studies we see that the cDNA molecules are sequenced as full length in the template strand of the sequenced reads. Additionally, the ONT MinION platform does not show any preference in sequencing short or long cDNA molecules from either the simple cDNA population of the ERCC mix or the complex cDNA population of the HEK-293 cells. No bias is also observed for high or low GC content transcripts.

We show that if we use the abundant low quality sequenced reads from the ONT MinION platform, as opposed to only the high quality sequenced reads, we get a consistent result comparable with the other long-read and short-read sequencing platforms. A good agreement between the abundance of targeted cDNA amplicons sequenced with the ONT MinION and the Illumina MiSeq platforms, has been observed in another study¹⁰. Given that the platform can be used for accurate quantification of isoform expression abundance it is very important to be able to use as many sequenced reads as possible with the caveat of being of a lower quality. A combination of the low quality template reads with the high quality 2D reads seems the ideal combination for quantitative accuracy, robustness, reproducibility and maximum usage of the MinION sequenced reads.

As the single-molecule sequencing platforms (PacBio RS II, ONT MinION) do not rely on PCR amplification for sequencing, they usually show lower biases than the short read sequencing platforms⁴. Indeed, in our data we see that the Illumina HiSeq 2500 platform shows an overrepresentation of tagmentation fragments with high GC content (within the 30–52% spectrum) or with short length. This is probably due to the PCR amplification biases^6,22 either at the PCR step after the tagmentation or at the cluster generation step on the Illumina flow cell²³.

By introducing commercially synthesized RNA spikes of a known sequence we were able to monitor the performance of the sequencing run, which seems necessary as the ONT MinION platform is still under development. In our case we were able to identify factors that potentially affected the full length sequencing of the cDNA molecules. For example, we saw that the increase in the “bias voltage” considerably affected the performance of the full length cDNA sequencing.

The ONT MinION permits the simultaneous quantification of short and long transcripts. On the contrary, the design of the PacBio RS II sequencing flow cell (SMRT Cells) has the disadvantage of biasing towards sequencing short cDNA molecules with the “diffusion loading method” or towards sequencing cDNA molecules longer than 700 bp with the “MagBead loading method”. This can considerably affect the absolute quantification of genes and isoforms as the abundance of short transcripts relative to the long ones cannot be accurately determined.

The ONT MinION platform has been shown to successfully resolve the different isoforms from genes with a complex alternative splicing pattern¹⁰. Further improvements in the MinION data quality and yield are expected and this will permit the generation of a sufficient number of reads necessary to identify low abundant transcripts but currently the technology is suitable to identify and quantify high abundant cDNA isoforms. cDNA normalization methods based on duplex-specific nuclease (DSN)²⁴, aimed at enhancing the detection of rare transcripts in eukaryotic cDNA libraries by decreasing the prevalence of highly abundant transcripts, can help increase the resolution in the median to low abundant transcript range at the current ONT MinION sequencing yield.

Overall nanopore sequencing, as represented by the ONT MinION platform, has proven itself a viable alternative, for cDNA quantification, to the Illumina short read sequencing platform and to the PacBio RS II long read sequencing platform (Supplementary Table S4). The ONT MinION platform is useful for cDNA quantification studies provided that it can achieve consistency of throughput and that it can sequence at a lower cost than the other sequencing platforms. For cDNA quantification, the ONT MinION platform can be used instead of the Illumina platform as it provides comparable quantitative data. It is also superior to the Illumina platform as it provides sequencing information for the full length of the cDNA molecules (from transcription start site to transcription end site). The full length sequencing of the cDNA molecules permits the unambiguous assignment of isoforms although the high error rate of the ONT MinION platform might not permit a clear assignment of isoforms in the case of isoforms with micro-exons as argued by Bolisetty et al.¹⁰. Additionally, the ONT MinION is superior for quantification from Illumina as it lacks the GC content bias.

The ONT MinION platform can be used instead of the PacBio RS II platform as it provides more accurate quantitative data in the short transcript range (<700bp in length). The quantitative data for the long transcript range (>700bp in length) is as good as the PacBio RS II platform.

The disadvantage of the ONT MinION platform is that the basecalling accuracy is not close to the Illumina basecalling accuracy or the basecalling accuracy achieved from the Consensus Circular Reads (CCS reads) from the PacBio RS II platform. In this study we saw that the highest quality reads of the ONT MinION have an average error rate similar with the lowest quality PacBio RS II reads (Supplementary Fig. S1). This indicates that identifying confidently variants and RNA editing events in transcripts or accurately assembling de novo transcripts with the ONT MinION platform, will need a higher read coverage than with the other platforms to cancel out non-systematic basecalling errors.

Another disadvantage of the ONT MinION platform is that the total amount of cDNA used in the standard ONT MinION library preparation protocols presented in this manuscript (~0.5–3 ug) is more than the one used in standard PacBio RS II library preparation protocols (~0.5 ug) or the low input PacBio RS II library preparation version used here (~0.03 ug) and considerably more than the 0.0004 ug needed with the Illumina platform. This can make the platform prohibiting for samples with small amount of cDNA material (for example cDNA from a single cell ~4–6 ng). Nevertheless, low input versions of the ONT MinION library preparation protocols have been developed (~20–100 ng of starting DNA material). The platform is under continuous improvement and some of these problems are expected to be resolved in newer versions of the platform.

Methods

Materials

The following materials were used: “ERCC RNA Spike-In Mix 1” stock (#4456740, Ambion, Thermo), RNA storage solution (Ambion, Thermo), Triton-X 100 (molecular biology grade, Sigma), dNTP mix (Advantage UltraPure PCR Deoxynucleotide Mix 10 mM each dNTP, Clontech), RNAse inhibitor (Clontech), Superscript III enzyme and Superscript III first-strand buffer (Invitrogen, Thermo), ultrapure DTT (Invitrogen, Thermo), Betaine (Sigma), MgCl₂ (Sigma), oligo-dT30V primer (IDT), template switch primer (IDT), cDNA amplification primer (IDT) (the sequences are given in Supplementary Fig. S30), 2X KAPA2G Robust HotStart ReadyMix (kapabiosystems), AMPure XP beads (Beckman Coulter), NEBNext End Repair Module (New England Biolabs, UK), NEBNext dA-Tailing Module (New England Biolabs, UK).

ERCC cDNA production

An adaptation of the Smart-seq2 method²⁵ was used to produce full length cDNA molecules from the ERCC RNA transcripts. Two different ERCC RNA concentrations were used mimicking conditions where the poly-A RNA was derived from either a bulk of cells or from a few cells (~200 cells/single cell). In the first case, the “ERCC RNA Spike-In Mix 1” stock (32 ngs/ul) was diluted 10 times in RNA storage solution down to 3.2 ngs/ul. In the second case, the “ERCC RNA Spike-In Mix 1” stock was diluted 500 times in RNA storage solution down to 0.064 ngs/ul. The poly-A hybridization step, of the first strand cDNA synthesis primer, was performed in a 3 ul volume reaction (3.2 ngs or 0.064 ngs ERCC RNA Spike-In, 0.12% Triton-X 100. 4.2 μM of oligo-dT30V, 2.8 mM dNTP mix, 1X Superscript III first-strand buffer, 20 U RNAse inhibitor) with the following thermocycler parameters 72 °C for 3 min, 4 °C for 10 min, 25 °C for 1 min. The synthesis of the first and second strand of cDNA was performed by topping the 3 ul reaction up to a 7 ul volume reaction (final concentration: 1X Superscript III first-strand buffer, 2.5 mM DTT, 1.2 μM template switch primer, 27 U RNAse inhibitor, 70 U SuperScript III reverse transcriptase, 0.5 M betaine, 3.6 mM MgCl₂). The cDNA synthesis was performed with the following thermocycling parameters: 1 cycle of [50 °C for 90 min], 10 cycles of [55 °C for 2 min, 50 °C for 2 min], 5 cycles of [60 °C for 2 min, 55 °C for 2 min], 1 cycle of [70 °C for 15 min]. Afterwards the double stranded cDNA in the 7 ul reaction was amplified in a 70 ul PCR reaction containing 12 μM cDNA amplification primers and 1x KAPA2G Robust HotStart ReadyMix. Depending on the initial amount of RNA in the reaction we used 14 cycles of amplification for 3.2 ngs of RNA or 21 cycles of amplification for 0.064 ngs of RNA. In the case of 14 cycles of cDNA amplification the cDNA amplification protocol used was the following: 1 cycle of [95 °C for 1 min], 5 cycles of [95 °C for 20 seconds, 58 °C for 4 minutes, 68 °C for 6 minutes], 9 cycles of [95 °C for 20 seconds, 64 °C for 30 seconds, 68 °C for 6 minutes], 1 cycle of [72 °C for 10 minutes]. In case of 21 cycles of cDNA amplification the cDNA amplification protocol used was the following: 1 cycle of [95 °C for 1 min], 5 cycles of [95 °C for 20 seconds, 58 °C for 4 minutes, 68 °C for 6 minutes], 9 cycles of [95 °C for 20 seconds, 64 °C for 30 seconds, 68 °C for 6 minutes], 7 cycles of [95 °C for 30 seconds, 64 °C for 30 seconds, 68 °C for 7 minutes], 1 cycle of [72 °C for 10 minutes]. The amplified product was subsequently cleaned with 0.9X sample volume AMPure XP beads and eluted in H₂0 for a total of 2400 ngs of cDNA per 3.2 ngs of initial RNA material (or 5400 ngs of cDNA per 0.064 ngs of initial RNA material). For sequencing on the Illumina HiSeq 2500 or MiSeq platforms, the cDNA was tagmented using the Nextera XT DNA Library Preparation Kit (Illumina Inc) as described in supplementary methods.

ONT MinION sequencing of the ERCC cDNA population

The full length cDNA libraries were sequenced with version r7 and version r7.3 MinION flow cells. The ONT MinION Genomic DNA Sequencing Kit reagents (SQK-MAP0003 for the r7 flow cell, SQK-MAP0004 or SQK-MAP0005 for the r7.3 flow cells as indicated in Table 1 and the 2D cDNA sequencing protocol were used to prepare the libraries. The amount of starting material used is presented in Supplementary Table S3. In each case the cDNA was end-repaired in a 100 ul reaction containing: 0.45–2 ug cDNA (depending on the run) in a 80 ul solution, 10 ul of 10X End-repair buffer (from the NEBNext End Repair Module), 5ul End-repair enzyme mix (from the NEBNext End Repair Module) and 5 ul of nuclease-free water. In the case where we started with 3 ug of cDNA, two reactions with 1.5 ug of cDNA material were used (ERCC experiment 4, Supplementary Table S3). The end-repair reaction was then incubated at 25 °C in a thermocycler for 30 minutes. Afterwards we added 1X sample volume Ampure XP beads to the End-Repair reaction. The solution was mixed by pipetting and the DNA was allowed to bind to the beads by rotating for 5 minutes on a HulaMixer (Thermo). Then the beads were pelleted on a magnet, the supernatant was aspired off and the beads, while they stayed on the magnet, were washed twice with 200 μl of freshly prepared 70% ethanol. The tube was centrifuged to collect residual liquid at the bottom and the residual wash solution was aspirated. The DNA was eluted from the beads by resuspending the beads in 26 ul of 10 mM Tris-HCl pH 8.5 and incubating them for 5 minutes at room temperature. The isolated DNA was quantified on the Qubit fluorimeter. For the subsequent dA-tailing step all the eluted DNA was used in the following reaction: cDNA library solution from the previous step (>750 ng), 3 ul 10x dA-tailing buffer (from the NEBNext dA-Tailing Module), 2 ul dA-tailing enzyme (from the NEBNext dA-Tailing Module). Subsequently, depending on the protocol version the DNA was cleaned with the Ampure XP beads as described above (SQK-MAP0005 kit version) or used immediately in the ligation reaction (SQK-MAP0004, SQK-MAP0003 kit version). In the ligation reaction the DNA was mixed with the following reagents: 30 ul dA-tailed DNA, 10 ul Adapter Mix, 10 ul HP adapter, 50 ul Blunt/TA ligase Master Mix. The reaction was incubated at room temperature for 10 minutes. Depending on the protocol the cDNA library was either loaded on the flow cell as is (SQK-MAP0003 kit version) or enriched for sequences that bear the hairpin (SQK-MAP0004, SQK-MAP0005 kit version) as follows.10 ul of His-beads (Dynabeads His-Tag, Thermo) were washed two times with 1x Bead Binding Buffer (SQK-MAP0005 kit), then resuspended in 100 μl of the 2x Bead Binding Buffer (SQK-MAP0005 kit) and added to the adaptor ligated DNA. The bead bound DNA was then washed two times with 200 μl 1x Bead Binding Buffer (SQK-MAP0005 kit) and eluted with Elution Buffer (SQK-MAP0005 kit) with the amount presented in Supplementary Table S3. The parameters used to align the ONT MinION reads are presented in supplementary methods. Considerations on the amount of cDNA library to adaptor molecules used are also presented in supplementary methods.

HEK-293 cDNA production and ONT MinION sequencing

cDNA was produced from 200 HEK-293 cells expressing shRNAs targeting PBRM1 (the cells were kindly provided by Dr. Yasser Riazalhosseini). The HEK-293 cDNA production was based on the Single-Cell cDNA Library preparation protocol for mRNA Sequencing for the Fluidigm C1 machine as described in detail in supplementary methods. The HEK-293 cDNA library was sequenced with a version r7.3 MinION flow cell. The ONT MinION Genomic DNA Sequencing Kit reagents (SQK-MAP0005) and the 2D cDNA sequencing protocol were used to prepare the libraries. The library preparation method was performed similarly to the ERCC cDNA and is presented in the supplementary methods.

PacBio RS II library preparation and sequencing of the HEK-293 and ERCC cDNA

The amplification product from the HEK-293 cDNA preparation was prepared for sequencing using 28ng of input material. The protocol we used²⁶ is a modified version of the PacBio RS II library preparation (SMRTbell template prep Kit). First a DNA damage repair step and end repair step is performed followed by purification (AMPure XP beads). The hairpin is then ligated to the fragments. Here 500ng of carrier DNA (Clontech pBR322) is added to the library before exonuclease treatment and additional purification. Following this a sequencing primer with a poly-A tail is hybridized to the hairpin ligated to the fragments. Magnetic dT beads are then used to both separate the library from the carrier and also to load the fragments onto the SMRT cells. The parameters used to align the PacBio RS II reads are presented in supplementary methods.

The PacBio RS II library for the ERCC cDNA was prepared using the same protocol as the HEK-293 cDNA. For the ERCC cDNA PacBio RS II library we used material from the ERCC cDNA aliquot from the 14 cycles of cDNA amplification which is the same aliquot as the one used in the ERCC experiments number 2, 3, 4. We prepared two libraries starting from 10ng of cDNA each. Each library was then run on one SMRTcell.

Data access

All sequencing data sets have been submitted to the ENA database under the accession number PRJEB11818. A description of each file in the ENA archive and in the supplementary data folder is provided in the supplementary excel file.