Abstract

The concordance of RNA-sequencing (RNA-seq) with microarrays for genome-wide analysis of differential gene expression has not been rigorously assessed using a range of chemical treatment conditions. Here we use a comprehensive study design to generate Illumina RNA-seq and Affymetrix microarray data from the same liver samples of rats exposed in triplicate to varying degrees of perturbation by 27 chemicals representing multiple modes of action (MOAs). The cross-platform concordance in terms of differentially expressed genes (DEGs) or enriched pathways is linearly correlated with treatment effect size (R20.8). Furthermore, the concordance is also affected by transcript abundance and biological complexity of the MOA. RNA-seq outperforms microarray (93% versus 75%) in DEG verification as assessed by quantitative PCR, with the gain mainly due to its improved accuracy for low-abundance transcripts. Nonetheless, classifiers to predict MOAs perform similarly when developed using data from either platform. Therefore, the endpoint studied and its biological complexity, transcript abundance and the genomic application are important factors in transcriptomic research and for clinical and regulatory decision making.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Accessions

Primary accessions

Gene Expression Omnibus

Sequence Read Archive

References

  1. 1.

    Advancing regulatory science. Science 331, 987 (2011).

  2. 2.

    , , & A decade of toxicogenomic research and its contribution to toxicological science. Toxicol. Sci. 130, 217–228 (2012).

  3. 3.

    et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

  4. 4.

    et al. Rat toxicogenomic study reveals analytical consistency across microarray platforms. Nat. Biotechnol. 24, 1162–1169 (2006).

  5. 5.

    et al. The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat. Biotechnol. 28, 827–838 (2010).

  6. 6.

    et al. Consistency of predictive signature genes and classifiers generated using different microarray platforms. Pharmacogenomics J. 10, 247–257 (2010).

  7. 7.

    , , , & Mapping and quantifying mammalian transcriptomes by RNA-Seq. Nat. Methods 5, 621–628 (2008).

  8. 8.

    , , , & RNA-seq: an assessment of technical reproducibility and comparison with gene expression arrays. Genome Res. 18, 1509–1517 (2008).

  9. 9.

    et al. Evaluating gene expression in C57BL/6J and DBA/2J mouse striatum using RNA-Seq and microarrays. PLoS ONE 6, e17820 (2011).

  10. 10.

    et al. A comparison of massively parallel nucleotide sequencing with oligonucleotide microarrays for global transcription profiling. BMC Genomics 11, 282 (2010).

  11. 11.

    , & Comparative study of RNA-seq- and microarray-derived coexpression networks in Arabidopsis thaliana. Bioinformatics 29, 717–724 (2013).

  12. 12.

    & Microarrays, deep sequencing and the true measure of the transcriptome. BMC Biol. 9, 34 (2011).

  13. 13.

    et al. RNA-seq profiling reveals novel hepatic gene expression pattern in Aflatoxin B1 treated rats. PLoS ONE 8, e61768 (2013).

  14. 14.

    et al. A comprehensive comparison of RNA-Seq-based transcriptome analysis from reads to differential gene expression and cross-comparison with microarrays: a case study in Saccharomyces cerevisiae. Nucleic Acids Res. 40, 10084–10097 (2012).

  15. 15.

    et al. A systematic comparison and evaluation of high density exon arrays and RNA-seq technology used to unravel the peripheral blood transcriptome of sickle cell disease. BMC Med. Genomics 5, 28 (2012).

  16. 16.

    , , & RNA-Seq vs dual- and single-channel microarray data: sensitivity analysis for differential expression and clustering. PLoS ONE 7, e50986 (2012).

  17. 17.

    et al. Comparing next-generation sequencing and microarray technologies in a toxicological study of the effects of aristolochic acid on rat kidneys. Chem. Res. Toxicol. 24, 1486–1493 (2011).

  18. 18.

    & Gene-expression measurement: variance-modeling considerations for robust data analysis. Nat. Immunol. 13, 199–203 (2012).

  19. 19.

    et al. RNA sequencing shows no dosage compensation of the active X-chromosome. Nat. Genet. 42, 1043–1047 (2010).

  20. 20.

    et al. Human transcriptome array for high-throughput clinical studies. Proc. Natl. Acad. Sci. USA 108, 3707–3712 (2011).

  21. 21.

    et al. Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling. Bioinformatics 27, i383–i391 (2011).

  22. 22.

    et al. RNA-seq: technical variability and sampling. BMC Genomics 12, 293 (2011).

  23. 23.

    et al. Comparative RNA-Seq and microarray analysis of gene expression changes in B-cell lymphomas of Canis familiaris. PLoS ONE 8, e61088 (2013).

  24. 24.

    et al. A global view of gene activity and alternative splicing by deep sequencing of the human transcriptome. Science 321, 956–960 (2008).

  25. 25.

    A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nat. Biotechnol. 10.1038/nbt.2957 (24 August 2014).

  26. 26.

    & AceView: a comprehensive cDNA-supported gene and transcripts annotation. Genome Biol. 7 (suppl. 1), S12.1–14 (2006).

  27. 27.

    et al. Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res. 31, e15 (2003).

  28. 28.

    & Model-based analysis of oligonucleotide arrays: expression index computation and outlier detection. Proc. Natl. Acad. Sci. USA 98, 31–36 (2001).

  29. 29.

    Bioinformatics and Computational Biology Solutions Using R and Bioconductor. (eds. Gentleman, R., Carey, V., Huber, W., Irizarry, R. & Dudoit, S.) 397–420 (Springer, 2005).

  30. 30.

    , & edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics 26, 139–140 (2010).

  31. 31.

    & Differential expression analysis for sequence count data. Genome Biol. 11, R106 (2010).

  32. 32.

    et al. Microarray scanner calibration curves: characteristics and implications. BMC Bioinformatics 6 (suppl. 2), S11 (2005).

  33. 33.

    et al. Ontology-based meta-analysis of global collections of high-throughput public data. PLoS ONE 5, e13066 (2010).

  34. 34.

    & Dynamic expression of 3′ UTRs revealed by Poisson hidden Markov modeling of RNA-Seq: implications in gene expression profiling. Gene 527, 616–623 (2013).

  35. 35.

    , , & Analysis and design of RNA sequencing experiments for identifying isoform regulation. Nat. Methods 7, 1009–1015 (2010).

  36. 36.

    et al. The External RNA Controls Consortium. a progress report. Nat. Methods 2, 731–734 (2005).

  37. 37.

    et al. Revisiting global gene expression analysis. Cell 151, 476–482 (2012).

  38. 38.

    et al. Development of a large-scale chemogenomics database to improve drug candidate selection and to understand mechanisms of chemical toxicity and action. J. Biotechnol. 119, 219–244 (2005).

  39. 39.

    et al. Analysis of high density expression microarrays with signed-rank call algorithms. Bioinformatics 18, 1593–1599 (2002).

  40. 40.

    Affymetrix Technical Note. Guide to Probe Logarithmic Intensity Error (PLIER) Estimation () (2005).

  41. 41.

    , , , & A model-based background adjustment for oligonucleotide expression arrays. J. Am. Stat. Assoc. 99, 909–917 (2004).

  42. 42.

    & An R Companion to Applied Regression (Sage, Thousand Oaks, CA, 2011).

  43. 43.

    et al. The TRANSFAC system on gene expression regulation. Nucleic Acids Res. 29, 281–283 (2001).

  44. 44.

    et al. TRANSFAC: an integrated system for gene expression regulation. Nucleic Acids Res. 28, 316–319 (2000).

  45. 45.

    , , & Signal transduction pathway profiling of individual tumor samples. BMC Bioinformatics 6, 163 (2005).

  46. 46.

    , & TopHat: discovering splice junctions with RNA-Seq. Bioinformatics 25, 1105–1111 (2009).

Download references

Acknowledgements

We thank M. Arana and D. Mendrick for their critical review of the manuscript. This research was supported, in part, by the Intramural Research Program of the National Institutes of Health (NIH), National Institute of Environmental Health Sciences (NIEHS) (ES102345-04 and ES023026) and National Library of Medicine. P.P.Ł. and D.P.K. acknowledge support by the Vienna Scientific Cluster (VSC), the Vienna Science and Technology Fund (WWTF), Baxter AG, Austrian Research Centres (ARC) Seibersdorf and the Austrian Centre of Biopharmaceutical Technology (ACBT).

Author information

Author notes

    • Charles Wang
    • , Binsheng Gong
    •  & Pierre R Bushel

    These authors contributed equally to this work.

Affiliations

  1. Center for Genomics and Division of Microbiology & Molecular Genetics, School of Medicine, Loma Linda University, Loma Linda, California, USA.

    • Charles Wang
  2. Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, Arkansas, USA.

    • Binsheng Gong
    • , Joshua Xu
    • , Huixiao Hong
    • , Jie Shen
    • , Zhenqiang Su
    • , Joe Meehan
    • , Leming Shi
    •  & Weida Tong
  3. Microarray and Genome Informatics Group, National Institute of Environmental Health Sciences, Research Triangle Park, North Carolina, USA.

    • Pierre R Bushel
    •  & Jianying Li
  4. Biostatistics Branch, National Institute of Environmental Health Sciences, Research Triangle Park, North Carolina, USA.

    • Pierre R Bushel
  5. National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA.

    • Jean Thierry-Mieg
    •  & Danielle Thierry-Mieg
  6. The Office of Scientific Coordination, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, Arkansas, USA.

    • Hong Fang
  7. Functional Genomics Core, Department of Molecular Medicine, Beckman Research Institute, City of Hope, Duarte, California, USA.

    • Xiaojin Li
    • , Lu Yang
    •  & Haiqing Li
  8. Chair of Bioinformatics Research Group, Boku University Vienna, Vienna, Austria.

    • Paweł P Łabaj
    •  & David P Kreil
  9. University of Warwick, Coventry, UK.

    • David P Kreil
  10. CMINDS Research Center, Department of Electrical and Computer Engineering, Francis College of Engineering, University of Massachusetts, Lowell, Massachusetts, USA.

    • Dalila Megherbi
  11. Department of Toxicogenomics, Maastricht University, Maastricht, the Netherlands.

    • Stan Gaj
    • , Florian Caiment
    • , Joost van Delft
    •  & Jos Kleinjans
  12. Australian Genome Research Facility Ltd., The Walter and Eliza Hall Institute of Medical Research, Melbourne, Australia.

    • Andreas Scherer
  13. AbbVie, Inc., North Chicago, Illinois, USA.

    • Viswanath Devanarayan
  14. Research Informatics and Statistics, Eli Lilly and Company, Lilly Corporate Center, Indianapolis, Indiana, USA.

    • Jian Wang
    • , Yong Yang
    •  & Hui-Rong Qian
  15. Thomson Reuters, IP & Science, Carlsbad, California, USA.

    • Lee J Lancashire
    •  & Marina Bessarabova
  16. Vavilov Institute of General Genetics, Russian Academy of Science, Moscow, Russia.

    • Yuri Nikolsky
  17. Fondazione Bruno Kessler, Trento, Italy.

    • Cesare Furlanello
    • , Marco Chierici
    • , Davide Albanese
    • , Giuseppe Jurman
    • , Samantha Riccadonna
    • , Michele Filosi
    •  & Roberto Visintainer
  18. Computational Biology Department, Research and Innovation Centre, Fondazione Edmund Mach (FEM), San Michele all'Adige, Italy.

    • Davide Albanese
    •  & Samantha Riccadonna
  19. Bioinformatics core, Department of Pathology, University of North Dakota, Grand Forks, North Dakota, USA.

    • Ke K Zhang
  20. Kelly Government Solutions, Inc., Durham, North Carolina, USA.

    • Jianying Li
  21. Biomolecular Screening Branch, Division of the National Toxicology Program, National Institute of Environmental Health Sciences, Research Triangle Park, North Carolina, USA.

    • Jui-Hua Hsieh
    •  & Scott S Auerbach
  22. SRA International, Durham, North Carolina, USA.

    • Daniel L Svoboda
  23. Division of Systems Biology, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, Arkansas, USA.

    • James C Fuscoe
  24. Department of Internal Medicine and Biochemistry, Rush University Medical Center, Chicago, Illinois, USA.

    • Youping Deng
  25. State Key Laboratory of Genetic Engineering and MOE Key Laboratory of Contemporary Anthropology, Schools of Life Sciences and Pharmacy, Fudan University, Shanghai, China (L.S.'s primary affiliation).

    • Leming Shi
  26. Laboratory of Toxicology and Pharmacology, National Institute of Environmental Health Sciences, Research Triangle Park, North Carolina, USA.

    • Richard S Paules

Authors

  1. Search for Charles Wang in:

  2. Search for Binsheng Gong in:

  3. Search for Pierre R Bushel in:

  4. Search for Jean Thierry-Mieg in:

  5. Search for Danielle Thierry-Mieg in:

  6. Search for Joshua Xu in:

  7. Search for Hong Fang in:

  8. Search for Huixiao Hong in:

  9. Search for Jie Shen in:

  10. Search for Zhenqiang Su in:

  11. Search for Joe Meehan in:

  12. Search for Xiaojin Li in:

  13. Search for Lu Yang in:

  14. Search for Haiqing Li in:

  15. Search for Paweł P Łabaj in:

  16. Search for David P Kreil in:

  17. Search for Dalila Megherbi in:

  18. Search for Stan Gaj in:

  19. Search for Florian Caiment in:

  20. Search for Joost van Delft in:

  21. Search for Jos Kleinjans in:

  22. Search for Andreas Scherer in:

  23. Search for Viswanath Devanarayan in:

  24. Search for Jian Wang in:

  25. Search for Yong Yang in:

  26. Search for Hui-Rong Qian in:

  27. Search for Lee J Lancashire in:

  28. Search for Marina Bessarabova in:

  29. Search for Yuri Nikolsky in:

  30. Search for Cesare Furlanello in:

  31. Search for Marco Chierici in:

  32. Search for Davide Albanese in:

  33. Search for Giuseppe Jurman in:

  34. Search for Samantha Riccadonna in:

  35. Search for Michele Filosi in:

  36. Search for Roberto Visintainer in:

  37. Search for Ke K Zhang in:

  38. Search for Jianying Li in:

  39. Search for Jui-Hua Hsieh in:

  40. Search for Daniel L Svoboda in:

  41. Search for James C Fuscoe in:

  42. Search for Youping Deng in:

  43. Search for Leming Shi in:

  44. Search for Richard S Paules in:

  45. Search for Scott S Auerbach in:

  46. Search for Weida Tong in:

Contributions

W.T. coordinated the consortium study and manuscript preparation. W.T., S.S.A. and C.W. designed the study. C.W. conducted sequencing and qPCR experiments. S.S.A. provided rat tissue samples, gene expression data and contributed to the data analysis. P.R.B. was involved heavily in manuscript preparation and data analysis. B.G. and J.X. conducted the majority of data analysis and prepared various figures and supplementary materials. J.T.M. and D.T.M. constructed the mapping table between microarray and RNA-seq along with other data analysis and interpretation. All the co-authors contributed to various components of the study, including data analysis and preparation of text, figures, tables and supplementary materials.

Competing interests

The authors declare no competing financial interests.

Corresponding authors

Correspondence to Pierre R Bushel or Scott S Auerbach or Weida Tong.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–8, Supplementary Tables 1, 2, 4–9, 12, 13 and Supplementary Notes 1–6

Excel files

  1. 1.

    Supplementary Table 3

    RNA-seq data and mapping status summary based on data analysis pipeline P1

  2. 2.

    Supplementary Table 10

    List of transcripts with shortened 3' UTRs detected from the samples treated by chemicals PHE and PIR

  3. 3.

    Supplementary Table 11

    List of differentially spliced isoforms detected in samples treated by chemicals PHE and PIR

  4. 4.

    Supplementary Table 14

    Master table for mapping Affymetrix probesets to RNA-seq gene annotations

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nbt.3001

Further reading