Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium

Nature Biotechnology volume 32, pages 903–914 (2014)Cite this article

Subjects

This article has been updated

Abstract

We present primary results from the Sequencing Quality Control (SEQC) project, coordinated by the US Food and Drug Administration. Examining Illumina HiSeq, Life Technologies SOLiD and Roche 454 platforms at multiple laboratory sites using reference RNA samples with built-in controls, we assess RNA sequencing (RNA-seq) performance for junction discovery and differential expression profiling and compare it to microarray and quantitative PCR (qPCR) data using complementary metrics. At all sequencing depths, we discover unannotated exon-exon junctions, with >80% validated by qPCR. We find that measurements of relative expression are accurate and reproducible across sites and platforms if specific filters are used. In contrast, RNA-seq and microarrays do not provide accurate absolute measurements, and gene-specific biases are observed for all examined platforms, including qPCR. Measurement performance depends on the platform and data analysis pipeline, and variation is large for transcript-level profiling. The complete SEQC data sets, comprising >100 billion reads (10Tb), provide unique resources for evaluating RNA-seq analyses for clinical and regulatory settings.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The SEQC (MAQC-III) project and experimental design.
Figure 2: Gene detection and junction discovery.
Figure 3: Sensitivity, specificity and reproducibility of differential expression calls.
Figure 4: Built-in truths for assessing RNA-seq.
Figure 5: Cross-platform agreement of expression levels.
Figure 6: Multiple performance metrics for the quantification of genes and alternative transcripts.

Similar content being viewed by others

Accession codes

Primary accessions

Gene Expression Omnibus

Change history

  • 09 September 2014

    In the version of this article initially published online, the superscript 95 for the footnote for “these authors contributed equally to this work” was omitted for the first three authors. The error has been corrected for the print, PDF and HTML versions of this article.

References

  1. Wang, E.T. et al. Alternative isoform regulation in human tissue transcriptomes. Nature 456, 470–476 (2008).

    Article  CAS  Google Scholar 

  2. Mortazavi, A., Williams, B.A., McCue, K., Schaeffer, L. & Wold, B. Mapping and quantifying mammalian transcriptomes by RNA-seq. Nat. Methods 5, 621–628 (2008).

    Article  CAS  Google Scholar 

  3. Łabaj, P.P. et al. Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling. Bioinformatics 27, i383–i391 (2011).

    Article  Google Scholar 

  4. Liu, S., Lin, L., Jiang, P., Wang, D. & Xing, Y. A comparison of RNA-Seq and high-density exon array for detecting differential gene expression between closely related species. Nucleic Acids Res. 39, 578–588 (2011).

    Article  CAS  Google Scholar 

  5. McIntyre, L.M. et al. RNA-seq: technical variability and sampling. BMC Genomics 12, 293 (2011).

    Article  CAS  Google Scholar 

  6. Toung, J.M., Morley, M., Li, M. & Cheung, V.G. RNA-sequence analysis of human B-cells. Genome Res. 21, 991–998 (2011).

    Article  CAS  Google Scholar 

  7. Djebali, S. et al. Landscape of transcription in human cells. Nature 489, 101–108 (2012).

    Article  CAS  Google Scholar 

  8. Cancer Genome Atlas Research Network. Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature 455, 1061–1068 (2008).

  9. International Cancer Genome Consortium. International network of cancer genome projects. Nature 464, 993–998 (2010).

  10. Shi, L. et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

    Article  CAS  Google Scholar 

  11. Li, S. et al. Detecting and correcting systematic variation in large-scale RNA sequencing data. Nat. Biotechnol. 10.1038/nbt.3000 (24 August 2014).

  12. Wang, C. et al. The concordance between RNA-seq and microarray data depends on chemical treatment and transcript abundance. Nat. Biotechnol. 10.1038/nbt.3001 (24 August 2014).

  13. Yu, Y. et al. A rat RNA-seq transcriptomic Bodymap across eleven organs and four developmental stages. Nat. Commun. 5, 3230 (2014).

    Article  Google Scholar 

  14. Baker, S.C. et al. The External RNA Controls Consortium: a progress report. Nat. Methods 2, 731–734 (2005).

    Article  CAS  Google Scholar 

  15. Pruitt, K.D., Tatusova, T., Brown, G.R. & Maglott, D.R. NCBI Reference Sequences (RefSeq): current status, new features and genome annotation policy. Nucleic Acids Res. 40, D130–D135 (2012).

    Article  CAS  Google Scholar 

  16. Harrow, J. et al. GENCODE: The reference human genome annotation for The ENCODE Project. Genome Res. 22, 1760–1774 (2012).

    Article  CAS  Google Scholar 

  17. Thierry-Mieg, D. & Thierry-Mieg, J. AceView: a comprehensive cDNA-supported gene and transcripts. Genome Biol. 7, S12 (2006).

    Article  Google Scholar 

  18. Dobin, A. et al. STAR: ultrafast universal RNA-seq aligner. Bioinformatics 29, 15–21 (2013).

    Article  CAS  Google Scholar 

  19. Liao, Y., Smyth, G.K. & Shi, W. The Subread aligner: fast, accurate and scalable read mapping by seed-and-vote. Nucleic Acids Res. 41, e108 (2013).

    Article  Google Scholar 

  20. Kim, D. et al. TopHat2: accurate alignment of transcriptomes in the presence of insertions, deletions and gene fusions. Genome Biol. 14, R36 (2013).

    Article  Google Scholar 

  21. Li, S. et al. Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study. Nat. Biotechnol. 10.1038/nbt.2972 (24 August 2014).

  22. Xu, W. et al. Human transcriptome array for high-throughput clinical studies. Proc. Natl. Acad. Sci. USA 108, 3707–3712 (2011).

    Article  CAS  Google Scholar 

  23. Marioni, J.C., Mason, C.E., Mane, S.M., Stephens, M. & Gilad, Y. RNA-seq: an assessment of technical reproducibility and comparison with gene expression arrays. Genome Res. 18, 1509–1517 (2008).

    Article  CAS  Google Scholar 

  24. VanGuilder, H., Vrana, K. & Freeman, W. Twenty-five years of quantitative PCR for gene expression analysis. Biotechniques 44 (suppl.) 619–626 (2008).

    Article  CAS  Google Scholar 

  25. Aird, D. et al. Analyzing and minimizing PCR amplification bias in Illumina sequencing libraries. Genome Biol. 12, R18 (2011).

    Article  CAS  Google Scholar 

  26. Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).

    Article  CAS  Google Scholar 

  27. Trapnell, C. et al. Differential analysis of gene regulation at transcript resolution with RNA-seq. Nat. Biotechnol. 31, 46–53 (2013).

    Article  CAS  Google Scholar 

  28. Pickrell, J.K., Pai, A.A., Gilad, Y. & Pritchard, J.K. Noisy splicing drives mRNA isoform diversity in human cells. PLoS Genet. 6, e1001236 (2010).

    Article  Google Scholar 

  29. Dai, M. et al. Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data. Nucleic Acids Res. 33, e175 (2005).

    Article  Google Scholar 

  30. Liu, Y. et al. Evaluating the impact of sequencing depth on transcriptome profiling in human adipose. PLoS ONE 8, e66883 (2013).

    Article  CAS  Google Scholar 

  31. Levin, J.Z. et al. Targeted next-generation sequencing of a cancer transcriptome enhances detection of sequence variants and novel fusion transcripts. Genome Biol. 10, R115 (2009).

    Article  Google Scholar 

  32. Agarwal, A. et al. Comparison and calibration of transcriptome data from RNA-Seq and tiling arrays. BMC Genomics 11, 383 (2010).

    Article  Google Scholar 

  33. Raghavachari, N. et al. A systematic comparison and evaluation of high density exon arrays and RNA-seq technology used to unravel the peripheral blood transcriptome of sickle cell disease. BMC Med. Genomics 5, 28 (2012).

    Article  CAS  Google Scholar 

  34. Qing, T., Yu, Y., Du, T. & Shi, L. mRNA enrichment protocols determine the quantification characteristics of external RNA spike-in controls in RNA-seq studies. Sci. China Life Sci. 56, 134–142 (2013).

    Article  CAS  Google Scholar 

  35. Benjamini, Y. & Speed, T.P. Summarizing and correcting the GC content bias in high-throughput sequencing. Nucleic Acids Res. 40, e72 (2012).

    Article  CAS  Google Scholar 

  36. Robinson, M.D. & Oshlack, A. A scaling normalization method for differential expression analysis of RNA-seq data. Genome Biol. 11, R25 (2010).

    Article  Google Scholar 

  37. Smyth, G.K. in Bioinformatics and Computational Biology Solutions Using R Bioconductor (eds. Gentleman, R., Carey, V.J., Huber, W., Irizarry, R.A. & Dudoit, S.) 397–420 (Springer, New York, 2005).

  38. Huber, W., von Heydebreck, A., Sültmann, H., Poustka, A. & Vingron, M. Variance stabilization applied to microarray data calibration and to the quantification of differential expression. Bioinformatics 18, S96–S104 (2002).

    Article  Google Scholar 

  39. Wu, Z., Irizarry, R., Gentleman, R., Murillo, F.M. & Spencer, F. A model based background adjustment for oligonucleotide expression arrays. J. Amer. Stat. Assoc. 99, 909–917 (2004).

    Article  Google Scholar 

  40. Hochreiter, S., Clevert, D.-A. & Obermayer, K. A new summarization method for affymetrix probe level data. Bioinformatics 22, 943–949 (2006).

    Article  CAS  Google Scholar 

  41. Fasold, M., Stadler, P.F. & Binder, H. G-stack modulated probe intensities on expression arrays–sequence corrections and signal calibration. BMC Bioinformatics 11, 207 (2010).

    Article  Google Scholar 

  42. Mueckstein, U., Leparc, G.G., Posekany, A., Hofacker, I. & Kreil, D.P. Hybridization thermodynamics of NimbleGen Microarrays. BMC Bioinformatics 11, 35 (2010).

    Article  Google Scholar 

  43. Sykacek, P. et al. The impact of quantitative optimization of hybridization conditions on gene expression analysis. BMC Bioinformatics 12, 73 (2011).

    Article  Google Scholar 

  44. Rapaport, F. et al. Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data. Genome Biol. 14, R95 (2013).

    Article  Google Scholar 

  45. Xu, J. et al. Cross-platform ultradeep transcriptomic profiling of human reference RNA samples by RNA-Seq. Sci. Data (in the press).

  46. Liu, S. et al. A comparison of RNA-seq and high-density exon array for detecting differential gene expression between closely related species. Nucleic Acids Res. 39, 578–588 (2011).

    Article  CAS  Google Scholar 

  47. Munro, S. et al. Nat. Commun. (in the press).

  48. David, M., Dzamba, M., Lister, D., Ilie, L. & Brudno, M. SHRiMP2: Sensitive yet practical short read mapping. Bioinformatics 27, 1011–1012 (2011).

    Article  CAS  Google Scholar 

  49. Glaus, P., Honkela, A. & Rattray, M. Identifying differentially expressed transcripts from RNA-seq data with biological variation. Bioinformatics 28, 1721–1728 (2012).

    Article  CAS  Google Scholar 

  50. Liao, Y., Smyth, G.K. & Shi, W. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. Bioinformatics 30, 923–930 (2014).

    Article  CAS  Google Scholar 

  51. Rasmussen, C.E. Gaussian Processes for Machine Learning (MIT Press, 2006).

  52. Law, C.W. et al. Voom: precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biol. 15, R29 (2014).

    Article  Google Scholar 

  53. Dillies, M.-A. et al. A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis. Brief. Bioinform. 14, 671–683 (2013).

    Article  CAS  Google Scholar 

  54. Robinson, M.D., McCarthy, D.J. & Smyth, G.K. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics 26, 139–140 (2010).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

All SEQC (MAQC-III) participants freely donated their time and reagents for the completion and analyses of the project. Many participants contributed to the sometimes-heated discussions on the topic of this paper during numerous e-mail exchanges, teleconferences and face-to-face project meetings. The common conclusions and recommendations reported in this paper evolved from this extended discourse. The authors gratefully acknowledge support by the National Center for Biotechnology Information (NCBI)'s Supercomputing Center, the FDA's Supercomputing Center, China's National Supercomputing Center of Tianjin, the Vienna Scientific Cluster High Performance Computing Facility (VSC), the Vienna Science and Technology Fund (WWTF), Baxter, the Austrian Institute of Technology, and the Austrian Centre of Biopharmaceutical Technology. This work was supported in part by China's Program of Global Experts. This work was supported in part by the US National Institutes of Health (NIH) grants R01CA163256, R01HG006798, R01NS076465, R44HG005297, U54CA119338, PO1HG00205, R24GM102656 and the Intramural Research Program of the NIH, National Library of Medicine, National Institute of Environmental Health Sciences (NIEHS) Z01 ES102345-04, Shriners Research Grant 85500, an Australia National Health and Medical Research Council (NH&MRC) Project grant (1023454) and Victorian State Government Operational Infrastructure Support (Australia), the National 973 Key Basic Research Program of China (2010CB945401), the National Natural Science Foundation of China (31240038 and 31071162), and the Science and Technology Commission of Shanghai Municipality (11DZ2260300). We greatly appreciate SAS Institute, Inc. for kindly hosting several face-to-face meetings of the SEQC (MAQC-III) project.

Author information

Authors and Affiliations

  1. FDA/NCTR, Jefferson, Arkansas, USA

    Zhenqiang Su, Reagan Kelly, Joshua Xu, Huixiao Hong, Ching-Wei Chang, Tao Chen, Hong Fang, James C Fuscoe, Weigong Ge, Binsheng Gong, Bridgett Green, Lei Guo, Li-Wu Guo, Yan Li, Joseph Meehan, Nan Mei, Baitang Ning, Roger G Perkins, Feng Qian, Jie Shen, Liqing Wan, Jiekun Xuan, Wenqian Zhang, Weida Tong & Leming Shi

  2. Chair of Bioinformatics Research Group, Boku University Vienna, Vienna, Austria

    Paweł P Łabaj, Peter Sykacek, Nancy Stralis-Pavese & David P Kreil

  3. Department of Physiology and Biophysics, Weill Cornell Medical College, New York, New York, USA

    Sheng Li, Jorge Gandara, Paul Zumbo & Christopher E Mason

  4. The HRH Prince Alwaleed Bin Talal Bin Abdulaziz Alsaud Institute for Computational Biomedicine, Weill Cornell Medical College, New York, New York, USA

    Sheng Li, Paul Zumbo & Christopher E Mason

  5. NIH/NCBI, Bethesda, Maryland, USA

    Jean Thierry-Mieg & Danielle Thierry-Mieg

  6. Bioinformatics Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia

    Wei Shi, Yang Liao & Gordon K Smyth

  7. Department of Computing and Information Systems, The University of Melbourne, Parkville, Victoria, Australia

    Wei Shi

  8. Center for Genomics and Division of Microbiology & Molecular Genetics, School of Medicine, Loma Linda University, Loma Linda, California, USA

    Charles Wang

  9. Illumina Inc., Hayward, California, USA

    Gary P Schroth

  10. Life Technologies Corporation, Austin, Texas, USA

    Robert A Setterquist

  11. Claritas Genomics, Cambridge, Massachusetts, USA

    John F Thompson

  12. Expression Analysis Inc., Durham, North Carolina, USA

    Wendell D Jones

  13. Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA

    Wenzhong Xiao

  14. Stanford Genome Technology Center, Palo Alto, California, USA

    Wenzhong Xiao & Weihong Xu

  15. Department of Biological Sciences, Virginia Tech, Blacksburg, Virginia, USA

    Roderick V Jensen

  16. Centro de Investigación Príncipe Felipe (CIPF), Computational Genomics Program, Valencia, Spain

    Ana Conesa, Joaquin Dopazo & Pedro Furió-Tari

  17. Fondazione Bruno Kessler (FBK), Trento, Italy

    Cesare Furlanello & Marco Chierici

  18. DNA Sequencing/Solexa Core, Beckman Research Institute, City of Hope Comprehensive Cancer Center, City of Hope National Medical Center, Duarte, California, USA

    Hanlin Gao & Jinhui Wang

  19. Center for Genetic Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA

    Nadereh Jafari

  20. SynapDx Corporation, Lexington, Massachusetts, USA

    Stan Letovsky

  21. Department of Medical Biology, The University of Melbourne, Parkville, Victoria, Australia

    Yang Liao

  22. GE Healthcare SeqWright Genomics Services, Houston, Texas, USA

    Fei Lu, Xin-Xing Tan & Lee T Szkotnicki

  23. Novartis Institutes for Biomedical Research (NIBR), Basel, Switzerland

    Edward J Oakeley, Frank Staedtler & Anita Fernandez

  24. BGI-Shenzhen, Bei Shan Industrial Zone, Yantian District, Shenzhen, Guangdong, China.,

    Zhiyu Peng, Zirui Dong, Huan Gao, Meihua Gong, Zhuolin Gong, Min Jian, Shiyong Li, Bimeng Tu, Jun Wang, Ye Yin, Wenqian Zhang, Wenwei Zhang & Yanyan Zhang

  25. The Pennsylvania State University, University Park, Pennsylvania, USA

    Craig A Praul

  26. Medical Genome Project (MGP), Genomics and Bioinformatics Platform of Andalusia (GBPA), Sevilla, Spain

    Javier Santoyo-Lopez, Francisco J López, Javier Pérez-Florido & Alicia Vela-Boza

  27. Edinburgh Genomics, University of Edinburgh, Edinburgh, Scotland, UK

    Javier Santoyo-Lopez

  28. Australian Genome Research Facility Ltd., The Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia

    Andreas Scherer & Matthew Tinning

  29. Spheromics, Kontiolahti, Finland

    Andreas Scherer

  30. Center for Bioinformatics and Computational Biology, Shanghai Key Laboratory of Regulatory Biology, the Institute of Biomedical Sciences and School of Life Sciences, East China Normal University, Shanghai, China

    Tieliu Shi, Geng Chen, Peng Li & Chen Zhao

  31. Department of Mathematics and Statistics, The University of Melbourne, Parkville, Victoria, Australia

    Gordon K Smyth

  32. Department of Cancer Biology, Mayo Clinic Jacksonville, Jacksonville, Florida, USA

    E Aubrey Thompson

  33. Biogazelle, Zwijnaarde, Belgium

    Jo Vandesompele & Jan Hellemans

  34. Department of Biomedical Engineering, GeorgiaTech and Emory University, Atlanta, Georgia, USA

    May D Wang & John H Phan

  35. Research Informatics, Lilly Corporate Center, Eli Lilly and Company, Indianapolis, Indiana, USA

    Jian Wang & Yong Yang

  36. SAS Institute Inc., Cary, North Carolina, USA

    Russell D Wolfinger, Wenjun Bao, Tzu-Ming Chu & Li Li

  37. NYU Genome Technology Center, New York University Langone Medical Center, New York, NewYork, USA

    Jiri Zavadil & Elisa Venturini

  38. Department of Pathology, NYU Center for Health Informatics and Bioinformatics, New York University Langone Medical Center, New York, New York, USA

    Jiri Zavadil

  39. Molecular Mechanisms and Biomarkers Group, International Agency for Research on Cancer, Lyon, France

    Jiri Zavadil

  40. NIH/NIEHS, National Institute of Environmental Health Sciences, NIH, Research Triangle Park, North Carolina, USA

    Scott S Auerbach, Pierre R Bushel & Jianying Li

  41. Interdisciplinary Centre for Bioinformatics, University of Leipzig, Leipzig, Germany

    Hans Binder

  42. Division of Pulmonary and Critical Care Medicine, Department of Medicine, University of Toledo Health Sciences Campus, Medical College of Ohio, Toledo, Ohio, USA

    Thomas Blomquist & James C Willey

  43. Center for Human Genetics, Marshfield Clinic Research Foundation, Marshfield, Wisconsin, USA

    Murray H Brilliant

  44. School of Pharmacy, Fudan University, Shanghai, China

    Weimin Cai, Tingting Du, Meiwen Jia, Tao Qing, Ying Yu & Yuanting Zheng

  45. Office of Cellular, Tissue, and Gene Therapies, FDA/CBER, Bethesda, Maryland, USA

    Jennifer G Catalano

  46. Department of Genetics and Genomic Sciences, Icahn Institute for Genomics and Multiscale Biology, Icahn School of Medicine at Mount Sinai, New York, New York, USA

    Rong Chen & Li Li

  47. Institute of Bioinformatics, Johannes Kepler University Linz, Linz, Austria

    Djork-Arné Clevert & Sepp Hochreiter

  48. Department of Internal Medicine, Rush University Cancer Center, Chicago, Illinois, USA

    Youping Deng

  49. Novartis Institutes for Biomedical Research, Novartis, Cambridge, Massachusetts, USA

    Adnan Derti

  50. AbbVie Inc., Global Pharmaceutical R&D, Souderton, Pennsylvania, USA

    Viswanath Devanarayan

  51. CIBER de Enfermedades Raras (CIBERER) and Functional Genomics Node (INB) at CIPF, Valencia, Spain

    Joaquin Dopazo

  52. Centre for Genomic Research, University of Liverpool, Liverpool, UK

    Yongxiang Fang, Suzanne Kay & Lucille Rainbow

  53. ecSeq Bioinformatics, Leipzig, Germany

    Mario Fasold

  54. Department of Pediatric Oncology and Hematology and Center for Molecular Medicine (CMMC), University of Cologne, Cologne, Germany

    Matthias Fischer

  55. Department of Toxicogenomics, Maastricht University, Maastricht, the Netherlands

    Florian Caimet, Stan Gaj, Jos Kleinjans & Joost van Delft

  56. RIKEN BioResource Center, Tsukuba, Ibarak, Japan

    Yoichi Gondo

  57. Functional Genomics Core, Beckman Research Institute, City of Hope National Medical Center, Duarte, California, USA

    Chao Guo

  58. Cancer Research UK Cambridge Institute, University of Cambridge, Li Ka Shing Centre, Cambridge, UK

    James Hadfield

  59. Texas A&M AgriLife Research, College Station, Texas, USA

    Charles D Johnson & Scott Schwartz

  60. FDA/CBER, Rockville, Maryland, USA

    Samir Lababidi

  61. HudsonAlpha Institute for Biotechnology, Huntsville, Alabama, USA

    Shawn Levy

  62. University of Texas Southwestern Medical Center (UTSW), Dallas, Texas, USA

    Quan-Zhen Li

  63. Bioinformatics Core, Beckman Research Institute, City of Hope National Medical Center, Duarte, California, USA

    Haiqing Li, Bing Mu & Yate-Ching Yuan

  64. The Research Institute, Nationwide Children's Hospital, Columbus, Ohio, USA

    Simon M Lin

  65. AbbVie Inc., Global Pharmaceutical R&D, North Chicago, Illinois, USA

    Xin Lu

  66. UALR/UAMS Joint Bioinformatics Graduate Program, University of Arkansas at Little Rock, Little Rock, Arkansas, USA

    Heng Luo

  67. Discovery Statistics, Lilly Corporate Center, Eli Lilly and Company, Indianapolis, Indiana, USA

    Xiwen Ma

  68. Department of Electrical and Computer Engineering, CMINDS Research Center, University of Massachusetts at Lowell, Lowell, Massachusetts, USA

    Dalila B Megherbi

  69. McKusick-Nathans Institute of Genetic Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA

    Akhilesh Pandey

  70. Institute of Bioinformatics, Bangalore, India

    Akhilesh Pandey

  71. Partek Inc., St. Louis, Missouri, USA

    Ryan Peters

  72. Department of Psychiatry, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA

    Mehdi Pirooznia

  73. Oxford e-Research Centre, University of Oxford, Oxford, UK

    Philippe Rocca-Serra & Susanna-Assunta Sansone

  74. UJF-Grenoble 1/CNRS/TIMC-IMAG UMR 5525, Computational and Mathematical Biology (BCM), Grenoble, France

    Laure Sambourg

  75. SRA International Inc., Durham, North Carolina, USA

    Ruchir Shah

  76. Geospiza Inc., Seattle, Washington, USA

    Todd M Smith

  77. European Molecular Biology Laboratory, European Bioinformatics Institute, Hinxton, Cambridge, UK

    Oliver Stegle

  78. San Raffaele Scientific Institute, Center for Translational Genomics and Bioinformatics, Milano, Italy

    Elia Stupka

  79. Department of Medical Genome Sciences, The University of Tokyo, Chiba, Japan

    Yutaka Suzuki

  80. Wake Forest Institute for Regenerative Medicine, Wake Forest University, Health Sciences, Winston-Salem, North Carolina, USA

    Stephen J Walker

  81. Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, New Jersey, USA

    Wei Wang

  82. Department of Biology, University of Copenhagen, Copenhagen, Denmark

    Jun Wang

  83. King Abdulaziz University, Jeddah, Saudi Arabia

    Jun Wang

  84. The Novo Nordisk Foundation Center for Basic Metabolic Research, University of Copenhagen, Copenhagen, Denmark

    Jun Wang

  85. Department of Biochemistry and Molecular Biology, Mayo Clinic Rochester, Rochester, Minnesota, USA

    Eric D Wieben

  86. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA

    Po-Yen Wu

  87. Biomedical Informatics Research Center, Marshfield Clinic Research Foundation, Marshfield, Wisconsin, USA

    Zhan Ye

  88. Center of Big Data Research, Zhijiang College, Zhejiang University of Technology, Hangzhou, Zhejiang, China

    John Zhang

  89. Department of Pathology, School of Medicine, University of North Dakota, Grand Forks, North Dakota, USA

    Ke K Zhang

  90. Digomics LLC, Brookline, Massachusetts, USA

    Yiming Zhou

  91. University of Warwick, Coventry, UK

    David P Kreil

  92. State Key Laboratory of Genetic Engineering and MOE Key Laboratory of Contemporary Anthropology, Schools of Life Sciences and Pharmacy, Fudan University, Shanghai, China (L.S.'s primary affiliation).,

    Leming Shi

  93. Fudan-Zhangjiang Center for Clinical Genomics, Shanghai, China

    Leming Shi

  94. Zhanjiang Center for Translational Medicine, Shanghai, China

    Leming Shi

Consortia

SEQC/MAQC-III Consortium

  • Zhenqiang Su
  • , Paweł P Łabaj
  • , Sheng Li
  • , Jean Thierry-Mieg
  • , Danielle Thierry-Mieg
  • , Wei Shi
  • , Charles Wang
  • , Gary P Schroth
  • , Robert A Setterquist
  • , John F Thompson
  • , Wendell D Jones
  • , Wenzhong Xiao
  • , Weihong Xu
  • , Roderick V Jensen
  • , Reagan Kelly
  • , Joshua Xu
  • , Ana Conesa
  • , Cesare Furlanello
  • , Hanlin Gao
  • , Huixiao Hong
  • , Nadereh Jafari
  • , Stan Letovsky
  • , Yang Liao
  • , Fei Lu
  • , Edward J Oakeley
  • , Zhiyu Peng
  • , Craig A Praul
  • , Javier Santoyo-Lopez
  • , Andreas Scherer
  • , Tieliu Shi
  • , Gordon K Smyth
  • , Frank Staedtler
  • , Peter Sykacek
  • , Xin-Xing Tan
  • , E Aubrey Thompson
  • , Jo Vandesompele
  • , May D Wang
  • , Jian Wang
  • , Russell D Wolfinger
  • , Jiri Zavadil
  • , Scott S Auerbach
  • , Wenjun Bao
  • , Hans Binder
  • , Thomas Blomquist
  • , Murray H Brilliant
  • , Pierre R Bushel
  • , Weimin Cai
  • , Jennifer G Catalano
  • , Ching-Wei Chang
  • , Tao Chen
  • , Geng Chen
  • , Rong Chen
  • , Marco Chierici
  • , Tzu-Ming Chu
  • , Djork-Arné Clevert
  • , Youping Deng
  • , Adnan Derti
  • , Viswanath Devanarayan
  • , Zirui Dong
  • , Joaquin Dopazo
  • , Tingting Du
  • , Hong Fang
  • , Yongxiang Fang
  • , Mario Fasold
  • , Anita Fernandez
  • , Matthias Fischer
  • , Pedro Furió-Tari
  • , James C Fuscoe
  • , Florian Caimet
  • , Stan Gaj
  • , Jorge Gandara
  • , Huan Gao
  • , Weigong Ge
  • , Yoichi Gondo
  • , Binsheng Gong
  • , Meihua Gong
  • , Zhuolin Gong
  • , Bridgett Green
  • , Chao Guo
  • , Lei Guo
  • , Li-Wu Guo
  • , James Hadfield
  • , Jan Hellemans
  • , Sepp Hochreiter
  • , Meiwen Jia
  • , Min Jian
  • , Charles D Johnson
  • , Suzanne Kay
  • , Jos Kleinjans
  • , Samir Lababidi
  • , Shawn Levy
  • , Quan-Zhen Li
  • , Li Li
  • , Li Li
  • , Peng Li
  • , Yan Li
  • , Haiqing Li
  • , Jianying Li
  • , Shiyong Li
  • , Simon M Lin
  • , Francisco J López
  • , Xin Lu
  • , Heng Luo
  • , Xiwen Ma
  • , Joseph Meehan
  • , Dalila B Megherbi
  • , Nan Mei
  • , Bing Mu
  • , Baitang Ning
  • , Akhilesh Pandey
  • , Javier Pérez-Florido
  • , Roger G Perkins
  • , Ryan Peters
  • , John H Phan
  • , Mehdi Pirooznia
  • , Feng Qian
  • , Tao Qing
  • , Lucille Rainbow
  • , Philippe Rocca-Serra
  • , Laure Sambourg
  • , Susanna-Assunta Sansone
  • , Scott Schwartz
  • , Ruchir Shah
  • , Jie Shen
  • , Todd M Smith
  • , Oliver Stegle
  • , Nancy Stralis-Pavese
  • , Elia Stupka
  • , Yutaka Suzuki
  • , Lee T Szkotnicki
  • , Matthew Tinning
  • , Bimeng Tu
  • , Joost van Delft
  • , Alicia Vela-Boza
  • , Elisa Venturini
  • , Stephen J Walker
  • , Liqing Wan
  • , Wei Wang
  • , Jinhui Wang
  • , Jun Wang
  • , Eric D Wieben
  • , James C Willey
  • , Po-Yen Wu
  • , Jiekun Xuan
  • , Yong Yang
  • , Zhan Ye
  • , Ye Yin
  • , Ying Yu
  • , Yate-Ching Yuan
  • , John Zhang
  • , Ke K Zhang
  • , Wenqian Zhang
  • , Wenwei Zhang
  • , Yanyan Zhang
  • , Chen Zhao
  • , Yuanting Zheng
  • , Yiming Zhou
  • , Paul Zumbo
  • , Weida Tong
  • , David P Kreil
  • , Christopher E Mason
  •  & Leming Shi

Contributions

Project coordination: US Food and Drug Administration.

Project lead: Weida Tong & Leming Shi.

Manuscript lead: David P. Kreil.

Scientific management: David P. Kreil, Christopher E. Mason, Weida Tong & Leming Shi.

Next-generation sequencing technology lead: Christopher E. Mason.

The following authors contributed to project leadership: Zhenqiang Su, Paweł P. Łabaj, Sheng Li, Jean Thierry-Mieg, Danielle Thierry-Mieg, Wei Shi, Charles Wang, Gary P. Schroth, Robert A. Setterquist, John F. Thompson, Wendell D. Jones, Wenzhong Xiao, Weihong Xu, Roderick V Jensen, Reagan Kelly, Joshua Xu, Ana Conesa, Cesare Furlanello, Hanlin Gao, Huixiao Hong, Nadereh Jafari, Stan Letovsky, Yang Liao, Fei Lu, Edward J. Oakeley, Zhiyu Peng, Craig A. Praul, Javier Santoyo-Lopez, Andreas Scherer, Tieliu Shi, Gordon K. Smyth, Frank Staedtler, Peter Sykacek, Xin-Xing Tan, E. Aubrey Thompson, Jo Vandesompele, May D. Wang, Jian Wang, Russell D. Wolfinger, Jiri Zavadil, Weida Tong, David P. Kreil, Christopher E. Mason & Leming Shi.

The following authors contributed equally to this work: Zhenqiang Su, Paweł P. Łabaj & Sheng Li.

Corresponding authors

Correspondence to David P Kreil, Christopher E Mason or Leming Shi.

Ethics declarations

Competing interests

Some of the SEQC (MAQC-III) Consortium members are employed by companies that provide services or manufacture products or equipment related to gene expression profiling, as can be seen from the affiliations provided by the manuscript authors.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–46, Supplementary Tables 1–15 and Supplementary Notes (PDF 24302 kb)

Supplementary Data 1

RNA-seq read coverage flanking all 250 candidate junctions considered for validation. (TXT 870 kb)

Supplementary Data 2

Employed qPCR primer sequences, qPCR results and expression level estimates, as well as the corresponding RNA-seq expression level estimates for the 173 performed assays. (XLS 121 kb)

Supplementary Data 3

Supplementary Data 3 (ZIP 38371 kb)

Supplementary Protocols (PDF 1467 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

SEQC/MAQC-III Consortium. A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nat Biotechnol 32, 903–914 (2014). https://doi.org/10.1038/nbt.2957

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt.2957

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing