Subjects

Abstract

We have evaluated the performance characteristics of three quantitative gene expression technologies and correlated their expression measurements to those of five commercial microarray platforms, based on the MicroArray Quality Control (MAQC) data set. The limit of detection, assay range, precision, accuracy and fold-change correlations were assessed for 997 TaqMan Gene Expression Assays, 205 Standardized RT (Sta)RT-PCR assays and 244 QuantiGene assays. TaqMan is a registered trademark of Roche Molecular Systems, Inc. We observed high correlation between quantitative gene expression values and microarray platform results and found few discordant measurements among all platforms. The main cause of variability was differences in probe sequence and thus target location. A second source of variability was the limited and variable sensitivity of the different microarray platforms for detecting weakly expressed genes, which affected interplatform and intersite reproducibility of differentially expressed genes. From this analysis, we conclude that the MAQC microarray data set has been validated by alternative quantitative gene expression platforms thus supporting the use of microarray platforms for the quantitative characterization of gene expression.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    et al. Transcript profiling of enzymes involved in detoxification of xenobiotics and reactive oxygen in human normal and simian virus 40 T antigen-immortalized oral keratinocytes. Int. J. Cancer 99, 776–782 (2002).

  2. 2.

    et al. Branched DNA amplification multimers for the sensitive, direct detection of human hepatitis virus. Nucleic Acids Symp. Ser. 24, 197–200 (1991).

  3. 3.

    et al. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25, 205–216 (2002).

  4. 4.

    (ed.). A-Z of Quantitative PCR. (International University Line Biotechnology Series, La Jolla, California, USA, 2004).

  5. 5.

    & Real-time PCR for mRNA quantitation. Biotechniques 39, 75–85 (2005).

  6. 6.

    , & Allelic discrimination by nick-translation PCR with fluorogenic probes. Nucleic Acids Res. 21, 3761–3766 (1993).

  7. 7.

    , , & Real time quantitative PCR. Genome Res. 6, 986–994 (1996).

  8. 8.

    , & A novel method for real time quantitative RT-PCR. Genome Res. 6, 995–1001 (1996).

  9. 9.

    et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).

  10. 10.

    et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).

  11. 11.

    et al. Large scale real-time PCR validation on gene expression measurements from two commercial long-oligonucleotide microarrays. BMC Genomics 7, 59 (2006).

  12. 12.

    et al. Standardized RT-PCR and the standardized expression measurement center. Methods Mol. Biol. 258, 13–41 (2004).

  13. 13.

    et al. mRNA expression levels of methotrexate resistance-related proteins in childhood leukemia as determined by a standardized competitive template-based RT-PCR method. Leukemia 14, 2166–2175 (2000).

  14. 14.

    et al. CEBPG transcription factor correlates with antioxidant and DNA repair genes in normal bronchial epithelial cells but not in individuals with bronchogenic carcinoma. BMC Cancer 5, 141 (2005).

  15. 15.

    et al. A multiplex branched DNA assay for parallel quantitative gene expression profiling. Anal. Biochem. 352, 50–60 (2006).

  16. 16.

    et al. Multicenter Evaluation of the VERSANT Hepatitis B Virus DNA 3.0 Assay. J. Clin. Microbiol. 42, 800–806 (2004).

  17. 17.

    et al. Multicenter Evaluation of the Performance Characteristics of the Bayer VERSANT HCV RNA 3.0 Assay (bDNA). J. Clin. Microbiol. 42, 563–569 (2004).

  18. 18.

    & Accuracy in amplification. Nat. Biotechnol. 19, 1011–1012 (2001).

  19. 19.

    Robust locally weighted regression and smoothing scatter plots. J. Am. Stat. Assoc. 74, 829–836 (1979).

  20. 20.

    MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

  21. 21.

    & Controlling the false discovery rate - a practical and powerful approach to multiple testing. J. R. Stat. Soc. B. Met. 57, 289–300 (1995).

  22. 22.

    et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61 (2004).

  23. 23.

    & Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCT Method. Methods 25, 402–408 (2001).

  24. 24.

    et al. An enhanced-sensitivity branched-DNA assay for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34, 3196–3202 (1996).

  25. 25.

    et al. Regulation of insulin preRNA splicing by glucose. Proc. Natl Acad. Sci. USA 94, 4360–4365 (1997).

  26. 26.

    et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).

  27. 27.

    et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).

Download references

Acknowledgements

We would like to acknowledge the contribution to this manuscript from the following members of the MAQC team: Shawn B. Baker, Anne Bergstrom Lucas, Jim Collins, Eugene Chudin, Stephanie Fulmer-Smentek, Damir Herman, Richard Shippy, Chunlin Xiao and Necip Mehmet.

Author information

Author notes

    • Roger D Canales
    • , Yuling Luo
    •  & James C Willey

    These authors contributed equally to this work.

Affiliations

  1. Applied Biosystems, 850 Lincoln Centre Dr., Foster City, California 94404, USA.

    • Roger D Canales
    • , Catalin C Barbacioru
    • , Kathryn Hunkapiller
    • , Kathleen Y Lee
    • , Karen Poulter
    • , Raymond R Samaha
    •  & Lu Zhang
  2. Panomics, Inc., 6519 Dumbarton Circle, Fremont, California 94555, USA.

    • Yuling Luo
    • , Yunqing Ma
    • , Botoul Maqsodi
    •  & Wen Yang
  3. University of Toledo, Toledo, Ohio 43614, USA.

    • James C Willey
    •  & Bradley Austermiller
  4. ViaLogy Corp., 2400 Lincoln Avenue, Altadena, California 91001, USA.

    • Cecilie Boysen
  5. University of Massachusetts-Boston, 100 Morrissey Blvd., Boston, Massachusetts 02125, USA.

    • Roderick V Jensen
    •  & Adam Papallo
  6. Gene Express, Inc., 975 Research Drive, Toledo, Ohio 43614, USA.

    • Charles R Knight
    •  & Elizabeth Herness Peters
  7. Innovative Analytics, 7107 Elm Valley Dr., Kalamazoo, Michigan 49009, USA.

    • Patricia L Ruppel
  8. National Center for Toxicological Research, US Food and Drug Administration, 3900 NCTR Rd., Jefferson, Arkansas 72079, USA.

    • Leming Shi
  9. Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Ave., Silver Spring, Maryland 20993, USA.

    • Federico M Goodsaid

Authors

  1. Search for Roger D Canales in:

  2. Search for Yuling Luo in:

  3. Search for James C Willey in:

  4. Search for Bradley Austermiller in:

  5. Search for Catalin C Barbacioru in:

  6. Search for Cecilie Boysen in:

  7. Search for Kathryn Hunkapiller in:

  8. Search for Roderick V Jensen in:

  9. Search for Charles R Knight in:

  10. Search for Kathleen Y Lee in:

  11. Search for Yunqing Ma in:

  12. Search for Botoul Maqsodi in:

  13. Search for Adam Papallo in:

  14. Search for Elizabeth Herness Peters in:

  15. Search for Karen Poulter in:

  16. Search for Patricia L Ruppel in:

  17. Search for Raymond R Samaha in:

  18. Search for Leming Shi in:

  19. Search for Wen Yang in:

  20. Search for Lu Zhang in:

  21. Search for Federico M Goodsaid in:

Competing interests

J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Corresponding author

Correspondence to Federico M Goodsaid.

Supplementary information

Word documents

  1. 1.

    Supplementary Fig. 1

    TaqMan® assays SD vs. CT plot.

  2. 2.

    Supplementary Fig. 2

    TaqMan® assays R2 distribution plot.

  3. 3.

    Supplementary Fig. 3

    Boxplot of fold change as a function of signal.

  4. 4.

    Supplementary Fig. 4

    Expression characteristics of EPHA7.

  5. 5.

    Supplementary Fig. 5

    Endogenous control expression.

  6. 6.

    Supplementary Table 1

    Gene lists used for the analysis of performance metrics of alternative quantitative platforms and for comparison with microarrays.

  7. 7.

    Supplementary Table 2

    Inter-site discordance in detection among genes expressed at low level.

  8. 8.

    Supplementary Table 3

    TPR, FDR tables.

  9. 9.

    Supplementary Table 4

    Discordant probes among the 997 genes.

  10. 10.

    Supplementary Table 5

    Discordant gene expression values in alternative quantitative platforms FC expression values for all discordant genes.

  11. 11.

    Supplementary Methods

About this article

Publication history

Published

DOI

https://doi.org/10.1038/nbt1236

Further reading