Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Evaluation of DNA microarray results with quantitative gene expression platforms

Abstract

We have evaluated the performance characteristics of three quantitative gene expression technologies and correlated their expression measurements to those of five commercial microarray platforms, based on the MicroArray Quality Control (MAQC) data set. The limit of detection, assay range, precision, accuracy and fold-change correlations were assessed for 997 TaqMan Gene Expression Assays, 205 Standardized RT (Sta)RT-PCR assays and 244 QuantiGene assays. TaqMan is a registered trademark of Roche Molecular Systems, Inc. We observed high correlation between quantitative gene expression values and microarray platform results and found few discordant measurements among all platforms. The main cause of variability was differences in probe sequence and thus target location. A second source of variability was the limited and variable sensitivity of the different microarray platforms for detecting weakly expressed genes, which affected interplatform and intersite reproducibility of differentially expressed genes. From this analysis, we conclude that the MAQC microarray data set has been validated by alternative quantitative gene expression platforms thus supporting the use of microarray platforms for the quantitative characterization of gene expression.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Effect of the number of transcript molecules on assay precision.
Figure 2: Analysis of assay accuracy.
Figure 3: Correlation of fold change between alternative quantitative platforms.
Figure 4: Performance of microarray platforms relative to alternative quantitative platforms.
Figure 5: Assessment of true positive rates and false discovery rates using TaqMan assays.
Figure 6: Resolution of fold-change discrepancy results.

References

  1. 1

    Vondracek, M. et al. Transcript profiling of enzymes involved in detoxification of xenobiotics and reactive oxygen in human normal and simian virus 40 T antigen-immortalized oral keratinocytes. Int. J. Cancer 99, 776–782 (2002).

    CAS  Article  Google Scholar 

  2. 2

    Urdea, M. et al. Branched DNA amplification multimers for the sensitive, direct detection of human hepatitis virus. Nucleic Acids Symp. Ser. 24, 197–200 (1991).

    CAS  Google Scholar 

  3. 3

    Gleaves, C.A. et al. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25, 205–216 (2002).

    CAS  Article  Google Scholar 

  4. 4

    Bustin, S.A. (ed.). A-Z of Quantitative PCR. (International University Line Biotechnology Series, La Jolla, California, USA, 2004).

    Google Scholar 

  5. 5

    Wong, M.L. & Medrano, J.F. Real-time PCR for mRNA quantitation. Biotechniques 39, 75–85 (2005).

    CAS  Article  Google Scholar 

  6. 6

    Lee, L.G., Connell, C.R. & Bloch, W. Allelic discrimination by nick-translation PCR with fluorogenic probes. Nucleic Acids Res. 21, 3761–3766 (1993).

    CAS  Article  Google Scholar 

  7. 7

    Heid, C.A., Stevens, J., Livak, K.J. & Williams, P.M. Real time quantitative PCR. Genome Res. 6, 986–994 (1996).

    CAS  Article  Google Scholar 

  8. 8

    Gibson, U.E., Heid, C.A. & Williams, P.M. A novel method for real time quantitative RT-PCR. Genome Res. 6, 995–1001 (1996).

    CAS  Article  Google Scholar 

  9. 9

    Qin, L.X. et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).

    Article  Google Scholar 

  10. 10

    Kuo, W.P. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).

    CAS  Article  Google Scholar 

  11. 11

    Wang, Y. et al. Large scale real-time PCR validation on gene expression measurements from two commercial long-oligonucleotide microarrays. BMC Genomics 7, 59 (2006).

    Article  Google Scholar 

  12. 12

    Willey, J.C. et al. Standardized RT-PCR and the standardized expression measurement center. Methods Mol. Biol. 258, 13–41 (2004).

    CAS  PubMed  Google Scholar 

  13. 13

    Rots, M.G. et al. mRNA expression levels of methotrexate resistance-related proteins in childhood leukemia as determined by a standardized competitive template-based RT-PCR method. Leukemia 14, 2166–2175 (2000).

    CAS  Article  Google Scholar 

  14. 14

    Mullins, D.N. et al. CEBPG transcription factor correlates with antioxidant and DNA repair genes in normal bronchial epithelial cells but not in individuals with bronchogenic carcinoma. BMC Cancer 5, 141 (2005).

    Article  Google Scholar 

  15. 15

    Flagella, M. et al. A multiplex branched DNA assay for parallel quantitative gene expression profiling. Anal. Biochem. 352, 50–60 (2006).

    CAS  Article  Google Scholar 

  16. 16

    Yao, J.D. et al. Multicenter Evaluation of the VERSANT Hepatitis B Virus DNA 3.0 Assay. J. Clin. Microbiol. 42, 800–806 (2004).

    CAS  Article  Google Scholar 

  17. 17

    Elbeik, T. et al. Multicenter Evaluation of the Performance Characteristics of the Bayer VERSANT HCV RNA 3.0 Assay (bDNA). J. Clin. Microbiol. 42, 563–569 (2004).

    CAS  Article  Google Scholar 

  18. 18

    Stenman, J. & Orpana, A. Accuracy in amplification. Nat. Biotechnol. 19, 1011–1012 (2001).

    CAS  Article  Google Scholar 

  19. 19

    Cleveland, W. Robust locally weighted regression and smoothing scatter plots. J. Am. Stat. Assoc. 74, 829–836 (1979).

    Article  Google Scholar 

  20. 20

    MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

  21. 21

    Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate - a practical and powerful approach to multiple testing. J. R. Stat. Soc. B. Met. 57, 289–300 (1995).

    Google Scholar 

  22. 22

    Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61 (2004).

    Article  Google Scholar 

  23. 23

    Livak, K.J. & Schmittgen, T.D. Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCT Method. Methods 25, 402–408 (2001).

    CAS  Article  Google Scholar 

  24. 24

    Kern, D. et al. An enhanced-sensitivity branched-DNA assay for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34, 3196–3202 (1996).

    CAS  Article  Google Scholar 

  25. 25

    Wang, J. et al. Regulation of insulin preRNA splicing by glucose. Proc. Natl Acad. Sci. USA 94, 4360–4365 (1997).

    CAS  Article  Google Scholar 

  26. 26

    Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).

    CAS  Article  Google Scholar 

  27. 27

    Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the contribution to this manuscript from the following members of the MAQC team: Shawn B. Baker, Anne Bergstrom Lucas, Jim Collins, Eugene Chudin, Stephanie Fulmer-Smentek, Damir Herman, Richard Shippy, Chunlin Xiao and Necip Mehmet.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Federico M Goodsaid.

Ethics declarations

Competing interests

J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Supplementary information

Supplementary Fig. 1

TaqMan® assays SD vs. CT plot. (DOC 31 kb)

Supplementary Fig. 2

TaqMan® assays R2 distribution plot. (DOC 166 kb)

Supplementary Fig. 3

Boxplot of fold change as a function of signal. (DOC 43 kb)

Supplementary Fig. 4

Expression characteristics of EPHA7. (DOC 91 kb)

Supplementary Fig. 5

Endogenous control expression. (DOC 204 kb)

Supplementary Table 1

Gene lists used for the analysis of performance metrics of alternative quantitative platforms and for comparison with microarrays. (DOC 51 kb)

Supplementary Table 2

Inter-site discordance in detection among genes expressed at low level. (DOC 156 kb)

Supplementary Table 3

TPR, FDR tables. (DOC 100 kb)

Supplementary Table 4

Discordant probes among the 997 genes. (DOC 36 kb)

Supplementary Table 5

Discordant gene expression values in alternative quantitative platforms FC expression values for all discordant genes. (DOC 44 kb)

Supplementary Methods (DOC 48 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Canales, R., Luo, Y., Willey, J. et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat Biotechnol 24, 1115–1122 (2006). https://doi.org/10.1038/nbt1236

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing