Abstract
We have evaluated the performance characteristics of three quantitative gene expression technologies and correlated their expression measurements to those of five commercial microarray platforms, based on the MicroArray Quality Control (MAQC) data set. The limit of detection, assay range, precision, accuracy and fold-change correlations were assessed for 997 TaqMan Gene Expression Assays, 205 Standardized RT (Sta)RT-PCR assays and 244 QuantiGene assays. TaqMan is a registered trademark of Roche Molecular Systems, Inc. We observed high correlation between quantitative gene expression values and microarray platform results and found few discordant measurements among all platforms. The main cause of variability was differences in probe sequence and thus target location. A second source of variability was the limited and variable sensitivity of the different microarray platforms for detecting weakly expressed genes, which affected interplatform and intersite reproducibility of differentially expressed genes. From this analysis, we conclude that the MAQC microarray data set has been validated by alternative quantitative gene expression platforms thus supporting the use of microarray platforms for the quantitative characterization of gene expression.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Vondracek, M. et al. Transcript profiling of enzymes involved in detoxification of xenobiotics and reactive oxygen in human normal and simian virus 40 T antigen-immortalized oral keratinocytes. Int. J. Cancer 99, 776–782 (2002).
Urdea, M. et al. Branched DNA amplification multimers for the sensitive, direct detection of human hepatitis virus. Nucleic Acids Symp. Ser. 24, 197–200 (1991).
Gleaves, C.A. et al. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25, 205–216 (2002).
Bustin, S.A. (ed.). A-Z of Quantitative PCR. (International University Line Biotechnology Series, La Jolla, California, USA, 2004).
Wong, M.L. & Medrano, J.F. Real-time PCR for mRNA quantitation. Biotechniques 39, 75–85 (2005).
Lee, L.G., Connell, C.R. & Bloch, W. Allelic discrimination by nick-translation PCR with fluorogenic probes. Nucleic Acids Res. 21, 3761–3766 (1993).
Heid, C.A., Stevens, J., Livak, K.J. & Williams, P.M. Real time quantitative PCR. Genome Res. 6, 986–994 (1996).
Gibson, U.E., Heid, C.A. & Williams, P.M. A novel method for real time quantitative RT-PCR. Genome Res. 6, 995–1001 (1996).
Qin, L.X. et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).
Kuo, W.P. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).
Wang, Y. et al. Large scale real-time PCR validation on gene expression measurements from two commercial long-oligonucleotide microarrays. BMC Genomics 7, 59 (2006).
Willey, J.C. et al. Standardized RT-PCR and the standardized expression measurement center. Methods Mol. Biol. 258, 13–41 (2004).
Rots, M.G. et al. mRNA expression levels of methotrexate resistance-related proteins in childhood leukemia as determined by a standardized competitive template-based RT-PCR method. Leukemia 14, 2166–2175 (2000).
Mullins, D.N. et al. CEBPG transcription factor correlates with antioxidant and DNA repair genes in normal bronchial epithelial cells but not in individuals with bronchogenic carcinoma. BMC Cancer 5, 141 (2005).
Flagella, M. et al. A multiplex branched DNA assay for parallel quantitative gene expression profiling. Anal. Biochem. 352, 50–60 (2006).
Yao, J.D. et al. Multicenter Evaluation of the VERSANT Hepatitis B Virus DNA 3.0 Assay. J. Clin. Microbiol. 42, 800–806 (2004).
Elbeik, T. et al. Multicenter Evaluation of the Performance Characteristics of the Bayer VERSANT HCV RNA 3.0 Assay (bDNA). J. Clin. Microbiol. 42, 563–569 (2004).
Stenman, J. & Orpana, A. Accuracy in amplification. Nat. Biotechnol. 19, 1011–1012 (2001).
Cleveland, W. Robust locally weighted regression and smoothing scatter plots. J. Am. Stat. Assoc. 74, 829–836 (1979).
MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).
Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate - a practical and powerful approach to multiple testing. J. R. Stat. Soc. B. Met. 57, 289–300 (1995).
Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61 (2004).
Livak, K.J. & Schmittgen, T.D. Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCT Method. Methods 25, 402–408 (2001).
Kern, D. et al. An enhanced-sensitivity branched-DNA assay for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34, 3196–3202 (1996).
Wang, J. et al. Regulation of insulin preRNA splicing by glucose. Proc. Natl Acad. Sci. USA 94, 4360–4365 (1997).
Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).
Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).
Acknowledgements
We would like to acknowledge the contribution to this manuscript from the following members of the MAQC team: Shawn B. Baker, Anne Bergstrom Lucas, Jim Collins, Eugene Chudin, Stephanie Fulmer-Smentek, Damir Herman, Richard Shippy, Chunlin Xiao and Necip Mehmet.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.
Supplementary information
Supplementary Fig. 1
TaqMan® assays SD vs. CT plot. (DOC 31 kb)
Supplementary Fig. 2
TaqMan® assays R2 distribution plot. (DOC 166 kb)
Supplementary Fig. 3
Boxplot of fold change as a function of signal. (DOC 43 kb)
Supplementary Fig. 4
Expression characteristics of EPHA7. (DOC 91 kb)
Supplementary Fig. 5
Endogenous control expression. (DOC 204 kb)
Supplementary Table 1
Gene lists used for the analysis of performance metrics of alternative quantitative platforms and for comparison with microarrays. (DOC 51 kb)
Supplementary Table 2
Inter-site discordance in detection among genes expressed at low level. (DOC 156 kb)
Supplementary Table 3
TPR, FDR tables. (DOC 100 kb)
Supplementary Table 4
Discordant probes among the 997 genes. (DOC 36 kb)
Supplementary Table 5
Discordant gene expression values in alternative quantitative platforms FC expression values for all discordant genes. (DOC 44 kb)
Rights and permissions
About this article
Cite this article
Canales, R., Luo, Y., Willey, J. et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat Biotechnol 24, 1115–1122 (2006). https://doi.org/10.1038/nbt1236
Published:
Issue Date:
DOI: https://doi.org/10.1038/nbt1236
This article is cited by
-
The Sequencing Quality Control 2 study: establishing community standards for sequencing in precision medicine
Genome Biology (2021)
-
Tutorial: using nanoneedles for intracellular delivery
Nature Protocols (2021)
-
The heat shock response, determined by QuantiGene multiplex, is impaired in HD mouse models and not caused by HSF1 reduction
Scientific Reports (2021)
-
Systematic comparison and assessment of RNA-seq procedures for gene expression quantitative analysis
Scientific Reports (2020)
-
aFold – using polynomial uncertainty modelling for differential gene expression estimation from RNA sequencing data
BMC Genomics (2019)