Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements

Abstract

Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Repeatability of expression signal within test sites.
Figure 2: Signal variation within and between test sites.
Figure 3: Concordance of detection calls within and between test sites.
Figure 4: Agreement of gene lists.
Figure 5: Agreement of log ratios across platforms and test sites.
Figure 6: Correlation between microarray and TaqMan data.

Accession codes

Accessions

Gene Expression Omnibus

References

  1. 1

    Lesko, L.J. & Woodcock, J. Translation of pharmacogenomics and pharmacogenetics: a regulatory perspective. Nat. Rev. Drug Discov. 3, 763–769 (2004).

    CAS  Article  Google Scholar 

  2. 2

    Frueh, F.W. Impact of microarray data quality on genomic data submissions to the FDA. Nat. Biotechnol. 24, 1105–1107 (2006).

    CAS  Article  Google Scholar 

  3. 3

    Dix, D.J. et al. A framework for the use of genomics data at the EPA. Nat. Biotechnol. 24, 1108–1111 (2006).

    CAS  Article  Google Scholar 

  4. 4

    Tan, P.K. et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 31, 5676–5684 (2003).

    CAS  Article  Google Scholar 

  5. 5

    Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R.C. & Melton, D.A. “Stemness”: transcriptional profiling of embryonic and adult stem cells. Science 298, 597–600 (2002).

    CAS  Article  Google Scholar 

  6. 6

    Ivanova, N.B. et al. A stem cell molecular signature. Science 298, 601–604 (2002).

    CAS  Article  Google Scholar 

  7. 7

    Miller, R.M. et al. Dysregulation of gene expression in the 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine-lesioned mouse substantia nigra. J. Neurosci. 24, 7445–7454 (2004).

    CAS  Article  Google Scholar 

  8. 8

    Fortunel, N.O. et al. Comment on “'Stemness': transcriptional profiling of embryonic and adult stem cells” and “a stem cell molecular signature”. Science 302, 393 author reply 393 (2003).

  9. 9

    Miklos, G.L. & Maleszka, R. Microarray reality checks in the context of a complex disease. Nat. Biotechnol. 22, 615–621 (2004).

    CAS  Article  Google Scholar 

  10. 10

    Frantz, S. An array of problems. Nat. Rev. Drug Discov. 4, 362–363 (2005).

    CAS  Article  Google Scholar 

  11. 11

    Marshall, E. Getting the noise out of gene arrays. Science 306, 630–631 (2004).

    CAS  Article  Google Scholar 

  12. 12

    Michiels, S., Koscielny, S. & Hill, C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet 365, 488–492 (2005).

    CAS  Article  Google Scholar 

  13. 13

    Ein-Dor, L., Zuk, O. & Domany, E. Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proc. Natl. Acad. Sci. USA 103, 5923–5928 (2006).

    CAS  Article  Google Scholar 

  14. 14

    Petersen, D. et al. Three microarray platforms: an analysis of their concordance in profiling gene expression. BMC Genomics 6, 63 (2005).

    Article  Google Scholar 

  15. 15

    Dobbin, K.K. et al. Interlaboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clin. Cancer Res. 11, 565–572 (2005).

    CAS  PubMed  Google Scholar 

  16. 16

    Irizarry, R.A. et al. Multiple-laboratory comparison of microarray platforms. Nat. Methods 2, 345–350 (2005).

    CAS  Article  Google Scholar 

  17. 17

    Larkin, J.E., Frank, B.C., Gavras, H., Sultana, R. & Quackenbush, J. Independence and reproducibility across microarray platforms. Nat. Methods 2, 337–344 (2005).

    CAS  Article  Google Scholar 

  18. 18

    Kuo, W.P. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).

    CAS  Article  Google Scholar 

  19. 19

    Shi, L. et al. QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev. Mol. Diagn. 4, 761–777 (2004).

    Article  Google Scholar 

  20. 20

    Shi, L. et al. Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 6 Suppl. 2, S12 (2005).

    Article  Google Scholar 

  21. 21

    Ji, H. & Davis, R.W. Data quality in genomics and microarrays. Nat. Biotechnol. 24, 1112–1113 (2006).

    CAS  Article  Google Scholar 

  22. 22

    Canales, R.D. et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat. Biotechnol. 24, 1115–1122 (2006).

    CAS  Article  Google Scholar 

  23. 23

    Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).

    CAS  Article  Google Scholar 

  24. 24

    Patterson, T.A. et al. Performance comparison of one-color and two-color platforms within the MicroArray Quality Control (MAQC) project. Nat. Biotechnol. 24, 1140–1150 (2006).

    CAS  Article  Google Scholar 

  25. 25

    Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).

    CAS  Article  Google Scholar 

  26. 26

    Guo, L. et al. Rat toxicogenomic study reveals analytical consistency across microarray platforms. Nat. Biotechnol. 24, 1162–1169 (2006).

    CAS  Article  Google Scholar 

  27. 27

    Mecham, B.H. et al. Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res. 32, e74 (2004).

    Article  Google Scholar 

  28. 28

    Carter, S.L., Eklund, A.C., Mecham, B.H., Kohane, I.S. & Szallasi, Z. Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics 6, 107 (2005).

    Article  Google Scholar 

  29. 29

    Draghici, S., Khatri, P., Eklund, A.C. & Szallasi, Z. Reliability and reproducibility issues in DNA microarray measurements. Trends Genet. 22, 101–109 (2006).

    CAS  Article  Google Scholar 

  30. 30

    Irizarry, R.A., Wu, Z. & Jaffee, H.A. Comparison of Affymetrix GeneChip expression measures. Bioinformatics 22, 789–794 (2006).

    CAS  Article  Google Scholar 

  31. 31

    Pruitt, K.D., Tatusova, T. & Maglott, D.R. NCBI Reference Sequence (RefSeq): a curated non-redundant sequence database of genomes, transcripts and proteins. Nucleic Acids Res. 33, D501–D504 (2005).

    CAS  Article  Google Scholar 

  32. 32

    Thierry-Mieg, D. & J, T.-M. AceView: a comprehensive cDNA-supported gene and transcripts annotation. Genome Biology 7, Suppl. 1, S12 (2006).

    Article  Google Scholar 

  33. 33

    Bammler, T. et al. Standardizing global gene expression analysis between laboratories and across platforms. Nat. Methods 2, 351–356 (2005).

    Article  Google Scholar 

  34. 34

    Harr, B. & Schlotterer, C. Comparison of algorithms for the analysis of Affymetrix microarray data as evaluated by co-expression of genes in known operons. Nucleic Acids Res. 34, e8 (2006).

    Article  Google Scholar 

  35. 35

    Tusher, V.G., Tibshirani, R. & Chu, G. Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. USA 98, 5116–5121 (2001).

    CAS  Article  Google Scholar 

  36. 36

    Thompson, K.L. et al. Use of a mixed tissue RNA design for performance assessments on multiple microarray formats. Nucleic Acids Res. 33, e187 (2005).

    Article  Google Scholar 

  37. 37

    Baker, S.C. et al. The External RNA Controls Consortium: a progress report. Nat. Methods 2, 731–734 (2005).

    CAS  Article  Google Scholar 

  38. 38

    Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).

    Google Scholar 

  39. 39

    Ball, C.A. et al. Standards for microarray data. Science 298, 539 (2002).

    CAS  Article  Google Scholar 

  40. 40

    Tong, W. et al. ArrayTrack–supporting toxicogenomic research at the U.S. Food and Drug Administration National Center for Toxicological Research. Environ. Health Perspect. 111, 1819–1826 (2003).

    CAS  Article  Google Scholar 

  41. 41

    Tong, W. et al. Development of public toxicogenomics software for microarray data management and analysis. Mutat. Res. 549, 241–253 (2004).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

All MAQC participants freely donated their time and reagents for the completion and analysis of the MAQC project. Participants from the National Institutes of Health (NIH) were supported by the Intramural Research Program of NIH, Bethesda, Maryland. D.H. thanks Ian Korf for BLAST discussions. This study utilized a number of computing resources, including the high-performance computational capabilities of the Biowulf PC/Linux cluster at the NIH (http://biowulf.nih.gov/) as well as resources at the analysis sites.

Author information

Affiliations

Consortia

Corresponding author

Correspondence to Leming Shi.

Ethics declarations

Competing interests

Many of the MAQC participants are employed by companies that manufacture gene expression products and/or perform testing services. J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Supplementary information

Supplementary Table 1

cRNA Size and Yields Collected from Microarray Platforms (XLS 45 kb)

Supplementary Table 2

Genes-to-NM Accession List for All Probes (TXT 2629 kb)

Supplementary Table 3

Most 3′ Probe-to-NM Accession List for 23,971 Common Probes (TXT 3272 kb)

Supplementary Table 4

One Probe-to-One Gene List for All Genes (TXT 1718 kb)

Supplementary Table 5

One Probe-to-One Gene List for 12,091 Common Genes (TXT 1211 kb)

Supplementary Methods

BLAST Analysis for the MAQC Project (PDF 67 kb)

Supplementary Notes

README for Probe Mapping Files (PDF 11 kb)

Supplementary Data

Supplementary Information on MAQC Study (PDF 663 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

MAQC Consortium., Shi, L., Shi, L. et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 24, 1151–1161 (2006). https://doi.org/10.1038/nbt1239

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing