Article | Published:

A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies

Abstract

Over the last decade, gene expression microarrays have had a profound impact on biomedical research. The diversity of platforms and analytical methods available to researchers have made the comparison of data from multiple platforms challenging. In this study, we describe a framework for comparisons across platforms and laboratories. We have attempted to include nearly all the available commercial and 'in-house' platforms. Using probe sequences matched at the exon level improved consistency of measurements across the different microarray platforms compared to annotation-based matches. Generally, consistency was good for highly expressed genes, and variable for genes with lower expression values as confirmed by quantitative real-time (QRT)-PCR. Concordance of measurements was higher between laboratories on the same platform than across platforms. We demonstrate that, after stringent preprocessing, commercial arrays were more consistent than in-house arrays, and by most measures, one-dye platforms were more consistent than two-dye platforms.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Accession codes

Accessions

Gene Expression Omnibus

References

  1. 1

    Edgar, R., Domrachev, M. & Lash, A.E. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res. 30, 207–210 (2002).

  2. 2

    Brazma, A. et al. ArrayExpress–a public repository for microarray gene expression data at the EBI. Nucleic Acids Res. 31, 68–71 (2003).

  3. 3

    Ali-Seyed, M. et al. Cross-platform expression profiling demonstrates that SV40 small tumor antigen activates Notch, Hedgehog, and Wnt signaling in human cells. BMC Cancer 6, 54–68 (2006).

  4. 4

    Bammler, T. et al. Standardizing global gene expression analysis between laboratories and across platforms. Nat. Methods 2, 351–356 (2005).

  5. 5

    Barczak, A. et al. Spotted long oligonucleotide arrays for human gene expression analysis. Genome Res. 13, 1775–1785 (2003).

  6. 6

    Barnes, M., Freudenberg, J., Thompson, S., Aronow, B. & Pavlidis, P. Experimental comparison and cross-validation of the Affymetrix and Illumina gene expression analysis platforms. Nucleic Acids Res. 33, 5914–5923 (2005).

  7. 7

    Irizarry, R.A. et al. Multiple-laboratory comparison of microarray platforms. Nat. Methods 2, 345–350 (2005).

  8. 8

    Kothapalli, R., Yoder, S.J., Mane, S. & Loughran, T.P., Jr. Microarray results: how accurate are they? BMC Bioinformatics 3, 22–32 (2002).

  9. 9

    Kuo, W.P., Jenssen, T.K., Butte, A.J., Ohno-Machado, L. & Kohane, I.S. Analysis of matched mRNA measurements from two different microarray technologies. Bioinformatics 18, 405–412 (2002).

  10. 10

    Larkin, J.E., Frank, B.C., Gavras, H., Sultana, R. & Quackenbush, J. Independence and reproducibility across microarray platforms. Nat. Methods 2, 337–344 (2005).

  11. 11

    Lee, J.K. et al. Comparing cDNA and oligonucleotide array data: concordance of gene expression across platforms for the NCI-60 cancer cells. Genome Biol. 4, R82–94 (2003).

  12. 12

    Li, J., Pankratz, M. & Johnson, J.A. Differential gene expression patterns revealed by oligonucleotide versus long cDNA arrays. Toxicol. Sci. 69, 383–390 (2002).

  13. 13

    Mecham, B.H. et al. Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res. 32, e74–82 (2004).

  14. 14

    Park, P.J. et al. Current issues for DNA microarrays: platform comparison, double linear amplification, and universal RNA reference. J. Biotechnol. 112, 225–245 (2004).

  15. 15

    Parrish, M.L. et al. A microarray platform comparison for neuroscience applications. J. Neurosci. Methods 132, 57–68 (2004).

  16. 16

    Petersen, D. et al. Three microarray platforms: an analysis of their concordance in profiling gene expression. BMC Genomics 6, 63–77 (2005).

  17. 17

    Pylatuik, J.D. & Fobert, P.R. Comparison of transcript profiling on Arabidopsis microarray platform technologies. Plant Mol. Biol. 58, 609–624 (2005).

  18. 18

    Rogojina, A.T., Orr, W.E., Song, B.K. & Geisert, E.E., Jr. Comparing the use of Affymetrix to spotted oligonucleotide microarrays using two retinal pigment epithelium cell lines. Mol. Vis. 9, 482–496 (2003).

  19. 19

    Schlingemann, J. et al. Patient-based cross-platform comparison of oligonucleotide microarray expression profiles. Lab. Invest. 85, 1024–1039 (2005).

  20. 20

    Shi, L. et al. Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 6 suppl. Suppl. 2, S12–S26 (2005).

  21. 21

    Tan, P.K. et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 31, 5676–5684 (2003).

  22. 22

    Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61–76 (2004).

  23. 23

    Walker, S.J., Wang, Y., Grant, K.A., Chan, F. & Hellmann, G.M. Long versus short oligonucleotide microarrays for the study of gene expression in nonhuman primates. J. Neurosci. Methods 152, 179–189 (2005).

  24. 24

    Wang, H., He, X., Band, M., Wilson, C. & Liu, L. A study of inter-lab and inter-platform agreement of DNA microarray data. BMC Genomics 6, 71–80 (2005).

  25. 25

    Wang, H.Y. et al. Assessing unmodified 70-mer oligonucleotide probe performance on glass-slide microarrays. Genome Biol. 4, R5–R18 (2003).

  26. 26

    Warnat, P., Eils, R. & Brors, B. Cross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes. BMC Bioinformatics 6, 265–280 (2005).

  27. 27

    Woo, Y. et al. A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J. Biomol. Tech. 15, 276–284 (2004).

  28. 28

    Yauk, C.L., Berndt, M.L., Williams, A. & Douglas, G.R. Comprehensive comparison of six microarray technologies. Nucleic Acids Res. 32, e124–e131 (2004).

  29. 29

    Yuen, T., Wurmbach, E., Pfeffer, R.L., Ebersole, B.J. & Sealfon, S.C. Accuracy and calibration of commercial oligonucleotide and custom cDNA microarrays. Nucleic Acids Res. 30, e48–e57 (2002).

  30. 30

    Zhu, B., Ping, G., Shinohara, Y., Zhang, Y. & Baba, Y. Comparison of gene expression measurements from cDNA and 60-mer oligonucleotide microarrays. Genomics 85, 657–665 (2005).

  31. 31

    Sherlock, G. Of fish and chips. Nat. Methods 2, 329–330 (2005).

  32. 32

    Lee, M.L., Kuo, F.C., Whitmore, G.A. & Sklar, J. Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc. Natl. Acad. Sci. USA 97, 9834–9839 (2000).

  33. 33

    Mecham, B.H. et al. Increased measurement accuracy for sequence-verified microarray probes. Physiol. Genomics 18, 308–315 (2004).

  34. 34

    Carter, S.L., Eklund, A.C., Mecham, B.H., Kohane, I.S. & Szallasi, Z. Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics 6, 107–122 (2005).

  35. 35

    Blackshaw, S., Fraioli, R.E., Furukawa, T. & Cepko, C.L. Comprehensive analysis of photoreceptor gene expression and the identification of candidate retinal disease genes. Cell 107, 579–589 (2001).

  36. 36

    Blackshaw, S. et al. Genomic analysis of mouse retinal development. PLoS Biol. 2, E247–E268 (2004).

  37. 37

    Velculescu, V.E., Zhang, L., Vogelstein, B. & Kinzler, K.W. Serial analysis of gene expression. Science 270, 484–487 (1995).

  38. 38

    Brenner, S. et al. Gene expression analysis by massively parallel signature sequencing (MPSS) on microbead arrays. Nat. Biotechnol. 18, 630–634 (2000).

  39. 39

    Pounds, S. & Cheng, C. Statistical development and evaluation of microarray gene expression data filters. J. Comput. Biol. 12, 482–495 (2005).

  40. 40

    Chu, T.M., Deng, S., Wolfinger, R., Paules, R.S. & Hamadeh, H.K. Cross-site comparison of gene expression data reveals high similarity. Environ. Health Perspect. 112, 449–455 (2004).

  41. 41

    Qin, L.X. et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).

  42. 42

    Roth, M.E. et al. Expression profiling using a hexamer-based universal microarray. Nat. Biotechnol. 22, 418–426 (2004).

  43. 43

    Gunderson, K.L. et al. Decoding randomly ordered DNA arrays. Genome Res. 14, 870–877 (2004).

  44. 44

    Workman, C. et al. A new non-linear normalization method for reducing variability in DNA microarray experiments. Genome Biol 3, research0048 (2002).

  45. 45

    Berger, J.A. et al. Optimized LOWESS normalization parameter selection for DNA microarray data. BMC Bioinformatics 5, 194–207 (2004).

  46. 46

    Bolstad, B.M., Irizarry, R.A., Astrand, M. & Speed, T.P. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 19, 185–193 (2003).

  47. 47

    Bussey, K.J. et al. MatchMiner: a tool for batch navigation among gene and gene product identifiers. Genome Biol. 4, R27–34 (2003).

  48. 48

    Kent, W.J. BLAT–the BLAST-like alignment tool. Genome Res. 12, 656–664 (2002).

  49. 49

    Liu, G. et al. NetAffx: Affymetrix probesets and annotations. Nucleic Acids Res. 31, 82–86 (2003).

  50. 50

    Gentleman, R.C. et al. Bioconductor: open software development for computational biology and bioinformatics. Genome Biol. 5, R80–R96 (2004).

Download references

Acknowledgements

We would like to thank vendors, Applied Biosystems, GE Healthcare, and Mergen for providing and running microarrays as part of this large-scale evaluation. In addition, we would like to thank Applied Biosystems for running TaqMan assays and Exiqon for supplying us with the ProbeLibrary kit as well as Roche Diagnostics for allowing us to use their 480 LightCycler. We thank Robert A. Greenes for reviewing the manuscript. W.P.K. was supported by the National Institutes of Health (NIH) EY014466 grant and by the Bioinformatics Division of the Harvard Center for Neurodegeneration and Repair. C.L.C. was supported by the Howard Hughes Medical Institute. F.L. and E.H. were supported by the functional genomics program (FUGE) in the Research council of Norway. G.M.C. was supported by NIH-NHGRI-CEGS. M.W.F., B.S. and G.F.S. were supported by Programs for Genomic Applications grants HL66678 and HL72358. R.B. was supported by NIH grants HL072370 and ES011387.

Author information

Correspondence to Winston Patrick Kuo.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Assessment of cross-platform data agreement using CAT plots. (PDF 760 kb)

Supplementary Fig. 2

Assessment of cross-laboratory data agreement using PCA. (PDF 471 kb)

Supplementary Table 1

Evaluation of data consistency within platforms. (PDF 106 kb)

Supplementary Table 2

Evaluation of data consistency across platforms. (PDF 145 kb)

Supplementary Table 3

QRT-PCR primer sequences and validation results. (PDF 154 kb)

Supplementary Data 1

Study design and experimental protocols of microarray platforms and QRT-PCR technologies. (PDF 150 kb)

Rights and permissions

Reprints and Permissions

About this article

Further reading

Figure 1: Cross-platform agreement of probes matched within one exon.
Figure 2: Cross-platform PCA plot.
Figure 3: Scatter plot of QRT-PCR versus all microarrays.