Main

The College of American Pathologists (CAP)/American College of Medical Genetics (ACMG) Cytogenetics Resource Committee (CyRC) began a pilot study of proficiency testing for constitutional cytogenomic microarray analysis in 2007. The Survey, named “CYCGH” and currently titled “Microarray-Genomic Copy Number Assay Survey” was designed to include all platforms (i.e., array CGH and single nucleotide polymorphism) that may be used to evaluate genomic copy number changes by microarray. During these 3 years, results obtained through the Survey demonstrate significant growth in both the technological aspects of the assay and its utilization by the clinical community. We present a description of the proficiency test (PT), a summary of results from participating laboratories, and a collation of answers that were provided to supplemental questions included in the PT Surveys. These PT data constitute one of the largest studies of interlaboratory reproducibility available using this technology and provide insight into the evolving interpretative and reporting practices within the cytogenetic and molecular genetic communities. Results from the PT showed a high level of concordance for the identification of clinically significant copy number changes among all laboratories across all array platforms.

The introduction of microarray assays for whole genome analysis of copy number changes has clearly been one of the most significant advances ever for the field of clinical cytogenetics.1,2 Historically, the discovery of the human diploid chromosome number, the advent of banding techniques for the identification of individual chromosomes and chromosome segments, and the use of fluorescence in situ hybridization technology for visualizing specific DNA sequences along chromosomes and in interphase nuclei have provided a wealth of new discoveries in clinical genetics.3 As each of these technologies moved into routine clinical diagnostics, proficiency testing programs were developed as a means for laboratories to gauge their technical and interpretive performance against that of their peers, and requirements for PT were established by various governmental and accrediting agencies. The recent rapidly expanding clinical use of cytogenomic microarray testing in diagnostic laboratories warranted the development of an array-based PT for the genetics community.

The goal of the CyRC was to create a PT that would examine how laboratories process, analyze, and interpret data from genomic microarray studies for constitutional abnormalities. Before offering this test as an official CAP Survey, the committee developed a pilot test and distributed it to seven volunteer laboratories that routinely performed this testing on a clinical basis. The purpose of the pilot test was to (1) determine the feasibility of administering a cytogenomic microarray PT; (2) evaluate potential vendors as a source of DNA for the actual PT; (3) develop an acceptable test result form for data entry by participants; and (4) solicit input from participants regarding the design and implementation of the pilot test. After a successful and informative pilot test in 2007, the first PT Survey for CGH microarray was offered in early 2008. The main aim of the PT was to provide a valid assessment of the reproducibility and concordance of results among laboratories. Each Survey consisted of two prepared DNA samples to be analyzed using the laboratory's routine methods for assessing copy number changes. Participants were asked to report only copy number changes of known clinical significance using current nomenclature.4,5 Since the initiation of this PT, 2 Surveys per year have been distributed, for a total of 5 surveys and 10 different challenges as of this writing. A summary of responses from participants in those Surveys will be presented.

MATERIALS AND METHODS

Specimens

Briefly, after institutional review board approval by the laboratory of origin, peripheral blood specimens from patients previously determined to have a clinically significant copy number abnormality were immortalized by Epstein-Barr virus transformation. The karyotype of the transformed cells was evaluated to eliminate cases with acquired cytogenetic changes, and DNA from the immortalized cells was evaluated by cytogenomic array to eliminate cases with acquired submicroscopic cytogenetic changes. Consistent with the tendency for immortalized cell lines to be monoclonal or oligoclonal, deletions representing the expected rearrangement of immunoglobulin genes (IGH@, IGK@, and IGL@) were observed in most of the cell lines tested; no cell line was rejected for this reason. Cell lines were deemed acceptable if they showed a clinically significant copy number abnormality. These lines were expanded, and sufficient DNA was extracted to distribute to the laboratories subscribing to the Survey. DNA specimens for the two Survey cases with a normal array result were isolated from whole blood. This DNA was analyzed at one or more laboratories under the direction of CyRC committee members using the array platform in clinical use in each of the respective laboratories. If the DNA was deemed acceptable by the CyRC committee members (i.e., relatively low background noise and consensus for a given abnormality or lack thereof), it was distributed to Survey subscribers via the CAP.

Survey materials

Distribution of surveys

Surveys were mailed twice per year (in May and October). Participants were given approximately 4 weeks to complete the analyses and return the response forms. The CyRC reviewed all responses within a few weeks of receipt and generated a participant summary report shortly thereafter.

Evaluation criteria

One of the challenges in developing this Survey was to design a Test Result Form that would encompass all potential answers. When we initiated the Survey, we anticipated that continued rapid developments in the field were likely, and thus we attempted to prepare a form that would be both easy for participants to use and unambiguous for the committee to interpret and grade. As the aim of each challenge was for the participants to identify clinically significant copy number alterations in the DNA provided and distinguish them from likely benign copy number variants, we designed a result form permitting responses for both gains and losses on each chromosome arm. In addition, as with other CAP/ACMG Cytogenetic Surveys, we asked that current International System for Cytogenetic Nomenclature be used to describe any abnormalities detected. Distinguishing clinically significant changes from known benign copy number alterations is imperative in clinical practice, and that goal was also reflected in the Surveys: participants were asked to report only those alterations of known clinical significance. Before selection for use in the Survey, DNA from specimens was ascertained to include no more than two clinically significant abnormalities using currently established criteria. These criteria are generally accepted in the literature to include evidence-based association of a copy number alteration in a known genomic region with an abnormal phenotype.1,6–9

Goals of the survey

The primary goal of the Survey was to evaluate individual laboratories for overall proficiency in processing, evaluation, and interpretation of genomic copy number changes. The CyRC also solicited feedback from participants by including supplemental questions with each mailing. In the short time that this Survey has been distributed, it has become apparent that the field has evolved significantly: array platforms and probe densities have changed, and consensus is emerging on which copy number changes are of clinical significance.

RESULTS

All the laboratories participating in the pilot Survey obtained and reported a result for each sample and identified the abnormalities targeted in the Survey. An acceptable vendor was identified for the preparation and distribution of DNA samples, and the Test Result Form was deemed appropriate for evaluating the responses entered by the testing laboratories; the pilot Survey was therefore considered successful. All pilot respondents indicated that they were satisfied with the Survey, and their suggestions about DNA quality, notation for including International System for Cytogenetic Nomenclature designations, and formatting of the response form were considered as the CyRC prepared the first Survey to be offered to the broader scientific community.

A summary of all responses to date for the Surveys is shown in Table 1. A total of 10 different challenges have been distributed since 2008. Per CAP standards, results were evaluated according to a consensus response, defined as that reported by at least 80% of participants. Overall, approximately 96% of all challenge responses were concordant for an abnormality involving a particular chromosomal region. Most individual responses were at or near complete (100%) agreement. The Survey was formally graded for the first time with the initial 2010 mailing.

Table 1 Summary of abnormality and participant responses for all CYCGH surveys
Full size table

DNA specimens used for these Surveys included abnormalities ranging from duplicated or deleted regions visible by G-banding (such as complete trisomy for chromosome 13) to submicroscopic gains and losses (Table 1). There was an approximately 0.8 Mb copy number change in 22q11.22 (∼20.7–21.5 Mb, hg18) in specimen 04 from the 2008 Survey. Seven of 33 participant laboratories included that copy number change in their list of clinically significant abnormalities. Copy number changes that overlap this entire region are present in multiple studies of phenotypically normal individuals, as indicated in the Database of Genomic Variants (http://projects.tcag.ca/variation/). Important aspects of distinguishing benign from clinically significant copy number changes were discussed in the Participant Summary Report from that Survey and are discussed below in more detail. Errors in reporting results, which were few, included designating an opposite change (gain versus loss or loss versus gain), reporting a copy number change that was not clinically significant, or missing a clinically significant abnormality (and reporting a normal result). Complete responses from all participants, including their reported nomenclature, can be seen in the Tables, Supplemental Digital Content 1, http://links.lww.com/GIM/A181.

The answers to supplemental questions in each Survey help document evolutionary changes in clinical laboratory practice. Changes in the number of laboratories offering microarray assays for diagnostics, in the types of reagents used, and in the interpretation of results are indicated in these responses. First, the number of laboratories subscribing to this Survey has steadily increased from 28 participants in 2008 to 74 participants in 2010. This reflects the growing use of cytogenomic microarrays in clinical practice and more widespread adoption of testing platforms. Second, the percentage of laboratories using bacterial artificial chromosome-based platforms has dropped from approximately 43 to 5%, and those using oligonucleotide and SNP arrays have increased from 50 to 72% and 11 to 23%, respectively (Fig. 1). Third, the type of control DNAs used continues to vary. The number of laboratories reporting that they use sex-matched control DNA has increased from 50% to approximately 79%, with a concomitant decrease in the use of sex-mismatched controls from 50 to 21%; this has been reported to improve the sensitivity of detection of sex chromosome abnormalities.10 Of those participants who responded, there did not seem to be a change in the use of DNAs from a pool of individuals (∼65%) to laboratories using DNA from only one individual (∼35%). There was a slight increase in the number of laboratories reporting that they offer microarray testing for prenatal and oncology specimens, with 28% and 12%, respectively, currently offering each test.

Fig. 1
figure 1

The types of array platforms used by participants evolved with changing laboratory practices through the course of the proficiency testing period. The x-axis shows the individual mailing for each Survey year (A = first and B = second) and N = the number of participants subscribed to the respective Survey. The y-axis shows the percentage of those laboratories using respective array platforms. Light gray lines represent bacterial artificial chromosome platforms, black lines represent oligonucleotide platforms, and gray lines represent SNP platforms.

Full size image

Laboratories reporting the use of analysis software provided by the same vendor supplying the arrays have decreased from approximately 89–72%, whereas the use of laboratory-designed software has increased from 7 to 24%. Those using software obtained from a different vendor than that which supplies the arrays remained stable at approximately 4% of participants. Almost all laboratories refer to the Database of Genomic Variants (http://projects.tcag.ca/variation/) when interpreting the clinical significance of copy number alterations, and approximately 60% use the DECIPHER database (http://decipher.sanger.ac.uk/); the number of laboratories using internal databases increased from 46 to 78%.

In the last two Surveys, we asked for more specific information regarding recognition of benign copy number alterations. The number of benign copy number changes reported by participants ranged from 0 to 36 per specimen, and details of the large number of individual responses were included in the Participant Summary Reports through CAP for the respective Surveys.

Responses from the supplemental questions indicated that approximately 81% of array testing is performed in cytogenetics laboratories and approximately 15% in molecular genetics laboratories. Four percent of responses indicated that array testing was performed in an “other” category. Finally, approximately 84% of directors who sign array reports are American Board of Medical Genetics (ABMG)-certified in Clinical Cytogenetics, and approximately 40% are ABMG-certified in Clinical Molecular Genetics (some are certified in both Clinical Cytogenetics and Molecular Genetics but information regarding the number of dual-certified individuals was not available); 1.4% of directors note that they are board certified by the American Board of Pathology/ABMG in Molecular Genetic Pathology, 11 and 1.4% are boarded by the Canadian Board of Medical Genetics and Molecular Genetics, respectively, and 9.5% noted “other” relating to director certification.

DISCUSSION

As the use of cytogenomic microarray technology for constitutional diagnoses becomes more prevalent, geneticists interpreting results have faced a degree of uncertainty in the significance of some copy number alterations. Of primary concern is whether a change seen in a patient is clinically relevant, benign, or of unknown significance. To address this uncertainty, initial guidelines for processing and evaluating cytogenomic microarray data have been published.6,7 The status of inheritance, size of copy number alteration, gene content in the affected region, reported association with syndromic conditions, and whether the finding is described in benign copy number databases are all factors that need to be considered when interpreting the clinical significance of a copy number change that is discovered in a cytogenomic microarray.1 Additional guidelines developed by the ACMG for copy number variant interpretation have been recently published.9

For CYCGH Surveys, the CyRC intentionally chose specimens previously interpreted as having known clinically significant abnormalities or specimens previously interpreted as normal. An internal quality control in selecting Survey challenges involves pretesting of Survey materials in committee members' laboratories to assure that the DNA quality is acceptable. In addition, the specimens are tested on a variety of array platforms likely to be representative of those used in respondents' testing laboratories. The CyRC has focused on DNA copy number abnormalities that are likely to be identified by all array platforms in clinical use. It is the intent of the committee to select cases that will have a high likelihood of achieving a consensus result yet remain challenging to the participants.

To our knowledge, this is the largest study of interlaboratory reproducibility for cytogenomic array-based testing and provides valuable information to the diagnostic community. A comparison of array results from more than 70 laboratories testing the same DNA is unprecedented, and the high rate of concordance (almost 96%) provides tremendously encouraging information in support of the reproducibility of microarray testing in clinical diagnostics laboratories. Particularly, as this technology is becoming a first-tier analysis for detecting constitutional abnormalities,1 assurances that laboratories can achieve consistency in detection of copy number changes and interpretation of results using a variety of platforms are critical as the field moves forward.

A recent study showed some of the quandaries faced by different laboratories when evaluating and interpreting the same rare copy number changes.11 As the number of chromosomal microarray studies has increased, more data have become available to correlate findings with phenotypes, and databases cataloging this information are becoming more readily available (e.g., DECIPHER and Database of Genomic Variants). The difference between the high concordance of result interpretation among laboratories participating in the CYCGH Surveys and the variability seen in the study reported by Tsuchiya et al. likely reflects a difference in survey design. The CYCGH Surveys have been specifically designed to test each participant's ability to detect and interpret known clinically significant copy number changes. The CAP/ACMG committee defined clinically significant copy number changes as deletions and duplications of critical regions for which there are well-documented associations with abnormal phenotypes in the literature (e.g., deletion of the common Williams syndrome critical region), or those involving deletions and duplications of many megabases of DNA that include many dozens of genes (e.g., 2009 CYCGH-03; Table 1). In addition, the CAP/ACMG committee graded the surveys on the basis of at least 80% of participants reaching a consensus response. Therefore, if at least 80% identified a particular variant, it is considered clinically significant for grading purposes. In contrast, the survey reported by Tsuchiya et al. focused on copy number changes that are not well documented as clinically significant. Therefore, although laboratories have minimal difficulty with the interpretation of known clinically significant copy number changes reported in the literature, those that are currently of undocumented clinical significance pose a greater challenge. As array platforms increase in density, these undocumented copy number changes also increase in frequency and are now more commonly encountered than the known clinically significant findings. To address this challenge, the ACMG has recently published guidelines for interpreting and reporting copy number changes.9

The higher probe densities available on newer platforms has resulted in an increase in the detection of small (e.g., <100 kb) copy number alterations and is anticipated to increase. Similarly, the number of benign copy number changes recognized would also be expected to increase, and the number of clinically significant alterations recognized could also potentially increase. Because the intent of the CYCGH Survey was to assess overall processing, analysis, and interpretation of array specimens by individual laboratories, it was necessary to standardize the Survey sufficiently to minimize cross-platform variability and avoid discrepancies that could result from the use of platforms with higher probe density. The consistency with which the majority of laboratories have reported the presence or absence of abnormalities indicates that this effort has been successful. We believe that the high level of concordance in interpretations (>95% for almost 500 analyses) indicates the robust power of the array procedure and supports the belief that this testing, regardless of the platform used, is reliable, reproducible, and sensitive.