Introduction

The number of available genetic tests is rapidly increasing, as is the number of genes included in any given test.1 At the same time that clinicians are handling a much larger volume of variants, it has become clear that classifying variants is more challenging than previously thought.2 Insights from large-scale sequencing efforts such as gnomAD have called into question historical variant classification standards and have led to the reclassification of many variants.3, 4 Data-sharing efforts such as ClinVar have revealed that disagreement about the classification of variants is not infrequent.1, 5, 6, 7, 8, 9, 10 These developments have sparked changes in variant classification approaches, including a shift to greater stringency.6, 7, 11

There is evidence that the frequency of disagreements in classifications is particularly high in cardiovascular genetics.7, 8, 9 The 2015 American College of Medical Genetics and Genomics variant classification guidelines were designed to be broadly applicable to all areas of genetics.11 The lack of gene and disease specificity has made application of these guidelines in specialty areas such as cardiology problematic.12 In response to these challenges, many clinicians in cardiovascular genetics have taken a more active role in the classification of variants. The majority (81%) of clinical cardiovascular genetic counselors report that their clinical teams reevaluate variant classifications received from genetic testing laboratories.13

While previous studies have compared classifications by different laboratories, differences in classifications made by laboratories and clinicians have not been examined. We aimed to delineate the frequency and nature of differences in variant classifications between cardiovascular genetics clinicians and genetic-testing laboratories.

Materials and methods

The Stanford Center for Inherited Cardiovascular Disease provides interdisciplinary cardiovascular genetics care for cardiomyopathies, arrhythmias, dyslipidemias, and aortopathies. The data set included all variants reviewed by clinicians in our center following clinical genetic testing between May 2007 and August 2016. Study methods are outlined in Supplementary Figure S1 online.

Clinicians’ classifications

As part of routine clinical care, members of the clinical team reviewed and classified variants reported via clinical genetic testing by accredited genetic-testing laboratories. The clinical team’s variant classification process is detailed in Supplementary Figure S2.

Comparison of the clinicians’ and laboratory’s classifications

Classifications of variants by clinicians were compared with those reported by the genetic-testing laboratory on the most recently issued clinical genetic-test report. To facilitate such comparisons, all classifications were translated into the American College of Medical Genetics and Genomics (ACMG) five-tier system (pathogenic (P), likely pathogenic (LP), variant of uncertain significance (VUS), likely benign (LB), benign (B)).11

Clinician–laboratory classification pairs were categorized as discordant if the clinicians’ classification and the laboratory’s classification differed in terms of a major classification category (for example, P/LP versus VUS) (Supplementary Figure S3a). Classification pairs were considered concordant if classifications were exactly the same (for example, LB versus LB) or differed only in the degree of confidence (for example, LP versus P).

We also sought to determine whether classification differences would affect clinical care of the patient or family. Clinically significant discordance was defined as P/LP versus VUS/LB/B, since P/LP variants are likely to be used to direct care while VUS/LB/B typically are not (Supplementary Figure S3b). In defining clinically significant discordance we took into consideration the fact that many clinicians treat P and LP variants in the same way.

ClinVar comparisons

For the subset of variants with discordant classifications, the clinicians’ and the testing laboratory’s classifications were compared to classifications submitted to ClinVar by other clinical genetic-testing laboratories. Data in ClinVar were accessed on 5 November 2016.

Results

Summary of data set

The data set included a total of 639 unique variants with 688 laboratory variant classifications (Table 1). Fourteen genetic-testing laboratories were represented (Supplementary Table S1), though 95% of classifications came from the five most frequently used laboratories.

Table 1 Variant characteristics
Full size table

Discordance between clinicians and testing laboratory

The clinicians’ and testing laboratory’s variant classifications were discordant 18% (126/688) of the time (Figure 1a). Discordance that would affect clinical care was found in 83% (105/126) of discordant classifications and 15% (105/688) of all classifications (Figure 1a). In most cases discordance arose because the clinical team was more conservative; clinicians were less likely than the laboratory to consider a variant pathogenic or likely pathogenic (P < 0.00001; Figure 1b). The most frequently observed difference was discordance between a laboratory classification of LP/P and a clinician classification of VUS, which occurred in 78% (82/105) of clinically significant instances of discordance (Figure 1b).

Figure 1
figure 1

Discordance between classifications by cardiovascular genetics clinicians and the genetic testing laboratory. (a) Frequency of discordance between clinicians and testing laboratory. (b) Nature of discordance. Classification pairs noted as clinicians:lab—VUS:LP/P (82/125, 65.6%); LP/P:VUS (23/125, 18.4%); VUS:LB/B (13/125, 10.4%); LB/B:VUS (6/125, 4.8%); LB/B:LP/P (1/125, 0.8%), LP/P:LB/B (0/125, 0%); P < 0.0001 for Fisher’s exact test across clinicians–laboratory classification pairs. (c) Frequency of discordance varies by testing laboratory’s classification. P < 0.0001 for Fisher’s exact test across three testing-lab classification categories. (d) Frequency of discordance varies by testing laboratory. VUS = variant of uncertain significance. P < 0.0001 for Fisher’s exact test across the top five most frequently used laboratories.

Full size image

Discordance varies by laboratory classification and laboratory

The frequency of discordance varied depending on the testing laboratory’s classification (Figure 1c). Discordance was less common when the testing laboratory defined the variant as a VUS (8%) compared with L/LP (29%) and B/LB (68%) (P < 0.0001). Discordance also varied depending on the testing laboratory used, ranging from 3% to 33% in the five most commonly used genetic-testing laboratories (P < 0.00001) (Figure 1d).

Of note, variant characteristics such as type of variation, novelty, case count, gene, or segregation data were not associated with discordance. We saw a bivariate association between timing of testing and frequency of discordance. However, this was confounded by changes in laboratories used over the course of the study period. On multivariate logistic regression, including both timing of testing and testing laboratory, timing variables were not significant.

ClinVar comparisons and discrepancies

Fifty-seven of the variants with discordant classifications in our data set had a classification by at least one other clinical genetic testing laboratory in ClinVar.

We first sought to assess the level of discordance for these variants among genetic-testing laboratories in ClinVar (including the testing laboratory). Over half of the discordant variants with entries in ClinVar from more than one genetic-testing laboratory (25/45, 55.6%) had discordant classifications between laboratories in ClinVar, and most of these were clinically significant (88.0%, 22/25).

We then compared the classifications by the clinical team and by the testing laboratory to the classifications by other clinical genetic testing laboratories provided in ClinVar. The clinical team’s classification was discordant with one or more laboratory in ClinVar in 49.1% (28/57) of cases. The testing laboratory’s classification was discordant with another laboratory in ClinVar in 82.5% of cases (47/57, P = 0.0002). More laboratories in ClinVar agreed with the clinical team’s classification than with the testing laboratory’s classification (mean 1.4 vs. mean 0.6, P = 0.0005).

Discussion

Clinicians’ variant classifications differed from the classification reported by the genetic-testing laboratory 18% of the time, and most of these differences (83%) would affect clinical care for the patient and/or family (15% of all classifications). Clinicians tended to be more cautious in their classifications, calling fewer variants LP/P. Notably, the clinical team’s classifications were more in line with classifications by other laboratories, as represented in ClinVar.

It is impossible to know which classification is correct, the clinicians’ or the laboratory’s. Variant classification is inherently probabilistic and to date most methods remain subjective. Without a gold standard or a method for knowing the ground truth regarding a variant’s pathogenicity, we cannot assess which classification is accurate. It is notable that when the clinicians’ classification differed from the laboratory’s classification, the clinicians’ classification was more in line with classifications by other laboratories. It is also important to consider the time frame for the study (2007–2016) when interpreting our finding that the most common form of disagreement was when the laboratory classified a variant as LP/P and the clinicians classified it as a VUS. During this time period variant classifications across the field of genetics have become more stringent.14 It is likely that if these variants were reassessed now, some of them would be reclassified by the laboratory as VUS, thus resolving the discordance in classification.

The frequency of disagreement between classifications by the clinical team and laboratory classifications that we observed (18%) is comparable to the frequency of disagreement observed between genetic-testing laboratories (12–53%) in previous studies.1, 5, 6, 7, 8, 9 Discordance between laboratories arises from a variety of factors, most notably from lack of data sharing, outdated classifications, and inconsistencies in the use of classification criteria.5, 6, 7, 10 Recent efforts have shown that when these key issues are addressed and classifications are discussed on an “as needed” basis, discordance can often be resolved.5, 6, 7, 10 However, it is notable that even with these efforts, 13–44% of discordant classifications cannot currently be resolved.5, 6, 7, 10 These data point to the probabilistic nature of variant classification and they also suggest that a small portion of variants is particularly challenging or controversial. We found that variants with discordance in our data set had a particularly high level of discordance between clinical genetic-testing laboratories in ClinVar (55.6% compared to 17–19% across all of ClinVar).1, 8 This suggests that at least some of the variants that were discordant in our data set are especially challenging to classify.

The majority of cardiovascular genetics groups assess variant classifications after receiving a genetic test report, motivated by a sense of responsibility for the result as well as previous experiences with reclassifications and disagreements between laboratories.13 These motivations are in line with studies that suggest discordance in genetic-test interpretation is more common in cardiovascular genetics.7, 8, 9 They also conform with our finding that clinicians approach variant classification with caution; when they disagree with the testing laboratory they tend to downgrade a LP/P classification to a VUS. A certain amount of caution in response to challenges in variant classification may be particularly apt in the setting of cardiovascular genetics, where misclassifying a variant as LP/P could lead to inappropriately discharging variant-negative healthy relatives from cardiac screening and leaving them at risk of sudden death. This tendency of clinicians to be less likely to classify a variant as LP/P is also consistent with recent increases in stringency in variant classification practices in the field as a whole.14

In addition, our data show that when the clinicians’ classification differed from the testing laboratory’s classification, more laboratories in ClinVar agreed with the clinicians’ classification than with the testing laboratory’s classification. This indicates that the clinicians’ classification is aligned with the rest of the field. Our data suggest that there is some benefit from clinician involvement in variant classification, and that the expertise of specialized clinicians may be a more important factor in variant interpretation than has been previously established. Periodic reassessment of classifications provides a key opportunity for clinicians to participate in variant classification, whether by performing reassessments themselves or by prompting the testing laboratory to do so. Automatic periodic reassessment of variants is not currently among the services provided by genetic-testing laboratories; however, data from several studies demonstrate that clinically significant reclassifications occur when variants are reevaluated.6, 7, 15 When considering the role of clinicians in variant assessment, it is important to note that not all clinicians have the necessary training or expertise to confidently and successfully classify variants. Genetic testing may be ordered by cardiologists who do not have the needed expertise in variant classification to go beyond the assessment provided by the laboratory. Guidelines recommend that genetics evaluations for inherited cardiovascular disease be performed by centers with appropriate expertise and staffing, including cardiovascular genetic counselors as part of the multidisciplinary team.16, 17 Finally, the not insignificant amount of time required for clinicians to assess variant classifications must be considered, particularly with the increasing demand for genetics services and the need to improve efficiency and increase clinical volumes.

While discordance in genetic test interpretation has only recently been broadly recognized, discordance in clinical interpretation within the medical field as a whole is not novel.8 Examples include differences in interpretation of ventricular tachycardia on electrocardiograms and assessment of anatomic pathology slides.18, 19 Genetics clinicians and laboratories can learn from insights and processes developed in other fields, such as the use of mandatory second-opinion review in surgical pathology.19 Such review could be performed by the clinical team if they have sufficient expertise, by another genetic testing laboratory, or, more passively, by checking ClinVar. As we learn more about how to minimize, manage, and resolve differences in variant classification, our field will undoubtedly uncover insights and develop standards that could benefit other areas of medicine, such as the movement toward data sharing.

While the uncertainties of variant classification are challenging for clinicians to manage, the ultimate impact is, of course, on patients and families. Of utmost importance are the ways in which clinicians help families navigate the uncertainties inherent in genetic-test interpretation, inconsistencies in classification, and changes in classification over time.20

Conclusion

Within our cardiovascular genetics center we found that our clinical team’s classifications of variants differed from the genetic testing laboratory’s classifications 18% of the time, with most of those differences impacting clinical care. This rate of disagreement is comparable to the rate of disagreement reported between genetic testing laboratories, reflecting the need for efforts to decrease discordance and increase accuracy in genetic-test interpretation. Our data suggest there may be a role for expert clinicians in the assessment of variant classification.