Introduction

Cystic fibrosis (CF; OMIM 217900) represents one of the most commonly tested monogenic disorders within routine DNA diagnostic services in Europe and North America. CF and CFTR-related disorders are associated with a high number of different alleles in the CFTR gene (OMIM 602421).1, 2, 3 Thus, analysis of a considerable number of CFTR mutations is required to achieve a sufficiently high (ie, >85%) population-specific mutation detection rate.1 Utilization of complex commercial assays for routine CFTR gene testing is gradually increasing,2 as these assays provide ready-to-use solutions for the detection of up to 90% of CF-causing alleles in European-derived Caucasians.3

EuroGentest (www.eurogentest.org) is a FP6 EU Network of Excellence that primarily focuses on the harmonization and improvement of the quality of genetic services in Europe.4 Within the frame of its activities, we conducted a pilot survey whereby participants of the 2005 annual European CF external quality assessment (EQA) scheme (www.cfnetwork.be) were asked to share their experience with the use of CE-marked in vitro diagnostic (IVD) assays within routine CF DNA diagnostics. The aim of this study was to provide evidence supporting the necessity of verification procedures in qualitative commercial assays before their use in routine DNA diagnostics and for drafting of respective guidelines.

Methods

Variability in the use of commercial assays was evaluated by a structured questionnaire. This questionnaire was sent to all participants of the annual EQA scheme and its content was focused on the most commonly used commercial assays. These assays represent three different technical approaches to the detection of known mutations in the CFTR gene: (1) ELUCIGENETM assays (Tepnel Diagnostics) – multiplex ARMS-based PCR followed by horizontal agarose gel electrophoresis of amplicons (versions: CF29, CF30, CF Poly-T, CF7, CF-MEP, including CF-HT, that relies on capillary electrophoresis); (2) INNO-LiPATM assays (Innogenetics) – reverse blot hybridization and post-PCR analysis based on biotin–streptavidin–peroxidase sandwich hybridization and colorimetric detection with a chromogen (versions: CFTR19, CFTR17+Tn update, CFTR 17+Tn, CFTR Italian Regional, CFTR12); and (3) CF Genotyping AssayTM (Abbott Diagnostics) – based on the OLA-PCR and capillary electrophoresis (versions 2 and 3). All analysed assays were CE marked.5 Results of this survey were subsequently discussed with users and/or respective manufacturer representatives.

Results

Altogether 125 responses from DNA diagnostic laboratories (further only laboratories) representing 20 countries were received. With regard to the assays tested: 17.6% of respondents used ELUCIGENE, 40.8% used INNO-LiPA and 41.6% used the CF Genotyping Assay. From the total of 197 laboratories participating in the 2005 CF EQA scheme, 169 used one or more of these assays. As there is a similar distribution in the utilization of respective assays within the entire EQA scheme and within our survey (Figure 1), our results are representative.

Figure 1
figure 1

Proportion of the most commonly used CE-marked IVD commercial assays for CF DNA diagnostics within the survey and within the entire 2005 European CF EQA scheme.

Full size image

However, less than half of the respondents performed in-house verification of CE-marked IVD assays before their use in routine diagnostics (30% for ELUCIGENE, 31% for INNO-LiPA and 47% for CF Genotyping Assay). Moreover, owing to a variety of reasons, almost half of the respondents (43.6%) changed the manufacturer-recommended protocols: 32% for ELUCIGENE, 41% for INNO-LiPA and 58% for CF Genotyping Assay. The most common types of modifications are listed in Figure 2.

Figure 2
figure 2

Examples of modifications of manufacturer-recommended protocols. Other: use of different DNA templates (eg, single-strand DNA), utilization of different software or different hybridization conditions and/or temperatures.

Full size image

Despite such a broad scale of modifications of recommended protocols, only 60.4% of respondents properly verified respective deviations before their implementation in routine practice. Users of CF Genotyping Assay verified protocol modifications in 73.3% of all cases, whereas users of ELUCIGENE and INNO-LiPA assays have done so in 55.5% and 52.4%, respectively.

Almost 23% of respondents found sporadic and/or recurrent problems with correct genotyping of certain alleles and/or with the overall assay robustness (data not shown). These problems were variable and have been observed both in unmodified and/or modified assays in 16 and 84% of the cases, respectively. In addition, only 3 out of the 10 laboratories that made genotyping errors in the EQA scheme participated in this survey. Therefore, we could not assess whether there is any difference in error rate in those who modified protocols versus those who adhered to manufacturer's recommendations.

On the basis of these preliminary data, we have contacted company representatives to discuss observed problems with assay performance, discussed all issues related to the modifications of recommended protocols and suggested ways for improvement of their assays. Although this approach was highly regarded by all companies, we have not been informed whether their assays will be modified to reflect user's comments due to confidentiality measures from their side.

Discussion

In DNA diagnostics, genotyping is usually performed only once in a patient's lifetime. Therefore, it is essential to develop highly accurate testing platforms for commercial assays. This prerequisite is fulfilled by proper pre-market industrial assay development and subsequently thorough validation procedures so that manufacturers fulfil CE-marking requirements. Recently, laboratories have increasingly shifted from ‘home-brew’ techniques towards commercial assays and had established them as primary genotyping platforms.6 Furthermore, the growing impetus for the implementation of quality assurance and accreditation in laboratories have further accelerated this trend.

However, users should be aware of the fact that the routine implementation of commercial assays alone does not ensure high accuracy of genotyping.7 Our data gathered from the 2005 CF EQA scheme together with previous reports6, 8 suggest that a considerable error rate still persists and needs to be taken into account by laboratories.

Our survey has also demonstrated that less than half of the respondents verified commercial assays before their implementation in routine diagnostic practice. Moreover, proper verification was not performed after modification(s) of manufacturer-recommended protocols. Intriguingly, the observed rate of modifications of these protocols was rather high, that is, close to 50%. Although each modification should have been verified before its implementation in routine testing by laboratories,6, 9 results of this survey demonstrate that only 60.4% of the respondents addressed this important issue.

In general, the rate of pre-diagnostic implementation verification of commercial assays is still insufficient and laboratories have an unsubstantiated a priori assumption that the use of a commercial assay alone assures high-quality results. Moreover, laboratories frequently modified manufacturer-recommended protocols (Figure 2), but did not verify these changes. Cost-saving measures were often the main reason for modifications, for example, by cutting strips in half and correspondingly decreasing the PCR master-mix volume. In other instances, decreasing of recommended DNA template concentration was intended to avoid ‘high background’. Many laboratories did not consider using recommended thermal cyclers, genetic analysers or DNA isolation techniques as essential for the overall quality of their results. Although respective modifications could be considered as neutral, advantageous or even deleterious, these should always have been verified/validated by laboratories.

In summary, presented survey results highlight the necessity to strengthen DNA diagnostic laboratory awareness of validation/verification usefulness in all types of genetic tests, even if these comply with CE-marking requirements.