INTRODUCTION

Emergence of liquid-based Pap tests in clinical screening for the precursors of cervical carcinoma has the potential for revolutionizing the traditional Pap test. Recent clinical series (1, 2, 3) have suggested that the ThinPrep® PapTest™ (Cytyc Corporation, Boxborough, Massachusetts) is more sensitive than the traditional Pap smear for the detection of cervical squamous intraepithelial lesions (SIL). In addition, the ThinPrep® Pap test has the added benefit that a residual sample, especially in problematic cases, can be used to test for human papillomavirus (HPV) DNA using the Hybrid Capture II technique (4). Because of the high correlation of HPV results (using high-risk HPV probes) with SIL (5, 6, 7), the results of this test can be used to direct the follow-up of women with ASCUS (atypical squamous cells of uncertain significance) Pap results (5, 8). This report describes our experience with the use of the HPV test as a quality assurance monitor for cytologic interpretation. Use of HPV DNA testing in quality assurance monitoring was first suggested by Sherman et al. (9). At present, correlation of cytology and histology is performed using those cases coming to biopsy. Although the current approach certainly gathers substantive information, biopsy correlation is laborious and suffers from similar problems of observer variability (10, 11) and sampling error (11) as cytologic interpretation. In some cases, colposcopic biopsies can be falsely negative, which significantly complicates the correlation process. Furthermore, tissue studies are only performed on a subset of patients, that is, those with abnormal results who return for follow-up in the same institution. This leaves significant gaps in the quality assurance process, particularly involving the incidence of falsely negative cytologic interpretations. Use of this molecular test as an additional monitor can offer a complementary, objective measure of the effectiveness of cytologic interpretation. This study was undertaken to evaluate the potential for this approach using our clinical cases.

METHODS

Patient Selection

This study was approved by the Institutional Review Board of the University of Oklahoma Health Sciences Center. The samples were derived from the ThinPrep® Pap cases submitted for cytologic interpretation to the Cytopathology Laboratory of University Hospital as part of routine patient care. The patient population is predominantly that of a screened, low-risk group. However, there is a tendency for our referring clinicians to offer the ThinPrep® Pap test to patients with a history of abnormal smears, so that the samples included here represent a higher risk subset of the population served. The 397 test cases were unselected patients with a spectrum of cytologic interpretations, including those interpreted as negative for tumor or dysplasia. The diagnoses used are the original results as signed out by one of three rotating pathologists or one of nine cytotechnologists. The cytology personnel were not aware of plans to send samples for HPV DNA analysis, and the molecular pathology laboratory personnel had no access to the cytologic interpretation.

HPV DNA Testing

The residual PreservCyt (Cytyc Corporation, Boxborough, MA) vial from the test cases was sent to the Molecular Pathology Laboratory for HPV testing. This occurred immediately before routine disposal of the residual sample after the cytologic report was rendered and within 2 weeks of receipt in the laboratory. The HPV DNA test, using the Hybrid Capture II Microplate (HCII) System (Digene Corporation, Beltsville, MD; 12), was performed on cell pellets derived from 4 mL of residual volumes of PreservCyt. This chemiluminescent signal-amplified hybridization assay uses an RNA probe cocktail that detects the high-risk HPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68. The cocktail for the low-risk HPV subtypes was not used in this study. The HCII microplate was read on a Dynex MLX luminometer (Dynatech Technologies, Chantilly, VA). For the assay to be valid, positive and negative calibrators must meet set criteria; samples with readings above the mean value of the positive calibrators (1 pg/mL) are considered to be positive. Test results are considered to be equivocal when their values are below the mean cutoff value of the positive controls by less than 15%. For the purposes of statistical comparison with cytology in this report, equivocal cases were deleted from further analysis.

RESULTS

The results of the cytologic study and the HPV analysis for the 397 samples tested are shown in Table 1. The HPV DNA results of five cases were equivocal and included four interpreted as negative by cytology and one as ASCUS by cytology. Because of the uncertain significance of this result with respect to the presence or absence of HPV DNA, these cases (1.3% in this series) were excluded from further analysis in this study; however, this percentage may be a helpful indicator in monitoring the validity of the test conditions in the laboratory.

TABLE 1 Distribution of Cytologic Interpretations of ThinPreps with HPV DNA Analysis of Residual PreservCyt Samples
Full size table

The percentage of high-risk HPV DNA-positive cases in the series increased with the severity of the cytologic interpretation, showing a high association with SIL, particularly with diagnoses of high-grade dysplasia. It should be noted that because the subset of low-risk HPV subtypes (i.e., HPV 6, 11, 42, 43, 44) was not included in this study, some lesions associated with these viruses would not be detected in our series. Other studies (5) have indicated that these cases would be few and unlikely to be associated with high-grade squamons intraepithelial lesion.

Using the HCII test results as the gold standard, the results of the evaluation of the cytologic diagnoses for the ThinPrep® Pap cases from our laboratory are shown in Table 2. Eliminating ASCUS and equivocal HPV results from this population, there is strong agreement (80.8%) between cytologic interpretation of negative/SIL and HPV positive/negative results for the remaining 266 diagnostic cases in this study (Cohen's kappa =.59; 95% confidence interval [CI] =.48–.68; 13). The ASCUS cases were excluded from the comparison statistics because of the heterogeneity inherent in that designation. However, the percentage of ASCUS cases with high-risk HPV DNA-positive Hybrid Capture results (43.7% in this series) serves as a useful benchmark in itself. The percent of negative cytology cases that had HPV DNA-positive results (17.1% in this report) is a similarly useful monitor.

TABLE 2 Analysis of Cytologic Interpretations of Thin Prep Pap Test Result Using HPV DNA Positivity of Residual PreservCyt Samples as the Gold Standard
Full size table

DISCUSSION

Cytologic-histologic correlation has traditionally been used as a quality assurance monitor for cytologic interpretations of cervical smears to the extent that it is mandated by CLIA '88. However, histologic assessment, the so-called gold standard, suffers from similar difficulties as cytologic analysis. Both are subject to sampling variation and subjective interpretation (10, 11, 14). In addition, cytohistologic correlation is extremely labor intensive, taking place over several months and reflecting only the subset of patients undergoing histologic evaluation in the same laboratory. As a molecular test, HPV DNA testing using HCII has an endpoint, expressed as a ratio of the test sample result compared with that of a known positive sample (1 pg/mL), that is independent of subjective morphologic interpretation. Because of the high correlation of HPV DNA results generated by HCII with PCR-based HPV testing (7) and the presence of SIL (5, 15, 16, 17), this approach (see Fig. 1 for flow chart) promises to provide accurate and reproducible information for quality assurance purposes in a highly efficient manner. An added benefit is that the HPV DNA results and the cytologic interpretation are generated from aliquots of the same patient sample, thus minimizing the problem of sample variation. The comparison values generated in this pilot study are being used to establish an initial baseline for this proposed quality assurance program. Data generated through subsequent periodic monitoring will be compared and added to previous data to evaluate trends. Although these results were generated from the screening activities of all personnel in the cytopathology laboratory, they could similarly be used to monitor the diagnostic performance of the individuals in the laboratory.

FIGURE 1
figure 1

Chart depicting the work flow pattern for using HPV analysis into the quality assurance program in the cytopathology laboratory. *, HPV positive with cytologic interpretation of SIL or HPV negative with negative cytology; **, HPV negative with cytologic interpretation of SIL.

Full size image

Because the percentage of HPV DNA positive cases in a laboratory will be determined by the patient population served by that laboratory (9), results probably will vary from laboratory to laboratory. However, the percentages in the individual cytologic categories should be reproducible for each laboratory, given an adequate number of cases. In addition, national target ranges can be established as guidelines for individual laboratory performance using input from other laboratories. Although the percentage of abnormal cases in a laboratory will vary with the population served, the percentage of HPV DNA-positive ASCUS, negative, or SIL cases should ideally fall within a limited range. Significant deviations from consensus values should trigger a reassessment of cytologic criteria within the laboratory. On a case-by-case basis, noncorrelation between the cytologic and HPV results should trigger rescreening of the cytology sample, and a determination should be made as to whether there was a screening/interpretive error using standard cytologic criteria. Unexplained variances should be noted and monitored.

Although clearly involving more objective data than that of visual interpretation, the use of HCII HPV results as the gold standard for cytologic interpretation requires additional validation and clarification. The results in the literature to date are promising. Comparison studies between HCII and PCR-based HPV assays have shown excellent results with agreement of 91.4% (kappa =.65) for low-grade squamons intraepithelial lesions lesions in the ALTS trial (7). Reithmuller et al. (18) found that HCII and PCR identified nearly equivalent prevalences of HPV in cervical smear specimens. Rigorous quality control standards must be in place in the molecular laboratory. In a comparison study between three laboratories using HC I, an earlier version of the current method, Schiffman et al. (19) found strong concordance between interlaboratory correlations and the HPV DNA reference standard as well as with the concurrent cytopathologic diagnoses.

However, there are also ambiguities in data interpretation that remain to be addressed. For example, the clinical significance of the “equivocal” HPV DNA result using the HCII is unclear. Possible causes of these equivocal results are an extremely low copy number of HPV DNA in the cell sample that is beneath the detection threshold for the HCII or undefined variances in the cell samples that generate nondiagnostic, spurious results. When used clinically to determine follow-up of a patient with ThinPrep® Pap interpreted as ASCUS, we currently recommend that a new sample be obtained for retesting from patients with equivocal HPV DNA results.

A second unresolved issue relates to the significance of the cases defined as false-negative cytologic interpretations on the basis of positive HPV DNA results. It is generally accepted that some women test positive for HPV who do not have SIL. Using the HC I method, an earlier version of the current method, Hall et al. (17) found that 35% of women testing positive for HPV DNA were disease negative, whereas Clavel (6) reported that 8.8% of women with negative cytologies tested positive for high-risk HPV subtypes. Riethmuller et al. (18) found that 14.3% of their cases with negative cytology contained high-risk HPV DNA using the HCII, compared with 25.1% by PCR. The significance of HPV DNA-positive/cytology-negative cases in our study (17.1%) is unclear. Although classified as false-negative cytologies for the purposes of this analysis, these cases may be negative because of a multitude of factors including cytologic undercalls, subclinical HPV infection, relative insensitivity of cytologic detection compared with the HCII, or other, as yet undefined, methodological problems. Occasional sporadic false-positive results have been reported using the earlier HC I test (19). We intend to re-evaluate our cases in this category (as well as the HPV-defined false-positive cases), which will be the subject of a subsequent publication. For quality assurance purposes, such noncorrelating cases (as well as HPV DNA-negative cases with cytologic interpretations of SIL) should be referred to the cytopathology supervisor and the ThinPrep® Pap slides reviewed for diagnostic errors.

In summary, this study illustrates the substantial utility of HPV DNA testing of the residual volume of liquid-based Pap tests in a quality assurance program for the cytopathology laboratory that offers these tests.