An audit of breast cancer pathology reporting in Australia in 1995

To measure the quality of pathology reporting of breast cancer and establish a baseline against which future changes can be measured, we audited item completeness in breast cancer reports in Australia in 1995 before the release of specific recommendations from the Australian Cancer Network. Tumour type and size were given in reports of invasive breast cancer for 93% of women, 70% had, in addition, grade and clearance of the margins while only 28% had all recommended information. The most complete items in reports were histological type of breast cancer (99.6% of cases), tumour size (94%, 95% confidence interval (CI) 92–95) and margins of excision (87%, 95% CI 85–89). Histological grade (84%, 95% CI 82–86 of cases) and presence or absence of ductal carcinoma in situ (DCIS) (79%, 95% CI 77–81) were less complete and vessel invasion (61%, 95% CI 58–63) and changes in non-neoplastic breast tissue adjacent to the breast cancer (68%, 95% CI 66–71) the least complete. Less than half the reports of DCIS reported on tumour size (49%, 95% CI 42–57), presence or absence of necrosis (41%, 95% CI 34–49) or nuclear grade (39%, 95% CI 31–46). Around 1500 reports were identified as issued by 147 laboratories and 392 pathologists; 69% of pathologists issued fewer than two reports a month in the audit. We concluded that infrequency of reporting may have contributed to incompleteness of reporting. In addition, we found significant variation across Australian states with some indication that reporting was consistently poor in one state. The audit highlighted areas for improvement for breast cancer reporting in Australia. Research evidence suggests that multifaceted strategies are needed to assist practitioners with implementing more uniform reporting standards. © 1999 Cancer Research Campaign


MATERIALS AND METHODS
We obtained copies of pathology reports, identified by number only, from Australian state and territory cancer registries for cases of invasive breast cancer or DCIS diagnosed in April and May 1995 (and June in the smallest states and territories). In one State, only three of six laboratories agreed to participate, supplying 57% of the cases expected in that State.
Completeness of reporting of invasive breast cancer was examined for the key items: tumour size, histological type, histological grade, margins of excision, vessel invasion and changes in the adjacent breast tissue. We examined reporting of changes in adjacent breast tissue in two separate items, the presence or absence of DCIS and changes in adjacent non-neoplastic tissue. In addition, we examined the reporting of nodal status of all breast cancers and the presence or absence of an extensive intraductal component (EIC) associated with infiltrating ductal carcinomas nitric oxide synthase (NOS) (no special type).
Reports were classified by the type of diagnostic or therapeutic procedure from which the specimen came. Analyses were based on reports from biopsies and mastectomies which contained information about the breast cancer and excluded fine needle aspirations (FNA), core biopsies and slide reviews.
Completeness was defined as the percentage of reports in which a definite statement about an item, either positive or negative, was made. A data dictionary and coding manual and data collection forms were developed for use by three pathology registrars who were the auditors. Typically, the data form sought to classify each pathology item as present, absent, uncertain whether present or absent or not mentioned. In addition, the grade of a tumour and actual measurements of size or margins were noted when given. One data collection form was completed for each pathology report and there were separate forms for invasive breast cancer and DCIS. The sample size (1600) was chosen to give completeness percentages that had 95% confidence bounds that were not more than 5% on either side of the point estimate.
The extent of agreement among auditors was examined in a sample of 10% of pathology reports that were audited independently by all auditors. The kappa statistic was used to examine variation between pairs of auditors after adjustment for the extent of agreement that would be expected by chance (Cohen, 1960).
Three sets of analyses were undertaken which addressed all reports for each woman, all reports from biopsies and mastectomies and the effects of reporting 'volume' on completeness. For each woman with invasive breast cancer, an item was regarded as complete if the biopsy or mastectomy report gave the relevant information. It is possible that the sets of reports available for some women were incomplete sets. Analyses by reporting 'volume' were based on the numbers of reports issued by each laboratory or pathologist in April and May 1995.
Ninety-five per cent confidence intervals (CI) were calculated for percentages of complete items using the Normal distribution for numbers of ten or more and exact limits using the F distribution for smaller numbers (Armitage and Berry, 1991). Homogeneity of proportions was tested using the χ 2 distribution and trends in percentages across ordered groups were assessed by the Mantel-Haenszel test for trend (Armitage and Berry, 1991).

RESULTS
We examined 1563 pathology reports of invasive breast cancer from biopsies and mastectomies for 1409 women diagnosed with breast cancer in all Australian states and territories in 1995. A total of 195 reports of re-excision specimens which showed no residual cancer were excluded. More than half (56%) the women had only one report, 37% had two reports and 7% had three or more.

Agreement between the auditors
Auditor 1 completed data collection forms for 879 (34%) pathology reports of invasive breast cancer, Auditor 2 completed 1312 (50%) and Auditor 3 completed 399 (15%). Agreement was calculated in 140 pathology reports examined by both Auditors 1 and 2 and 38 reports audited by all three.
The observed agreement between pairs of auditors in classifying completeness was very high for tumour type, size, grade and vessel invasion (89% or more) and lower for clearance of the margins of excision, the presence or absence of DCIS and changes in adjacent non-neoplastic tissue (between 66% and 88%). The values of kappa were slightly lower than the observed percentages of agreement.

Reports of invasive cancer for each woman
Completeness of reporting invasive cancer for each woman was highest for histological type of cancer (99.6% of cases), tumour size (94%, 95% CI 92-95) and margins of excision (87%, 95% CI 85-89). Histological grade (84%, 95% CI 82-86 of cases) and presence or absence of DCIS (79%, 95% CI 77-81) were reported less while vessel invasion and changes in non-neoplastic breast tissue adjacent to the breast cancer were reported for fewer than 70% of women. Tumour type and size were both stated in 93% of reports and 70% of reports had, in addition, grade and clearance of the margins. Only 28% of all women had all key items reported ( Figure 1).

Reports of invasive breast cancers from biopsies and mastectomies
Item completeness was examined in greater detail in the 1563 individual reports of invasive breast cancer from biopsies and mastectomies. Of these, 21% gave a definite finding for all key items, 51% had six or seven, and 73% had five or less items complete. As might be expected, less complete information was given in reports which followed a re-excision, especially tumour size (23 percentage points less) and histological grade (22 percentage points less), but also vessel invasion and presence of additional DCIS (10 percentage points less). Reporting of tumour size was high (89%, 95% CI 87-90), and higher (91%, 95% CI 90-93) when there were one or two tumours than when there were three or more (65%, 95% CI 57-73%).
Histological grade was given in the majority (80%, 95% CI 78-82) of reports. Overall, 31% of reports named the Bloom and Richardson grading system, as recommended by the ACN Working Party. Reports with breast cancer size more often gave grade (85% of 1175 reports) than those without (45% of 173). Grade was given less often (33% of 42 reports) for smaller (< 4 mm) than larger (4 mm or more) cancers (85% of 1348 reports).
Given the importance of clear margins for prognosis and treatment decision, reporting was surprisingly variable. Comments about excision margins were made in 83% (95% CI 81-85) of reports but only in 62% (95% CI 59-65) of 1038 reports with clear margins was a measurement of the margin given. Moreover, the auditors indicated that, for 33 of 121 reports (27%, 95% CI 19-35) with a stated margin of 1 mm or less, clearance was, in fact, uncertain.
There was statistically significant variation (P < 0.05) by Australian state in completeness of reporting of histological type and grade, size, vessel invasion and changes in adjacent breast tissue (Table 1). Differences in reporting histological type were based on a handful of cases only, while tumour size varied by seven percentage points, histological grade by eleven, changes in adjacent non-neoplastic tissue by 17, and vessel invasion by 18 percentage points. One Australian state had the lowest item completeness in all items (Table 1).
Completeness of reports of tumour size increased significantly (P < 0.001) with increasing age of the women, from 85% at < 50 years of age to 93% at 70+ years of age, while there was significantly (P < 0.001) less complete reporting of DCIS in women 70+ years (69%) compared with women younger than 70 years (77-78%).
The Pathology Working Party defined an extensive intraductal component (EIC) principally as the presence of 'a significant amount' of DCIS in an infiltrating ductal cancer NOS, together with any DCIS in the adjacent breast tissue, noting that there has been confusion about its definition. Few of the 1199 reports of infiltrating ductal carcinomas made comments about EIC status. Thirty-nine per cent described a DCIS component in or adjacent to the invasive breast cancer as 'less than or greater than 25%', 'EIC+ or EIC-', or 'extensive, minimal, or moderate'. Six per cent measured the extent of DCIS but did not indicate its EIC status, 20% indicated only that DCIS was present or absent and 35% of reports did not mention DCIS at all.
A statement about lymph nodes was given in 55% (95% CI 52-57) of reports. Most reports mentioning lymph nodes gave the number of nodes examined (97%) and the number positive (98%; 835 of 852 reports).

Completeness of reports of invasive breast cancer by number of reports for pathologists and laboratories
There were 1316 reports in April and May assigned to categories of reporting volume; the pathologist was unknown for an additional 46 cases. Nearly half the pathologists (47%) had issued only one (29%) or two (18%) reports. The majority (69%) of pathologists issued less than five reports of invasive breast cancer during these 2 months.
Only vessel invasion showed some evidence of increasing completeness with increasing volume of reports. Presence or absence of vessel invasion was described more often by pathologists who had issued five or more reports (59-61%) than less than five (44-57%; P-value for trend = 0.05) ( Table 4).
Of the 129 laboratories identified with 1362 invasive breast cancer reports, most (91%) had issued less than 15 reports in total over the 2-month period while only six had issued 40 or more reports (an average of five a week). The high volume laboratories had more complete information on tumour size (P for trend = 0.05), margins of excision and vessel invasion (P for trend < 0.05 in each case) (Table 4).    each of the four items -histological type, size, grade and clearance of the margins -widely regarded as the key to informed patient management (Nakhleh et al, 1997) and vital for prognosis (Elston and Ellis, 1991;Henson et al, 1991;Wijetunga et al, 1996). Only 28%, however, had reports which gave all the Pathology Working Party's recommended items of information. The deficiencies in individual items such as tumour size (missing for 6% of women), clearance of the margins (missing for 13%) and histological grade (missing for 16%) gave cause for concern. Compared with earlier Australian audits (Table 5), 1995 pathology reports were much more complete, particularly for histological grade (+15 percentage points), excision margins (+10 percentage points) and DCIS (as much as +36 percentage points). Completeness of reporting of tumour size, in comparison, appeared to have reached a plateau, with little change (+1 percentage point) from 1992 after a larger improvement (+10 percentage points) from 1989 ( Table 5).

Seventy per cent of women with invasive breast cancer in 1995 in
Reporting of DCIS was relatively poor. Although most reports gave architechture and three-quarters stated whether margins were involved or clear, less than half gave size, nuclear grade or presence or absence of necrosis, suggesting a lack of a uniform approach to histological reporting of these tumours. A 1995 survey of specialist clinicians, mainly in the USA but including a handful in Australia, ranked size and margin status (99-100% of surgeons and oncologists), tumour architecture (93-98%) and nuclear grade (88-94%) as the most desired information for clinical management of DCIS (Nakhleh et al, 1997).
The ACN's Pathology Working Party recommended that all invasive carcinomas have 'the Elston modification of Bloom and Richardson's grading … applied' (ACN Pathology Working Party, 1997). The differences in completeness of reporting grade between ductal (91% graded), lobular (34%) and other carcinomas (55%) suggests that the grading of other than ductal carcinomas may be a relatively new approach in Australian laboratories. Relative newness may also account for the incompleteness of DCIS reporting when it is present alone and as a component of infiltrating ductal carcinomas NOS. Reports with breast cancer size more often stated grade than those without. This coincidence may suggest that when standard procedures include assessment of one of these features, routine assessment of the other is also likely.
Pathology reports of invasive breast cancers in 1995 in Australia had higher levels of information of size (complete for 94% of women) and histological grade (84%) than a UK health region in 1992 (size and grade each 74% complete; Ma et al, 1997). The UK study, on the other hand, had more complete information (94%) on surgical margins of excision than in Australia (84%; Ma et al, 1997).
We found some evidence that completeness of reporting varied with reporting volume particularly for laboratories rather than individual pathologists. Reporting of cancer size, margins of excision and vessel invasion all increased significantly with laboratory reporting volume. A similar trend was found in the 1992 audit in NSW in which reporting was more complete from pathologists who issued at least 2-3 reports a month or were based in teaching hospitals (Bilous et al, 1995). Volume of institutional reporting in the UK was also associated with item completeness, although inconsistently since reporting of tumour size fell as volume increased while margins of excision and histological grade were more completely reported as volume increased (Ma et al, 1997).
Many pathologists, in assessing breast cancers, would have carried out appropriate procedures (Rippey, 1996;Bull et al, 1997). Failure to state key information in pathology reports has been attributed to inadvertant omission (Bull et al, 1997), the tendency to substitute telephone consultation for completeness in reporting (Miller and Slater, 1996;Hammond and Flinner, 1997), perceptions of clinicians' information needs (Nakhleh et al, 1997) and significant uncertainty about best practice because of changing knowledge of the importance of items for choices about management and prognosis (Bull et al, 1997).
There was important variation among Australian states in completeness of reporting of margins of excision, vessel invasion and changes in adjacent tissue. Although margins of excision and vessel invasion were increasingly completely reported as volume of reporting in laboratories increased, three States had consistently low levels of information. A strong State-level influence was suggested by the observation that one State had the lowest levels of reporting on all items other than tumour type (Table 1). We do not know the reason for this difference between States.
The change from more to less radical surgical procedures has meant that pathology information is not only important in giving a prognosis and deciding on post-operative adjuvant therapy for the woman with breast cancer, but is also important for monitoring the quality of health care. Measurement of quality in surgery, by auditing surgical margins and the numbers of nodes excised, and the performance of early detection programmes by monitoring size, grade and nodal status rely on pathology information. Given the higher incidence of DCIS since the introduction of mammographic screening, it is important also that reporting of the pathology of these cancers is more uniform and of high quality.
Use of a standard form or checklist is the one practice found to be significantly associated with an increased completeness of reporting of selected items in pathology reports of cancer (Zarbo, 1992;Bilous et al, 1995;Wijetunga et al, 1996;Rippey, 1996;Shepherd and Quirke, 1997;Bull et al 1997). Synoptic reporting was recommended by the ACN's Pathology Working Party and its widespread adoption would be expected to lead to further increases in completeness of reporting. These increases would, in turn, be expected to lead to improved outcomes and quality of care. One option to increase the use of synoptic reporting in Australia might be to consider its adoption as an accreditation requirement for pathologists and laboratories.
By measuring completeness of item information, we have undertaken the first steps in ascertaining the quality of pathology reporting. We believe that this simple process has highlighted important gaps in reporting breast cancers in Australia in 1995. We have circulated a summary of audit outcomes to all members of the Royal College of Pathologists Australia and the complete report to major health organizations. Passive, dissemination-only strategies, however, are known to be generally ineffective in improving practice (Bero et al, 1998). While more intensive, 'multifaceted' strategies that combine methods, such as locallybased consensus processes and participation in audit with feedback, would probably be more effective with pathologists and laboratories (Bero et al, 1998), our audit was designed to assess performance at a national or state, and not an individual, level. A much larger sample size would be needed to obtain sufficiently precise estimates of individual performance that could be supplied as individual feedback.
The existence of the ACN's recommendations speaks to a local consensus on the value of key items in breast cancer care. What is needed is a mechanism to assist laboratories and individuals to implement more uniform reporting standards (Bull et al, 1997;Shepherd and Quirke, 1997;Nakhleh et al, 1997). Research evidence suggests that consistently effective interventions for change in health professionals are multifaceted, flexible, systematized, focussed on everyday practice and locally based, and include self-assessment and feedback, practice reminders, consensus processes and interactive educational meetings (du Boulay, 1997;Bero et al, 1998). Currently, no one organization has a mandate to ensure that research findings about pathology reporting standards and interventions to improve professional performance are translated into improvements in everyday practice.